What is Live Unit Testing

Live Unit Testing is a brand-new technology, made available in Visual Studio 2017 version 15.3 or above. Live unit testing enables the IDE to execute unit tests automatically in real time without cod being built and as you make changes to the code.

Which frameworks and tooling support Live Unit Testing?

Live Unit Testing is available for C# and Visual Basic projects using MSTest, NUnit, or xUnit Test frameworks that target the .NET Core or .NET Framework in the Enterprise Edition of Visual Studio 2017.

Problem Statement

Unit Tests, and code have a very close relationship, and both depend heavily on each other. I.e. a change in source code must impact the unit test(s) code for what was being tested. Similarly, a change in a unit test, must impact the code coverage of the source code, which is being exercised via unit tests.

In the light of above mentioned problem statement, there is no efficient and developer friendly solution which a traditional unit tests can solve. Moreover, validating code changes through unit tests may quickly turn out to be a tedious task, as you must run all the unit tests from Test Explore after every code change.

Benefits of Live Unit Testing

It empower the developers to refactor and change their code with greater confidence. Live Unit Testing graphically depicts code coverage in real time and provides a quick visual view of the code coverage, and which code statements are passing in unit tests. As per the name “Live Unit Test”, you don’t have to build the code, and run the tests again via Test Explorer to validate the changes. I.e. soon after code changes are made, unit tests will be automatically executed to reflect the impact (pass, fail).

Quick look at a traditional Unit Test

As explained in my previous article Inside Out : TDD using C# there is always a system under test or code which needs to be exercised via unit tests. This code is available at my GitHub, and this code will work just fine for Unit Test, and Live Unit Test (if you have the required Visual Studio 2017) purposes.

The code below, shows unit tests written for Account.cs class’s IsAccountActive() function.

What is being tested?

As shown in the image above, Account.cs class’s IsAccountActive() method is being exercised. I.e. Account.cs is our System Under Test.

Let’s begin with Red, Refactor, Green

As this code is showcasing Test Driven Development, I.e. write a failing test, and then write code to make that test pass, and so on.

First, let’s write/refactor code to make TestAccountStatus_Active_Success() test case pass. To do so, copy the code from TestCase#1 code snippet from TextFile1.txt (included in the UnitTestBankApplication at my github

Account.cs code when TestAccountStatus_Active_Success() was failing:

Account.cs code changes to make TestAccountStatus_Active_Success() pass:

Now, if you will run the Tests again, by clicking “Run All” option in the Test Explorer, then you shall observe that 1/3 Test case is passing, as shown in the image below.

But what exactly is the problem here?

Unit Test code or source code which is being tested doesn’t explicitly tell which code lines are being covered, not covered, or passed via unit tests. I.e. whenever developer will change the code, test cases need to be run over and over to identify the impact, and that is without any visual clue in the code files. Hence, with traditional unit testing, a developer can’t clearly tell which code statements are covered, or not covered, through passed or failing unit tests.

Turn-on Live Unit Testing

Navigate to the Test menu, expand Live Unit Testing, and then click on Start, as shown in the image below.

If you don’t see this option then you might not have the appropriate version of Visual Studio 2017 Enterprise Edition, which offers the Live Unit Testing feature. However, if you have Visual Studio 2017 enterprise edition, and you don’t see this option, then you can add Live Unit Testing tool from the setup menu, and selecting it from the individual components tab.

Live Unit Testing in action

As soon as you have started the Live Unit Testing, your IDE will instantly start tracking the code changes in the background, as shown by X, , and symbols on the left of code statements in the IDE, as shown in the image below.

As of now, there is only one test passing, let’s take the Test Case#2 code from TestFile1.txt

and put it in Account.cs class’s IsAccountActive() function, and soon after putting the code in there, without building the code explicitly, you shall observe the changes, and notice that now two tests are passing. Alongside, you shall observe the changes to the code line symbols X, , and on the left.

Now, you may want to work on passing the third unit test, and to make that happen as per TDD guidelines, you code should refactor the code logic to accomplish that. Put the Test Case#3 code from TestFile1.txt, into Account.cs class’s IsAccountActive() function

As soon as the code is refactored, again without any explicit build and running tests from Test Explorer, Live Unit Testing observed the code changes and ran the tests. As shown in the image below, now all three unit tests are passing, and all code lines in Account.cs class’s IsAccountActive() shows symbol on their left. The  symbol means that all code statements in this function or block are covered by the Unit Tests.

Now, what’s next?

As you can see in the image above towards the bottom-half, ─ symbol in front of other code blocks, which means that these code blocks are covered by 0 test. I.e. no unit test case(s) exist for these code blocks. You may want to take it to next level by taking code from my GitHub repo, and expand it.

Pause or Stop Live Unit Testing

Once Live Unit Testing has been started, now it can be either paused or stopped. You can navigate to the Test menu, select Live Unit Testing, and then choose either Pause or Stop, as shown in the image below. Based on the choice you make; Live Unit Testing will be either paused or stopped completely.

Excluding individual Unit Tests from Live Unit Testing

Ideally, all the tests in all the projects will be covered once Live Unit Testing has been started.

But, you can use the following attributes to specify in the source code that you want to exclude targeted test methods from Live Unit Testing:

  • For MSTest: [TestCategory(“SkipWhenLiveUnitTesting”)]
  • For xUnit: [Trait(“Category”, “SkipWhenLiveUnitTesting”)]
  • For NUnit: [Category(“SkipWhenLiveUnitTesting”)]

Include/Exclude Unit Test Projects or Test class files

Live Unit Testing is IDE level, and it works for all test projects and all class files which are part of a test project. To include/exclude either a test project or a test class file, you can use the context menu and choose Live Unit Testing, Include or Exclude options. The image below shows these options on the UnitTestBankApplication Test Project, and you may want to try it on test class file as well.

Wrapping up

Live Unit Testing appears to be a milestone in fast-paced, technology-heavy software development field. This certainly doesn’t take away the importance of knowing Unit Test fundamentals, rather, it enforces the TDD (Test Driven Development) mindset in an effective, and productive manner to the developer community, by showing visual clues in the IDE code window, and running the tests while a developer is refactoring the code.

Related Articles:

Inside Out: TDD using C#

Visual Studio and .NET Unit Test Framework

Unit Test Automation with Visual Studio

Introduction

Unit Tests play an important role in delivering high quality software solutions. An ideal unit test is a piece of code (automated test) written by a developer which exercises a small but specific area of code functionality to ensure that it works as expected.

Why Unit Test

According to the Test Triangle, in a software project the largest number of tests must be Unit Tests. Because multiple individual Unit Tests exercises small units of functionality which spans over entire various areas of functionality offered by the software solution.

 

Unit Test Check-List

While writing Unit Test(s) following points must be considered and followed by the Developers and SDETs.

Check

Description

  √

Self-Describing Names

Unit Test method names must be Self-Describing and Pascal case.  For example choose Add_New_Customer_With_Valid_AcccountNumber over AddCustomer_with_validAcc or addCustomer etc.Also focus on naming style, keep the naming style consistent across all the tests methods and test.

   [ ]

A3 (Arrange, Asset, Act)

Make sure that all the Test Methods are designed around Arrange, Act and Assert.If required Refactor your code to fall into these three sections.

   [ ]

Test Happy and Sad Path

The unit test should cover all possible scenarios and strive for high code coverage and ensuring good quality metrics. Unit Test methods must exercise all possible use case scenarios to test the input validations, interactions, errors messages, edge cases and exceptions etc.

   [ ]

Make use of Attributes

Use Test Framework provided Attributes like :

[TestCategory(“<describe if its Unit or Integration Test>”)]

[TestCategory(“<Which section of the application is being tested>”)]

[Priority(n), TestCategory(“<define if it’s Gated or not-Gated build Tests>”)]

[WorkItem(xxxx)]

[Owner(“<who wrote this test>”)]

   [ ]

Have least number of asserts per test (if applicable)

A good Unit test should only have limited # of assert statements. It should unit test the very functionality, as indicated in its descriptive name.A well-defined Unit Test should contain only one assert statement per test and must not try to exercise all the validation/boundary checks etc. by multiple Assert() in one Unit Test method. .

   [ ]

Keep assert messages descriptive

Use descriptive messages to improve readability of code and the build log.Assert.IsTrue(customer.IsExist,”The Customer is not found in the Database”);

   [ ]

Unit != Integration

There is a very fine line between Unit and Integration, if you happen to go beyond what your function is supposed to be doing then you are not writing a Unit Test. I.e. Unit Test doesn’t focus on interaction with external systems and software layers/dependencies.Test Doubles (for example Microsoft Fakes framework) comes into the picture to write unit tests which has dependencies on external libraries and systems etc.

   [ ]

Follow OOP Design and Adopt DI

Following Dependency Injection will allow to convert potential Integration Tests into small and quickly testable Unit Tests by taking advantages of Test Doubles (e.g. Microsoft Fakes, Moq, FakeItEasy frameworks etc.)

   [ ]

Should be thorough

Unit Tests are supposed to test all the possible areas of functionality that are subject to failure due to incorrect input/validation checks/boundary checks etc. for given function/method.

   [ ]

Must be Repeatable

Unit Tests must be repeatable for every build and must produce the same results. The development best practice suggests that if you are working on code that is impacting a Unit Test then you must fix the affected Unit Test as well and ensure that it passes.

   [ ]

Have to be Independent

Unit Tests must be independent of another test. In other words, no collateral damage. Hence, a Unit Test must focus only on a small aspect of big functionality. When this Unit Test fails, it should be easy to discover where the issue is in the code. I.e. can be tested in isolation

   [ ]

Keep it Professional

Even though at times Unit Tests may appear to be very simple and small, you must write Unit Tests with coding practices as good as you use for your main development coding. You may want to follow Refactoring, Code Analysis and Code Review practices and so on as for your Test Projects as well.

  [ ]

No Collateral Damage

Make Sure to Run all related Unit Tests after any dev code change big or small; to verify and ensure that no collateral damage occurs or has been introduced.

  [ ]

If you break it, You bought it

If you are working on a feature and to verify no collateral damage, as a best practice run all the Unit Tests. If you observe that some Unit Tests started failing because of your code changes then you own to Fix those broken Unit Tests to make sure that continue to pass.

  [ ]

Track and  maintain the tests

The test code changes should be tracked and maintained as on-going effort. Continue to follow thedesign principles and coding guidelines.

  [ ]

Code Review the Tests

Just like any other Dev code, Unit Tests also needs to be code reviewed by peer. Regardless of size of the test; follow the process.Code review might include reviewing the name of test method, scenarios covered, conditions checked, scope of assertions and code / design smells etc.