Welcome!

Microsoft Cloud Authors: Kevin Benedict, Pat Romanski, Liz McMillan, Lori MacVittie, Elizabeth White

Related Topics: Microsoft Cloud

Microsoft Cloud: Article

Automated Unit Testing with NUnit - If testing is so important, why do so many developers do without formalized tests?

Automated Unit Testing with NUnit - If testing is so important, why do so many developers do without formalized tests?

Ask any group of developers if testing matters and they will tell you "Of course!" But ask to see the tests for their current project and you will likely get a series of excuses ranging from, "Well I just, you know, use the application to see if anything goes wrong" to "The dog ate my test cases." If testing is so important, why do so many developers do without formalized tests? The typical answer is that formal testing is too hard or tedious.

Developers are not solely to blame for the lack of testing. Look at how we learn. College textbooks typically devote only one chapter to testing, and even then they fail to explain how to test. Look at the .NET bookshelves in your local bookstore and pick out the titles about testing. Failing that, find the titles that have a good section on how to test. Go to the MSDN or GotDotNet Web sites and search on "testing". You probably won't find much. You can find countless gigabytes of information on how to implement all manner of programming tasks, but very little on how to implement testing.

This article is one step toward helping you understand how to implement testing in your projects. In addition, I include some resources on testing so you can spend less time searching for in-depth discussions and more time coding (and testing).

Why Test?
So, why exactly should you test? You should test to ensure that the software is behaving exactly as intended. Tests allow you to detect the defects that cause the software to behave in an unexpected fashion. As tests ensure behavior and the requirements section of the software specification defines that behavior, the tests must reflect the requirements.

Formal tests provide you with consistency of your testing effort. One of the scariest tasks developers perform is modifying old code that they didn't write or that they haven't looked at for a while. If you change one part of the program and do not test the "unaffected" areas, how do you know that your change did not affect other areas of the code? With formal tests available you can ensure that any changes you make produce no side effects that cause the software to behave in unexpected ways.

One other side effect of writing tests: it provides another form of documentation on how your code should be called. What better example of how a class library should be used than the code that tests the class library's code!

Test-First Versus Test-Last
Simply deciding to test is not enough; the decision you make on when to test, before or after coding ("test-first" versus "test-last"), can impact the quality of your testing.

The advocates of a "test-first" philosophy - where tests are planned, designed, and created before the code that the tests operate upon is written - say that there are several advantages over writing the tests after the code:
1.  Testing will not be put off. We all know that with the pressures of project schedules the second task to get dropped is testing (documentation is the first). Writing the tests first allows us to do it when we have more time. As we will see later, automating testing will also allow us to spend less time debugging and more time coding.

2.  Code meets expectation (not expectation meets code). If the test cases are written after the code being tested, there is the danger that the tests will be written to validate the behavior of the code written and not necessarily the behavior expected.

3.  Engenders courage. If you first write a new test case that will adequately test the new functionality (expected behavior) you are to add to the code, seeing the results of passing all the tests after the change is made encourages you to make additional changes to this area of the code. This is especially encouraging in maintenance programming, where you are coming back to code that you did not originally write or that you wrote some time ago. It helps reinforce your (likely incorrect) belief that you remember how the code really works.

4.  Adequate time, adequate coverage. Perhaps the most important reason of all is to ensure that you have the necessary time to write an adequate number of tests for complete code coverage. Without the time to write an adequate number of tests, all other advantages of testing are gone.

Unit Testing in .NET
For each complex class you develop, you should write unit test cases to test the internals and isolated features of that class, ensuring complete code coverage (all lines of code are executed during the tests). You can use these tests to quickly detect, pinpoint, and fix bugs that might otherwise take a long debugging session to uncover.

There are several products out there for automating unit testing on the .NET Framework. In this article, I will show how to install and use NUnit version 2.0.

NUnit is a unit-testing framework styled after the xUnit unit-testing framework, which has been ported to a number of development platforms and a variety of programming languages. The more famous JUnit (Java unit-testing framework) and SUnit (Smalltalk unit-testing framework) products are based on the principles of xUnit.

Installing NUnit
Installing NUnit could not be simpler; you can download it as a standard .msi package. Once it is installed, you can access the samples, (sparse) documentation, and the NUnit Test Runner GUI from items in the Start menu.

One activity the installer does in the background for you is to add the main assembly, nunit.framework.dll, to the Global Assembly Cache so that it can be easily used with any project. One step that it does not take, however, is to add the installation directory to the registry so VS.NET can automatically display the assembly when you want to add a reference to your project.

Adding VS.NET Support
When you want to add a reference to an external assembly to your project, you select the Add Reference menu (see Figure 1) and select the assembly from the list in the dialog. As NUnit did not add an entry to the registry, you must click the Browse button, then find and select the nunit.framework.dll in order to add the reference to your project. If you are going to make unit testing a regular part of your development process, it is better to have the nunit.framework.dll automatically appear in the Add Reference dialog (see Figure 2).

 

 

Fortunately, the decision of which folders VS.NET will search for .NET assemblies is controlled by a few simple registry entries. Here is how you can add NUnit's bin directory to the folders searched:

  1. Open the Registry Editor.
  2. Go to the key [HKEY_LOCAL_MACHINESOFTWAREMicrosoft.NETFrameworkAssemblyFolders]. Each key under the AssemblyFolders key will tell VS.NET where to look for additional assemblies for display. Note that you should already have at least one key named Primary Interop Assemblies.
  3. Click on the AssemblyFolder icon.
  4. Go to the Edit menu, choose the New sub-menu, and click on the Key menu item.
  5. Name the new key (the folder icon) "NUnit". Your registry folders should now look similar to Figure 3.

     

  6. With the NUnit icon selected, change the (default) value to the path of the nunit.framework.dll. This will be in the bin directory under the directory in which NUnit was installed. Note that you should not put quotes around the directory path, even if there are spaces.
After closing the registry, and restarting Visual Studio, you should now see the NUnit assemblies in the Add Reference dialog, as shown in Figure 2.

Using NUnit
Listing 1 shows three small sample classes. For purposes of convenient display, the classes are all included in a single file and all comments are removed. In this example I will show how to create a test project within our solution, the tests for the USAddress class, and how the tests are used.

Adding a Test Project
Figure 4 shows the start of the solution. We will add a new empty project to the solution to build our test cases as a DLL. Then, using the NUnit Test Runner GUI, we will automatically run all of the test cases.

 

Start by going to the File menu, choosing the Add Project submenu, and selecting the New Project menu item. In the Add New Project dialog, select New Project In Existing Folder. Name the project USAddressTests. In the next dialog, select the folder where your current project is located.

Now that we have a new, empty project, we want to make several changes to the defaults of the project. Open the property page of the USAddressTests project and change the Output Type to Class Library and the Default Namespace to ContactInfo (the namespace of the class under test). Figure 5 shows the section of the property page changed.

 

In the USAddressTests project, we want to add the necessary source files so we can build the tests. In this example we simply need to add the AssemblyInfo.cs, SampleCode.cs, and USAddressTests.cs files. The last source file contains the actual test cases, which we will look at in a moment.

We will also need to add a reference to the NUnit Framework assembly. Good thing we added the registry key! Simply right-click on the References folder in the project, select the Add Reference menu, and select the nunit.frame.dll assembly.

One final step is to make the USAddressTests project the startup project. This saves us the step of selecting the project before building. After the first build, your solution should look like Figure 6.

 

Writing Tests Cases
Listing 2 shows two very basic tests used to test the USAddress class. This example uses C#, but the test code can be written in any .NET language. These tests are not designed to show you what good test cases should consist of - for that I refer you to some of the resources at the end of this article - but rather what test code looks like and how NUnit uses and accesses it.

The first item of note, especially if you have ever used JUnit for testing Java code, is that NUnit uses custom attributes to specify the use of each class and method in the test code. Compare the class and method names in Listing 2 with the tree view in NUnit (see Figure 7). You can see that the assembly under test is the first level of the tree; the ContactInfo namespace is the second level; the USAddressTests class is the third; and the PropertyTests and PropertyChangeTests methods are lowest level. This shows you that a class marked with the class attribute [TestFixture] represents a group of tests, and methods marked with the method attribute [Test] represent a single test under that group.

 

You might be wondering why we have more than one group of tests? This is where the [SetUp] and [TearDown] method attributes come into play. Again referring to the test code in Listing 2, you will notice that the test methods start with assertions regarding properties of the Address class. But without additional code in the method, static class initialization code, or a class constructor, how are these values initialized? The answer is in the method marked [SetUp]. NUnit calls this method before each [Test] method is invoked. In addition, if a [TestFixture] class has a [TearDown] method, it is called after each [Test] method is invoked. So, using the code in Listing 2 as an example, the methods are called in the following order: SetupTest, PropertyTests, TearDown Test, SetupTest, PropertyChangeTests, and TearDownTest.

There are three additional attributes in the NUnit Framework: [ExpectedException], [Ignore], and [Suite]. Here is a quick synopsis of their uses:

  • [ExpectedException]: This method attribute is used in conjunction with the [Test] attribute and indicates to NUnit that it is expected that an exception will be thrown during the course of a particular test. If the exception is not thrown, or if an exception of a type other than the one specified is thrown, it is considered a failed test.

  • [Ignore]: This method attribute is used in conjunction with the [Test] attribute and indicates that a test should not be run. It is a convenient way to avoid running a test and is preferable to commenting out large blocks of code or removing the [Test] attribute.

  • [Suite]: This method attribute is included for backward compatibility with the previous testing model.

    Assertions
    Assertions are simply one-line tests that evaluate to true or false. Depending upon the type of assertion you specify, the test will succeed or fail based upon the test value. Here is a list of the assertions that NUnit supports:

    • Assert: If the assertion is true, the test passes.
    • AssertEquals: This compares an expected result to an actual result, and if the two are equal, the test passes.
    • AssertNotNull: If the object passed is not null, the test passes.
    • AssertNull: If the object passed is null, the test passes.
    • AssertSame: This compares two objects and determines if they are the same object. If they are, the test passes. Note that this is not the same as AssertEquals, which uses the object's Equals method to determine if the two objects are equivalent.
    • Fail: Automatically fails a test.
    All assertion methods take an optional string argument (in the first position), which is used as a custom error message if the test fails.

    Adding an NUnit Template to VS.NET
    It is very easy to quickly and simply add a new template to VS.NET. As NUnit test code has a definite pattern, it is helpful to use a template as a starting point. Listing 3 shows what the NUnit template code looks like. This example will use C#.
    1.  Go to the Visual Studio .NET installation directory. By default it is "C:Program FilesMicrosoft Visual Studio .NET". Under that directory, find the language directory. For C# this directory is called "VC#".

    2.  Under the language directory look for the project items directory. For C# this directory is called "CSharpProjectItems". It contains a number of "VSZ" files, which define the wizards used by VS.NET. Copy the file "CSharpAddClassWiz.vsz" and paste it into the same directory. Rename the duplicated file to "CSharpAddNUnitWiz.vsz".

    3.  Open the "CSharpAddNUnitWiz. vsz" file in Notepad and change the line:

    Param="WIZARD_NAME = CSharpAddClassWiz"

    to:

    Param="WIZARD_NAME = CSharpAddNUnitWiz"

    Now save and close the file.

    4.  Inside the project items directory is the "LocalProjectItems" directory. In that directory is a file, "LocalProject Items.vsdir". Open that file in Notepad.

    5.  Ugly file, isn't it? Each row in this file is a separate record and the "|" character separates each field in the record. Basically, each record denotes a different project item in the Add New Item dialog in VS.NET. Find the line that begins with:

    ..CSharpAddClassWiz.vsz

    (It should be the eighth line in your file, but your file may be different depending on the version of your software.) Copy that entire line and paste it at the bottom of the file. (Make a backup of the file first, if you like.)

    6.  We want to change the new entry to point to the NUnit template we are building. The record should read:

    ..CSharpAddNUnitWiz.vsz|{FAE04EC1-301F-11d3-BF4B-00C04F79EFBC}
    |NUnit Class|25|A class for implementing NUnit tests|
    {FAE04EC1-301F-11d3-BF4B-00C04F79EFBC}|4515|0|Tests.cs

    The changes made were to field 1 (pointing to our new VSZ file), field 3 (the name of the item in the Add New Item dialog), field 4 (used for sorting the entries), field 5 (the long description displayed when the NUnit Class icon is selected), and field 9 (used to determine the default name of the file added to the project). After making the changes, save and close the file.

    7.  Under the language directory is a wizards directory (for C# this directory is called "VC#Wizards"). Open that directory.

    8.  Under the wizards directory will be one directory for each wizard. Find the directory "CSharpAddClass Wiz", then select, copy, and paste it. Rename the duplicate directory to "CSharpAddNUnitWiz".

    9.  Inside the new CSharpAddNUnit Wiz directory is a Templates directory. Inside that directory is another - named "1033" if you are using the English language version of VS.NET - which we will call the template files directory.

    10.  Inside the template files directory are two files: "NewCSharpFile.cs" and "Templates.inf". You can delete the "NewCSharpFile.cs" file and create a new file, "NewNUnitFile. cs". This file should contain the code in Listing 3.

    11.  Open the file "Templates.inf" and change the single line from:

    NewCSharpFile.cs

    to:

    NewNUnitFile.cs

    Now save and close the file.

    At this point, you have a brand new template available that adds a new NUnit item to your project when you select Add New Item in VS.NET (see Figure 8).

     

    To recap, in order to quickly create a new template, you need to:

    1. Create a wizard definition (VSZ) file. This defines the name of the wizard, which in turn directs the wizard engine to the directory containing the wizard code and templates.
    2. Modify the VSDIR file, which determines what items show in the Add New Item dialog. This points to the wizard definition file.
    3. Create a directory for the new wizard, which contains the new template to be used for the project item to be added.
    Other Testing Frameworks for .NET
    There are competing unit testing frameworks available for .NET. One in particular, csUnit (www.csunit.org), looks as good as NUnit and appears to work very similarly to NUnit. It has a bit more support for VS.NET, taking care of adding the registry keys and even creating new menus to access the testing GUI quickly.

    Given the strong similarity between the two frameworks, you might wonder why you would want to use one over the other. This was not an answer I discovered while researching on the Internet. I found NUnit first, became used to the interface, and stuck with it.

    Resources
    To find out more about the xUnit testing framework, test-first development, and test-driven development, I recommend the following books and online resources:

  • Beck, K. (2002.) Test-Driven Development: By Example. Addison-Wesley. This book gives examples of how to work through development issues using test-driven development. In the course of the exercises it gives you a better understanding of how the framework is structured.

    Although the code is in Java and Python, the examples are clear.

  • Link, J. (2003.) Unit Testing in Java: How Tests Drive the Code. Morgan Kaufmann. As indicated in the title, the examples are in Java, but three-quarters of the book describes testing techniques that can easily be carried over to .NET.
  • XProgramming.com: www.xprogramming.com. This Web site by Ron Jeffries contains a number of articles on C# development using NUnit as the testing framework. In addition, a listing of the various flavors of xUnit and their extensions can be found here.
  • Comments (0)

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    IoT & Smart Cities Stories
    DXWorldEXPO | CloudEXPO are the world's most influential, independent events where Cloud Computing was coined and where technology buyers and vendors meet to experience and discuss the big picture of Digital Transformation and all of the strategies, tactics, and tools they need to realize their goals. Sponsors of DXWorldEXPO | CloudEXPO benefit from unmatched branding, profile building and lead generation opportunities.
    Disruption, Innovation, Artificial Intelligence and Machine Learning, Leadership and Management hear these words all day every day... lofty goals but how do we make it real? Add to that, that simply put, people don't like change. But what if we could implement and utilize these enterprise tools in a fast and "Non-Disruptive" way, enabling us to glean insights about our business, identify and reduce exposure, risk and liability, and secure business continuity?
    In this Women in Technology Power Panel at 15th Cloud Expo, moderated by Anne Plese, Senior Consultant, Cloud Product Marketing at Verizon Enterprise, Esmeralda Swartz, CMO at MetraTech; Evelyn de Souza, Data Privacy and Compliance Strategy Leader at Cisco Systems; Seema Jethani, Director of Product Management at Basho Technologies; Victoria Livschitz, CEO of Qubell Inc.; Anne Hungate, Senior Director of Software Quality at DIRECTV, discussed what path they took to find their spot within the tec...
    The deluge of IoT sensor data collected from connected devices and the powerful AI required to make that data actionable are giving rise to a hybrid ecosystem in which cloud, on-prem and edge processes become interweaved. Attendees will learn how emerging composable infrastructure solutions deliver the adaptive architecture needed to manage this new data reality. Machine learning algorithms can better anticipate data storms and automate resources to support surges, including fully scalable GPU-c...
    Nicolas Fierro is CEO of MIMIR Blockchain Solutions. He is a programmer, technologist, and operations dev who has worked with Ethereum and blockchain since 2014. His knowledge in blockchain dates to when he performed dev ops services to the Ethereum Foundation as one the privileged few developers to work with the original core team in Switzerland.
    DXWorldEXPO LLC announced today that Telecom Reseller has been named "Media Sponsor" of CloudEXPO | DXWorldEXPO 2018 New York, which will take place on November 11-13, 2018 in New York City, NY. Telecom Reseller reports on Unified Communications, UCaaS, BPaaS for enterprise and SMBs. They report extensively on both customer premises based solutions such as IP-PBX as well as cloud based and hosted platforms.
    "Akvelon is a software development company and we also provide consultancy services to folks who are looking to scale or accelerate their engineering roadmaps," explained Jeremiah Mothersell, Marketing Manager at Akvelon, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
    "Space Monkey by Vivent Smart Home is a product that is a distributed cloud-based edge storage network. Vivent Smart Home, our parent company, is a smart home provider that places a lot of hard drives across homes in North America," explained JT Olds, Director of Engineering, and Brandon Crowfeather, Product Manager, at Vivint Smart Home, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
    DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI, Machine Learning and WebRTC to one location.
    The current age of digital transformation means that IT organizations must adapt their toolset to cover all digital experiences, beyond just the end users’. Today’s businesses can no longer focus solely on the digital interactions they manage with employees or customers; they must now contend with non-traditional factors. Whether it's the power of brand to make or break a company, the need to monitor across all locations 24/7, or the ability to proactively resolve issues, companies must adapt to...