Page 1 of 2   next page »

One of the most important and yet overlooked aspects of a software development project is the concept of regression testing during implementation. Regression testing is the practice of running tests for previously tested code following modification to ensure that faults have not been introduced or uncovered as a result of the changes made. In this article I will outline the need for continuous integration, automated builds, and testing, which support the tenets of regression testing and provide automation for the process. I will also introduce some of the free open source tools that can be used to effectively employ a continuous integration process, and we'll take a look at a scenario that demonstrates their use.

The Case for Continuous Integration
If you are a project manager, lead developer, or perhaps a tester on a software development project you may have heard the buzz surrounding the idea of continuous integration. The general idea is somewhat simple: continuously build as often as possible using source control, an automated build process, and automated unit testing. A combination of all this should produce a working version of the software that at any given time on demand compiles cleanly and passes all unit tests. A tremendous benefit of continuous integration is instant feedback on the state of the build. As code is checked into the main source tree, a build is initiated and either succeeds or fails. If the build fails, a report can be viewed to see what was checked in that might have broken the build and corrective action is taken immediately. With traditional integration methods, mistakes that break builds and cause tests to fail might not be caught until days or months down the road when it is much harder to track them down and rectify them. Continuous integration also provides automation of regression testing to reduce defects and ensure quality. A nice side effect is confidence in the build at any given time, which translates into confidence and high morale within the development team. (see Figure 1)

Let's take a look at the components of a continuous integration system and the tools that support it. Most of the tools mentioned in this article are free and open source (see sidebar2).

CruiseControl.NET
The CruiseControl.NET continuous integration server is at the heart of my continuous build process. In a nutshell, CruiseControl.NET provides a monitoring system for projects by checking for modifications in source control on a set cycle. If changes are detected, the server will take the appropriate action for the project as defined in the configuration file. CruiseControl.NET also provides a very handy "Web dashboard" that allows you to have an "at-a-glance" view of the build state of your projects. The dashboard also allows you to force a build rather than wait for the timed cycle to run the next build. You can right-click on a project to see its build history as well as a hot link for the most recent build report. Installation of CruiseControl.NET is very simple and automated. A virtual directory called "ccnet" is created for the Web dashboard and a Windows service is installed to start and stop the server. If you wish to not have a Windows service installed or you want to set up the Web dashboard virtual directory manually, you can do so. The installation documentation has more information on this process.

NAnt and NAntContrib
NAnt is a tool that can be used to automate the entire build process, including compilation and running unit tests. NAnt is the .NET port of Ant for Java. It is similar to make, but it's easily extended through classes and it runs from an XML configuration file. NAnt can run stand-alone from the CruiseControl.NET server, and while CruiseControl.NET does provide many of the same features of NAnt such as a NUnit task, Visual Studio task, etc., I would recommend using the NAnt task of CruiseControl.NET and putting all of your build specific tasks in the NAnt build file using the NAnt and NAntContrib set of tasks. This will allow you to keep your builds autonomous of a build server and provide a mechanism for your projects to be built from the command line using NAnt.

NAntContrib is a project for tasks and tools that have not made it into the main NAnt distribution, or that just don't belong there. In our scenario we are using it for the source control support provided. In particular we will be using the <vssget> task to connect to Microsoft Visual Source Safe and retrieve the latest version of the source for the build. NAntContrib also has other extended tasks that are very useful in conjunction with NAnt. Complete documentation for the NAnt and NAntContrib tasks is included in the distribution.

NUnit
It goes without saying that unit testing is an essential part of a successful project. NUnit is a .NET port of JUnit for Java. It provides a way for developers to write unit tests in .NET code, decorate them with attributes marking them as tests, and then use assertions such as AreEqual, AreSame, IsTrue, IsFalse, etc., to check if certain conditions in the test code are in the expected state after execution of code. A developer can then compile the tests and run the resulting assembly in the NUnit GUI application, which shows green or red for pass or fail, respectively. For example, if you have a bank account with a balance of $150 and you insert $50, you might write a test to ensure that the balance of the account is $200 after the transaction; if not, the test would fail and you would know that something is broken. The real benefit of this sort of test code comes when changes have to be made to the code that could potentially cause breakage. If you have unit tests in place, you will know that the existing functionality that was working fine before changes still produces the expected results. This is known as regression testing. There is no better feeling when making changes to production code than knowing that you are not breaking what already works! Installation of NUnit is simple and the tool is very intuitive. Another free tool that is not open source, but is very handy for unit testing, is TestDriven.NET (see sidebar1).

The Robot Framework Scenario
So let's put these tools into practice with a scenario. Our project is to build a Robot Framework that consists of a BaseRobot class and a set of unit tests to ensure that the BaseRobot can perform its job properly. The main goal of this scenario is to get the project building in a continuous integration environment, doing an automated build, and running unit tests automatically anytime anyone makes changes to the source code.

Our robot can perform simple maneuvers such as turning and moving forward. It is designed to move on a Cartesian grid with a set of X and Y axes. It also has a name and can report at any time its name and current X and Y coordinates on the grid, as well as the direction it is facing: north, south, east, or west (see Listing 1).

The BaseRobotTest class defines unit tests that run the functionality of our robot through its paces, thereby ensuring that expected conditions exist after execution of code. A class is decorated with the [TestFixture] attribute and individual tests or methods within the class are decorated with the [Test] attribute. When NUnit executes the assembly it recognizes and executes the tests (see Listing 2).

We now need a build script for NAnt. Our build file is called default.build and consists of several items, including a project definition, targets, and tasks within those targets. In essence, our build file needs to delete all existing assemblies that were previously built, pull the latest code from Visual Source Safe, perform a build using the Visual Studio.NET solution file, and then run unit tests against the newly compiled assemblies.

Continuous Integration of the Robot Framework
The Robot Framework project is now ready to be placed under continuous integration by following a few simple procedures. First we need to ensure that CruiseControl.NET is installed and the service is running. To test this, we can simply try to load the Web dashboard by going to http://servername/ccnet (this assumes you have followed the default installation as described above and the install has created the "ccnet" virtual directory for you). If CruiseControl.NET has been installed successfully and is running, you should see the Web dashboard with no projects (see Figure 2).

Our next step is to put our solution into Microsoft Visual Source Safe. (Note: NAnt can be used with other source control systems such as PVCS, Star Team, and Surround SCM. See the NAntContrib documentation for more information.) Once the project is in VSS a <project> element entry can be defined in the ccnet.config file that is used for configuration of the CruiseControl.NET server (see Listing 3). Once this file is modified with the project entry, the CruiseControl.NET server will automatically pick up the changes and restart. Once the changes are recognized, a refresh of the Web dashboard will now show our project under continuous integration along with a status. Click on the "Force Build" button to trigger the build process. You can also check out one of the source files, modify it, and check it back in. When the next cycle for checking modifications occurs, CruiseControl.NET will check and automatically start the build cycle.

Page 1 of 2   next page »