Automating processes is critical to the success of any software project. Because computers can perform redundant tasks faster and more reliably than people, automation becomes more necessary as the processes become larger and more complicated. This is one of the main drivers behind Test Driven Development - constantly rerunning an automated build with Unit Tests. This article will provide techniques to affectively automate large processes, with a case study using NAnt.

Figure 1 shows a simple process with four tasks and various flow controls. This illustrates how a process has two parts: the tasks that do work, and the scripting that connects those tasks. Automating a process requires automating both the tasks and the scripts. For ease of explanation we'll first illustrate scripting techniques with .NET, and then show how to call almost any task from the command line using these scripts.

Step 1: Flow-controlling scripts
A process can only be as powerful as the script that connects its individual parts. Therefore we want the best scripting possible: ideally something with clear syntax, comprehensive programming power, a supporting IDE, modularity, and access to the .NET Framework.

Neither DOS, nor VBScript, nor third-party tools designed to script a specific task (such as NAnt for automating the build process) alone meets these criteria. We really want a script with the full power of a .NET Language and Visual Studio - we want "C# Script." Fortunately .NET lets us do just that!

Making C# "Script"
We can put all of our complex logic in a C# class file, have a DOS batch compile that file, and then call the resulting executable. For example, Class1.cs is a simple console application:

using System;
 namespace MyProcess
 {
     class Class1
     {
         [STAThread]
         static void Main(string[] args)
         {
             Console.WriteLine("Running a C# program");
         }
     }
 }

The batch script, Run.bat, first calls the C# compiler, csc.exe, to compile Class1.cs into MyProcess.exe. It can now call MyProcess.exe just like any other executable.

%windir%\Microsoft.NET\Framework\v1.1.4322\csc /out:MyProcess.exe Class1.cs
MyProcess.exe

The script as a whole is run simply by calling Run.bat. This technique provides the best of both worlds. It can still be considered scripting because it can be edited in any text editor, and doesn't manually invoke the compiler (the DOS batch now does that). Yet it has all the power of C# - including access to the .NET Framework classes, OOP features, and a supporting IDE with debugging and intellisense. This DOS-C# system only requires the .NET Framework and SDK to be installed.

Besides accessing just the existing .Net Framework, this approach lets you reference your own Class Libraries too. Therefore we can create our own utility library with methods that are useful for automated processes, such as quick Xml reading and writing, file manipulation with regular expressions, database administration tasks, etc. This utility library can be compiled with the main application by using the /reference option:

%windir%\Microsoft.NET\Framework\v1.1.4322\csc /out:MyProcess.exe /reference:ClassLibrary1.dll Class1.cs

Passing Data Between Scripts
Passing data between external processes is usually limited to simple scalars, and can have different syntax for each scripting tool. Therefore to minimize this limitation, we want the brunt of our automation script contained in our C# file. This lets us focus on just passing data into and out of the C# file. We can achieve this via reading and writing to the file system, getting the C# Main method's input arguments, getting the environmental variables, returning ErrorLevels from C#, and calling external processes with System.Diagnostics.Process.

First, we can always use the underlying file system as a global data store. DOS, VBScript, NAnt, and certainly C# can all read and write files. For example, C# might parse an NAnt log file for error messages. Scripts should also be able to pass parameters directly to each other, not just indirectly via the file system.

We can pass parameters into the C# script from the string[] parameter of the Main method. We can also use System.Environment.GetEnvironmentVariable to get the DOS script's environmental variables. We can pass parameters out of the C# script by returning an integer from the Main method, which sets the ErrorLevel in DOS.

While DOS syntax is available simply by typing "Help" in the command line, as a refresher, note that you can pass parameters between DOS scripts by appending them after the file being called. You can then reference them in the child script with %n%, where "n" is the parameter index (starting at 1).

The .NET Framework provides process handling from within the System.Diagnostics.Process class. The static Start method of this class lets us run any program. Because we must often wait for that program to exit before continuing, we can set the Process's WaitForExit property to true. We could encapsulate this into a reusable method:

public static void RunConsole(string strFileName, 
string strArguments, bool blnWaitForExit) 
{
	ProcessStartInfo psi = new ProcessStartInfo();
	psi.FileName = strFileName;
	psi.Arguments = strArguments;

	Process p = System.Diagnostics.Process.Start(psi);

	if (blnWaitForExit)
		p.WaitForExit();
}

The C# snippet below combines all of this. It gets the first command line argument, and an environmental variable "var1." It then runs an external process, "Process2.bat," waits for that process to exit, and then returns an error code.

static int Main(string[] args)
{
	string strInputArg = args[0];
	string strEnvArg = Environment.GetEnvironmentVariable("var1");
	ClassLibrary1.Class1.RunConsole("Process2.bat", strEnvArg, true);
	return 0;
}

This snippet can be compiled and called by the DOS script shown below. This script also sets the environmental variable used by the C# script, passes in a parameter "Hello," and gets the returned ErrorLevel.

%windir%\Microsoft.NET\Framework\v1.1.4322\csc /out:MyProcess.exe Class1.cs
set var1=World
MyProcess.exe "Hello"
echo ErrorLevel=%ErrorLevel%

Step 2: Isolating Each Task
Most processes consist of three types of tasks. First, the process may need to run applications that already have existing command lines - like NAnt, NUnit, or the lesser-known OSQL for database manipulation. Second, the process may need to access utility methods from a class library. Third, the process may require modifying system objects like changing the folder permissions or creating virtual directories. All of these can ultimately be called from the command line, or from C# with System.Diagnostics.Process.

The first case is trivial - executables can already be run from the command line. Almost any Microsoft or open-source application will provide a command line interface. For the second case, while a benefit of C# "Script" is that it can already access Class Libraries, there may be legitimate cases where non-C# scripts need to access those class libraries as well. For example, perhaps a task embedded in the middle of an NAnt script needs to call a utility method. While it isn't reasonable to have a separate console application to wrap every utility method you write, it is reasonable to have a single console application use System.Reflection to dynamically access static members. You can build a console app that takes in the DLL, type, method name, and its arguments, and then uses Reflection in the method below to call other static methods. Note that this method is the simplest case, and has no exception handling or input validation.

public static object DynamicLoad(string strDll, string strType, 
string strMethod, object [] p) 
{
	Assembly a = Assembly.LoadFrom(strDll);
	System.Type t = a.GetType(strType);
	MethodInfo m = t.GetMethod(strMethod, 
		BindingFlags.Static | BindingFlags.Public);
	return m.Invoke(null, p);
}

VBScript can solve the third case. Many system objects, such as setting the folder permissions, creating IIS virtual directories, or even accessing ASP.NET performance tests with Application Center Test (ACT) all expose a COM interface that can be manipulated through VBScript's CreateObject and GetObject commands. While you can use the .Net Framework for many of these tasks, VBScript already has literally hundreds of existing scripts written (see www.microsoft.com/technet/scriptcenter/default.mspx).

The Role of Each Scripting Technique
We've discussed four different kinds of scripting techniques. Table 1 shows the pros and cons of each. The strengths and weaknesses of each technique can supplement each other to form a powerful automation process, as shown in Figure 2. We initially start the process with DOS. This can be done either from a single main DOS script, or by a wrapper script that passes in a hard-coded parameter into the main script to configure the process. For example Run.bat could expect a parameter; Run_A.bat could pass it value "A," and Run_B.bat could pass it value "B." Run.bat could then compile the C# code, passing along the parameter. The C# script could then use that parameter to configure the process, as well as do any complicated logic, call VBScripts or external applications, and then generate a summary report from the process. We want to maximize the C# script's responsibility while minimizing the DOS and VBScript.

Case Study: Extending the Build Process with NAnt
Theory is good, but putting theory into practice keeps us employed. Therefore this article concludes by applying these techniques to a popular automated process - the NAnt build. We can apply the model from Figure 2 to extend this process.

First we want to abstract all environment-specific values to an external config file, such as any program paths or source control and database connection information. We can use NAnt's <include> task to include this file in the build.

Our build may need to run under several different scopes, such as merely compiling the code (for continuous integration), running everything locally (which means not getting the latest from source control), or running the full master build. We can pass in this scope parameter from the wrapper scripts (i.e., have "Run_Compile.bat," "Run_Local.bat," "Run_Master.bat"), through the main DOS script, and into the C# script.

The C# script initializes the system by reading the Build's XML config file, thus creating a custom version # and creating a directory to store the NAnt output products, as well as whatever else your build needs. It then dynamically assembles the command line needed to call NAnt, including passing in any parameters (such as the scope - which NAnt can use to include a corresponding config file). The C# script then calls NAnt, waits for its exit, and then creates a custom HTML report. The data for such a report can come from the timespan needed to run NAnt, the build log, the existence of output products (such as an MSI installer), and the summary results of all of the applications run by your build: NUnit tests, NCoverage results, FxCop rules, NDoc link, etc.

Conclusion
The easiest time to automate a process is when first designing it. Because all of these techniques require only the .NET platform, you can start benefiting from them in your design right away.