Cloud Limitations
Before getting into detail on how we used Skytap to improve our agile development process, it's worth noting what types of testing the Cloud is not suitable for. Most Infrastructure-as-a-Service vendors offer standard infrastructure; however, some limit the configurability of processors, amount of memory, and network settings of machines (for instance, Amazon only offers Small, Large, and Extra Large specifications of machines). This can be an issue for specific test cases where low-spec machines need to be created. Other types of test scenarios include unit testing, functional testing, system testing, integration testing, and load testing of applications and the Cloud is suitable for these tests assuming your Cloud provider supports your application platform.

There are a few use cases where the Cloud is not recommended for testing (see Figure 1). These include tests that require specific x86 hardware (e.g., BIOS driver tests) and some types of performance and stress testing. If an application requires an onsite Web Service behind a firewall, this can usually be accessed using a VPN connection.

Extending Agile Methods to the Cloud
We follow an agile development philosophy and were keen to explore ways to improve our development process and quality using the Cloud. Since we made the move, we've seen numerous ways to improve our agile approach. Here's what we've learned:

Faster Iterations
The cloud is inherently self-service, which allows developers to get immediate access to development resources whenever they're needed. Activities such as buying, installing, and configuring physical machines are a thing of the past and we can now deploy a new development environment in a matter of minutes versus several hours (or days) previously.

Developers are rarely blocked because an IT operations request is taking too long and they can scale virtual machines up and down as needed. We've found that by removing these friction points in our development process we've reduced our iteration time by about 25%.

Customer Feedback
We've always tried to get customer feedback early and often in our development process. The challenge we had was providing customer access to applications easily. Typically we'd have to find a test stack (several machines), install a new build, and give a customer detailed instructions on how to RDP into these machines using a VPN connection. Usually, these machines couldn't be used for anything else while a customer was testing a build, which tied up machines and was inefficient.

Using a Cloud-based virtual lab, we can publish an entire development stack and give a customer a URL to access these machines. We only pay for the hours a customer uses the machines and can set auto-suspend rules after a period of inactivity. This not only improves the ease of getting customers involved earlier in the development process, but also cuts costs and improves flexibility.

Test Contexts
Many of our projects involve very specific test contexts, such as named domains, active directories with specific user accounts, and SQL server installations with users and data that closely mirror production. Our test cases include changing active directory entries, updating database entries, and changing configuration settings and we found setting up an isolated test environment very challenging.

Spinning up a test environment in the Cloud solves many the issues involved with replicating a complex multi-machine environment. Network fencing ensures environments are isolated, but that they still retain domain names, subnets, IP addresses, etc. We can run multiple copies of the same test environment side-by-side and roll back changes to a clean state quickly and easily. This capability has saved many tedious hours of trying to replicate test contexts and ensure test cases are adequately covered.

Team Interaction
We have a regionally distributed development team and this poses challenges with team communication. We've found many of these challenges have been solved with a Cloud-based virtual lab. We can now easily instant-message a URL to each other to collaborate in real-time on an application stack or to get advice. These "configuration URLs" point to entire multi-machine stacks and can only be accessed by approved team members.

In addition, our QA team can now snapshot entire multi-machine configurations and save them in the Cloud. This practically eliminates the "it works on my machine" complaint developers often pass back to the test team. Just like a TiVo can pause and replay TV programs, a tester can pause an application at the point of failure using the virtualization capabilities of the Cloud and shoot this over to the development team. This saves hours of trying to replicate issues for complex bugs.

Benefits
Overall, moving our development and test environments to the Cloud has been very successful for Buildingi. I'm often asked how to quantify the benefits of adopting a Cloud-based development environment. Every team is different, but analyzing total cost of ownership (TCO), time savings, and quality of code are good areas to focus on. Here are some of the specific benefits we've measured:

• Deployed the complete virtual lab solution in two days (including developer training)
• Saved $100,000 in capital expenses
• Reduced time to reproduce complex bugs by 80%
• Cut development iteration cycle time by 25%
• Improved set-up and tear-down of new test environments by 75% (two hours versus eight+ hours)
• Reduced time to deploy a customer user acceptance testing (UAT) environment from six+ hours to a few minutes

In summary, I'd definitely recommend looking into a Cloud-based lab environment to augment or replace your existing development and test infrastructure. We found Skytap worked best for us, but it's always important to complete a thorough evaluation based on your company's requirements. I'd recommend looking carefully at the capabilities of the Clouds you evaluate, especially with respect to supported operating systems, Web-based UIs, virtual lab automation, and configurable networking. It's a leap of faith moving to the cloud, but I'm confident you'll find significant benefits for typical .NET application development. Given Cloud vendors don't have any upfront charges and can be cancelled at any time, there's very little to lose and a lot to gain!