Click here to close now.

Welcome!

.NET Authors: Michael Kanasoot, Greg O'Connor, Jayaram Krishnaswamy, Adine Deford, Peter Silva

Related Topics: .NET

.NET: Article

Programmatically Posting Data to ASP .NET Web Applications

Create powerful automated utilities

Programmatically posting data to a traditional ASP page is an extremely useful and well-known technique. However, I recently discovered that posting data to an ASP .NET Web application from another program is not well understood and requires several new programming tricks. In this article I'll show you how to use the HttpWebRequest class and the ASP .NET ViewState mechanism to programmatically send form data to an ASP .NET Web application and then capture the response. This will provide you with a powerful new way to write utility programs for your Web applications. (Note: This article assumes you are familiar with creating ASP .NET Web applications, using classes in the .NET Framework, and have intermediate familiarity with the C# language.)

The best way to demonstrate what we will accomplish is with two screenshots. Figure 1 shows a simple ASP .NET Web application. If the user types "red" in the upper textbox control and clicks the submit button, the application displays "roses are red" in the lower textbox as shown. If the user types and sends anything other than "red," "blue," or "green," the application responds with "unknown color."

Suppose we want to post data programmatically and examine the response. In other words, we want a console application or some other type of program to simulate typing a value in the Enter-a-color textbox and clicking the submit button, and then determine what the response is. Figure 2 shows a console application that does just that. You can see the value of TextBox2 is "roses are red."

Even though this example is a console application, the underlying code can be used in any .NET program. In the following sections I'll walk you through the core code that posts data to ASP .NET and explain it in detail, then we'll briefly look at some variations to show you how to deal with server-side controls other than TextBox. I'll conclude with a discussion of some of the ways you can use this technique in a production environment.

The Problem
Listing 1 shows the source code for the simple ASP .NET Web application shown in Figure 1. For simplicity, I used a basic text editor (yes, Notepad) and put the logic code in the same file as the HTML display code. I used the Visual Basic .NET language for the Web application, but I could have used any .NET language.

If we want to send "red" to this application, we have to deal with three problems. How do we assign "red" to the TextBox1 control; how do we tell the ASP .NET server that Button1 has been clicked; and how do we capture the response from the server?

The ViewState Value
Listing 2 shows the complete console application source code that generated the output shown in Figure 2. As you can see I used C# but I could have used Visual Basic .NET or any other .NET language. I'll walk you through this code in the next three sections and explain exactly how it works.

After declaring five namespaces so I wouldn't have to fully qualify each class, I declared and assigned a variable for the URL of the Web application:

string url = "http://localhost/PostToASPdotNET/colors.aspx";

Next comes the trickiest part and the key to the entire programmatic post technique:

string viewstate = InitialViewState(url);

I declare and assign a value for the Web application's ViewState. What is this? Even though HTTP is a stateless protocol - that is, each Request-Response pair is an isolated transaction - ASP .NET works behind the scenes to simulate a stateful environment. One of the ways ASP .NET does this is through the use of an HTML hidden input named __VIEWSTATE. It's a Base64-encoded string value that represents the state of the page when the server last processed it. In this way pages can retain values between successive calls. To correctly post information to an ASP .NET application, we need to send the ViewState value to the server. Where do we get this value from?

The easiest way to get the initial ViewState value for a Web application is to launch Internet Explorer, get the page, and then do a View -> Source from the menu bar (see Figure 3).

As you can see, the initial value of __VIEWSTATE is "dDw0NjgyMT gwODQ7Oz7iZ4VvNPCaZ3SL6rj+ucrH9CSs8Q==", so I could have written:

string viewstate = "dDw0NjgyMTgwODQ7Oz7iZ4VvNPCaZ3SL6rj+ucrH9CSs8Q==";

This manual determination of the ViewState value is awkward because the whole point of our technique is to post data programmatically rather than manually.

I wrote a helper method InitialViewState() that queries the Web application for its initial __VIEWSTATE value and returns it as a string. I'll discuss the InitialViewState() method in the next section; for now, just assume that we can get that value. The raw ViewState value needs processing from the UrlEncode() method:

viewstate = HttpUtility.UrlEncode(viewstate);

UrlEncode() converts characters that the ASP .NET server would misinterpret into escaped sequences. For example, the "/" characters are converted into %2F sequences. After we have the ViewState value, we can construct the full data string that we will post to the Web application, copy the post data to a byte array, and deal with a proxy server if there is one:

string data = "TextBox1=red&TextBox2=empty&Button1=clicked&__VIEWSTATE=" +
viewstate;
byte[] buffer = Encoding.UTF8.GetBytes(data);
string proxy = null;

In this example, there are four name-value pairs. The first pair, TextBox1=red, is exactly what you might expect. It assigns "red" to the control with an ID attribute "TextBox1". The second pair, TextBox2=empty, is probably not expected if you have experience posting data to traditional ASP pages. Because TextBox2 is a server-side control, it contributes to the ViewState value and we must post it to the server to keep the ViewState value valid. The value we assign, "empty," is irrelevant and we could have used anything. I prefer to use values like "empty" that are somewhat self-documenting. The third name-value pair, Button1= clicked, is a little more subtle than it appears. Because Button1 is a server-side control we must post it to keep the ViewState value synchronized. Assigning any value to it has no effect so we could have written Button1= by itself. I like to assign a value like "clicked" as it makes the code more readable. The fourth name-value pair is the __VIEWSTATE pair (note: two underscores) I discussed earlier and the real key to programmatically posting to ASP .NET servers.

After we set up the data string we copy it into a byte array using the GetBytes() method in the System.Text namespace because the method that will post the data later requires the data to be stored as bytes. Then we set up a value for a proxy server. Because I ran my example on a local machine, I just assigned null for its value.

The HttpWebRequest Class
Let's continue walking through the source code by examining the HttpWebRequest class that actually posts our data.

Start by creating an HttpWebRequest object. Notice that, rather unusually, we use an explicit Create() method rather than calling a constructor with the new keyword:

HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);

After creating the HttpWebRequest, we supply values for five of its properties.

req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
req.ContentLength = buffer.Length;
req.Proxy = new WebProxy(proxy, true); // ignore for local addresses
req.CookieContainer = new CookieContainer(); // enable cookies

We use the POST method because we're sending form data, and we set the ContentType property to "application/x-www-form-urlencoded". This is a MIME (Multipurpose Internet Mail Extension) type that essentially tells the ASP .NET server to expect form data. We set the ContentLength property to the number of bytes of post data that we stored in a byte array buffer earlier.

The Boolean true parameter in the WebProxy constructor means to ignore the proxy for local addresses as I'm doing in this example. Assigning a value to the CookieContainer property is required. Notice that we assign an empty CookieContainer object. This is one of the little details that caused me a lot of trouble when I was figuring out this technique.

Before we can post our request to the ASP .NET application, we have to add the post data to the Request object.

Stream reqst = req.GetRequestStream(); // add form data to request stream
reqst.Write(buffer, 0, buffer.Length);
reqst.Flush();
reqst.Close();

After we declare a Stream object, we get the RequestStream from the HttpWebRequestObject and then add to it using the Write() method. Write() requires a Byte array instead of a string; that's why we converted the post data from a string into bytes earlier. Another important detail is to remember to call the Flush() method to actually add the post data to the Request stream.

Finally we're ready to post to our application:

Console.WriteLine("\nPosting 'red' to " + url);
HttpWebResponse res = (HttpWebResponse)req.GetResponse(); // send request, get
response

If you're new to this type of programming, you might have expected something like "req.SendData()" instead of declaring an HttpWebResponse and assigning its value with the GetResponse() method like we actually do. The "send" is implicit. Getting the response from the request follows the usual .NET Framework stream pattern:

Console.WriteLine("\nResponse stream is: \n");
Stream resst = res.GetResponseStream(); // display HTTP response
StreamReader sr = new StreamReader(resst);
Console.WriteLine(sr.ReadToEnd());

To summarize, in order to programmatically post data to an ASP .NET Web application, you must send values for all server-side controls and the __VIEWSTATE input. The methods and properties of the HttpWeb Request class are the primary means we use. You must also be aware of several details that aren't particularly well documented like the necessity of setting the CookieContainer property of your HttpWebRequest object.

The InitialViewState() Helper Method
In the previous sections, we assumed we had a method named InitialViewState() that returns the critical ViewState value. Let's walk through that helper method.

The InitialViewState() accepts as input a string that represents the URL of the Web application of which we are determining the ViewState. The key is to use the WebClient class that is similar to the HttpWebRequest class in that it can send an HTTP request and get the response. However, the WebClient class is much simpler; it can send only a basic HTTP GET request and accept the response stream, but that's all we need to get the initial value of the __VIEWSTATE variable.

First we declare and create a WebClient object using its only default constructor:

WebClient wc = new WebClient();

Next we use that object to create a stream of information to the URL of the Web application:

Stream st = wc.OpenRead(url);
StreamReader sr = new StreamReader(st);

Now we read through the response stream of the initial Web application page line by line, looking for the line that contains the __VIEWSTATE variable so we can parse out its value. Recall that the ViewState is contained in a single line that looks like:

<input type="hidden"
name="__VIEWSTATE"
value="dDw0NjgyMTgwODQ7Oz7iZ4VvNPCaZ3SL6rj+ucrH9CSs8Q==" />

We can use the String.Index Of() method to determine if we are at the target line.

 


if (line.IndexOf("__VIEWSTATE") != -1) // found line
{
  sr.Close();
  st.Close();
  int startIndex = line.IndexOf("value=") + 7;
  int endIndex = line.IndexOf("\"", startIndex);
  int count = endIndex - startIndex;
  return line.Substring(startIndex, count);
}

When we have found the line that contains the __VIEWSTATE value, we can extract it using the String.Substring() method, looking for the characters that follow "value=" and are before the next double-quote character.

Discussion
The example presented and explained in the previous sections accepts user input into a TextBox control and also displays output in a TextBox. What if your application uses other controls? Consider an ASP .NET Web application that has a checkbox, a button, and a label control:

 


<form method="post" runat="server" >
  <asp:CheckBox id="CheckBox1" runat="server"
  Text="Check Me" /> <br>
  <asp:Button id="Button1" runat="server" Text="Button"
  OnClick="Button1_Click"/> <br>
  <asp:Label id="Label1" runat="server" />
</form>

Imagine that the page has some logic that displays a message on the Label control according to whether or not the CheckBox control is checked. To simulate a user checking the CheckBox and clicking the submit Button control, construct a data string like:

data = "CheckBox1=checked&Button1=clicked&Label1=empty&__VIEWSTATE=" & viewstate

To simulate checking the CheckBox control, we put CheckBox1=checked in the data string. Somewhat surprisingly, "checked" is not relevant and as long as we specify any value there, the server will interpret the control as checked. If we want to simulate an unchecked CheckBox, we leave out any value and put CheckBox1= in the string:

data = "CheckBox1=&Button1=clicked&Label1=empty&__VIEWSTATE=" & viewstate

As explained before, we need to supply any value for the Button control, and "clicked" is nicely self-documenting. We must include the Label control too because it contributes to the __VIEWSTATE value, but in this case it doesn't matter whether we supply a value or not (so I used "empty"). Of course we need the initial __VIEWSTATE value.

Depending on which controls you have in your ASP .NET Web application, there are many variations possible, but with a little experimentation you can programmatically post data to any combination of controls.

Now that you can programmatically post data to an ASP .NET Web application, what are some of the ways you can employ this technique in a production environment? I have used this technique in virtually every Web-based product I've worked on. One valuable use is to construct Developer Regression Tests (DRTs) for your Web application. DRTs are a sequence of automated tests that are run after you make changes to your application. They are designed to determine if your new code has broken existing functionality before you check in the code.

For the example application in this article, you could create a simple text file of test cases:

001:red:roses are red
002:blue:the sky is blue
003:green:grass is green
etc.

The first line of this file has a test case ID, followed by the input, and then the expected result. Read this file line by line, parse out the three fields, programmatically post the input string, examine the response stream to see if the actual response matches the expected result or not, and then log the pass or fail result. Notice that test case 003 would catch a logic error in our application (see Listing 1).

There are many ways you can modify the code I've presented here. I have left out all error-checking for the sake of clarity and you'll want to add all the usual checks; the .NET try-catch mechanism is powerful and adding it to this code will save you a lot of time in the long run. In particular, the InitialViewState() method is quite brittle.

I have hard-coded most values. To make the code in this article more flexible you can parameterize the code in several ways. For example, a parameterization I often use takes the form:

bool ResponseStringHasTarget(string url, string proxy, string data, string target)

where the method returns true if the response from data sent to the application URL using proxy server proxy contains a string target. It could then be called like:

 


if (ResponseStringHasTarget(url, proxy, data, target) == true)
  LogToFile("Pass");
else
  LogToFile("Fail");

There are many other ways you can customize and modify the code in this article to suit your own particular needs.

Conclusion
The ability to programmatically post data to a traditional ASP Web page is extremely useful and fairly well understood. But the techniques required to post data to an ASP .NET Web application are not well documented or generally known. This article has shown you how to programmatically post data to a Web application from any .NET program. The key class that posts the request and gets the response is the HttpWebRequest class. However, the technique that we use to post to traditional ASP pages must be significantly modified by taking into account the application's initial __VIEWSTATE attribute and the state of all server-side controls.

By using the techniques in this article you can create extremely powerful automated tests for any ASP .NET Web application and write a wide range of useful utility tools.

Resources and References

  • Warren, S. "Taking a Bite Out of ASP.NET ViewState." November 27, 2001: Taking a Bite Out of ASP.NET ViewState
  • "Client-Side Functionality in a Server Control." .NET Framework Developer's Guide: Client-Side Functionality in a Server Control
  • "HttpWebRequest Class." .NET Framework Class Library: HttpWebRequest Class
  • "Control.ViewState Property." .NET Framework Class Library: Control.ViewState Property
  • More Stories By James McCaffrey

    Dr. James McCaffrey works for Volt Information Sciences, Inc., where he manages technical training for software engineers working at Microsoft's Redmond, WA campus. He has worked on several Microsoft products, including Internet Explorer and MSN Search. James can be reached at [email protected] or [email protected]

    Comments (1) View Comments

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    Most Recent Comments
    portsj 08/19/08 07:19:18 PM EDT

    This code does not work for me. I created a new website and a C# console application in VS.NET 2005. HttpWebResponse res = (HttpWebResponse)req.GetResponse(); throws a 500 error. Also, viewstate = HttpUtility.UrlEncode(viewstate); should be
    viewstate = HttpUtility.UrlDecode(viewstate);

    Can you please test your code and provide me with working code and/or advise how to make it work?

    Thanks in advance,
    p

    @ThingsExpo Stories
    SYS-CON Events announced today that robomq.io will exhibit at SYS-CON's @ThingsExpo, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. robomq.io is an interoperable and composable platform that connects any device to any application. It helps systems integrators and the solution providers build new and innovative products and service for industries requiring monitoring or intelligence from devices and sensors.
    Internet of Things (IoT) will be a hybrid ecosystem of diverse devices and sensors collaborating with operational and enterprise systems to create the next big application. In their session at @ThingsExpo, Bramh Gupta, founder and CEO of robomq.io, and Fred Yatzeck, principal architect leading product development at robomq.io, will discuss how choosing the right middleware and integration strategy from the get-go will enable IoT solution developers to adapt and grow with the industry, while at the same time reduce Time to Market (TTM) by using plug and play capabilities offered by a robust I...
    After making a doctor’s appointment via your mobile device, you receive a calendar invite. The day of your appointment, you get a reminder with the doctor’s location and contact information. As you enter the doctor’s exam room, the medical team is equipped with the latest tablet containing your medical history – he or she makes real time updates to your medical file. At the end of your visit, you receive an electronic prescription to your preferred pharmacy and can schedule your next appointment.
    SYS-CON Events announced today that Solgenia will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY, and the 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. Solgenia is the global market leader in Cloud Collaboration and Cloud Infrastructure software solutions. Designed to “Bridge the Gap” between Personal and Professional Social, Mobile and Cloud user experiences, our solutions help large and medium-sized organizations dr...
    While not quite mainstream yet, WebRTC is starting to gain ground with Carriers, Enterprises and Independent Software Vendors (ISV’s) alike. WebRTC makes it easy for developers to add audio and video communications into their applications by using Web browsers as their platform. But like any market, every customer engagement has unique requirements, as well as constraints. And of course, one size does not fit all. In her session at WebRTC Summit, Dr. Natasha Tamaskar, Vice President, Head of Cloud and Mobile Strategy at GENBAND, will explore what is needed to take a real time communications ...
    The world's leading Cloud event, Cloud Expo has launched Microservices Journal on the SYS-CON.com portal, featuring over 19,000 original articles, news stories, features, and blog entries. DevOps Journal is focused on this critical enterprise IT topic in the world of cloud computing. Microservices Journal offers top articles, news stories, and blog posts from the world's well-known experts and guarantees better exposure for its authors than any other publication. Follow new article posts on Twitter at @MicroservicesE
    SYS-CON Events announced today that Litmus Automation will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Litmus Automation’s vision is to provide a solution for companies that are in a rush to embrace the disruptive Internet of Things technology and leverage it for real business challenges. Litmus Automation simplifies the complexity of connected devices applications with Loop, a secure and scalable cloud platform.
    SYS-CON Events announced today the IoT Bootcamp – Jumpstart Your IoT Strategy, being held June 9–10, 2015, in conjunction with 16th Cloud Expo and Internet of @ThingsExpo at the Javits Center in New York City. This is your chance to jumpstart your IoT strategy. Combined with real-world scenarios and use cases, the IoT Bootcamp is not just based on presentations but includes hands-on demos and walkthroughs. We will introduce you to a variety of Do-It-Yourself IoT platforms including Arduino, Raspberry Pi, BeagleBone, Spark and Intel Edison. You will also get an overview of cloud technologies s...
    Containers and microservices have become topics of intense interest throughout the cloud developer and enterprise IT communities. Accordingly, attendees at the upcoming 16th Cloud Expo at the Javits Center in New York June 9-11 will find fresh new content in a new track called PaaS | Containers & Microservices Containers are not being considered for the first time by the cloud community, but a current era of re-consideration has pushed them to the top of the cloud agenda. With the launch of Docker's initial release in March of 2013, interest was revved up several notches. Then late last...
    The WebRTC Summit 2015 New York, to be held June 9-11, 2015, at the Javits Center in New York, NY, announces that its Call for Papers is open. Topics include all aspects of improving IT delivery by eliminating waste through automated business models leveraging cloud technologies. WebRTC Summit is co-located with 16th International Cloud Expo, @ThingsExpo, Big Data Expo, and DevOps Summit.
    SOA Software has changed its name to Akana. With roots in Web Services and SOA Governance, Akana has established itself as a leader in API Management and is expanding into cloud integration as an alternative to the traditional heavyweight enterprise service bus (ESB). The company recently announced that it achieved more than 90% year-over-year growth. As Akana, the company now addresses the evolution and diversification of SOA, unifying security, management, and DevOps across SOA, APIs, microservices, and more.
    Wearable technology was dominant at this year’s International Consumer Electronics Show (CES) , and MWC was no exception to this trend. New versions of favorites, such as the Samsung Gear (three new products were released: the Gear 2, the Gear 2 Neo and the Gear Fit), shared the limelight with new wearables like Pebble Time Steel (the new premium version of the company’s previously released smartwatch) and the LG Watch Urbane. The most dramatic difference at MWC was an emphasis on presenting wearables as fashion accessories and moving away from the original clunky technology associated with t...
    The list of ‘new paradigm’ technologies that now surrounds us appears to be at an all time high. From cloud computing and Big Data analytics to Bring Your Own Device (BYOD) and the Internet of Things (IoT), today we have to deal with what the industry likes to call ‘paradigm shifts’ at every level of IT. This is disruption; of course, we understand that – change is almost always disruptive.
    SYS-CON Events announced today that SafeLogic has been named “Bag Sponsor” of SYS-CON's 16th International Cloud Expo® New York, which will take place June 9-11, 2015, at the Javits Center in New York City, NY. SafeLogic provides security products for applications in mobile and server/appliance environments. SafeLogic’s flagship product CryptoComply is a FIPS 140-2 validated cryptographic engine designed to secure data on servers, workstations, appliances, mobile devices, and in the Cloud.
    GENBAND has announced that SageNet is leveraging the Nuvia platform to deliver Unified Communications as a Service (UCaaS) to its large base of retail and enterprise customers. Nuvia’s cloud-based solution provides SageNet’s customers with a full suite of business communications and collaboration tools. Two large national SageNet retail customers have recently signed up to deploy the Nuvia platform and the company will continue to sell the service to new and existing customers. Nuvia’s capabilities include HD voice, video, multimedia messaging, mobility, conferencing, Web collaboration, deskt...
    SYS-CON Media announced today that @WebRTCSummit Blog, the largest WebRTC resource in the world, has been launched. @WebRTCSummit Blog offers top articles, news stories, and blog posts from the world's well-known experts and guarantees better exposure for its authors than any other publication. @WebRTCSummit Blog can be bookmarked ▸ Here @WebRTCSummit conference site can be bookmarked ▸ Here
    SYS-CON Events announced today that Cisco, the worldwide leader in IT that transforms how people connect, communicate and collaborate, has been named “Gold Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Cisco makes amazing things happen by connecting the unconnected. Cisco has shaped the future of the Internet by becoming the worldwide leader in transforming how people connect, communicate and collaborate. Cisco and our partners are building the platform for the Internet of Everything by connecting the...
    Temasys has announced senior management additions to its team. Joining are David Holloway as Vice President of Commercial and Nadine Yap as Vice President of Product. Over the past 12 months Temasys has doubled in size as it adds new customers and expands the development of its Skylink platform. Skylink leads the charge to move WebRTC, traditionally seen as a desktop, browser based technology, to become a ubiquitous web communications technology on web and mobile, as well as Internet of Things compatible devices.
    Docker is an excellent platform for organizations interested in running microservices. It offers portability and consistency between development and production environments, quick provisioning times, and a simple way to isolate services. In his session at DevOps Summit at 16th Cloud Expo, Shannon Williams, co-founder of Rancher Labs, will walk through these and other benefits of using Docker to run microservices, and provide an overview of RancherOS, a minimalist distribution of Linux designed expressly to run Docker. He will also discuss Rancher, an orchestration and service discovery platf...
    SYS-CON Events announced today that Vitria Technology, Inc. will exhibit at SYS-CON’s @ThingsExpo, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Vitria will showcase the company’s new IoT Analytics Platform through live demonstrations at booth #330. Vitria’s IoT Analytics Platform, fully integrated and powered by an operational intelligence engine, enables customers to rapidly build and operationalize advanced analytics to deliver timely business outcomes for use cases across the industrial, enterprise, and consumer segments.