Welcome!

.NET Authors: Esmeralda Swartz, Matt Hester, Elizabeth White, Lori MacVittie, Adine Deford

Related Topics: .NET

.NET: Article

Programmatically Posting Data to ASP .NET Web Applications

Create powerful automated utilities

Programmatically posting data to a traditional ASP page is an extremely useful and well-known technique. However, I recently discovered that posting data to an ASP .NET Web application from another program is not well understood and requires several new programming tricks. In this article I'll show you how to use the HttpWebRequest class and the ASP .NET ViewState mechanism to programmatically send form data to an ASP .NET Web application and then capture the response. This will provide you with a powerful new way to write utility programs for your Web applications. (Note: This article assumes you are familiar with creating ASP .NET Web applications, using classes in the .NET Framework, and have intermediate familiarity with the C# language.)

The best way to demonstrate what we will accomplish is with two screenshots. Figure 1 shows a simple ASP .NET Web application. If the user types "red" in the upper textbox control and clicks the submit button, the application displays "roses are red" in the lower textbox as shown. If the user types and sends anything other than "red," "blue," or "green," the application responds with "unknown color."

Suppose we want to post data programmatically and examine the response. In other words, we want a console application or some other type of program to simulate typing a value in the Enter-a-color textbox and clicking the submit button, and then determine what the response is. Figure 2 shows a console application that does just that. You can see the value of TextBox2 is "roses are red."

Even though this example is a console application, the underlying code can be used in any .NET program. In the following sections I'll walk you through the core code that posts data to ASP .NET and explain it in detail, then we'll briefly look at some variations to show you how to deal with server-side controls other than TextBox. I'll conclude with a discussion of some of the ways you can use this technique in a production environment.

The Problem
Listing 1 shows the source code for the simple ASP .NET Web application shown in Figure 1. For simplicity, I used a basic text editor (yes, Notepad) and put the logic code in the same file as the HTML display code. I used the Visual Basic .NET language for the Web application, but I could have used any .NET language.

If we want to send "red" to this application, we have to deal with three problems. How do we assign "red" to the TextBox1 control; how do we tell the ASP .NET server that Button1 has been clicked; and how do we capture the response from the server?

The ViewState Value
Listing 2 shows the complete console application source code that generated the output shown in Figure 2. As you can see I used C# but I could have used Visual Basic .NET or any other .NET language. I'll walk you through this code in the next three sections and explain exactly how it works.

After declaring five namespaces so I wouldn't have to fully qualify each class, I declared and assigned a variable for the URL of the Web application:

string url = "http://localhost/PostToASPdotNET/colors.aspx";

Next comes the trickiest part and the key to the entire programmatic post technique:

string viewstate = InitialViewState(url);

I declare and assign a value for the Web application's ViewState. What is this? Even though HTTP is a stateless protocol - that is, each Request-Response pair is an isolated transaction - ASP .NET works behind the scenes to simulate a stateful environment. One of the ways ASP .NET does this is through the use of an HTML hidden input named __VIEWSTATE. It's a Base64-encoded string value that represents the state of the page when the server last processed it. In this way pages can retain values between successive calls. To correctly post information to an ASP .NET application, we need to send the ViewState value to the server. Where do we get this value from?

The easiest way to get the initial ViewState value for a Web application is to launch Internet Explorer, get the page, and then do a View -> Source from the menu bar (see Figure 3).

As you can see, the initial value of __VIEWSTATE is "dDw0NjgyMT gwODQ7Oz7iZ4VvNPCaZ3SL6rj+ucrH9CSs8Q==", so I could have written:

string viewstate = "dDw0NjgyMTgwODQ7Oz7iZ4VvNPCaZ3SL6rj+ucrH9CSs8Q==";

This manual determination of the ViewState value is awkward because the whole point of our technique is to post data programmatically rather than manually.

I wrote a helper method InitialViewState() that queries the Web application for its initial __VIEWSTATE value and returns it as a string. I'll discuss the InitialViewState() method in the next section; for now, just assume that we can get that value. The raw ViewState value needs processing from the UrlEncode() method:

viewstate = HttpUtility.UrlEncode(viewstate);

UrlEncode() converts characters that the ASP .NET server would misinterpret into escaped sequences. For example, the "/" characters are converted into %2F sequences. After we have the ViewState value, we can construct the full data string that we will post to the Web application, copy the post data to a byte array, and deal with a proxy server if there is one:

string data = "TextBox1=red&TextBox2=empty&Button1=clicked&__VIEWSTATE=" +
viewstate;
byte[] buffer = Encoding.UTF8.GetBytes(data);
string proxy = null;

In this example, there are four name-value pairs. The first pair, TextBox1=red, is exactly what you might expect. It assigns "red" to the control with an ID attribute "TextBox1". The second pair, TextBox2=empty, is probably not expected if you have experience posting data to traditional ASP pages. Because TextBox2 is a server-side control, it contributes to the ViewState value and we must post it to the server to keep the ViewState value valid. The value we assign, "empty," is irrelevant and we could have used anything. I prefer to use values like "empty" that are somewhat self-documenting. The third name-value pair, Button1= clicked, is a little more subtle than it appears. Because Button1 is a server-side control we must post it to keep the ViewState value synchronized. Assigning any value to it has no effect so we could have written Button1= by itself. I like to assign a value like "clicked" as it makes the code more readable. The fourth name-value pair is the __VIEWSTATE pair (note: two underscores) I discussed earlier and the real key to programmatically posting to ASP .NET servers.

After we set up the data string we copy it into a byte array using the GetBytes() method in the System.Text namespace because the method that will post the data later requires the data to be stored as bytes. Then we set up a value for a proxy server. Because I ran my example on a local machine, I just assigned null for its value.

The HttpWebRequest Class
Let's continue walking through the source code by examining the HttpWebRequest class that actually posts our data.

Start by creating an HttpWebRequest object. Notice that, rather unusually, we use an explicit Create() method rather than calling a constructor with the new keyword:

HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);

After creating the HttpWebRequest, we supply values for five of its properties.

req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
req.ContentLength = buffer.Length;
req.Proxy = new WebProxy(proxy, true); // ignore for local addresses
req.CookieContainer = new CookieContainer(); // enable cookies

We use the POST method because we're sending form data, and we set the ContentType property to "application/x-www-form-urlencoded". This is a MIME (Multipurpose Internet Mail Extension) type that essentially tells the ASP .NET server to expect form data. We set the ContentLength property to the number of bytes of post data that we stored in a byte array buffer earlier.

The Boolean true parameter in the WebProxy constructor means to ignore the proxy for local addresses as I'm doing in this example. Assigning a value to the CookieContainer property is required. Notice that we assign an empty CookieContainer object. This is one of the little details that caused me a lot of trouble when I was figuring out this technique.

Before we can post our request to the ASP .NET application, we have to add the post data to the Request object.

Stream reqst = req.GetRequestStream(); // add form data to request stream
reqst.Write(buffer, 0, buffer.Length);
reqst.Flush();
reqst.Close();

After we declare a Stream object, we get the RequestStream from the HttpWebRequestObject and then add to it using the Write() method. Write() requires a Byte array instead of a string; that's why we converted the post data from a string into bytes earlier. Another important detail is to remember to call the Flush() method to actually add the post data to the Request stream.

Finally we're ready to post to our application:

Console.WriteLine("\nPosting 'red' to " + url);
HttpWebResponse res = (HttpWebResponse)req.GetResponse(); // send request, get
response

If you're new to this type of programming, you might have expected something like "req.SendData()" instead of declaring an HttpWebResponse and assigning its value with the GetResponse() method like we actually do. The "send" is implicit. Getting the response from the request follows the usual .NET Framework stream pattern:

Console.WriteLine("\nResponse stream is: \n");
Stream resst = res.GetResponseStream(); // display HTTP response
StreamReader sr = new StreamReader(resst);
Console.WriteLine(sr.ReadToEnd());

To summarize, in order to programmatically post data to an ASP .NET Web application, you must send values for all server-side controls and the __VIEWSTATE input. The methods and properties of the HttpWeb Request class are the primary means we use. You must also be aware of several details that aren't particularly well documented like the necessity of setting the CookieContainer property of your HttpWebRequest object.

The InitialViewState() Helper Method
In the previous sections, we assumed we had a method named InitialViewState() that returns the critical ViewState value. Let's walk through that helper method.

The InitialViewState() accepts as input a string that represents the URL of the Web application of which we are determining the ViewState. The key is to use the WebClient class that is similar to the HttpWebRequest class in that it can send an HTTP request and get the response. However, the WebClient class is much simpler; it can send only a basic HTTP GET request and accept the response stream, but that's all we need to get the initial value of the __VIEWSTATE variable.

First we declare and create a WebClient object using its only default constructor:

WebClient wc = new WebClient();

Next we use that object to create a stream of information to the URL of the Web application:

Stream st = wc.OpenRead(url);
StreamReader sr = new StreamReader(st);

Now we read through the response stream of the initial Web application page line by line, looking for the line that contains the __VIEWSTATE variable so we can parse out its value. Recall that the ViewState is contained in a single line that looks like:

<input type="hidden"
name="__VIEWSTATE"
value="dDw0NjgyMTgwODQ7Oz7iZ4VvNPCaZ3SL6rj+ucrH9CSs8Q==" />

We can use the String.Index Of() method to determine if we are at the target line.

 


if (line.IndexOf("__VIEWSTATE") != -1) // found line
{
  sr.Close();
  st.Close();
  int startIndex = line.IndexOf("value=") + 7;
  int endIndex = line.IndexOf("\"", startIndex);
  int count = endIndex - startIndex;
  return line.Substring(startIndex, count);
}

When we have found the line that contains the __VIEWSTATE value, we can extract it using the String.Substring() method, looking for the characters that follow "value=" and are before the next double-quote character.

Discussion
The example presented and explained in the previous sections accepts user input into a TextBox control and also displays output in a TextBox. What if your application uses other controls? Consider an ASP .NET Web application that has a checkbox, a button, and a label control:

 


<form method="post" runat="server" >
  <asp:CheckBox id="CheckBox1" runat="server"
  Text="Check Me" /> <br>
  <asp:Button id="Button1" runat="server" Text="Button"
  OnClick="Button1_Click"/> <br>
  <asp:Label id="Label1" runat="server" />
</form>

Imagine that the page has some logic that displays a message on the Label control according to whether or not the CheckBox control is checked. To simulate a user checking the CheckBox and clicking the submit Button control, construct a data string like:

data = "CheckBox1=checked&Button1=clicked&Label1=empty&__VIEWSTATE=" & viewstate

To simulate checking the CheckBox control, we put CheckBox1=checked in the data string. Somewhat surprisingly, "checked" is not relevant and as long as we specify any value there, the server will interpret the control as checked. If we want to simulate an unchecked CheckBox, we leave out any value and put CheckBox1= in the string:

data = "CheckBox1=&Button1=clicked&Label1=empty&__VIEWSTATE=" & viewstate

As explained before, we need to supply any value for the Button control, and "clicked" is nicely self-documenting. We must include the Label control too because it contributes to the __VIEWSTATE value, but in this case it doesn't matter whether we supply a value or not (so I used "empty"). Of course we need the initial __VIEWSTATE value.

Depending on which controls you have in your ASP .NET Web application, there are many variations possible, but with a little experimentation you can programmatically post data to any combination of controls.

Now that you can programmatically post data to an ASP .NET Web application, what are some of the ways you can employ this technique in a production environment? I have used this technique in virtually every Web-based product I've worked on. One valuable use is to construct Developer Regression Tests (DRTs) for your Web application. DRTs are a sequence of automated tests that are run after you make changes to your application. They are designed to determine if your new code has broken existing functionality before you check in the code.

For the example application in this article, you could create a simple text file of test cases:

001:red:roses are red
002:blue:the sky is blue
003:green:grass is green
etc.

The first line of this file has a test case ID, followed by the input, and then the expected result. Read this file line by line, parse out the three fields, programmatically post the input string, examine the response stream to see if the actual response matches the expected result or not, and then log the pass or fail result. Notice that test case 003 would catch a logic error in our application (see Listing 1).

There are many ways you can modify the code I've presented here. I have left out all error-checking for the sake of clarity and you'll want to add all the usual checks; the .NET try-catch mechanism is powerful and adding it to this code will save you a lot of time in the long run. In particular, the InitialViewState() method is quite brittle.

I have hard-coded most values. To make the code in this article more flexible you can parameterize the code in several ways. For example, a parameterization I often use takes the form:

bool ResponseStringHasTarget(string url, string proxy, string data, string target)

where the method returns true if the response from data sent to the application URL using proxy server proxy contains a string target. It could then be called like:

 


if (ResponseStringHasTarget(url, proxy, data, target) == true)
  LogToFile("Pass");
else
  LogToFile("Fail");

There are many other ways you can customize and modify the code in this article to suit your own particular needs.

Conclusion
The ability to programmatically post data to a traditional ASP Web page is extremely useful and fairly well understood. But the techniques required to post data to an ASP .NET Web application are not well documented or generally known. This article has shown you how to programmatically post data to a Web application from any .NET program. The key class that posts the request and gets the response is the HttpWebRequest class. However, the technique that we use to post to traditional ASP pages must be significantly modified by taking into account the application's initial __VIEWSTATE attribute and the state of all server-side controls.

By using the techniques in this article you can create extremely powerful automated tests for any ASP .NET Web application and write a wide range of useful utility tools.

Resources and References

  • Warren, S. "Taking a Bite Out of ASP.NET ViewState." November 27, 2001: Taking a Bite Out of ASP.NET ViewState
  • "Client-Side Functionality in a Server Control." .NET Framework Developer's Guide: Client-Side Functionality in a Server Control
  • "HttpWebRequest Class." .NET Framework Class Library: HttpWebRequest Class
  • "Control.ViewState Property." .NET Framework Class Library: Control.ViewState Property
  • More Stories By James McCaffrey

    Dr. James McCaffrey works for Volt Information Sciences, Inc., where he manages technical training for software engineers working at Microsoft's Redmond, WA campus. He has worked on several Microsoft products, including Internet Explorer and MSN Search. James can be reached at [email protected] or [email protected]

    Comments (1) View Comments

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    Most Recent Comments
    portsj 08/19/08 07:19:18 PM EDT

    This code does not work for me. I created a new website and a C# console application in VS.NET 2005. HttpWebResponse res = (HttpWebResponse)req.GetResponse(); throws a 500 error. Also, viewstate = HttpUtility.UrlEncode(viewstate); should be
    viewstate = HttpUtility.UrlDecode(viewstate);

    Can you please test your code and provide me with working code and/or advise how to make it work?

    Thanks in advance,
    p