Welcome!

.NET Authors: Srinivasan Sundara Rajan, Sergio Varga, Adine Deford, David Fletcher, Pat Romanski

Related Topics: .NET

.NET: Article

Programmatically Posting Data to ASP .NET Web Applications

Create powerful automated utilities

Programmatically posting data to a traditional ASP page is an extremely useful and well-known technique. However, I recently discovered that posting data to an ASP .NET Web application from another program is not well understood and requires several new programming tricks. In this article I'll show you how to use the HttpWebRequest class and the ASP .NET ViewState mechanism to programmatically send form data to an ASP .NET Web application and then capture the response. This will provide you with a powerful new way to write utility programs for your Web applications. (Note: This article assumes you are familiar with creating ASP .NET Web applications, using classes in the .NET Framework, and have intermediate familiarity with the C# language.)

The best way to demonstrate what we will accomplish is with two screenshots. Figure 1 shows a simple ASP .NET Web application. If the user types "red" in the upper textbox control and clicks the submit button, the application displays "roses are red" in the lower textbox as shown. If the user types and sends anything other than "red," "blue," or "green," the application responds with "unknown color."

Suppose we want to post data programmatically and examine the response. In other words, we want a console application or some other type of program to simulate typing a value in the Enter-a-color textbox and clicking the submit button, and then determine what the response is. Figure 2 shows a console application that does just that. You can see the value of TextBox2 is "roses are red."

Even though this example is a console application, the underlying code can be used in any .NET program. In the following sections I'll walk you through the core code that posts data to ASP .NET and explain it in detail, then we'll briefly look at some variations to show you how to deal with server-side controls other than TextBox. I'll conclude with a discussion of some of the ways you can use this technique in a production environment.

The Problem
Listing 1 shows the source code for the simple ASP .NET Web application shown in Figure 1. For simplicity, I used a basic text editor (yes, Notepad) and put the logic code in the same file as the HTML display code. I used the Visual Basic .NET language for the Web application, but I could have used any .NET language.

If we want to send "red" to this application, we have to deal with three problems. How do we assign "red" to the TextBox1 control; how do we tell the ASP .NET server that Button1 has been clicked; and how do we capture the response from the server?

The ViewState Value
Listing 2 shows the complete console application source code that generated the output shown in Figure 2. As you can see I used C# but I could have used Visual Basic .NET or any other .NET language. I'll walk you through this code in the next three sections and explain exactly how it works.

After declaring five namespaces so I wouldn't have to fully qualify each class, I declared and assigned a variable for the URL of the Web application:

string url = "http://localhost/PostToASPdotNET/colors.aspx";

Next comes the trickiest part and the key to the entire programmatic post technique:

string viewstate = InitialViewState(url);

I declare and assign a value for the Web application's ViewState. What is this? Even though HTTP is a stateless protocol - that is, each Request-Response pair is an isolated transaction - ASP .NET works behind the scenes to simulate a stateful environment. One of the ways ASP .NET does this is through the use of an HTML hidden input named __VIEWSTATE. It's a Base64-encoded string value that represents the state of the page when the server last processed it. In this way pages can retain values between successive calls. To correctly post information to an ASP .NET application, we need to send the ViewState value to the server. Where do we get this value from?

The easiest way to get the initial ViewState value for a Web application is to launch Internet Explorer, get the page, and then do a View -> Source from the menu bar (see Figure 3).

As you can see, the initial value of __VIEWSTATE is "dDw0NjgyMT gwODQ7Oz7iZ4VvNPCaZ3SL6rj+ucrH9CSs8Q==", so I could have written:

string viewstate = "dDw0NjgyMTgwODQ7Oz7iZ4VvNPCaZ3SL6rj+ucrH9CSs8Q==";

This manual determination of the ViewState value is awkward because the whole point of our technique is to post data programmatically rather than manually.

I wrote a helper method InitialViewState() that queries the Web application for its initial __VIEWSTATE value and returns it as a string. I'll discuss the InitialViewState() method in the next section; for now, just assume that we can get that value. The raw ViewState value needs processing from the UrlEncode() method:

viewstate = HttpUtility.UrlEncode(viewstate);

UrlEncode() converts characters that the ASP .NET server would misinterpret into escaped sequences. For example, the "/" characters are converted into %2F sequences. After we have the ViewState value, we can construct the full data string that we will post to the Web application, copy the post data to a byte array, and deal with a proxy server if there is one:

string data = "TextBox1=red&TextBox2=empty&Button1=clicked&__VIEWSTATE=" +
viewstate;
byte[] buffer = Encoding.UTF8.GetBytes(data);
string proxy = null;

In this example, there are four name-value pairs. The first pair, TextBox1=red, is exactly what you might expect. It assigns "red" to the control with an ID attribute "TextBox1". The second pair, TextBox2=empty, is probably not expected if you have experience posting data to traditional ASP pages. Because TextBox2 is a server-side control, it contributes to the ViewState value and we must post it to the server to keep the ViewState value valid. The value we assign, "empty," is irrelevant and we could have used anything. I prefer to use values like "empty" that are somewhat self-documenting. The third name-value pair, Button1= clicked, is a little more subtle than it appears. Because Button1 is a server-side control we must post it to keep the ViewState value synchronized. Assigning any value to it has no effect so we could have written Button1= by itself. I like to assign a value like "clicked" as it makes the code more readable. The fourth name-value pair is the __VIEWSTATE pair (note: two underscores) I discussed earlier and the real key to programmatically posting to ASP .NET servers.

After we set up the data string we copy it into a byte array using the GetBytes() method in the System.Text namespace because the method that will post the data later requires the data to be stored as bytes. Then we set up a value for a proxy server. Because I ran my example on a local machine, I just assigned null for its value.

The HttpWebRequest Class
Let's continue walking through the source code by examining the HttpWebRequest class that actually posts our data.

Start by creating an HttpWebRequest object. Notice that, rather unusually, we use an explicit Create() method rather than calling a constructor with the new keyword:

HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);

After creating the HttpWebRequest, we supply values for five of its properties.

req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
req.ContentLength = buffer.Length;
req.Proxy = new WebProxy(proxy, true); // ignore for local addresses
req.CookieContainer = new CookieContainer(); // enable cookies

We use the POST method because we're sending form data, and we set the ContentType property to "application/x-www-form-urlencoded". This is a MIME (Multipurpose Internet Mail Extension) type that essentially tells the ASP .NET server to expect form data. We set the ContentLength property to the number of bytes of post data that we stored in a byte array buffer earlier.

The Boolean true parameter in the WebProxy constructor means to ignore the proxy for local addresses as I'm doing in this example. Assigning a value to the CookieContainer property is required. Notice that we assign an empty CookieContainer object. This is one of the little details that caused me a lot of trouble when I was figuring out this technique.

Before we can post our request to the ASP .NET application, we have to add the post data to the Request object.

Stream reqst = req.GetRequestStream(); // add form data to request stream
reqst.Write(buffer, 0, buffer.Length);
reqst.Flush();
reqst.Close();

After we declare a Stream object, we get the RequestStream from the HttpWebRequestObject and then add to it using the Write() method. Write() requires a Byte array instead of a string; that's why we converted the post data from a string into bytes earlier. Another important detail is to remember to call the Flush() method to actually add the post data to the Request stream.

Finally we're ready to post to our application:

Console.WriteLine("\nPosting 'red' to " + url);
HttpWebResponse res = (HttpWebResponse)req.GetResponse(); // send request, get
response

If you're new to this type of programming, you might have expected something like "req.SendData()" instead of declaring an HttpWebResponse and assigning its value with the GetResponse() method like we actually do. The "send" is implicit. Getting the response from the request follows the usual .NET Framework stream pattern:

Console.WriteLine("\nResponse stream is: \n");
Stream resst = res.GetResponseStream(); // display HTTP response
StreamReader sr = new StreamReader(resst);
Console.WriteLine(sr.ReadToEnd());

To summarize, in order to programmatically post data to an ASP .NET Web application, you must send values for all server-side controls and the __VIEWSTATE input. The methods and properties of the HttpWeb Request class are the primary means we use. You must also be aware of several details that aren't particularly well documented like the necessity of setting the CookieContainer property of your HttpWebRequest object.

The InitialViewState() Helper Method
In the previous sections, we assumed we had a method named InitialViewState() that returns the critical ViewState value. Let's walk through that helper method.

The InitialViewState() accepts as input a string that represents the URL of the Web application of which we are determining the ViewState. The key is to use the WebClient class that is similar to the HttpWebRequest class in that it can send an HTTP request and get the response. However, the WebClient class is much simpler; it can send only a basic HTTP GET request and accept the response stream, but that's all we need to get the initial value of the __VIEWSTATE variable.

First we declare and create a WebClient object using its only default constructor:

WebClient wc = new WebClient();

Next we use that object to create a stream of information to the URL of the Web application:

Stream st = wc.OpenRead(url);
StreamReader sr = new StreamReader(st);

Now we read through the response stream of the initial Web application page line by line, looking for the line that contains the __VIEWSTATE variable so we can parse out its value. Recall that the ViewState is contained in a single line that looks like:

<input type="hidden"
name="__VIEWSTATE"
value="dDw0NjgyMTgwODQ7Oz7iZ4VvNPCaZ3SL6rj+ucrH9CSs8Q==" />

We can use the String.Index Of() method to determine if we are at the target line.

 


if (line.IndexOf("__VIEWSTATE") != -1) // found line
{
  sr.Close();
  st.Close();
  int startIndex = line.IndexOf("value=") + 7;
  int endIndex = line.IndexOf("\"", startIndex);
  int count = endIndex - startIndex;
  return line.Substring(startIndex, count);
}

When we have found the line that contains the __VIEWSTATE value, we can extract it using the String.Substring() method, looking for the characters that follow "value=" and are before the next double-quote character.

Discussion
The example presented and explained in the previous sections accepts user input into a TextBox control and also displays output in a TextBox. What if your application uses other controls? Consider an ASP .NET Web application that has a checkbox, a button, and a label control:

 


<form method="post" runat="server" >
  <asp:CheckBox id="CheckBox1" runat="server"
  Text="Check Me" /> <br>
  <asp:Button id="Button1" runat="server" Text="Button"
  OnClick="Button1_Click"/> <br>
  <asp:Label id="Label1" runat="server" />
</form>

Imagine that the page has some logic that displays a message on the Label control according to whether or not the CheckBox control is checked. To simulate a user checking the CheckBox and clicking the submit Button control, construct a data string like:

data = "CheckBox1=checked&Button1=clicked&Label1=empty&__VIEWSTATE=" & viewstate

To simulate checking the CheckBox control, we put CheckBox1=checked in the data string. Somewhat surprisingly, "checked" is not relevant and as long as we specify any value there, the server will interpret the control as checked. If we want to simulate an unchecked CheckBox, we leave out any value and put CheckBox1= in the string:

data = "CheckBox1=&Button1=clicked&Label1=empty&__VIEWSTATE=" & viewstate

As explained before, we need to supply any value for the Button control, and "clicked" is nicely self-documenting. We must include the Label control too because it contributes to the __VIEWSTATE value, but in this case it doesn't matter whether we supply a value or not (so I used "empty"). Of course we need the initial __VIEWSTATE value.

Depending on which controls you have in your ASP .NET Web application, there are many variations possible, but with a little experimentation you can programmatically post data to any combination of controls.

Now that you can programmatically post data to an ASP .NET Web application, what are some of the ways you can employ this technique in a production environment? I have used this technique in virtually every Web-based product I've worked on. One valuable use is to construct Developer Regression Tests (DRTs) for your Web application. DRTs are a sequence of automated tests that are run after you make changes to your application. They are designed to determine if your new code has broken existing functionality before you check in the code.

For the example application in this article, you could create a simple text file of test cases:

001:red:roses are red
002:blue:the sky is blue
003:green:grass is green
etc.

The first line of this file has a test case ID, followed by the input, and then the expected result. Read this file line by line, parse out the three fields, programmatically post the input string, examine the response stream to see if the actual response matches the expected result or not, and then log the pass or fail result. Notice that test case 003 would catch a logic error in our application (see Listing 1).

There are many ways you can modify the code I've presented here. I have left out all error-checking for the sake of clarity and you'll want to add all the usual checks; the .NET try-catch mechanism is powerful and adding it to this code will save you a lot of time in the long run. In particular, the InitialViewState() method is quite brittle.

I have hard-coded most values. To make the code in this article more flexible you can parameterize the code in several ways. For example, a parameterization I often use takes the form:

bool ResponseStringHasTarget(string url, string proxy, string data, string target)

where the method returns true if the response from data sent to the application URL using proxy server proxy contains a string target. It could then be called like:

 


if (ResponseStringHasTarget(url, proxy, data, target) == true)
  LogToFile("Pass");
else
  LogToFile("Fail");

There are many other ways you can customize and modify the code in this article to suit your own particular needs.

Conclusion
The ability to programmatically post data to a traditional ASP Web page is extremely useful and fairly well understood. But the techniques required to post data to an ASP .NET Web application are not well documented or generally known. This article has shown you how to programmatically post data to a Web application from any .NET program. The key class that posts the request and gets the response is the HttpWebRequest class. However, the technique that we use to post to traditional ASP pages must be significantly modified by taking into account the application's initial __VIEWSTATE attribute and the state of all server-side controls.

By using the techniques in this article you can create extremely powerful automated tests for any ASP .NET Web application and write a wide range of useful utility tools.

Resources and References

  • Warren, S. "Taking a Bite Out of ASP.NET ViewState." November 27, 2001: Taking a Bite Out of ASP.NET ViewState
  • "Client-Side Functionality in a Server Control." .NET Framework Developer's Guide: Client-Side Functionality in a Server Control
  • "HttpWebRequest Class." .NET Framework Class Library: HttpWebRequest Class
  • "Control.ViewState Property." .NET Framework Class Library: Control.ViewState Property
  • More Stories By James McCaffrey

    Dr. James McCaffrey works for Volt Information Sciences, Inc., where he manages technical training for software engineers working at Microsoft's Redmond, WA campus. He has worked on several Microsoft products, including Internet Explorer and MSN Search. James can be reached at [email protected] or [email protected]

    Comments (1) View Comments

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    Most Recent Comments
    portsj 08/19/08 07:19:18 PM EDT

    This code does not work for me. I created a new website and a C# console application in VS.NET 2005. HttpWebResponse res = (HttpWebResponse)req.GetResponse(); throws a 500 error. Also, viewstate = HttpUtility.UrlEncode(viewstate); should be
    viewstate = HttpUtility.UrlDecode(viewstate);

    Can you please test your code and provide me with working code and/or advise how to make it work?

    Thanks in advance,
    p

    @ThingsExpo Stories
    “In the past year we've seen a lot of stabilization of WebRTC. You can now use it in production with a far greater degree of certainty. A lot of the real developments in the past year have been in things like the data channel, which will enable a whole new type of application," explained Peter Dunkley, Technical Director at Acision, in this SYS-CON.tv interview at @ThingsExpo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
    The BPM world is going through some evolution or changes where traditional business process management solutions really have nowhere to go in terms of development of the road map. In this demo at 15th Cloud Expo, Kyle Hansen, Director of Professional Services at AgilePoint, shows AgilePoint’s unique approach to dealing with this market circumstance by developing a rapid application composition or development framework.
    The major cloud platforms defy a simple, side-by-side analysis. Each of the major IaaS public-cloud platforms offers their own unique strengths and functionality. Options for on-site private cloud are diverse as well, and must be designed and deployed while taking existing legacy architecture and infrastructure into account. Then the reality is that most enterprises are embarking on a hybrid cloud strategy and programs. In this Power Panel at 15th Cloud Expo (http://www.CloudComputingExpo.com), moderated by Ashar Baig, Research Director, Cloud, at Gigaom Research, Nate Gordon, Director of T...
    "BSQUARE is in the business of selling software solutions for smart connected devices. It's obvious that IoT has moved from being a technology to being a fundamental part of business, and in the last 18 months people have said let's figure out how to do it and let's put some focus on it, " explained Dave Wagstaff, VP & Chief Architect, at BSQUARE Corporation, in this SYS-CON.tv interview at @ThingsExpo, held Nov 4-6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
    SYS-CON Events announced today that Windstream, a leading provider of advanced network and cloud communications, has been named “Silver Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9–11, 2015, at the Javits Center in New York, NY. Windstream (Nasdaq: WIN), a FORTUNE 500 and S&P 500 company, is a leading provider of advanced network communications, including cloud computing and managed services, to businesses nationwide. The company also offers broadband, phone and digital TV services to consumers primarily in rural areas.
    The Internet of Things is not new. Historically, smart businesses have used its basic concept of leveraging data to drive better decision making and have capitalized on those insights to realize additional revenue opportunities. So, what has changed to make the Internet of Things one of the hottest topics in tech? In his session at @ThingsExpo, Chris Gray, Director, Embedded and Internet of Things, discussed the underlying factors that are driving the economics of intelligent systems. Discover how hardware commoditization, the ubiquitous nature of connectivity, and the emergence of Big Data a...

    ARMONK, N.Y., Nov. 20, 2014 /PRNewswire/ --  IBM (NYSE: IBM) today announced that it is bringing a greater level of control, security and flexibility to cloud-based application development and delivery with a single-tenant version of Bluemix, IBM's platform-as-a-service. The new platform enables developers to build ap...

    SYS-CON Events announced today that IDenticard will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. IDenticard™ is the security division of Brady Corp (NYSE: BRC), a $1.5 billion manufacturer of identification products. We have small-company values with the strength and stability of a major corporation. IDenticard offers local sales, support and service to our customers across the United States and Canada. Our partner network encompasses some 300 of the world's leading systems integrators and security s...
    DevOps Summit 2015 New York, co-located with the 16th International Cloud Expo - to be held June 9-11, 2015, at the Javits Center in New York City, NY - announces that it is now accepting Keynote Proposals. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long development cycles that produce software that is obsolete at launch. DevOps may be disruptive, but it is essential.
    "People are a lot more knowledgeable about APIs now. There are two types of people who work with APIs - IT people who want to use APIs for something internal and the product managers who want to do something outside APIs for people to connect to them," explained Roberto Medrano, Executive Vice President at SOA Software, in this SYS-CON.tv interview at Cloud Expo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
    Nigeria has the largest economy in Africa, at more than US$500 billion, and ranks 23rd in the world. A recent re-evaluation of Nigeria's true economic size doubled the previous estimate, and brought it well ahead of South Africa, which is a member (unlike Nigeria) of the G20 club for political as well as economic reasons. Nigeria's economy can be said to be quite diverse from one point of view, but heavily dependent on oil and gas at the same time. Oil and natural gas account for about 15% of Nigera's overall economy, but traditionally represent more than 90% of the country's exports and as...
    The Internet of Things is a misnomer. That implies that everything is on the Internet, and that simply should not be - especially for things that are blurring the line between medical devices that stimulate like a pacemaker and quantified self-sensors like a pedometer or pulse tracker. The mesh of things that we manage must be segmented into zones of trust for sensing data, transmitting data, receiving command and control administrative changes, and peer-to-peer mesh messaging. In his session at @ThingsExpo, Ryan Bagnulo, Solution Architect / Software Engineer at SOA Software, focused on desi...
    "At our booth we are showing how to provide trust in the Internet of Things. Trust is where everything starts to become secure and trustworthy. Now with the scaling of the Internet of Things it becomes an interesting question – I've heard numbers from 200 billion devices next year up to a trillion in the next 10 to 15 years," explained Johannes Lintzen, Vice President of Sales at Utimaco, in this SYS-CON.tv interview at @ThingsExpo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
    "For over 25 years we have been working with a lot of enterprise customers and we have seen how companies create applications. And now that we have moved to cloud computing, mobile, social and the Internet of Things, we see that the market needs a new way of creating applications," stated Jesse Shiah, CEO, President and Co-Founder of AgilePoint Inc., in this SYS-CON.tv interview at 15th Cloud Expo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
    SYS-CON Events announced today that Gridstore™, the leader in hyper-converged infrastructure purpose-built to optimize Microsoft workloads, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Gridstore™ is the leader in hyper-converged infrastructure purpose-built for Microsoft workloads and designed to accelerate applications in virtualized environments. Gridstore’s hyper-converged infrastructure is the industry’s first all flash version of HyperConverged Appliances that include both compute and storag...
    Today’s enterprise is being driven by disruptive competitive and human capital requirements to provide enterprise application access through not only desktops, but also mobile devices. To retrofit existing programs across all these devices using traditional programming methods is very costly and time consuming – often prohibitively so. In his session at @ThingsExpo, Jesse Shiah, CEO, President, and Co-Founder of AgilePoint Inc., discussed how you can create applications that run on all mobile devices as well as laptops and desktops using a visual drag-and-drop application – and eForms-buildi...
    We certainly live in interesting technological times. And no more interesting than the current competing IoT standards for connectivity. Various standards bodies, approaches, and ecosystems are vying for mindshare and positioning for a competitive edge. It is clear that when the dust settles, we will have new protocols, evolved protocols, that will change the way we interact with devices and infrastructure. We will also have evolved web protocols, like HTTP/2, that will be changing the very core of our infrastructures. At the same time, we have old approaches made new again like micro-services...
    Code Halos - aka "digital fingerprints" - are the key organizing principle to understand a) how dumb things become smart and b) how to monetize this dynamic. In his session at @ThingsExpo, Robert Brown, AVP, Center for the Future of Work at Cognizant Technology Solutions, outlined research, analysis and recommendations from his recently published book on this phenomena on the way leading edge organizations like GE and Disney are unlocking the Internet of Things opportunity and what steps your organization should be taking to position itself for the next platform of digital competition.
    The 3rd International Internet of @ThingsExpo, co-located with the 16th International Cloud Expo - to be held June 9-11, 2015, at the Javits Center in New York City, NY - announces that its Call for Papers is now open. The Internet of Things (IoT) is the biggest idea since the creation of the Worldwide Web more than 20 years ago.
    As the Internet of Things unfolds, mobile and wearable devices are blurring the line between physical and digital, integrating ever more closely with our interests, our routines, our daily lives. Contextual computing and smart, sensor-equipped spaces bring the potential to walk through a world that recognizes us and responds accordingly. We become continuous transmitters and receivers of data. In his session at @ThingsExpo, Andrew Bolwell, Director of Innovation for HP's Printing and Personal Systems Group, discussed how key attributes of mobile technology – touch input, sensors, social, and ...