Welcome!

Microsoft Cloud Authors: Pat Romanski, Jnan Dash, Andreas Grabner, Lori MacVittie, Jim Kaskade

Related Topics: Microsoft Cloud

Microsoft Cloud: Article

Tablet-Enable Your Windows Application Without Changing the Code

A look at the Context Tagging Tool

Today, millions of Windows applications exist around the world. They are written with various development tools, with different development languages, and are based on different libraries. Let's suppose that you are responsible for one of these - and then along comes the Tablet PC. Your users say that your application would be great if it were adapted for the Tablet platform. You then do some research and find out there is a nice Tablet API, including managed support and controls that capture Ink. You tell your manager about the possibilities, but he clamps down. He points out that there is no budget and no time for these modifications this year. He might even question the business value of the Tablet platform. Your users are whining and IT management is not budging. How can you please the users without the significant cost of a code rewrite? Are there options beyond rewriting your application?

Levels of Ink-ness

Let's step back for a moment and clear up a misconception about supporting the Tablet PC platform. There are various levels of Ink-ness, i.e., the richness or degree of integration of Ink via the Tablet API and provided controls. (Warning: Ink-ness is a word that I made up and not an official Microsoft term.) There have been various conversations about Ink integration on the Internet and in books, but my viewpoint is that it typically comes down to the lifetime of Ink in an application. It is obvious that the most pen-enabled Tablet PC applications will have Ink controls, store Ink somewhere, and keep Ink as Ink. But can your existing application be considered pen-enabled even without knowing about Ink?

Thanks to the Tablet PC Input Panel (otherwise known as the TIP), your application will appear to interact with Ink even though it knows nothing about Ink. When the focus of the application is in a text input control for example, the TIP will appear, allow the user to write, convert it to text, and then insert into the text input control. So, the Ink-ignorant application will survive. Although most people will not consider this as being Ink-enabled, this is my first level of Ink-ness which I titled "Do Nothing," and basically means that we will depend on the TIP to recognize the user's handwriting as best as it can (see Table 1). Even though the TIP has an advanced recognition algorithm that is more than capable, the Ink-ness of this type of application is not impressive.

Most developers would then consider the next level of Ink-ness to be working with Tablet PC API and using the included Ink-enabled Windows controls. While this is broken down into my third and fourth levels, it is not the next level of Ink-ness in my definition. The second level is the place between doing nothing and working with the Tablet API. It is a place that most technical managers do not realize exists but it is perfect for the scenario presented in the beginning of this article. It is a way to help the TIP recognize your handwriting by giving it context hints in a declarative manner. Moreover, it does not require rewriting any code or doing redeployment. Want to Tablet-enable your application with minimal effort? Read on.

Context

The TIP is a graphical control that interfaces to a high-quality recognition engine (called a recognizer in Tablet PC terms). The recognizer does an unbelievable feat by interpreting handwriting and converting it to the appropriate text. However, because human handwriting is varied, in a few cases it does not understand what the user is trying to write. But, the good news is that the accuracy can be greatly improved by informing the TIP of the "context" of the input. For example, the developer could tell the TIP that a particular field is a number, a date, a time, a zip code, or even a URL. The Tablet platform has a predefined list of these hints, called Common Input Scopes, which number around 50. It also supports custom context definitions, which we will discuss later.

One might not realize that Windows XP Tablet PC Edition 2005 came with several applications that have context awareness for certain controls. The most demonstrated example is that of Internet Explorer's address bar. Another example is in Outlook 2003's "To:" textbox control. And more are on the way, from Microsoft and from other vendors as well.

There are three ways to communicate the context of a control to the TIP. If you're doing WIN32 programming, check out the SetInputScope API. If you're writing in .NET, then look at the InkEdit control that exposes a RecognizerContext property. Notice that these mechanisms require that the communication exists in the application code. The third way to communicate context is via a special file called a context manifest file. This non-programmatic option is a special XML file created with the Context Tagging Tool.

The Context Tagging Tool

The Context Tagging Tool provides an interesting way of setting context information for individual input controls in an application. Provided in the Tablet SDK v1.7 (download from the Tablet PC Developer Center), this Windows application is very easy to use. After selecting an application to "tag" (see Figure 1), a person simply picks each input control, assigns a context to it, and saves the selections. This also shows that you can see what applications on a machine include context manifest files.

One neat feature is that the tool fires up the selected application so that you can select the controls and then closes the application upon saving the selections. Another tool benefit is in the way that controls are selected from the running application. I simply drag an icon from the tool onto the interested control in my application and the tool gathers the required information (See Figure 2).

It is important to note that not all controls are tag-able. The selected control has to be uniquely identified. This uniqueness is created by the control's AccessbileName property, window class name, and a consistent Window ID. This also implies that the identification could be version specific. For most environments, these issues are not a problem but there are exceptions.

The output of saving the selections is a context manifest, an XML file that has a "CTM" extension (i.e., ExeName.EXE.CTM). Created in the application directory, the data is read at run time by the TIP and communicated to the recognition engine. (One can view the XML of the file, but it is much easier to use the tool than to edit the file.) Upon successful testing, the file then can be copied to another Tablet PC running the same application and the user will enjoy an improved Tablet PC experience.

Another interesting point is that you can create a new context manifest file or edit an existing manifest with the Context Tagging Tool. The tool informs the user of an existing manifest for the application (notice the "Context File" column in Figure 1). Furthermore, you can view and edit the manifest created by someone else. Also, as long as I have the executable, I can even create a manifest for someone else's application.

An interesting story here is that of Thong Nguyen, a software engineer in New Zealand who has written some free Tablet PC utilities (see the Resource section). Thong has been evaluating a browser called Maxthon that does not support the Tablet PC. Since he wanted the browser to work better on his Tablet PC, Thong created his own context manifest and distributed it to the community. The lesson here is that any Windows application that includes someone else's is a candidate for tagging. There are many stories like this going around the Internet.

Input Scopes

As mentioned earlier, there are approximately 50 predefined context hints known as Common Input Scopes. These cover a wide variety of input contexts and are listed in the Tagging Tool Help. Figure 3 shows a control that has multiple Common Input Scopes selected. If the list is not enough, the Tablet PC team has provided Custom Input Scopes via Phrase Lists and Regular Expressions.

Phrase Lists are a way to limit or extend the list of words in a specific context. For example, maybe I have a baseball scoring application that has a textbox control for entering the batting outcome. Using the Manage Phrase Lists tab (see Figure 4), I could build a named list of possible traditional outcomes like K, 1B, 2B, 3B, HR, BB, E1, etc. Then later, when picking a control under the Tag Controls tab, I could assign that named list as a context for the selected control. Optionally, the Coerce to Input Scope definition checkbox will limit the recognition possibilities to the assigned Phrase List.

Using the Manage Regular Expressions tab, I am able to create named contexts using a Regular Expression syntax. This feature would allow for customized entries that can optionally include the re-use of Common Input Scopes (see image 5). The Tagging Tool Help has an informative section on this topic that includes samples. There are a few extra rules on the syntax, so be sure to refer to the related Help topic.

Deployment

After creating the context manifest, the next step is to get this to your clients. The final destination is in the application runtime folder, but the challenge is how to communicate this to your clients, distribute the file, and have it land in the appropriate folder. Therefore, you could create an installation program, simply promote this new feature via e-mail, including attaching the CTM, promote and distribute this from a known Web site, and then let the user copy the file, or use your favorite enterprise file distribution mechanism such as SMS. The main issue here is to get the file to your users and into the application directory.

Tagging Resources

Obviously, this is an underutilized topic and you don't find a lot of information resources explaining how to do this. However, along with the Tagging Tool Help, there are a few key places to check out in order to get the best results with tagging. I recommend surfing over to the Tablet PC Development Center and checking out Frank Gocinski's Tablet 101 column, which includes an article titled "Adding Context Awareness to Improve Handwriting Recognition." Furthermore, be sure to check out Leszynski Group's Web site. Because they are one of the most active Tablet PC partners, these guys have been around the block in tagging engagements and thus their "Tagging Tips & Tricks" article is a must read. (See the Resource section below for the URLs.)

Conclusion

As I write this, there are not a lot of applications that take advantage of context manifests. That story is changing in 2005. For example, in December of 2004, Microsoft released an Office 2003 Tablet PC Update, titled Improved Ink Recognition update. Even though it came in an install package, the only files added to the desktop are six context manifest files - specifically for Excel, InfoPath, OneNote, Outlook, PowerPoint, and Word. This is a perfect example of when to use the Context Tagging Tool. Obviously, Microsoft wanted to have a light impact on existing applications that have a large deployment base. These files also provide examples of Custom Input Scopes for us to learn from.

So, if you're looking to have your Windows application Tablet-enabled without modifying the code or dealing with a redeployment of binaries, the Context Tagging Tool is the way to go. You wouldn't end up with a rich application like one of the Tablet PC Power Toys (which includes my favorite Crossword Puzzle application), but you would improve the Tablet experience for your users with only a small effort. Then you can always decide later to implement a more significant modification. Increasing your Ink-ness is always cool!

Resources

  • Tablet PC Developer Center: http://msdn.microsoft.com/tabletpc
  • Adding Context Awareness to Improve Handwriting Recognition, by Frank Gocinski: Tablet 101 Column, Tablet PC Developer Center
  • Tagging Tips & Tricks, Leszynski Group's Tablet Articles site: www.leszynski.com/tabletpc/tpcArticles.htm
  • Thong Nguyen's Web site: www.veridicus.com/tummy/
  • Maxthon Browser: www.maxthon.com
  • More Stories By Jon Box

    Jon Box is an Architect Evangelist in Developer & Platform Evangelism with the Microsoft Corporation. He coauthored Building Solutions with the Microsoft .NET Compact Framework, published by Addison-Wesley, and blogs at http://blogs.msdn.com/jonbox/default.aspx.

    Comments (0)

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    @ThingsExpo Stories
    Technology vendors and analysts are eager to paint a rosy picture of how wonderful IoT is and why your deployment will be great with the use of their products and services. While it is easy to showcase successful IoT solutions, identifying IoT systems that missed the mark or failed can often provide more in the way of key lessons learned. In his session at @ThingsExpo, Peter Vanderminden, Principal Industry Analyst for IoT & Digital Supply Chain to Flatiron Strategies, will focus on how IoT depl...
    Big Data, cloud, analytics, contextual information, wearable tech, sensors, mobility, and WebRTC: together, these advances have created a perfect storm of technologies that are disrupting and transforming classic communications models and ecosystems. In his session at @ThingsExpo, Erik Perotti, Senior Manager of New Ventures on Plantronics’ Innovation team, provided an overview of this technological shift, including associated business and consumer communications impacts, and opportunities it m...
    Manufacturers are embracing the Industrial Internet the same way consumers are leveraging Fitbits – to improve overall health and wellness. Both can provide consistent measurement, visibility, and suggest performance improvements customized to help reach goals. Fitbit users can view real-time data and make adjustments to increase their activity. In his session at @ThingsExpo, Mark Bernardo Professional Services Leader, Americas, at GE Digital, discussed how leveraging the Industrial Internet and...
    There will be new vendors providing applications, middleware, and connected devices to support the thriving IoT ecosystem. This essentially means that electronic device manufacturers will also be in the software business. Many will be new to building embedded software or robust software. This creates an increased importance on software quality, particularly within the Industrial Internet of Things where business-critical applications are becoming dependent on products controlled by software. Qua...
    Fact is, enterprises have significant legacy voice infrastructure that’s costly to replace with pure IP solutions. How can we bring this analog infrastructure into our shiny new cloud applications? There are proven methods to bind both legacy voice applications and traditional PSTN audio into cloud-based applications and services at a carrier scale. Some of the most successful implementations leverage WebRTC, WebSockets, SIP and other open source technologies. In his session at @ThingsExpo, Da...
    "Tintri was started in 2008 with the express purpose of building a storage appliance that is ideal for virtualized environments. We support a lot of different hypervisor platforms from VMware to OpenStack to Hyper-V," explained Dan Florea, Director of Product Management at Tintri, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
    A critical component of any IoT project is what to do with all the data being generated. This data needs to be captured, processed, structured, and stored in a way to facilitate different kinds of queries. Traditional data warehouse and analytical systems are mature technologies that can be used to handle certain kinds of queries, but they are not always well suited to many problems, particularly when there is a need for real-time insights.
    In his General Session at 16th Cloud Expo, David Shacochis, host of The Hybrid IT Files podcast and Vice President at CenturyLink, investigated three key trends of the “gigabit economy" though the story of a Fortune 500 communications company in transformation. Narrating how multi-modal hybrid IT, service automation, and agile delivery all intersect, he will cover the role of storytelling and empathy in achieving strategic alignment between the enterprise and its information technology.
    IoT is at the core or many Digital Transformation initiatives with the goal of re-inventing a company's business model. We all agree that collecting relevant IoT data will result in massive amounts of data needing to be stored. However, with the rapid development of IoT devices and ongoing business model transformation, we are not able to predict the volume and growth of IoT data. And with the lack of IoT history, traditional methods of IT and infrastructure planning based on the past do not app...
    WebRTC is bringing significant change to the communications landscape that will bridge the worlds of web and telephony, making the Internet the new standard for communications. Cloud9 took the road less traveled and used WebRTC to create a downloadable enterprise-grade communications platform that is changing the communication dynamic in the financial sector. In his session at @ThingsExpo, Leo Papadopoulos, CTO of Cloud9, discussed the importance of WebRTC and how it enables companies to focus o...
    The Internet of Things can drive efficiency for airlines and airports. In their session at @ThingsExpo, Shyam Varan Nath, Principal Architect with GE, and Sudip Majumder, senior director of development at Oracle, discussed the technical details of the connected airline baggage and related social media solutions. These IoT applications will enhance travelers' journey experience and drive efficiency for the airlines and the airports.
    With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo 2016 in New York. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be! Internet of @ThingsExpo, taking place June 6-8, 2017, at the Javits Center in New York City, New York, is co-located with 20th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry p...
    "LinearHub provides smart video conferencing, which is the Roundee service, and we archive all the video conferences and we also provide the transcript," stated Sunghyuk Kim, CEO of LinearHub, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
    Things are changing so quickly in IoT that it would take a wizard to predict which ecosystem will gain the most traction. In order for IoT to reach its potential, smart devices must be able to work together. Today, there are a slew of interoperability standards being promoted by big names to make this happen: HomeKit, Brillo and Alljoyn. In his session at @ThingsExpo, Adam Justice, vice president and general manager of Grid Connect, will review what happens when smart devices don’t work togethe...
    "There's a growing demand from users for things to be faster. When you think about all the transactions or interactions users will have with your product and everything that is between those transactions and interactions - what drives us at Catchpoint Systems is the idea to measure that and to analyze it," explained Leo Vasiliou, Director of Web Performance Engineering at Catchpoint Systems, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York Ci...
    The 20th International Cloud Expo has announced that its Call for Papers is open. Cloud Expo, to be held June 6-8, 2017, at the Javits Center in New York City, brings together Cloud Computing, Big Data, Internet of Things, DevOps, Containers, Microservices and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportunity. Submit your speaking proposal ...
    Discover top technologies and tools all under one roof at April 24–28, 2017, at the Westin San Diego in San Diego, CA. Explore the Mobile Dev + Test and IoT Dev + Test Expo and enjoy all of these unique opportunities: The latest solutions, technologies, and tools in mobile or IoT software development and testing. Meet one-on-one with representatives from some of today's most innovative organizations
    20th Cloud Expo, taking place June 6-8, 2017, at the Javits Center in New York City, NY, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy.
    WebRTC is the future of browser-to-browser communications, and continues to make inroads into the traditional, difficult, plug-in web communications world. The 6th WebRTC Summit continues our tradition of delivering the latest and greatest presentations within the world of WebRTC. Topics include voice calling, video chat, P2P file sharing, and use cases that have already leveraged the power and convenience of WebRTC.
    SYS-CON Events announced today that Super Micro Computer, Inc., a global leader in Embedded and IoT solutions, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 7-9, 2017, at the Javits Center in New York City, NY. Supermicro (NASDAQ: SMCI), the leading innovator in high-performance, high-efficiency server technology, is a premier provider of advanced server Building Block Solutions® for Data Center, Cloud Computing, Enterprise IT, Hadoop/Big Data, HPC and E...