Welcome!

Microsoft Cloud Authors: Pat Romanski, Srinivasan Sundara Rajan, Glenn Rossman, Janakiram MSV, Steven Mandel

Related Topics: Microsoft Cloud

Microsoft Cloud: Article

Tablet-Enable Your Windows Application Without Changing the Code

A look at the Context Tagging Tool

Today, millions of Windows applications exist around the world. They are written with various development tools, with different development languages, and are based on different libraries. Let's suppose that you are responsible for one of these - and then along comes the Tablet PC. Your users say that your application would be great if it were adapted for the Tablet platform. You then do some research and find out there is a nice Tablet API, including managed support and controls that capture Ink. You tell your manager about the possibilities, but he clamps down. He points out that there is no budget and no time for these modifications this year. He might even question the business value of the Tablet platform. Your users are whining and IT management is not budging. How can you please the users without the significant cost of a code rewrite? Are there options beyond rewriting your application?

Levels of Ink-ness

Let's step back for a moment and clear up a misconception about supporting the Tablet PC platform. There are various levels of Ink-ness, i.e., the richness or degree of integration of Ink via the Tablet API and provided controls. (Warning: Ink-ness is a word that I made up and not an official Microsoft term.) There have been various conversations about Ink integration on the Internet and in books, but my viewpoint is that it typically comes down to the lifetime of Ink in an application. It is obvious that the most pen-enabled Tablet PC applications will have Ink controls, store Ink somewhere, and keep Ink as Ink. But can your existing application be considered pen-enabled even without knowing about Ink?

Thanks to the Tablet PC Input Panel (otherwise known as the TIP), your application will appear to interact with Ink even though it knows nothing about Ink. When the focus of the application is in a text input control for example, the TIP will appear, allow the user to write, convert it to text, and then insert into the text input control. So, the Ink-ignorant application will survive. Although most people will not consider this as being Ink-enabled, this is my first level of Ink-ness which I titled "Do Nothing," and basically means that we will depend on the TIP to recognize the user's handwriting as best as it can (see Table 1). Even though the TIP has an advanced recognition algorithm that is more than capable, the Ink-ness of this type of application is not impressive.

Most developers would then consider the next level of Ink-ness to be working with Tablet PC API and using the included Ink-enabled Windows controls. While this is broken down into my third and fourth levels, it is not the next level of Ink-ness in my definition. The second level is the place between doing nothing and working with the Tablet API. It is a place that most technical managers do not realize exists but it is perfect for the scenario presented in the beginning of this article. It is a way to help the TIP recognize your handwriting by giving it context hints in a declarative manner. Moreover, it does not require rewriting any code or doing redeployment. Want to Tablet-enable your application with minimal effort? Read on.

Context

The TIP is a graphical control that interfaces to a high-quality recognition engine (called a recognizer in Tablet PC terms). The recognizer does an unbelievable feat by interpreting handwriting and converting it to the appropriate text. However, because human handwriting is varied, in a few cases it does not understand what the user is trying to write. But, the good news is that the accuracy can be greatly improved by informing the TIP of the "context" of the input. For example, the developer could tell the TIP that a particular field is a number, a date, a time, a zip code, or even a URL. The Tablet platform has a predefined list of these hints, called Common Input Scopes, which number around 50. It also supports custom context definitions, which we will discuss later.

One might not realize that Windows XP Tablet PC Edition 2005 came with several applications that have context awareness for certain controls. The most demonstrated example is that of Internet Explorer's address bar. Another example is in Outlook 2003's "To:" textbox control. And more are on the way, from Microsoft and from other vendors as well.

There are three ways to communicate the context of a control to the TIP. If you're doing WIN32 programming, check out the SetInputScope API. If you're writing in .NET, then look at the InkEdit control that exposes a RecognizerContext property. Notice that these mechanisms require that the communication exists in the application code. The third way to communicate context is via a special file called a context manifest file. This non-programmatic option is a special XML file created with the Context Tagging Tool.

The Context Tagging Tool

The Context Tagging Tool provides an interesting way of setting context information for individual input controls in an application. Provided in the Tablet SDK v1.7 (download from the Tablet PC Developer Center), this Windows application is very easy to use. After selecting an application to "tag" (see Figure 1), a person simply picks each input control, assigns a context to it, and saves the selections. This also shows that you can see what applications on a machine include context manifest files.

One neat feature is that the tool fires up the selected application so that you can select the controls and then closes the application upon saving the selections. Another tool benefit is in the way that controls are selected from the running application. I simply drag an icon from the tool onto the interested control in my application and the tool gathers the required information (See Figure 2).

It is important to note that not all controls are tag-able. The selected control has to be uniquely identified. This uniqueness is created by the control's AccessbileName property, window class name, and a consistent Window ID. This also implies that the identification could be version specific. For most environments, these issues are not a problem but there are exceptions.

The output of saving the selections is a context manifest, an XML file that has a "CTM" extension (i.e., ExeName.EXE.CTM). Created in the application directory, the data is read at run time by the TIP and communicated to the recognition engine. (One can view the XML of the file, but it is much easier to use the tool than to edit the file.) Upon successful testing, the file then can be copied to another Tablet PC running the same application and the user will enjoy an improved Tablet PC experience.

Another interesting point is that you can create a new context manifest file or edit an existing manifest with the Context Tagging Tool. The tool informs the user of an existing manifest for the application (notice the "Context File" column in Figure 1). Furthermore, you can view and edit the manifest created by someone else. Also, as long as I have the executable, I can even create a manifest for someone else's application.

An interesting story here is that of Thong Nguyen, a software engineer in New Zealand who has written some free Tablet PC utilities (see the Resource section). Thong has been evaluating a browser called Maxthon that does not support the Tablet PC. Since he wanted the browser to work better on his Tablet PC, Thong created his own context manifest and distributed it to the community. The lesson here is that any Windows application that includes someone else's is a candidate for tagging. There are many stories like this going around the Internet.

Input Scopes

As mentioned earlier, there are approximately 50 predefined context hints known as Common Input Scopes. These cover a wide variety of input contexts and are listed in the Tagging Tool Help. Figure 3 shows a control that has multiple Common Input Scopes selected. If the list is not enough, the Tablet PC team has provided Custom Input Scopes via Phrase Lists and Regular Expressions.

Phrase Lists are a way to limit or extend the list of words in a specific context. For example, maybe I have a baseball scoring application that has a textbox control for entering the batting outcome. Using the Manage Phrase Lists tab (see Figure 4), I could build a named list of possible traditional outcomes like K, 1B, 2B, 3B, HR, BB, E1, etc. Then later, when picking a control under the Tag Controls tab, I could assign that named list as a context for the selected control. Optionally, the Coerce to Input Scope definition checkbox will limit the recognition possibilities to the assigned Phrase List.

Using the Manage Regular Expressions tab, I am able to create named contexts using a Regular Expression syntax. This feature would allow for customized entries that can optionally include the re-use of Common Input Scopes (see image 5). The Tagging Tool Help has an informative section on this topic that includes samples. There are a few extra rules on the syntax, so be sure to refer to the related Help topic.

Deployment

After creating the context manifest, the next step is to get this to your clients. The final destination is in the application runtime folder, but the challenge is how to communicate this to your clients, distribute the file, and have it land in the appropriate folder. Therefore, you could create an installation program, simply promote this new feature via e-mail, including attaching the CTM, promote and distribute this from a known Web site, and then let the user copy the file, or use your favorite enterprise file distribution mechanism such as SMS. The main issue here is to get the file to your users and into the application directory.

Tagging Resources

Obviously, this is an underutilized topic and you don't find a lot of information resources explaining how to do this. However, along with the Tagging Tool Help, there are a few key places to check out in order to get the best results with tagging. I recommend surfing over to the Tablet PC Development Center and checking out Frank Gocinski's Tablet 101 column, which includes an article titled "Adding Context Awareness to Improve Handwriting Recognition." Furthermore, be sure to check out Leszynski Group's Web site. Because they are one of the most active Tablet PC partners, these guys have been around the block in tagging engagements and thus their "Tagging Tips & Tricks" article is a must read. (See the Resource section below for the URLs.)

Conclusion

As I write this, there are not a lot of applications that take advantage of context manifests. That story is changing in 2005. For example, in December of 2004, Microsoft released an Office 2003 Tablet PC Update, titled Improved Ink Recognition update. Even though it came in an install package, the only files added to the desktop are six context manifest files - specifically for Excel, InfoPath, OneNote, Outlook, PowerPoint, and Word. This is a perfect example of when to use the Context Tagging Tool. Obviously, Microsoft wanted to have a light impact on existing applications that have a large deployment base. These files also provide examples of Custom Input Scopes for us to learn from.

So, if you're looking to have your Windows application Tablet-enabled without modifying the code or dealing with a redeployment of binaries, the Context Tagging Tool is the way to go. You wouldn't end up with a rich application like one of the Tablet PC Power Toys (which includes my favorite Crossword Puzzle application), but you would improve the Tablet experience for your users with only a small effort. Then you can always decide later to implement a more significant modification. Increasing your Ink-ness is always cool!

Resources

  • Tablet PC Developer Center: http://msdn.microsoft.com/tabletpc
  • Adding Context Awareness to Improve Handwriting Recognition, by Frank Gocinski: Tablet 101 Column, Tablet PC Developer Center
  • Tagging Tips & Tricks, Leszynski Group's Tablet Articles site: www.leszynski.com/tabletpc/tpcArticles.htm
  • Thong Nguyen's Web site: www.veridicus.com/tummy/
  • Maxthon Browser: www.maxthon.com
  • More Stories By Jon Box

    Jon Box is an Architect Evangelist in Developer & Platform Evangelism with the Microsoft Corporation. He coauthored Building Solutions with the Microsoft .NET Compact Framework, published by Addison-Wesley, and blogs at http://blogs.msdn.com/jonbox/default.aspx.

    Comments (0)

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    @ThingsExpo Stories
    SYS-CON Events announced today that Pulzze Systems will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Pulzze Systems, Inc. provides infrastructure products for the Internet of Things to enable any connected device and system to carry out matched operations without programming. For more information, visit http://www.pulzzesystems.com.
    I'm a lonely sensor. I spend all day telling the world how I'm feeling, but none of the other sensors seem to care. I want to be connected. I want to build relationships with other sensors to be more useful for my human. I want my human to understand that when my friends next door are too hot for a while, I'll soon be flaming. And when all my friends go outside without me, I may be left behind. Don't just log my data; use the relationship graph. In his session at @ThingsExpo, Ryan Boyd, Engi...
    The Transparent Cloud-computing Consortium (abbreviation: T-Cloud Consortium) will conduct research activities into changes in the computing model as a result of collaboration between "device" and "cloud" and the creation of new value and markets through organic data processing High speed and high quality networks, and dramatic improvements in computer processing capabilities, have greatly changed the nature of applications and made the storing and processing of data on the network commonplace.
    From wearable activity trackers to fantasy e-sports, data and technology are transforming the way athletes train for the game and fans engage with their teams. In his session at @ThingsExpo, will present key data findings from leading sports organizations San Francisco 49ers, Orlando Magic NBA team. By utilizing data analytics these sports orgs have recognized new revenue streams, doubled its fan base and streamlined costs at its stadiums. John Paul is the CEO and Founder of VenueNext. Prior ...
    In his general session at 18th Cloud Expo, Lee Atchison, Principal Cloud Architect and Advocate at New Relic, discussed cloud as a ‘better data center’ and how it adds new capacity (faster) and improves application availability (redundancy). The cloud is a ‘Dynamic Tool for Dynamic Apps’ and resource allocation is an integral part of your application architecture, so use only the resources you need and allocate /de-allocate resources on the fly.
    SYS-CON Events announced today that Numerex Corp, a leading provider of managed enterprise solutions enabling the Internet of Things (IoT), will exhibit at the 19th International Cloud Expo | @ThingsExpo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Numerex Corp. (NASDAQ:NMRX) is a leading provider of managed enterprise solutions enabling the Internet of Things (IoT). The Company's solutions produce new revenue streams or create operating...
    WebRTC adoption has generated a wave of creative uses of communications and collaboration through websites, sales apps, customer care and business applications. As WebRTC has become more mainstream it has evolved to use cases beyond the original peer-to-peer case, which has led to a repeating requirement for interoperability with existing infrastructures. In his session at @ThingsExpo, Graham Holt, Executive Vice President of Daitan Group, will cover implementation examples that have enabled ea...
    IoT offers a value of almost $4 trillion to the manufacturing industry through platforms that can improve margins, optimize operations & drive high performance work teams. By using IoT technologies as a foundation, manufacturing customers are integrating worker safety with manufacturing systems, driving deep collaboration and utilizing analytics to exponentially increased per-unit margins. However, as Benoit Lheureux, the VP for Research at Gartner points out, “IoT project implementers often ...
    The Jevons Paradox suggests that when technological advances increase efficiency of a resource, it results in an overall increase in consumption. Writing on the increased use of coal as a result of technological improvements, 19th-century economist William Stanley Jevons found that these improvements led to the development of new ways to utilize coal. In his session at 19th Cloud Expo, Mark Thiele, Chief Strategy Officer for Apcera, will compare the Jevons Paradox to modern-day enterprise IT, e...
    Complete Internet of Things (IoT) embedded device security is not just about the device but involves the entire product’s identity, data and control integrity, and services traversing the cloud. A device can no longer be looked at as an island; it is a part of a system. In fact, given the cross-domain interactions enabled by IoT it could be a part of many systems. Also, depending on where the device is deployed, for example, in the office building versus a factory floor or oil field, security ha...
    SYS-CON Events announced today the Enterprise IoT Bootcamp, being held November 1-2, 2016, in conjunction with 19th Cloud Expo | @ThingsExpo at the Santa Clara Convention Center in Santa Clara, CA. Combined with real-world scenarios and use cases, the Enterprise IoT Bootcamp is not just based on presentations but with hands-on demos and detailed walkthroughs. We will introduce you to a variety of real world use cases prototyped using Arduino, Raspberry Pi, BeagleBone, Spark, and Intel Edison. Y...
    Is your aging software platform suffering from technical debt while the market changes and demands new solutions at a faster clip? It’s a bold move, but you might consider walking away from your core platform and starting fresh. ReadyTalk did exactly that. In his General Session at 19th Cloud Expo, Michael Chambliss, Head of Engineering at ReadyTalk, will discuss why and how ReadyTalk diverted from healthy revenue and over a decade of audio conferencing product development to start an innovati...
    Fifty billion connected devices and still no winning protocols standards. HTTP, WebSockets, MQTT, and CoAP seem to be leading in the IoT protocol race at the moment but many more protocols are getting introduced on a regular basis. Each protocol has its pros and cons depending on the nature of the communications. Does there really need to be only one protocol to rule them all? Of course not. In his session at @ThingsExpo, Chris Matthieu, co-founder and CTO of Octoblu, walk you through how Oct...
    SYS-CON Events announced today that Bsquare has been named “Silver Sponsor” of SYS-CON's @ThingsExpo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. For more than two decades, Bsquare has helped its customers extract business value from a broad array of physical assets by making them intelligent, connecting them, and using the data they generate to optimize business processes.
    Identity is in everything and customers are looking to their providers to ensure the security of their identities, transactions and data. With the increased reliance on cloud-based services, service providers must build security and trust into their offerings, adding value to customers and improving the user experience. Making identity, security and privacy easy for customers provides a unique advantage over the competition.
    There are several IoTs: the Industrial Internet, Consumer Wearables, Wearables and Healthcare, Supply Chains, and the movement toward Smart Grids, Cities, Regions, and Nations. There are competing communications standards every step of the way, a bewildering array of sensors and devices, and an entire world of competing data analytics platforms. To some this appears to be chaos. In this power panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, Bradley Holt, Developer Advocate a...
    If you’re responsible for an application that depends on the data or functionality of various IoT endpoints – either sensors or devices – your brand reputation depends on the security, reliability, and compliance of its many integrated parts. If your application fails to deliver the expected business results, your customers and partners won't care if that failure stems from the code you developed or from a component that you integrated. What can you do to ensure that the endpoints work as expect...
    So, you bought into the current machine learning craze and went on to collect millions/billions of records from this promising new data source. Now, what do you do with them? Too often, the abundance of data quickly turns into an abundance of problems. How do you extract that "magic essence" from your data without falling into the common pitfalls? In her session at @ThingsExpo, Natalia Ponomareva, Software Engineer at Google, provided tips on how to be successful in large scale machine learning...
    If you had a chance to enter on the ground level of the largest e-commerce market in the world – would you? China is the world’s most populated country with the second largest economy and the world’s fastest growing market. It is estimated that by 2018 the Chinese market will be reaching over $30 billion in gaming revenue alone. Admittedly for a foreign company, doing business in China can be challenging. Often changing laws, administrative regulations and the often inscrutable Chinese Interne...
    Enterprise IT has been in the era of Hybrid Cloud for some time now. But it seems most conversations about Hybrid are focused on integrating AWS, Microsoft Azure, or Google ECM into existing on-premises systems. Where is all the Private Cloud? What do technology providers need to do to make their offerings more compelling? How should enterprise IT executives and buyers define their focus, needs, and roadmap, and communicate that clearly to the providers?