Welcome!

Microsoft Cloud Authors: David H Deans, Yeshim Deniz, Janakiram MSV, Andreas Grabner, Stackify Blog

Related Topics: Microsoft Cloud

Microsoft Cloud: Article

Tablet-Enable Your Windows Application Without Changing the Code

A look at the Context Tagging Tool

Today, millions of Windows applications exist around the world. They are written with various development tools, with different development languages, and are based on different libraries. Let's suppose that you are responsible for one of these - and then along comes the Tablet PC. Your users say that your application would be great if it were adapted for the Tablet platform. You then do some research and find out there is a nice Tablet API, including managed support and controls that capture Ink. You tell your manager about the possibilities, but he clamps down. He points out that there is no budget and no time for these modifications this year. He might even question the business value of the Tablet platform. Your users are whining and IT management is not budging. How can you please the users without the significant cost of a code rewrite? Are there options beyond rewriting your application?

Levels of Ink-ness

Let's step back for a moment and clear up a misconception about supporting the Tablet PC platform. There are various levels of Ink-ness, i.e., the richness or degree of integration of Ink via the Tablet API and provided controls. (Warning: Ink-ness is a word that I made up and not an official Microsoft term.) There have been various conversations about Ink integration on the Internet and in books, but my viewpoint is that it typically comes down to the lifetime of Ink in an application. It is obvious that the most pen-enabled Tablet PC applications will have Ink controls, store Ink somewhere, and keep Ink as Ink. But can your existing application be considered pen-enabled even without knowing about Ink?

Thanks to the Tablet PC Input Panel (otherwise known as the TIP), your application will appear to interact with Ink even though it knows nothing about Ink. When the focus of the application is in a text input control for example, the TIP will appear, allow the user to write, convert it to text, and then insert into the text input control. So, the Ink-ignorant application will survive. Although most people will not consider this as being Ink-enabled, this is my first level of Ink-ness which I titled "Do Nothing," and basically means that we will depend on the TIP to recognize the user's handwriting as best as it can (see Table 1). Even though the TIP has an advanced recognition algorithm that is more than capable, the Ink-ness of this type of application is not impressive.

Most developers would then consider the next level of Ink-ness to be working with Tablet PC API and using the included Ink-enabled Windows controls. While this is broken down into my third and fourth levels, it is not the next level of Ink-ness in my definition. The second level is the place between doing nothing and working with the Tablet API. It is a place that most technical managers do not realize exists but it is perfect for the scenario presented in the beginning of this article. It is a way to help the TIP recognize your handwriting by giving it context hints in a declarative manner. Moreover, it does not require rewriting any code or doing redeployment. Want to Tablet-enable your application with minimal effort? Read on.

Context

The TIP is a graphical control that interfaces to a high-quality recognition engine (called a recognizer in Tablet PC terms). The recognizer does an unbelievable feat by interpreting handwriting and converting it to the appropriate text. However, because human handwriting is varied, in a few cases it does not understand what the user is trying to write. But, the good news is that the accuracy can be greatly improved by informing the TIP of the "context" of the input. For example, the developer could tell the TIP that a particular field is a number, a date, a time, a zip code, or even a URL. The Tablet platform has a predefined list of these hints, called Common Input Scopes, which number around 50. It also supports custom context definitions, which we will discuss later.

One might not realize that Windows XP Tablet PC Edition 2005 came with several applications that have context awareness for certain controls. The most demonstrated example is that of Internet Explorer's address bar. Another example is in Outlook 2003's "To:" textbox control. And more are on the way, from Microsoft and from other vendors as well.

There are three ways to communicate the context of a control to the TIP. If you're doing WIN32 programming, check out the SetInputScope API. If you're writing in .NET, then look at the InkEdit control that exposes a RecognizerContext property. Notice that these mechanisms require that the communication exists in the application code. The third way to communicate context is via a special file called a context manifest file. This non-programmatic option is a special XML file created with the Context Tagging Tool.

The Context Tagging Tool

The Context Tagging Tool provides an interesting way of setting context information for individual input controls in an application. Provided in the Tablet SDK v1.7 (download from the Tablet PC Developer Center), this Windows application is very easy to use. After selecting an application to "tag" (see Figure 1), a person simply picks each input control, assigns a context to it, and saves the selections. This also shows that you can see what applications on a machine include context manifest files.

One neat feature is that the tool fires up the selected application so that you can select the controls and then closes the application upon saving the selections. Another tool benefit is in the way that controls are selected from the running application. I simply drag an icon from the tool onto the interested control in my application and the tool gathers the required information (See Figure 2).

It is important to note that not all controls are tag-able. The selected control has to be uniquely identified. This uniqueness is created by the control's AccessbileName property, window class name, and a consistent Window ID. This also implies that the identification could be version specific. For most environments, these issues are not a problem but there are exceptions.

The output of saving the selections is a context manifest, an XML file that has a "CTM" extension (i.e., ExeName.EXE.CTM). Created in the application directory, the data is read at run time by the TIP and communicated to the recognition engine. (One can view the XML of the file, but it is much easier to use the tool than to edit the file.) Upon successful testing, the file then can be copied to another Tablet PC running the same application and the user will enjoy an improved Tablet PC experience.

Another interesting point is that you can create a new context manifest file or edit an existing manifest with the Context Tagging Tool. The tool informs the user of an existing manifest for the application (notice the "Context File" column in Figure 1). Furthermore, you can view and edit the manifest created by someone else. Also, as long as I have the executable, I can even create a manifest for someone else's application.

An interesting story here is that of Thong Nguyen, a software engineer in New Zealand who has written some free Tablet PC utilities (see the Resource section). Thong has been evaluating a browser called Maxthon that does not support the Tablet PC. Since he wanted the browser to work better on his Tablet PC, Thong created his own context manifest and distributed it to the community. The lesson here is that any Windows application that includes someone else's is a candidate for tagging. There are many stories like this going around the Internet.

Input Scopes

As mentioned earlier, there are approximately 50 predefined context hints known as Common Input Scopes. These cover a wide variety of input contexts and are listed in the Tagging Tool Help. Figure 3 shows a control that has multiple Common Input Scopes selected. If the list is not enough, the Tablet PC team has provided Custom Input Scopes via Phrase Lists and Regular Expressions.

Phrase Lists are a way to limit or extend the list of words in a specific context. For example, maybe I have a baseball scoring application that has a textbox control for entering the batting outcome. Using the Manage Phrase Lists tab (see Figure 4), I could build a named list of possible traditional outcomes like K, 1B, 2B, 3B, HR, BB, E1, etc. Then later, when picking a control under the Tag Controls tab, I could assign that named list as a context for the selected control. Optionally, the Coerce to Input Scope definition checkbox will limit the recognition possibilities to the assigned Phrase List.

Using the Manage Regular Expressions tab, I am able to create named contexts using a Regular Expression syntax. This feature would allow for customized entries that can optionally include the re-use of Common Input Scopes (see image 5). The Tagging Tool Help has an informative section on this topic that includes samples. There are a few extra rules on the syntax, so be sure to refer to the related Help topic.

Deployment

After creating the context manifest, the next step is to get this to your clients. The final destination is in the application runtime folder, but the challenge is how to communicate this to your clients, distribute the file, and have it land in the appropriate folder. Therefore, you could create an installation program, simply promote this new feature via e-mail, including attaching the CTM, promote and distribute this from a known Web site, and then let the user copy the file, or use your favorite enterprise file distribution mechanism such as SMS. The main issue here is to get the file to your users and into the application directory.

Tagging Resources

Obviously, this is an underutilized topic and you don't find a lot of information resources explaining how to do this. However, along with the Tagging Tool Help, there are a few key places to check out in order to get the best results with tagging. I recommend surfing over to the Tablet PC Development Center and checking out Frank Gocinski's Tablet 101 column, which includes an article titled "Adding Context Awareness to Improve Handwriting Recognition." Furthermore, be sure to check out Leszynski Group's Web site. Because they are one of the most active Tablet PC partners, these guys have been around the block in tagging engagements and thus their "Tagging Tips & Tricks" article is a must read. (See the Resource section below for the URLs.)

Conclusion

As I write this, there are not a lot of applications that take advantage of context manifests. That story is changing in 2005. For example, in December of 2004, Microsoft released an Office 2003 Tablet PC Update, titled Improved Ink Recognition update. Even though it came in an install package, the only files added to the desktop are six context manifest files - specifically for Excel, InfoPath, OneNote, Outlook, PowerPoint, and Word. This is a perfect example of when to use the Context Tagging Tool. Obviously, Microsoft wanted to have a light impact on existing applications that have a large deployment base. These files also provide examples of Custom Input Scopes for us to learn from.

So, if you're looking to have your Windows application Tablet-enabled without modifying the code or dealing with a redeployment of binaries, the Context Tagging Tool is the way to go. You wouldn't end up with a rich application like one of the Tablet PC Power Toys (which includes my favorite Crossword Puzzle application), but you would improve the Tablet experience for your users with only a small effort. Then you can always decide later to implement a more significant modification. Increasing your Ink-ness is always cool!

Resources

  • Tablet PC Developer Center: http://msdn.microsoft.com/tabletpc
  • Adding Context Awareness to Improve Handwriting Recognition, by Frank Gocinski: Tablet 101 Column, Tablet PC Developer Center
  • Tagging Tips & Tricks, Leszynski Group's Tablet Articles site: www.leszynski.com/tabletpc/tpcArticles.htm
  • Thong Nguyen's Web site: www.veridicus.com/tummy/
  • Maxthon Browser: www.maxthon.com
  • More Stories By Jon Box

    Jon Box is an Architect Evangelist in Developer & Platform Evangelism with the Microsoft Corporation. He coauthored Building Solutions with the Microsoft .NET Compact Framework, published by Addison-Wesley, and blogs at http://blogs.msdn.com/jonbox/default.aspx.

    Comments (0)

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    @ThingsExpo Stories
    From 2013, NTT Communications has been providing cPaaS service, SkyWay. Its customer’s expectations for leveraging WebRTC technology are not only typical real-time communication use cases such as Web conference, remote education, but also IoT use cases such as remote camera monitoring, smart-glass, and robotic. Because of this, NTT Communications has numerous IoT business use-cases that its customers are developing on top of PaaS. WebRTC will lead IoT businesses to be more innovative and address...
    In his session at @ThingsExpo, Sudarshan Krishnamurthi, a Senior Manager, Business Strategy, at Cisco Systems, discussed how IT and operational technology (OT) work together, as opposed to being in separate siloes as once was traditional. Attendees learned how to fully leverage the power of IoT in their organization by bringing the two sides together and bridging the communication gap. He also looked at what good leadership must entail in order to accomplish this, and how IT managers can be the ...
    SYS-CON Events announced today that Secure Channels, a cybersecurity firm, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Secure Channels, Inc. offers several products and solutions to its many clients, helping them protect critical data from being compromised and access to computer networks from the unauthorized. The company develops comprehensive data encryption security strategie...
    "DX encompasses the continuing technology revolution, and is addressing society's most important issues throughout the entire $78 trillion 21st-century global economy," said Roger Strukhoff, Conference Chair. "DX World Expo has organized these issues along 10 tracks with more than 150 of the world's top speakers coming to Istanbul to help change the world."
    Recently, WebRTC has a lot of eyes from market. The use cases of WebRTC are expanding - video chat, online education, online health care etc. Not only for human-to-human communication, but also IoT use cases such as machine to human use cases can be seen recently. One of the typical use-case is remote camera monitoring. With WebRTC, people can have interoperability and flexibility for deploying monitoring service. However, the benefit of WebRTC for IoT is not only its convenience and interopera...
    When shopping for a new data processing platform for IoT solutions, many development teams want to be able to test-drive options before making a choice. Yet when evaluating an IoT solution, it’s simply not feasible to do so at scale with physical devices. Building a sensor simulator is the next best choice; however, generating a realistic simulation at very high TPS with ease of configurability is a formidable challenge. When dealing with multiple application or transport protocols, you would be...
    What sort of WebRTC based applications can we expect to see over the next year and beyond? One way to predict development trends is to see what sorts of applications startups are building. In his session at @ThingsExpo, Arin Sime, founder of WebRTC.ventures, discussed the current and likely future trends in WebRTC application development based on real requests for custom applications from real customers, as well as other public sources of information.
    SYS-CON Events announced today that App2Cloud will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct. 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. App2Cloud is an online Platform, specializing in migrating legacy applications to any Cloud Providers (AWS, Azure, Google Cloud).
    SYS-CON Events announced today that Calligo has been named “Bronze Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Calligo is an innovative cloud service provider offering mid-sized companies the highest levels of data privacy. Calligo offers unparalleled application performance guarantees, commercial flexibility and a personalized support service from its globally located cloud platform...
    IoT is at the core or many Digital Transformation initiatives with the goal of re-inventing a company's business model. We all agree that collecting relevant IoT data will result in massive amounts of data needing to be stored. However, with the rapid development of IoT devices and ongoing business model transformation, we are not able to predict the volume and growth of IoT data. And with the lack of IoT history, traditional methods of IT and infrastructure planning based on the past do not app...
    To get the most out of their data, successful companies are not focusing on queries and data lakes, they are actively integrating analytics into their operations with a data-first application development approach. Real-time adjustments to improve revenues, reduce costs, or mitigate risk rely on applications that minimize latency on a variety of data sources. Jack Norris reviews best practices to show how companies develop, deploy, and dynamically update these applications and how this data-first...
    Intelligent Automation is now one of the key business imperatives for CIOs and CISOs impacting all areas of business today. In his session at 21st Cloud Expo, Brian Boeggeman, VP Alliances & Partnerships at Ayehu, will talk about how business value is created and delivered through intelligent automation to today’s enterprises. The open ecosystem platform approach toward Intelligent Automation that Ayehu delivers to the market is core to enabling the creation of the self-driving enterprise.
    Internet-of-Things discussions can end up either going down the consumer gadget rabbit hole or focused on the sort of data logging that industrial manufacturers have been doing forever. However, in fact, companies today are already using IoT data both to optimize their operational technology and to improve the experience of customer interactions in novel ways. In his session at @ThingsExpo, Gordon Haff, Red Hat Technology Evangelist, shared examples from a wide range of industries – including en...
    Consumers increasingly expect their electronic "things" to be connected to smart phones, tablets and the Internet. When that thing happens to be a medical device, the risks and benefits of connectivity must be carefully weighed. Once the decision is made that connecting the device is beneficial, medical device manufacturers must design their products to maintain patient safety and prevent compromised personal health information in the face of cybersecurity threats. In his session at @ThingsExpo...
    "The Striim platform is a full end-to-end streaming integration and analytics platform that is middleware that covers a lot of different use cases," explained Steve Wilkes, Founder and CTO at Striim, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
    "We're a cybersecurity firm that specializes in engineering security solutions both at the software and hardware level. Security cannot be an after-the-fact afterthought, which is what it's become," stated Richard Blech, Chief Executive Officer at Secure Channels, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
    SYS-CON Events announced today that IBM has been named “Diamond Sponsor” of SYS-CON's 21st Cloud Expo, which will take place on October 31 through November 2nd 2017 at the Santa Clara Convention Center in Santa Clara, California.
    SYS-CON Events announced today that Massive Networks will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Massive Networks mission is simple. To help your business operate seamlessly with fast, reliable, and secure internet and network solutions. Improve your customer's experience with outstanding connections to your cloud.
    Everything run by electricity will eventually be connected to the Internet. Get ahead of the Internet of Things revolution and join Akvelon expert and IoT industry leader, Sergey Grebnov, in his session at @ThingsExpo, for an educational dive into the world of managing your home, workplace and all the devices they contain with the power of machine-based AI and intelligent Bot services for a completely streamlined experience.
    With tough new regulations coming to Europe on data privacy in May 2018, Calligo will explain why in reality the effect is global and transforms how you consider critical data. EU GDPR fundamentally rewrites the rules for cloud, Big Data and IoT. In his session at 21st Cloud Expo, Adam Ryan, Vice President and General Manager EMEA at Calligo, will examine the regulations and provide insight on how it affects technology, challenges the established rules and will usher in new levels of diligence a...