Welcome!

Microsoft Cloud Authors: Andreas Grabner, Stackify Blog, Liz McMillan, David H Deans, Automic Blog

Related Topics: Microsoft Cloud

Microsoft Cloud: Article

Tablet-Enable Your Windows Application Without Changing the Code

A look at the Context Tagging Tool

Today, millions of Windows applications exist around the world. They are written with various development tools, with different development languages, and are based on different libraries. Let's suppose that you are responsible for one of these - and then along comes the Tablet PC. Your users say that your application would be great if it were adapted for the Tablet platform. You then do some research and find out there is a nice Tablet API, including managed support and controls that capture Ink. You tell your manager about the possibilities, but he clamps down. He points out that there is no budget and no time for these modifications this year. He might even question the business value of the Tablet platform. Your users are whining and IT management is not budging. How can you please the users without the significant cost of a code rewrite? Are there options beyond rewriting your application?

Levels of Ink-ness

Let's step back for a moment and clear up a misconception about supporting the Tablet PC platform. There are various levels of Ink-ness, i.e., the richness or degree of integration of Ink via the Tablet API and provided controls. (Warning: Ink-ness is a word that I made up and not an official Microsoft term.) There have been various conversations about Ink integration on the Internet and in books, but my viewpoint is that it typically comes down to the lifetime of Ink in an application. It is obvious that the most pen-enabled Tablet PC applications will have Ink controls, store Ink somewhere, and keep Ink as Ink. But can your existing application be considered pen-enabled even without knowing about Ink?

Thanks to the Tablet PC Input Panel (otherwise known as the TIP), your application will appear to interact with Ink even though it knows nothing about Ink. When the focus of the application is in a text input control for example, the TIP will appear, allow the user to write, convert it to text, and then insert into the text input control. So, the Ink-ignorant application will survive. Although most people will not consider this as being Ink-enabled, this is my first level of Ink-ness which I titled "Do Nothing," and basically means that we will depend on the TIP to recognize the user's handwriting as best as it can (see Table 1). Even though the TIP has an advanced recognition algorithm that is more than capable, the Ink-ness of this type of application is not impressive.

Most developers would then consider the next level of Ink-ness to be working with Tablet PC API and using the included Ink-enabled Windows controls. While this is broken down into my third and fourth levels, it is not the next level of Ink-ness in my definition. The second level is the place between doing nothing and working with the Tablet API. It is a place that most technical managers do not realize exists but it is perfect for the scenario presented in the beginning of this article. It is a way to help the TIP recognize your handwriting by giving it context hints in a declarative manner. Moreover, it does not require rewriting any code or doing redeployment. Want to Tablet-enable your application with minimal effort? Read on.

Context

The TIP is a graphical control that interfaces to a high-quality recognition engine (called a recognizer in Tablet PC terms). The recognizer does an unbelievable feat by interpreting handwriting and converting it to the appropriate text. However, because human handwriting is varied, in a few cases it does not understand what the user is trying to write. But, the good news is that the accuracy can be greatly improved by informing the TIP of the "context" of the input. For example, the developer could tell the TIP that a particular field is a number, a date, a time, a zip code, or even a URL. The Tablet platform has a predefined list of these hints, called Common Input Scopes, which number around 50. It also supports custom context definitions, which we will discuss later.

One might not realize that Windows XP Tablet PC Edition 2005 came with several applications that have context awareness for certain controls. The most demonstrated example is that of Internet Explorer's address bar. Another example is in Outlook 2003's "To:" textbox control. And more are on the way, from Microsoft and from other vendors as well.

There are three ways to communicate the context of a control to the TIP. If you're doing WIN32 programming, check out the SetInputScope API. If you're writing in .NET, then look at the InkEdit control that exposes a RecognizerContext property. Notice that these mechanisms require that the communication exists in the application code. The third way to communicate context is via a special file called a context manifest file. This non-programmatic option is a special XML file created with the Context Tagging Tool.

The Context Tagging Tool

The Context Tagging Tool provides an interesting way of setting context information for individual input controls in an application. Provided in the Tablet SDK v1.7 (download from the Tablet PC Developer Center), this Windows application is very easy to use. After selecting an application to "tag" (see Figure 1), a person simply picks each input control, assigns a context to it, and saves the selections. This also shows that you can see what applications on a machine include context manifest files.

One neat feature is that the tool fires up the selected application so that you can select the controls and then closes the application upon saving the selections. Another tool benefit is in the way that controls are selected from the running application. I simply drag an icon from the tool onto the interested control in my application and the tool gathers the required information (See Figure 2).

It is important to note that not all controls are tag-able. The selected control has to be uniquely identified. This uniqueness is created by the control's AccessbileName property, window class name, and a consistent Window ID. This also implies that the identification could be version specific. For most environments, these issues are not a problem but there are exceptions.

The output of saving the selections is a context manifest, an XML file that has a "CTM" extension (i.e., ExeName.EXE.CTM). Created in the application directory, the data is read at run time by the TIP and communicated to the recognition engine. (One can view the XML of the file, but it is much easier to use the tool than to edit the file.) Upon successful testing, the file then can be copied to another Tablet PC running the same application and the user will enjoy an improved Tablet PC experience.

Another interesting point is that you can create a new context manifest file or edit an existing manifest with the Context Tagging Tool. The tool informs the user of an existing manifest for the application (notice the "Context File" column in Figure 1). Furthermore, you can view and edit the manifest created by someone else. Also, as long as I have the executable, I can even create a manifest for someone else's application.

An interesting story here is that of Thong Nguyen, a software engineer in New Zealand who has written some free Tablet PC utilities (see the Resource section). Thong has been evaluating a browser called Maxthon that does not support the Tablet PC. Since he wanted the browser to work better on his Tablet PC, Thong created his own context manifest and distributed it to the community. The lesson here is that any Windows application that includes someone else's is a candidate for tagging. There are many stories like this going around the Internet.

Input Scopes

As mentioned earlier, there are approximately 50 predefined context hints known as Common Input Scopes. These cover a wide variety of input contexts and are listed in the Tagging Tool Help. Figure 3 shows a control that has multiple Common Input Scopes selected. If the list is not enough, the Tablet PC team has provided Custom Input Scopes via Phrase Lists and Regular Expressions.

Phrase Lists are a way to limit or extend the list of words in a specific context. For example, maybe I have a baseball scoring application that has a textbox control for entering the batting outcome. Using the Manage Phrase Lists tab (see Figure 4), I could build a named list of possible traditional outcomes like K, 1B, 2B, 3B, HR, BB, E1, etc. Then later, when picking a control under the Tag Controls tab, I could assign that named list as a context for the selected control. Optionally, the Coerce to Input Scope definition checkbox will limit the recognition possibilities to the assigned Phrase List.

Using the Manage Regular Expressions tab, I am able to create named contexts using a Regular Expression syntax. This feature would allow for customized entries that can optionally include the re-use of Common Input Scopes (see image 5). The Tagging Tool Help has an informative section on this topic that includes samples. There are a few extra rules on the syntax, so be sure to refer to the related Help topic.

Deployment

After creating the context manifest, the next step is to get this to your clients. The final destination is in the application runtime folder, but the challenge is how to communicate this to your clients, distribute the file, and have it land in the appropriate folder. Therefore, you could create an installation program, simply promote this new feature via e-mail, including attaching the CTM, promote and distribute this from a known Web site, and then let the user copy the file, or use your favorite enterprise file distribution mechanism such as SMS. The main issue here is to get the file to your users and into the application directory.

Tagging Resources

Obviously, this is an underutilized topic and you don't find a lot of information resources explaining how to do this. However, along with the Tagging Tool Help, there are a few key places to check out in order to get the best results with tagging. I recommend surfing over to the Tablet PC Development Center and checking out Frank Gocinski's Tablet 101 column, which includes an article titled "Adding Context Awareness to Improve Handwriting Recognition." Furthermore, be sure to check out Leszynski Group's Web site. Because they are one of the most active Tablet PC partners, these guys have been around the block in tagging engagements and thus their "Tagging Tips & Tricks" article is a must read. (See the Resource section below for the URLs.)

Conclusion

As I write this, there are not a lot of applications that take advantage of context manifests. That story is changing in 2005. For example, in December of 2004, Microsoft released an Office 2003 Tablet PC Update, titled Improved Ink Recognition update. Even though it came in an install package, the only files added to the desktop are six context manifest files - specifically for Excel, InfoPath, OneNote, Outlook, PowerPoint, and Word. This is a perfect example of when to use the Context Tagging Tool. Obviously, Microsoft wanted to have a light impact on existing applications that have a large deployment base. These files also provide examples of Custom Input Scopes for us to learn from.

So, if you're looking to have your Windows application Tablet-enabled without modifying the code or dealing with a redeployment of binaries, the Context Tagging Tool is the way to go. You wouldn't end up with a rich application like one of the Tablet PC Power Toys (which includes my favorite Crossword Puzzle application), but you would improve the Tablet experience for your users with only a small effort. Then you can always decide later to implement a more significant modification. Increasing your Ink-ness is always cool!

Resources

  • Tablet PC Developer Center: http://msdn.microsoft.com/tabletpc
  • Adding Context Awareness to Improve Handwriting Recognition, by Frank Gocinski: Tablet 101 Column, Tablet PC Developer Center
  • Tagging Tips & Tricks, Leszynski Group's Tablet Articles site: www.leszynski.com/tabletpc/tpcArticles.htm
  • Thong Nguyen's Web site: www.veridicus.com/tummy/
  • Maxthon Browser: www.maxthon.com
  • More Stories By Jon Box

    Jon Box is an Architect Evangelist in Developer & Platform Evangelism with the Microsoft Corporation. He coauthored Building Solutions with the Microsoft .NET Compact Framework, published by Addison-Wesley, and blogs at http://blogs.msdn.com/jonbox/default.aspx.

    Comments (0)

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    @ThingsExpo Stories
    SYS-CON Events announced today that Cloud Academy named "Bronze Sponsor" of 21st International Cloud Expo which will take place October 31 - November 2, 2017 at the Santa Clara Convention Center in Santa Clara, CA. Cloud Academy is the industry’s most innovative, vendor-neutral cloud technology training platform. Cloud Academy provides continuous learning solutions for individuals and enterprise teams for Amazon Web Services, Microsoft Azure, Google Cloud Platform, and the most popular cloud com...
    SYS-CON Events announced today that CA Technologies has been named "Platinum Sponsor" of SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CA Technologies helps customers succeed in a future where every business - from apparel to energy - is being rewritten by software. From planning to development to management to security, CA creates software that fuels transformation for companies in the applic...
    SYS-CON Events announced today that TMC has been named “Media Sponsor” of SYS-CON's 21st International Cloud Expo and Big Data at Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Global buyers rely on TMC’s content-driven marketplaces to make purchase decisions and navigate markets. Learn how we can help you reach your marketing goals.
    Internet of @ThingsExpo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devic...
    "MobiDev is a Ukraine-based software development company. We do mobile development, and we're specialists in that. But we do full stack software development for entrepreneurs, for emerging companies, and for enterprise ventures," explained Alan Winters, U.S. Head of Business Development at MobiDev, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
    SYS-CON Events announced today that IBM has been named “Diamond Sponsor” of SYS-CON's 21st Cloud Expo, which will take place on October 31 through November 2nd 2017 at the Santa Clara Convention Center in Santa Clara, California.
    SYS-CON Events announced today that Telecom Reseller has been named “Media Sponsor” of SYS-CON's 21st International Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Telecom Reseller reports on Unified Communications, UCaaS, BPaaS for enterprise and SMBs. They report extensively on both customer premises based solutions such as IP-PBX as well as cloud based and hosted platforms.
    We build IoT infrastructure products - when you have to integrate different devices, different systems and cloud you have to build an application to do that but we eliminate the need to build an application. Our products can integrate any device, any system, any cloud regardless of protocol," explained Peter Jung, Chief Product Officer at Pulzze Systems, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA
    Multiple data types are pouring into IoT deployments. Data is coming in small packages as well as enormous files and data streams of many sizes. Widespread use of mobile devices adds to the total. In this power panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists looked at the tools and environments that are being put to use in IoT deployments, as well as the team skills a modern enterprise IT shop needs to keep things running, get a handle on all this data, and deliver...
    The current age of digital transformation means that IT organizations must adapt their toolset to cover all digital experiences, beyond just the end users’. Today’s businesses can no longer focus solely on the digital interactions they manage with employees or customers; they must now contend with non-traditional factors. Whether it's the power of brand to make or break a company, the need to monitor across all locations 24/7, or the ability to proactively resolve issues, companies must adapt to...
    SYS-CON Events announced today that Enzu will exhibit at SYS-CON's 21st Int\ernational Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Enzu’s mission is to be the leading provider of enterprise cloud solutions worldwide. Enzu enables online businesses to use its IT infrastructure to their competitive advantage. By offering a suite of proven hosting and management services, Enzu wants companies to focus on the core of their ...
    In his session at @ThingsExpo, Eric Lachapelle, CEO of the Professional Evaluation and Certification Board (PECB), provided an overview of various initiatives to certify the security of connected devices and future trends in ensuring public trust of IoT. Eric Lachapelle is the Chief Executive Officer of the Professional Evaluation and Certification Board (PECB), an international certification body. His role is to help companies and individuals to achieve professional, accredited and worldwide re...
    With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend 21st Cloud Expo October 31 - November 2, 2017, at the Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY, and learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
    With the introduction of IoT and Smart Living in every aspect of our lives, one question has become relevant: What are the security implications? To answer this, first we have to look and explore the security models of the technologies that IoT is founded upon. In his session at @ThingsExpo, Nevi Kaja, a Research Engineer at Ford Motor Company, discussed some of the security challenges of the IoT infrastructure and related how these aspects impact Smart Living. The material was delivered interac...
    IoT solutions exploit operational data generated by Internet-connected smart “things” for the purpose of gaining operational insight and producing “better outcomes” (for example, create new business models, eliminate unscheduled maintenance, etc.). The explosive proliferation of IoT solutions will result in an exponential growth in the volume of IoT data, precipitating significant Information Governance issues: who owns the IoT data, what are the rights/duties of IoT solutions adopters towards t...
    New competitors, disruptive technologies, and growing expectations are pushing every business to both adopt and deliver new digital services. This ‘Digital Transformation’ demands rapid delivery and continuous iteration of new competitive services via multiple channels, which in turn demands new service delivery techniques – including DevOps. In this power panel at @DevOpsSummit 20th Cloud Expo, moderated by DevOps Conference Co-Chair Andi Mann, panelists examined how DevOps helps to meet the de...
    When growing capacity and power in the data center, the architectural trade-offs between server scale-up vs. scale-out continue to be debated. Both approaches are valid: scale-out adds multiple, smaller servers running in a distributed computing model, while scale-up adds fewer, more powerful servers that are capable of running larger workloads. It’s worth noting that there are additional, unique advantages that scale-up architectures offer. One big advantage is large memory and compute capacity...
    No hype cycles or predictions of zillions of things here. IoT is big. You get it. You know your business and have great ideas for a business transformation strategy. What comes next? Time to make it happen. In his session at @ThingsExpo, Jay Mason, Associate Partner at M&S Consulting, presented a step-by-step plan to develop your technology implementation strategy. He discussed the evaluation of communication standards and IoT messaging protocols, data analytics considerations, edge-to-cloud tec...
    "When we talk about cloud without compromise what we're talking about is that when people think about 'I need the flexibility of the cloud' - it's the ability to create applications and run them in a cloud environment that's far more flexible,” explained Matthew Finnie, CTO of Interoute, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
    The Internet giants are fully embracing AI. All the services they offer to their customers are aimed at drawing a map of the world with the data they get. The AIs from these companies are used to build disruptive approaches that cannot be used by established enterprises, which are threatened by these disruptions. However, most leaders underestimate the effect this will have on their businesses. In his session at 21st Cloud Expo, Rene Buest, Director Market Research & Technology Evangelism at Ara...