Welcome!

.NET Authors: Bruce Armstrong, Pat Romanski, Liz McMillan, Yeshim Deniz, Dmitry Sotnikov

Related Topics: .NET

.NET: Article

Smart Personal Object Technology (SPOT)

Part I: An introduction to hardware, network, and system software

Product groups at Microsoft generally develop platforms or specific applications that run on these platforms. SPOT is different. Our goal is to increase the usefulness of everyday objects that we can wear, carry, or that might be scattered throughout the environment, ultimately making some activity easier and/or more enjoyable. To enable this new category of solutions (products, technologies, platforms, and services), SPOT is required to encompass a much broader scope than is typical at Microsoft.

The first product developed according to this vision is the Smart Watch with MSN Direct (see Figure 1). Some commercial watchmakers that are shipping this product include Fossil, Suunto, Tissot, as well as other watch manufacturers. Wristwatches qualify as everyday personal objects serving some useful purpose, namely, telling time. To increase its usefulness and make it smart, the core function of this object has been improved by increasing its accuracy (to within a measurable error of 1 PPM, or ~15 mSec). We have also made it aware (the key to smartness) of its local time zone - a novel trick for watches that's convenient when traveling.

In addition to the features typically found in digital watches (e.g., chronograph, timers, alarms), we taught SPOT a few new tricks by providing effortless access to timely information, such as stock market updates, weather conditions, news wire services, in-game sports scores, Microsoft Outlook Calendar integration, MSN Messenger paging, and virtually anything else similarly imaginable. In this particular embodiment of a Smart Personal Object, the specific human activity being enhanced is that of staying informed with timely, reliable, secure, "glanceable," and personally relevant information.

Achieving all of this requires, among many other things:

  • A custom chipset for processing and wireless connectivity
  • A new hardware platform, small and energy-efficient enough to be practical and acceptable for a first-generation wristwatch
  • A persisted wireless connection, protocols, and service infrastructure delivering rich, personalized, and dynamic content across North America, and eventually other geographies worldwide
  • A new software stack using Microsoft technology, meeting a myriad of (often contradictory) requirements: reliable, secure, energy-efficient, safe, small, fast, extensible, portable, patchable, easy to develop for, etc.
  • A "glanceable" user interface model to accommodate a constrained viewing surface
  • A user-friendly Web site for configuration, user assistance, marketing, and the back-end integration with MSN billing, Passport, and dozens of other teams and services, both internal and external to Microsoft
  • 24x7 datacenter and network operations with high reliability, fault tolerance, monitoring, alerting, reporting, etc.
This article discusses the details of these solutions and describes many of the requirements that led to them. The next installment in this series will conclude the software description and detail the steps we are taking to improve and bring these technologies to you.

Hardware, Network, and Trade-Offs
The Smart Watch and similarly sized and capable devices require a small hardware module. This module must include a processing unit, potentially several types of memory, communications mechanisms (peripherals, networks, sensors, displays, kinetic inputs, etc.), power management, and all of the other components typically associated with self-contained computational devices. If the application is to be battery-powered, it must operate for an "acceptable" amount of time before recharging.

To meet the specific requirements of our v1 products, specifically in the areas of power, size, cost, and processing horsepower, we determined that two custom chips would need to be developed. They are affectionately known internally as Stan and Ollie.

Stan
Stan (see Figure 2) is a small (2.8 mm x 2.8 mm x 860 um) mixed analog/digital 6 metal layer CMOS radio. It is frequency agile (i.e., can [quickly] retune to a different frequency) and can capture radio signals in short bursts (5-50 ms). Being a radio, it operates within the FM band (87.6-107.6 MHz). The pre-existing FM network is a popular means of communicating to people all over the world. Most regulatory bodies permit the transmission of digital information using a subcarrier of a given frequency. This is most commonly used for RDS, the mechanism by which car stereos display call signs, music titles, and other ephemeral information. This means that the SPOT network is actually built out by utilizing the transmitters of the commercial radio stations that you listen to every day.

We work with many radio partners throughout North America to deploy custom hardware, sometimes into extremely harsh environments, enabling us to securely and reliably deliver data from a Microsoft Service Operation Center (SOC) to the individual FM transmitters (246 of them at present) and to modulate the information "over the air." We benefit greatly from the sustained and rigorous maintenance, redundancy, and physical security these facilities provide. This new network, along with the protocols and infrastructure, is called DirectBand.

Each transmitter in a geographic region continuously streams content at ~12 Kbits/sec (~120 MB/day). Information is sent in ~160 KB frames, which take ~113 seconds to transmit. In the typical case, the entire frame time must pass before the clients (endpoints) can make use of the information contained within. This is an artifact of our time diversity and error correction strategy intended to overcome losses introduced by intermittent reception.

Despite its many advantages, we must admit that the FM subcarrier approach has a few limitations, about which we are vigilant. The most obvious limitation is one-way communication; a wristwatch physically does not transmit at the necessary energy to form a back-channel.

It turns out that this is acceptable for the majority of common information services, which tend to follow the familiar push model. News stories are trickled out as soon as they arrive (~1400/day), the latest stock quotes are continuously streamed out during market hours, etc. While this doesn't cover all cases, it is ideal for enough of them to make it valuable, much in the same way TV and radio are good for what they provide in the face of the bi-directional Internet.

To overcome the functional limitations, we have focused design and user scenarios around services that fit nicely within the one-to-many model, yet allow for personalized content and filtering by placing the burden on our smart endpoints. After all, the client knows the things in which you are interested, where you are (e.g., local city), and perhaps other details about your current situation (e.g., calendar, heart rate), and it also happens to know what is going on around you (e.g., current weather, movie times). A SPOT client can integrate these things to deliver an optimal, timely, and relevant experience.

Aside from not being able to explicitly request information, this drawback has significant bearing on communication reliability, as there is no way for the sender to determine whether or not the receiver accurately received its relevant information. To mitigate this, we have implemented:

  • Multiple levels of error correction (bit, byte, and packet)
  • Opportunistic transmission scheduling algorithms that attempt to make data fairly available to the appropriate endpoints in each geographic region, according to the specific delivery requirements of the information being transmitted
  • Time-diverse information retransmissions to account for the client being out of reception for brief and extended periods (e.g., in an elevator, day trip to a remote location)
Broadcast also means that anyone with the proper equipment can "listen" to the information stream. We have implemented several layers of encryption covering both shared information, such as news and weather, as well as personal information, such as Outlook calendar appointments and MSN Messenger text messages. The security and privacy measures have received much scrutiny and have been taken very seriously throughout the development of the entire technology suite.

The only other real remaining drawback is that FM operates on the order of 3-meter wavelengths, which means that the optimal antenna is 3 meters long. This poses a number of antenna design challenges considering the diameter of the human wrist, or typical watch bezel.

Ollie
Ollie (see Figure 3) is a slightly larger piece of silicon (6.8 mm x 6.8 mm x 860 um), containing an ARM7 TDMI-S 32-bit processor core and three DSP accelerator cores for radio data processing: ARCTAN, MAC, and Viterbi FEC decoder. It also has 384K SRAM (single cycle access), 512K ROM, and 128-bits of laser-etched unique ID. The core operates at 1.8V, while I/O operates at 3.3V. Ollie's primary responsibility is to bridge the user experience with the management of hardware.

In the Smart Watch, Ollie operates at 27.6MHz and is completely halted while Stan captures radio data (several times per minute) to reduce RF interference. This presents some interesting challenges to our software stack, as we must have smooth animations and continuous sound despite being sporadically shutdown. The first watches have 1MB of Flash memory and sophisticated access management software to account for high latency erase operations and finite part lifetimes.

Module
The SPOT team has built several reference modules (see Figure 4) and has worked over the past few years with OEMs and IHVs to design new custom modules with various requirements, including diverse memory types, displays, heart rate and barometric pressure monitoring, touch screens, robotics, sensor networks, and beyond. We have also built a comparatively powerful system development board that can be used for prototyping and experimentation and which is most closely related to the Stamp category of devices. I will go into more detail about Stamp later.

Embedded Software with .NET
The embedded software project (see Figure 5) had many goals and requirements, some of which have already been described. To understand our design choices, it is necessary to explain a few more:

  • Build a soft real-time platform suitable for custom, highly constrained ROM and FLASH devices
  • Be extremely memory-conscious and safe, able to track every single allocated bit; low-memory conditions must be dealt with well before they degrade the user experience
  • Create an extensible architecture, where the programmer does not simply use the existing functionality but can also extend and customize it
  • Design for low duty-cycles and energy-efficiency
  • Heavily safeguard against crashes and create a mechanism for OTA updates
  • Enable rapid development using modern languages and environments, for internal development, OEMs, and ISVs
At the center of our solution sits TinyCLR, a clean-room implementation of the CLR/CLS/CLI derived from the ECMA standard and designed specifically to meet the goals of SPOT. The result is a very small (~132 KB) implementation of the runtime and trimmed framework library, programmable with Visual Studio.NET (C# and VB.NET have been tested), and suitable for single processor devices and embedded solutions. TinyCLR operates as an interpreter and an ARM JITter has also been developed.

TinyCLR is built on top of a thin hardware abstraction layer (<30 KB, including common drivers), TinyHAL, which has been designed to serve primarily as the bootstrap and hardware interface for the runtime. TinyCLR and TinyHAL were developed from the start to work together in creating a small, fast, and extensible platform to meet the needs of the v1 watch product as well as support next-generation wearable and other small form factors.

Having a managed runtime this low in the stack blurs the distinction between OS and application code (thread management, memory management, I/O management, etc.). The present design eliminates the duplication of facilities. The long-term goal is to further reduce native code by having the majority of device driver code written in C#, as is the case with the Stamp product. Instead of being a traditional OS, this can more accurately be thought of as ".NET on a chip."

TinyHAL Implementation
While TinyHAL has been designed specifically for optimal execution of the TinyCLR, it has been used for various radio, DSP, and test applications and is capable of supporting a variety of other specialized applications.

Key Decisions
The TinyHAL kernel supports two execution modes: single-threaded application and interrupt service routine (ISR) - each has its own set of unique rules that must be followed. There is no scheduler per se, as there is only a single application thread. In the TinyCLR case, all user applications (the shell, network protocol, alerts, games, etc.) execute within the runtime, which has its own facilities for providing traditional multithreading. Having a single application thread also removes locking overhead. However, this means that the application running on TinyHAL must explicitly yield execution periodically (idle time) in order for continuations to be processed (cooperative multitasking).

Another key design decision was to expect low CPU utilization. This has resulted in simpler code, lower power consumption, deferred service routines (DSR) versus "do it now" versus "sleep in place" approaches, reducing clock speed while doing short spins, and other tricks.

Continuations
Continuations are the primary mechanisms for deferring work from an ISR, which must be executed very quickly, to the application thread, which typically isn't under strict time constraints. Continuations are implemented as a FIFO queue that can be scheduled within the application thread. The typical target is to execute less than 10 mSec of work with specifiable minimum contiguous duration.

ISR/DSR
ISRs are maskable and have a worst case latency of ~100 uSec. The longest ISR in system is ~1mSec.

Deferred service routines (DSR) allow processing to be performed at a specific time or when a particular event occurs. They can be processed on either the APP thread or from within an ISR. A time-deferred call has a granularity of 40uSec. An idle CPU is treated as a consumable resource.

Callbacks and Events
TinyHAL supports callbacks, which will fire from continuation processing on the application thread. An "event conditional sleep" mechanism allows the application thread to place the system in a dormant but alertable state. This allows the system to save power during idle periods.

The TinyHAL can communicate with the TinyCLR through event masks. Before entering into the interpreter loop, the TinyCLR checks for signals in any of the event masks. Should it find activity, it will dispatch into the appropriate handler code for further context-specific processing.

HAL and Drivers
A hardware abstraction layer (HAL) provides generic access to all peripherals and drivers. We currently have 3 HALs (including Windows XP, which is used for rich emulation/simulation). Drivers are common across all platforms and operate through opaque queued I/O. The design shields the driver from ISR and queue-locking details.

The existing drivers include: buffered RS232 I/O, SPI (13.8MHz) as a shared resource for peripherals, DirectBand Radio (QPSK), monochrome LCD (120x96), battery monitoring (temperature + voltage = state of charge), Flash memory (parallel), EEPROM memory (serial), OEM-specific (heart rate monitor, air pressure sensor, touch screen), calibrated accurate time, boolean outputs (backlight, vibrator), boolean inputs (buttons), and PWM outputs (Piezo, slow-start backlight, vibrator).

Drivers have been written for a number of specific parts (various EEPROMs, LCDs, battery monitors, etc.) using different buses and interconnects (SPI, HPPI, GPIO, etc.). TinyHAL also supports DMA, but the CPU must be active due to a hardware constraint.

Challenges and Complexities
Clock-halting provided a unique challenge. The application silicon is placed in the lowest power idle sleep during radio reception due to RF noise issues for 1-54mSec at 1% known load (bursts). This has caused problems with serial communications (clean halt Tx, lose Rx), timer stoppage (except DSR/RTC timer), no piezo, PWM driver, non-breakable I/O (halt continuation queue for short durations), smooth animations (requires a minimum jitter algorithm), etc.

ISRs do not have access to the system heap. The heap is non-locked and TinyHAL allocations are pinned within the managed heap. Allocations must happen within the application thread portion of the drivers, which can cause a GC iteration (10-50mSec).

Stan imposes a hard real-time constraint, but our error correction scheme allowed us to settle for soft real-time.

TinyCLR Implementation
TinyCLR implements all of the major features of the full CLR. It omits a few features that were deemed inappropriate for this class of device, and adds a few features that are specific for this class of device. This subject could fill many pages, so we will only briefly list and describe the highlights.

Supported features include:

  • Numeric types, Class types, value types, arrays (mono-dimensional only), delegates, events, references, weak references
  • Synchronization, threads, timers
  • Reflection (extremely important, it's the basis for the extensibility of the system)
  • Serialization (not true .NET serialization, but a more compact version)
  • Garbage collection and defragmentation of memory (vital for our field of application)
  • Non-incremental mark and sweep (generational was too heavyweight)
  • Self-describing data (as opposed to separate metadata)
  • Exception handling (very important for fault tolerance)
  • Time classes: DateTime, TimeSpan, using INT64 Arithmetic
  • Time-sliced thread management (20 ms)
Additionally, the following exceptions/extensions were made:
  • Execution constraints (limits call durations; prevents lock-ups)
  • Strings are represented internally as UTF-8 and exposed as Unicode
  • Value types are emulated by runtime (not first class), allowing for GC optimization and saving RAM
  • Global, shared string table for well-known values (types, methods, fields) reduces RAM/ROM, and cross-references between assemblies
  • Patching, which includes assembly differencing, method and resource substitution, and new types
  • No virtual tables for method resolution saves RAM
  • "Not so weak" references are prioritized and persisted (RAM, EEPROM, FLASH)
  • Specialized garbage collector provides "automatic" data management (stale content, etc.) and persisted storage management
  • "Weak" delegates mean that runtime manages dangling references
  • Custom serialization engine relies heavily on new attributes to guide encoding
Along the way, several design and implementation decisions have been made to not support certain features for reasons of development time, appropriateness, size, or performance:
  • Machine-dependent types (unsafe instructions) prevent direct memory access by managed code
  • SMP involves simpler runtime and is inappropriate for our goals (single processors are the norm)
  • Reduced functionality MSCORLIB is not desirable primarily due to size and appropriateness (no need for HTTP, etc.); it would entail only 416 methods, 66 types, 12 interfaces, and 42 fields - out of the 22471 methods and 1432 types in the desktop framework
  • Multi-dimensional arrays (sparse, jagged)
  • No exception handling in native code; processor faults (traps) and external watchdog only
  • Only one (implicit) Application Domain, which is compensated by support for the dynamic loading/unloading of assemblies
TinyCLR is designed in such a way that any combination of IL and native code can be supported. A new class can have its methods implemented in C#, Visual Basic, or preferred .NET language, or it can be marked as internal and implemented in native code.

Execution can be directly from ROM. A small look-up table is used for resolving cross-references between assemblies. Due to various levels of indirection in the type system, patching is built-in: methods written in IL can be converted to replacement native code or the opposite.

Another notable difference between TinyCLR and the standard CLR is that there's no binary compatibility between assemblies. This is by design for space reasons: using the PE format for the executables introduces too much overhead in the file, primarily debugging and other metadata not appropriate for our tiny v1 interpreter.

One last difference worth calling out is the extensions we've made to the garbage collector, which has proven empirically to be non-disruptive to the rest of the system. While using a mark-and-sweep approach has various disadvantages compared to Generational for this class of device, we've managed to find a nice balance between speed, complexity, overhead, and flexibility. The emulation environment used for development includes an interactive heap visualizer and extensive diagnostic reporting.

Summary
Part I of this article has described the SPOT motivation and a good bit of the first generation technologies. Next month, we'll conclude the description of the client software stack by describing the user interface shell and rendering architecture. We'll also discuss information services and what outline what we're working on to bring the Smart Personal Object technologies to you.

    More Stories By Donald Thompson

    Donald Thompson is responsible for overseeing the end-to-end design and day-to-day management of the developers, software, protocols, and technology strategy fueling the SPOT initiative. During the Internet boom, he built the centralized ad serving system used by all MSN Web properties, including Hotmail, MSNBC, and MSN.com. Prior to joining Microsoft, he developed an automated loan kiosk and decisioning system for Citibank, a cellular billing and management system for Bell South, and wrote AI and 3D graphics algorithms for commercial game companies. Before committing to a life of software, he was a professional child actor working in movies and television.

    Comments (0)

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.