Microsoft Cloud Authors: Jim Kaskade, Lori MacVittie, Janakiram MSV, Andreas Grabner, Pat Romanski

Related Topics: Microsoft Cloud

Microsoft Cloud: Article

WinFS: The Windows Storage Platform of the Future

How to Write Applications that Take Advantage of WinFS

The next version of the Windows operating system (codenamed "Longhorn") has a new storage subsystem (codenamed "WinFS"). In this article, we will try to understand the need for "WinFS"; define "WinFS", its type system, and its data model; and learn how to write applications that take advantage of "WinFS".

Why Does Windows Need a New Storage Subsystem?
The hardware industry is confidently striding towards conquering the 3T challenge (a teraflops processor, a terabyte hard disk, and a terabits per sec bandwidth). Increase in the size of hard disk storage has been complemented by the exponential increase in the production of digital data. The amount of digital data being born daily is so phenomenal that the pen, the paper, and the typewriter have achieved the status of endangered stationery. Digital data is stored by operating systems on magnetic media - like hard disks. Windows uses file systems like NTFS and FAT32 to organize data on hard disks that are divided into logical drives. Each drive has a root folder, which can have one or more files and folders. Data is stored in files.

Data has personality. It can be documents (text, DOC, PDF, RTF, PostScript formats), images (bitmap, GIF, JPEG, PNG, WMF formats), audio (WAV, MP3, WMA, AIFF, formats), video (MOV, MPEG, AVI, ASF, QuickTime formats), and more. Along with data, these formats also store rich metadata. NTFS and FAT32 are capable of reading only basic metadata from files, like their name, size, date modified, and type (derived from the file extension). So, if I want to search all songs of a particular artist, I can't do it just by using the file system. I'll need a specialized application like Windows Media Player, which is capable of reading the rich metadata provided by all audio files.

What is "WinFS"?
"WinFS" is the Windows storage platform of the future. It solves many of the problems associated with current file systems by storing data and its metadata together. This way data can be organized, searched, shared, and related depending upon what it is and not what it's called (filename)! "WinFS" also makes it easy to deal with non-file-backed data like personal contacts, e-mail messages, and event calendar.

"WinFS" allows a user to organize data with great flexibility. Using "WinFS", a user can group data according to common characteristics, create a customized containment hierarchy to hold data, and associate one piece of data to another via relationships. This organizational flexibility provides the user with greater flexibility when searching for data. The user can search for data based on its attributes, its relationship with other data, and by its storage location.

Searching Data
Because of its ability to organize data based on various parameters, "WinFS" presents data as a directed acyclic graph (DAG) instead of as a tree (as in NTFS and FAT32). In NTFS and FAT32, since data is stored as a tree, it can be located based only on a single criteria (its directory path), while in "WinFS", since data is stored as a graph it can be searched based on multiple criteria.

Sharing Data
"WinFS" facilitates data sharing among various applications by giving file-backed data (documents, audio video, media, and so on) as well as non file-backed (e-mail, contacts, appointments, and so on), permanent residency status in the operating system and providing a unique API to access them. Now different applications can use these common pieces of data without worrying about maintaining individual data stores and the associated synchronization.

WinFS Building Blocks
Figure 1 illustrates the various building blocks of "WinFS".

Core "WinFS"
"WinFS" does not supplant NTFS; rather, it utilizes the basic file system services of NTFS. The core building block of "WinFS" includes the relational storage engine that provides file system services like ACL support, import/export, and quota management.

Data Model
"WinFS" defines a rich data model that resides on top of a relational storage engine. "WinFS" represents a piece of data (item) as a tuple in a relation. The attributes of the tuple describe the piece of data. Items can be related to each other by defining relationships between the tuples. "WinFS" also provides the ability to extend items and relationships.

"WinFS" has built-in schemas to understand the rich metadata associated with your data. Some of the built-in schemas are for common data like documents, e-mail, contacts, appointments, tasks, and more. "WinFS" also allows you to write custom schemas for your own data.

"WinFS" provides certain services like synchronization and rules. These services are layered on top of core "WinFS" and provide "WinFS" with extended functionality. For example, the synchronization service enables you to synchronize two or more "WinFS" stores.

"WinFS"provides support for multiple programming models: object-oriented, relational, and XML based. "WinFS" can also be programmed using the Win32 API.

"WinFS" Type System Basics
"WinFS" is a "strongly typed" storage system. All data stored in "WinFS" is typed; that is, it is an object of some "WinFS" type. To understand the "WinFS" type system, we first have to understand the following four concepts:

  • Items: All data is stored in WinFS  as a type specialized from Item (System.Storage.Item). Examples of types that derive from the Item are: Contact, Document, Task, and Event (all of which are found in the System.Storage.Core namespace)
  • ScalarTypes: An atomic piece of information that describes an item.
  • NestedTypes: A set of information that can be stored about an item. A nested type can have other nested types and scalar types in it.
  • Relationships: Relate one item (source) to another (target). Relationships can be of three types:
    - Holding relationship: The source controls the lifetime of the target. These relationships are many-to-many, that is, a source can "hold" multiple targets and a target can be "held" by multiple sources. If a target looses all sources, it's deleted.
    - Reference relationship: Similar to a holding relationship but without the lifetime management of the target.
    - Embedding relationship: The source embeds the target. Strictly one-to-one.

Programming "WinFS"

In this section, we'll take a brief look at the object-oriented managed API provided by "WinFS". This API allows us to search, relate, and act upon data stored by WinFS

.An installation of Longhorn will have a single instance of "WinFS" service running on it. A "WinFS" instance can maintain multiple data stores. Each data source is referenced by its UNC path syntax as shown below:


All "WinFS" instances have a default store (DefaultStore). The default store on your "Longhorn" system will be called:


To program "WinFS", you have to get hold of an ItemContext object. This is done by calling the static Open method of the ItemContext class. The method is passed the UNC path of the store you want to program. If you pass nothing, the ItemContext refers to the default store.

All the types that are required to program "WinFS" can be found in three assemblies: System.Storage. dll, System.Storage.Schema.dll, and WinCorLib.dll.

As an example, Listing 1 is code that prints out all folders in the default store.

A very useful class provided by "WinFS" is the ItemSearcher (System.Storage.ItemSearcher) class, which allows you to search for items in a particular item type. Each item type (for example, Contact, Document, Folder, Event, and so on) has a static method called GetSearcher that takes in the ItemContext object and returns the ItemSearcher. Using the ItemSearcher, you can search for the items you need.

For example, Listing 2 uses the ItemSearcher to search for all documents whose title begins with the string 'Result'.

In this brief survey of "WinFS", we have seen that the future of data is with context-based storage systems. "WinFS" will provide the basis on which advanced data-mining tools are going to be built for the personal computer.

More Stories By Mujtaba Syed

Mujtaba Syed works as a software architect with Marlabs Inc. He is an MCSD
(early achiever) and loves to speak about and write on Microsoft .NET. Mujtaba has been programming the Microsoft .NET Framework since its beta 1 release. His current interests are focused on Longhorn.

Comments (2) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

Most Recent Comments
Mujtaba Syed 06/08/04 03:11:22 PM EDT

It''s corrected now.

Mujtaba Syed 06/08/04 02:29:18 PM EDT

Somehow the online version is showing WinFX at all places it should show WinFS! We will get this rectified ASAP. Thanks.

@ThingsExpo Stories
A completely new computing platform is on the horizon. They’re called Microservers by some, ARM Servers by others, and sometimes even ARM-based Servers. No matter what you call them, Microservers will have a huge impact on the data center and on server computing in general. Although few people are familiar with Microservers today, their impact will be felt very soon. This is a new category of computing platform that is available today and is predicted to have triple-digit growth rates for some ...
DevOps is being widely accepted (if not fully adopted) as essential in enterprise IT. But as Enterprise DevOps gains maturity, expands scope, and increases velocity, the need for data-driven decisions across teams becomes more acute. DevOps teams in any modern business must wrangle the ‘digital exhaust’ from the delivery toolchain, "pervasive" and "cognitive" computing, APIs and services, mobile devices and applications, the Internet of Things, and now even blockchain. In this power panel at @...
The Internet of Things (IoT), in all its myriad manifestations, has great potential. Much of that potential comes from the evolving data management and analytic (DMA) technologies and processes that allow us to gain insight from all of the IoT data that can be generated and gathered. This potential may never be met as those data sets are tied to specific industry verticals and single markets, with no clear way to use IoT data and sensor analytics to fulfill the hype being given the IoT today.
SYS-CON Events announced today that Numerex Corp, a leading provider of managed enterprise solutions enabling the Internet of Things (IoT), will exhibit at the 19th International Cloud Expo | @ThingsExpo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Numerex Corp. (NASDAQ:NMRX) is a leading provider of managed enterprise solutions enabling the Internet of Things (IoT). The Company's solutions produce new revenue streams or create operating...
SYS-CON Events announced today that MathFreeOn will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. MathFreeOn is Software as a Service (SaaS) used in Engineering and Math education. Write scripts and solve math problems online. MathFreeOn provides online courses for beginners or amateurs who have difficulties in writing scripts. In accordance with various mathematical topics, there are more tha...
The best way to leverage your Cloud Expo presence as a sponsor and exhibitor is to plan your news announcements around our events. The press covering Cloud Expo and @ThingsExpo will have access to these releases and will amplify your news announcements. More than two dozen Cloud companies either set deals at our shows or have announced their mergers and acquisitions at Cloud Expo. Product announcements during our show provide your company with the most reach through our targeted audiences.
@ThingsExpo has been named the Top 5 Most Influential Internet of Things Brand by Onalytica in the ‘The Internet of Things Landscape 2015: Top 100 Individuals and Brands.' Onalytica analyzed Twitter conversations around the #IoT debate to uncover the most influential brands and individuals driving the conversation. Onalytica captured data from 56,224 users. The PageRank based methodology they use to extract influencers on a particular topic (tweets mentioning #InternetofThings or #IoT in this ...
The Internet of Things will challenge the status quo of how IT and development organizations operate. Or will it? Certainly the fog layer of IoT requires special insights about data ontology, security and transactional integrity. But the developmental challenges are the same: People, Process and Platform and how we integrate our thinking to solve complicated problems. In his session at 19th Cloud Expo, Craig Sproule, CEO of Metavine, will demonstrate how to move beyond today's coding paradigm ...
Although it has gained significant traction in the consumer space, IoT is still in the early stages of adoption in enterprises environments. However, many companies are working on initiatives like Industry 4.0 that includes IoT as one of the key disruptive technologies expected to reshape businesses of tomorrow. The key challenges will be availability, robustness and reliability of networks that connect devices in a business environment. Software Defined Wide Area Network (SD-WAN) is expected to...
OnProcess Technology has announced it will be a featured speaker at @ThingsExpo, taking place November 1 - 3, 2016, in Santa Clara, California. Dan Gettens, OnProcess’ Chief Analytics Officer, will discuss how Internet of Things (IoT) data can be leveraged to predict product failures, improve uptime and slash costly inventory stock. @ThingsExpo is an annual gathering of IoT and cloud developers, practitioners and thought-leaders who exchange ideas and insights on topics ranging from Big Data in...
SYS-CON Events announced today that CDS Global Cloud, an Infrastructure as a Service provider, will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. CDS Global Cloud is an IaaS (Infrastructure as a Service) provider specializing in solutions for e-commerce, internet gaming, online education and other internet applications. With a growing number of data centers and network points around the world, ...
Developing software for the Internet of Things (IoT) comes with its own set of challenges. Security, privacy, and unified standards are a few key issues. In addition, each IoT product is comprised of (at least) three separate application components: the software embedded in the device, the back-end service, and the mobile application for the end user’s controls. Each component is developed by a different team, using different technologies and practices, and deployed to a different stack/target –...
Virgil consists of an open-source encryption library, which implements Cryptographic Message Syntax (CMS) and Elliptic Curve Integrated Encryption Scheme (ECIES) (including RSA schema), a Key Management API, and a cloud-based Key Management Service (Virgil Keys). The Virgil Keys Service consists of a public key service and a private key escrow service. 

Data is the fuel that drives the machine learning algorithmic engines and ultimately provides the business value. In his session at Cloud Expo, Ed Featherston, a director and senior enterprise architect at Collaborative Consulting, will discuss the key considerations around quality, volume, timeliness, and pedigree that must be dealt with in order to properly fuel that engine.
Big Data has been changing the world. IoT fuels the further transformation recently. How are Big Data and IoT related? In his session at @BigDataExpo, Tony Shan, a renowned visionary and thought leader, will explore the interplay of Big Data and IoT. He will anatomize Big Data and IoT separately in terms of what, which, why, where, when, who, how and how much. He will then analyze the relationship between IoT and Big Data, specifically the drilldown of how the 4Vs of Big Data (Volume, Variety,...
From wearable activity trackers to fantasy e-sports, data and technology are transforming the way athletes train for the game and fans engage with their teams. In his session at @ThingsExpo, will present key data findings from leading sports organizations San Francisco 49ers, Orlando Magic NBA team. By utilizing data analytics these sports orgs have recognized new revenue streams, doubled its fan base and streamlined costs at its stadiums. John Paul is the CEO and Founder of VenueNext. Prior ...
Ask someone to architect an Internet of Things (IoT) solution and you are guaranteed to see a reference to the cloud. This would lead you to believe that IoT requires the cloud to exist. However, there are many IoT use cases where the cloud is not feasible or desirable. In his session at @ThingsExpo, Dave McCarthy, Director of Products at Bsquare Corporation, will discuss the strategies that exist to extend intelligence directly to IoT devices and sensors, freeing them from the constraints of ...
SYS-CON Events announced today that Embotics, the cloud automation company, will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Embotics is the cloud automation company for IT organizations and service providers that need to improve provisioning or enable self-service capabilities. With a relentless focus on delivering a premier user experience and unmatched customer support, Embotics is the fas...
In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, will discuss how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team a...
The Open Connectivity Foundation (OCF), sponsor of the IoTivity open source project, and AllSeen Alliance, which provides the AllJoyn® open source IoT framework, today announced that the two organizations’ boards have approved a merger under the OCF name and bylaws. This merger will advance interoperability between connected devices from both groups, enabling the full operating potential of IoT and representing a significant step towards a connected ecosystem.