Flow Genesis Terms I – defining Models and Snapshots.

What is a Snapshot?

A snapshot is an XML file that contains information about the structure (metadata) of an OLAP model.

Although it sounds like a simple idea, snapshots can have some very powerful applications. They allow the Flow applications to read and display information about a model without being connected to the TM1 Server via the TM1 API. This means tasks can be carried out “offline”, without the risk of disrupting day-to-day TM1 Server functions.

A snapshot file is also a version. That is to say, it is the model at a specific point in time. This makes tasks such as backup, restore, archiving and version tracking possible, and much easier than they would be with a typical TM1 implementation.

Snapshots have been designed to be OLAP database agnostic, meaning they could potentially also describe an OLAP structure in a product other than TM1. However, Flow’s focus is exclusively on TM1 at present.

When exported to a file, Snapshots have the file extension “.xssd”, which stands for “XML Snapshot Document”.

What is a Model?

It’s tempting to assume the terms Model and Snapshot are interchangeable, but they actually have very specific meanings in Genesis.

A Model is simply a container for one or more Snapshots. In its most common usage, you can think of a Model like a project, with the snapshots it contains being the many versions that the project goes through during the development and production life cycle.

Sometimes you might utilize the Model concept in your own way, and that’s ok. Genesis was designed as a tool that lets you work the way you want without prescribing a rigid methodology.

However, generally, we would recommend using the Category field to collect Models into wider groups, such as customer, department or purpose.

What’s inside a Snapshot?

A snapshot contains XML markup to describe the structure of an OLAP Model. From this information, it is possible to perform various analysis of the server structures, automatically recreate the server from scratch, or update an older server with newer structures and code.

To understand snapshots better, you can open one in a standard text editor and review the markup.

You can see standard XML structures within the file, with each “node” having a simple begin and end tag, with optional attributes. Each node can contain other nodes inside it, as in the following example:

<Node att1="val1" att2="val2">
  <Node att1="val1" att2="val2"></Node>
</Node>

Once you understand the basic XML structure, it becomes possible to decipher the contents of the snapshot file. It can be a little difficult at first, because the markup is designed to be as compact as possible, but let’s review the core elements.

The root node is always Database. Hence, the most simple Snapshot possible would be the following:

<Database ID="1" NM="TM1_Server_Name"></Database>

This simply declares the database with an ID and a Name (NM).

Within a Database, a Snapshot can also include Stats, which are pieces of information about the database, and then any Cubes, Dimensions and ETL Processes it contains.

So equally, the following is also a valid representation of an empty server:

<Database ID="1" NM="TM1_Server_Name">
  <Cubes></Cubes>
  <Dims></Dims>
  <Procs></Procs>
</Database>

Each of the “Cubes”, “Dims” and “Procs” nodes can contain multiple markup descriptions of Cubes, Dimensions and ETL Processes respectively. From this simple structure, we can build up a more complete description of a TM1 server, like so:

<Database ID="1" NM="TM1_Server_Name">
  <Cubes>
    <Cube ID="1" NM="Cube1" DimIDs="1 2 3">...</Cube>
    <Cube ID="2" NM="Cube2" DimIDs="1 2 3">...</Cube>
    <Cube ID="3" NM="Cube3" DimIDs="1 2 3">...</Cube>
    ...
  </Cubes>
  <Dims>
    <Dim ID="1" NM="Dim1">...</Dim>
    <Dim ID="2" NM="Dim2">...</Dim>
    <Dim ID="3" NM="Dim3">...</Dim>
    ...
  </Dims>
  <Procs>
    <Proc ID="1" NM="Proc1">...</Proc>
    <Proc ID="2" NM="Proc2">...</Proc>
    <Proc ID="3" NM="Proc3">...</Proc>
    ...
  </Procs>
</Database>

Note that each object has an ID assigned. This is to make the markup more efficient, so when one object needs to refer to another, it can use it’s ID instead of the full name of the object.

A good example of this is where a Cube defines the Dimensions it contains. This is denoted by a list of IDs of the Dimensions, as opposed to a list of names, which could become quite verbose.

It is beyond the scope of this article to discuss the detailed structures of each object in a Snapshot file, but hopefully this gives you enough of an overview to get a basic understanding of how a Snapshot file works, and what the various terms associated with them mean.

We are currently working on a detailed whitepaper detailing the complete Snapshot format, in the hope it may form the basis of an open standard within the OLAP modelling community.

What does a Snapshot NOT contain?

Snapshot files are data-agnostic, so do not contain any data. The reason for this is both to ensure data security and separation of purpose — it makes sense that the design of a cube should be separate from the data contained within it.

There are already plenty of other formats that can handle data well, such as CSV or Tab Delimited text files. We recommend using those standards to import and export data to and from your TM1 servers.

Hidden control objects (such as TM1 “}ElementAttrbutes” cubes and dimensions) are also excluded, but are represented by more generic XML structures so that the information is preserved in a database-agnostic way.

How can I use a Snapshot?

Once you have developed a TM1 model, you can create a snapshot file using the Flow Model Packager.

Snapshot files can then be uploaded to Flow Genesis to be used as the basis for Exploring, Diagnosing, Templating and Deploying TM1 Models.

You can also save the snapshot files for later use, or archive them. If you require version control, Snapshot files are ideal for adding into a Version Control repository, such as SVN, CVS or Microsoft Visual SourceSafe.

Because Snapshot files are implemented with an open standard, you can also build software to read, write and manipulate Snapshot files.

This opens up a whole wealth of application possibilities, which we hope will lead to some innovative software from the TM1 and wider OLAP developer community.

Conclusion

Hopefully this article will give you an insight into Model and Snapshots, how they are different and how they work within the context of Flow Genesis.

Part II of this article will cover the other file types and standards in Flow Genesis.

1 Trackback