Self Service versus Self Destruction

For years we have all known that there are many different parts to the BI jigsaw puzzle. Originally it was quite clear that a finance division was the clear leader when it came to driving a solution for a company. The initial work on these projects produced a static outcome that included information like a general ledger and the accompanying reports they needed. We can even use a uniformed cube for each of these models in todays environment.

The calling for “self-service” reporting, or ad-hoc reports was not as prevalent as it is now. The reason that this did not create too much work for developers was that most accountants had a great knowledge of excel.

The real underlying issue has always been that to be truly gifted in “self-service” reporting you needed to be a “power” user. The number of true business report developers was hampered by this hurdle. However, I don’t believe that this was necessarily a bad thing from a client perspective.

The reason I say that is we have all heard the term “paralysis by analysis” and if you give self-service to everyone that is exactly what you will get.

The loss of man hours due to an employee not being able to undertake their primary work function, due to a desire to create mind numbingly beautiful reports will become horrendous. It is very difficult to see a safe balance if all this coding work is required to be undertaken inside a clients business.

Therefore, offering a client greater freedom to build reports has to be tempered by the fact that some of their staff will always be willing to spend valuable time to build a better report. Clients need to be made aware of the issues that BI products can cause to their human resources and also need to understand that it is far more efficient to allow a dedicated developer to undertake this for them.

Obviously, the argument will be the cost of an external developer to the cost of an internal resource. However, you are not only investing in development expertise, but you are also getting a person with a wide array of similar projects. Thus, an external resource will always have a greater experience base than someone who is restricted to one company. They still will have expertise, however they will miss out on the spectrum of experience that a pure consulting company undertakes.

There is also a great argument for static generic reports that give clients close to a full solution, especially ones that give a little flexibility when it comes to the final offering. So next time a client puts “self-service as their number one aim, just take the time to find out why.

Techniques for Building Generic Models

What is a Generic Model?

Generic modelling has the goal of satisfying customer requirements in way that takes into account future needs and enhancements.

Often while gathering requirements, a BA will focus on the customer’s short term needs without considering wider-reaching goals, enhancements, and ongoing maintenance.

In order to use generic modelling techniques, we need to include as much information as possible in our BRDs and use it to design a system that will be more robust, adaptable, and easier to work with and use in the future.

But wait, I hear you say, customers are all different! In fact, each one demands a completely new implementation from scratch! Well, sure, customers will always tell you that their needs are unique and they do things in unconventional ways, and most of them DO have their quirks.

But at the end of the day financial applications end up as one or more lines in GL, no matter how complex the underlying calculations and logic. Isn’t a version a version? Isn’t a currency a currency? Isn’t a year a year, and a product a product?

Legislation and international business standards drive the types and techniques of financial reporting they are trying to produce, so there is bound to be commonality from one customer to the next, even if they are unaware of it.

The idea is to take this commonality and abstract it using generic modelling concepts, in order to reduce the painful redundancy of starting each development cycle from scratch.

This article focuses on the basic development techniques and disciplines you can use in TM1 design and development that will help you achieve a more generic result.

Benefits of Generic Models

Generic models are characterized by the following beneficial features.

Reusability

Ever feel like you’re writing the same calculation logic over and over again? Well, chances are, you are doing exactly that.

Let’s face it, currency conversion rules, for instance, are always eerily similar each time you do them. Likewise, rules to roll forward opening balances.

And what about cube structures? Have you ever noticed that a Loans module is similar to, say, a Project Capex module? Not exactly the same, but each is a list of repeated payments over time, with various other values calculated from that core structure.

If you build with generic principles, you can easily take your Capex module, copy it, add some measures and adapt the rules to satisfy the requirements of a Loan module. This saves hours of design and development time, and keeps the model consistent and familiar to users.

Adaptability

The more generic a model is, the easier it is to make changes.

For example, if you write generic currency rules for each cube, and even include currency conversion in cubes where the customer has said it is not necessary, it is much easier to support additional currencies if the customer requires them later.

And if you are attempting to take an Agile approach to an implementation, being able to change or update a design quickly, and have development respond, is one of the most valuable abilities you can have.

Maintainability

If a model is designed using generic techniques, it is built from the ground up with maintainability in mind. Your goal here is to allow an administrative user to adapt model behaviour without requiring advanced development skills.

Any rules or processes should remain unchanged, with only the inputs to their calculation logic — be those lookup cubes, attributes, or process parameters — requiring update. This helps the customer maintain their own application, without feeling overly reliant on the original consultants who implemented it.

Deployability

Often you’ll want to change the behaviour of a model from development to staging to production. Generic models only require a data change to update their behaviour, so are much easier to maintain in a multi-tiered deployment scenario.

Reduced Risk

When you reuse the same structures and techniques instead of making code changes to adapt a model’s behaviour, you’re inherently reducing the risk of introducing new bugs into the system. To some extent this also alleviates testing requirements, as you only need to test the reusable portion of the module once.

Downside of Generic Models

Speed/Efficiency

Writing rules or processes in a generic way will inevitably add some degree of overhead to your calculations. This is unavoidable, but can be mitigated with various strategies and intelligent design.

Readability & Handover

Often generic rules and TI processes involve more complex code, with frequent lookups and if statements that can confuse developers who might be new to the project.

Likewise, if a customer wants to hire an in-house developer to maintain the model you’ve built for them, they may have some difficulty understand what was done and why in the original design.

This is why it is important that the code is formatted well and commented with meaningful explanations.

How to Build Generic Models

When all is said and done, building models in a generic way is really just a set of disciplines. You need to discard some of your old habits and begin to think generically with every rule and TI you write!

Use lookup cubes and attributes

Hard-coding is the enemy of generic models. If you have a literal string in your rules or TI, you better be sure the thing that string represents is never going to change!

For instance, imagine you are writing a rule that copies the closing balance of the previous year to the current year. Your first thought might be to write a rule like this:

['Opening Bal', '2014'] = N: DB('This Cube', ...,
    'YTD Dec',
    '2013',
    !This Cube Measure
);

Of course this would work, but why limit it to one year? Use an attribute so you don’t have to write the rule again for next year, like so:

['Opening Bal'] = N: DB('This Cube', ...,
    'YTD Dec',
    ATTRS('Year Dim', !Year Dim, 'Previous Year'),
    !This Cube Measure
);

Looks pretty generic, now. But what if you wanted to take this model and deploy it in Australia? The fiscal year starts in July in Australia, so using ‘YTD Dec’ isn’t going to cut it. Assuming you’re using consolidations to tally up your YTD figures, you can restructure the dimension easily enough, but you need to draw your opening balance from YTD Jun.

Again, we can use a lookup to help:

['Opening Bal'] = N: DB('This Cube', ...,
    DB('Lookup Cube', !Year Dim, 'Fiscal Year End'),
    ATTRS('Year Dim', !Year Dim, 'Previous Year'),
    !This Cube Measure
);

You could take these ideas even further. If you want to explore more specific techniques for supporting flexible time dimensions, check out our previous article “Flow’s Date Lookup cube – Back to the Future!” and its sequel, “Back to the Future, Part II“.

Never depend on a particular structure in code

The problem above arises because we have assumed that our YTD values will be structured in a certain way. To some extent, you can avoid assuming a few fundamentals about your design as you code rules and TI, but you can certainly minimize them.

For instance, what if, instead of changing the structure when we want to set a different end of fiscal year month, we decided to name our month elements generically? If we simply called our months, ‘M00’, ‘M01’, ‘M02’, etc, up to ‘M12’, would we have a more generic result?

Well, yes, we could directly reference ‘M12’ in our opening balance rule, which would be attached to ‘M00’. This would mean we wouldn’t need to adjust our YTD consolidations to accommodate a different fiscal year start month.

Use aliases as display labels

However, the above solution is a little aesthetically ugly and not particularly intuitive!

To solve this, we could add aliases to each of these elements to make the display names fit in with our fiscal year period. When deploying in Australia, we could simply name our ‘M01’ element ‘Jul’ to update the behaviour, and remove the need for the costly lookups. In this way, our consolidations could remain static, and would need to be changed to deploy to a different region.

You can apply this technique to many other dimensions, where you substitute an arbitrary code for the primary element name and use aliases to determine the display value. This can make models that were appropriate for one client more transferable to another, and makes your development more agile should a customer change their naming preferences.

Just make sure you save all your subsets with the display alias turned on, and use those subsets in all your important views!

Use standard time dimensions

There has been much discussion about standard time dimensions in the TM1 community, and it seems not everyone agrees. This is just a matter of emphasis. Some place more emphasis on the browsability of cubes and views, while others seek a more flexible and maintainable approach.

Whichever time dimension structure you choose, it is of great benefit to standardize these across your entire module. Even if your users would like to vary the form in which they input data, you can standardize reporting cubes to ensure users get a consistent view.

Check out our previous article “An Item-based Approach to TM1 Model Design” for more information on standardizing reporting cubes across the entire model.

Use a common dimension order

Likewise, it is preferable to keep the order of dimensions as standard as possible. Some developers like to optimize each cube individually by tweaking the dimension order, but in the age of large amounts of available RAM, I believe it is much more preferable to have a predictable dimension order.

This cuts down development time and removes the need for developers to be constantly opening up cubes or using the DB formula generator to ensure the dimension order is correct. It also makes rules much more readable and assist handover to new developers or on-site admins.

Use parameters & global variables to define the behaviour of TI processes

TI processes are generally purpose-built, especially those generated with the wizard. But if you put in a bit of extra time to write these processes generically, you can often use them on future jobs and will save time overall.

Have a look at the our articles “Array Types in Turbo Integrator” and “Returning values from Turbo Integrator processes”  for techniques and tricks to make processes work in more generic ways.

Use dynamic (MDX) subsets and saved subsets effectively

Well-designed dynamic subsets are a great way to ensure changes in your element names or dimension structures will have a minimal impact on saves views and the reports that are based upon them.

Likewise, saved subsets can be utilized to set values such as current month and year. This way, all you have to do is update the named subset, and all the reports and views based on it will reflect the change.

Try to avoid unnamed subsets as much as possible, as these will just lead to confusion and errors down the road.

Create dynamic reports

This is probably a whole article in itself, but when you are writing Excel-based reports, it’s very easy to create them in such a way that they will break if underlying structures or names change.

Create named lists with SUBNM formulae to ensure rows and columns do not get out of sync. Use dynamic paging techniques to display small subsets of data that the user can scroll through. And finally, judicious use of active forms (yes judicious, as Active Forms can become very cumbersome very quickly if you overuse them!) can keep your reports and templates dynamic and responsive.

There are many other tricks you can use, but the main point is, think through the impact of changes to the underlying model and try to develop strategies to help your reports survive the turmoil!

Conclusion

This article has only scratched the surface of the various ways you can create more generic models, just by adopting a few habits and disciplines in your modelling work.

Sure, there is a bit of extra work up front, but the pay-off is huge, and increases exponentially over time! Each implementation, you’ll add to your bag of tricks, and this will leave you with more time to attend to the finer details and give your customer a higher degree of satisfaction.

Team preparation should be a Methodology

What is team preparation?

Is preparation for your team part of your strategy or is it a tactic that needs to be undertaken everytime you are about to start a project?

I believe it is all about team collaboration which on a macro scale definitely falls under a strategy. So what do I mean by team preparation and collaboration?

Everytime you undertake a project you gain not only valuable experience but methodologies for undertaking other projects. For example, if you undertake a Revenue Model for a finance area, the basic structure is the same for every other similar model at other companies.

 

Preparation is not isolated from Project Management

Thus, if you decide on a generic model for doing the projects you can already utilise standard cubes for every Revenue project, standard processes and objects. In fact the only thing that realistically needs to change is the naming convention for the new cubes and dimensions. To ensure that all of your team benefits from this experience you need to ensure there is some sort of centralised area where you can keep these universal cubes and obviously the documentation that assists them to use these tools.

 

Your Toolbox

If you are a person who is tasked with implementing a particular project, then taking a fully equipped toolbox can certainly reduce time and increase the chance of customer satisfaction. So what do I mean by toolbox?

1. Almost all projects that are undertaken in BI usually end up as a line in the General Ledger somewhere, so always keep in mind that other projects need to dovetail into yours. Thus, things like naming conventions can be critical, so developing a generalised naming convention manual will enable you to quickly and successfully standardise names across a business. Remember that this manual is important to be shown to a client upfront and left on site as part of the documentation, here is an example.

2. The iterative approach really means that you are looking to the client as a member of your team. If that is so, then it is very essential to deploy each iteration as soon as it is complete. There are a number of good processes to aid this practice, such as standardising server infrastructure on each project. Ensure you have three types of servers; Production, Development and Staging. The staging server is used for UAT and thus won’t interfere with your next iteration, as with an Agile approach UAT does not necessarily need to be at the end of the programme. With the three server structure you have three “parallel streams” that can be independent and unreliant on other tasks that are being undertaken.

3. Developing some inhouse software solutions that enable you to deploy iteratives fast and efficiently can be of great value. These are usually developed by people with great experience and enable you to deploy each step as it is complete. With this method, it will also help you to decide which parts of each project are going to be undertaken in each step. Don’t be apprehensive about researching online to see if someone else has already developed the tools you need.

4. Make sure your toolbox includes as much uniform documentation as you possibly can. For example standard BRD’s, UAT Questionnaires and Technical Requirement Documents will save you hours of work for the BA’s and Developers when piecing the project together. Also, many of the projects you undertake will be of a similar nature so having pre-developed cubes for say GL projects (such as Capex, Look-up cubes and Revenue are good examples) and grouping them under different industries will be a godsend. They will help you do speedy and efficient Proof of Concepts and will save you mountains of time with your implementation. If you can find a way to enable  these models to be stored as an object and develop your own or buy tools that allow you to modify them (ie quickly change names), it will ensure you are way ahead of the curve.

The last thing is to ensure you have other tools to enable you to make your solution perfect. Tools that enable you to set-up email notifications, tools like Winmerge for comparing and detecting any changes that have been undertaken and also maybe something like SVN for version control.

 

Conclusion

As you can see a well equipped Toolbox will greatly assist your development team as they will always benefit from all the other projects and experience that the whole company have previously undertaken.