Big Data is like teenage sex

There is a saying in the marketing world at the moment and it is quite topical for us as an industry, the saying is:

“Big Data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it.”

I’m not alone in these thoughts other bloggers have discussed this before and it always makes for good reading when someone outside of the business intelligence world discusses what they actually see.

If you have been in and around BI for many years you will remember the transition, the acronyms and the “BI speak” that has constantly been evolving.

Ten years ago very few people in C-level roles had even heard of business intelligence, let alone the term BI, yet we used it . Then there were the cipher codes (acronyms and jargon) we invented by abbreviating all of the terms or inventing new ones. BI we already mentioned, then there was ETL, BRD, TRD, FPM, CPM, Datamart and many others that sometimes we let slip in front of non-technical clients. Added to this there were also the slogans that we liked to impress people with, such as “one version of the truth” or “real time reporting” and even “self-service reporting”

However, Big Data has to be one right up there with the most creative and befuddling of terminologies that has been invented. Think back to the first time you heard the term and what it actually meant to you. I pictured it as the ones and zero’s being far bigger than in normal data, therefore it was a very fat bytes.

I bet many people who read this will recall as much jargon as I have.

So is it technical people who come up with these terms? I doubt that very much, it is undoubtedly us marketing people who invent “sound bytes” that distinguish what we do from the competition. Our industry is not the only one that re-invents terminology, there used to be something called a bureau service and now that is called “in-the-cloud”.

What are the most unusual terms you have heard in the IT world?

Self Service versus Self Destruction

For years we have all known that there are many different parts to the BI jigsaw puzzle. Originally it was quite clear that a finance division was the clear leader when it came to driving a solution for a company. The initial work on these projects produced a static outcome that included information like a general ledger and the accompanying reports they needed. We can even use a uniformed cube for each of these models in todays environment.

The calling for “self-service” reporting, or ad-hoc reports was not as prevalent as it is now. The reason that this did not create too much work for developers was that most accountants had a great knowledge of excel.

The real underlying issue has always been that to be truly gifted in “self-service” reporting you needed to be a “power” user. The number of true business report developers was hampered by this hurdle. However, I don’t believe that this was necessarily a bad thing from a client perspective.

The reason I say that is we have all heard the term “paralysis by analysis” and if you give self-service to everyone that is exactly what you will get.

The loss of man hours due to an employee not being able to undertake their primary work function, due to a desire to create mind numbingly beautiful reports will become horrendous. It is very difficult to see a safe balance if all this coding work is required to be undertaken inside a clients business.

Therefore, offering a client greater freedom to build reports has to be tempered by the fact that some of their staff will always be willing to spend valuable time to build a better report. Clients need to be made aware of the issues that BI products can cause to their human resources and also need to understand that it is far more efficient to allow a dedicated developer to undertake this for them.

Obviously, the argument will be the cost of an external developer to the cost of an internal resource. However, you are not only investing in development expertise, but you are also getting a person with a wide array of similar projects. Thus, an external resource will always have a greater experience base than someone who is restricted to one company. They still will have expertise, however they will miss out on the spectrum of experience that a pure consulting company undertakes.

There is also a great argument for static generic reports that give clients close to a full solution, especially ones that give a little flexibility when it comes to the final offering. So next time a client puts “self-service as their number one aim, just take the time to find out why.

Team preparation should be a Methodology

What is team preparation?

Is preparation for your team part of your strategy or is it a tactic that needs to be undertaken everytime you are about to start a project?

I believe it is all about team collaboration which on a macro scale definitely falls under a strategy. So what do I mean by team preparation and collaboration?

Everytime you undertake a project you gain not only valuable experience but methodologies for undertaking other projects. For example, if you undertake a Revenue Model for a finance area, the basic structure is the same for every other similar model at other companies.


Preparation is not isolated from Project Management

Thus, if you decide on a generic model for doing the projects you can already utilise standard cubes for every Revenue project, standard processes and objects. In fact the only thing that realistically needs to change is the naming convention for the new cubes and dimensions. To ensure that all of your team benefits from this experience you need to ensure there is some sort of centralised area where you can keep these universal cubes and obviously the documentation that assists them to use these tools.


Your Toolbox

If you are a person who is tasked with implementing a particular project, then taking a fully equipped toolbox can certainly reduce time and increase the chance of customer satisfaction. So what do I mean by toolbox?

1. Almost all projects that are undertaken in BI usually end up as a line in the General Ledger somewhere, so always keep in mind that other projects need to dovetail into yours. Thus, things like naming conventions can be critical, so developing a generalised naming convention manual will enable you to quickly and successfully standardise names across a business. Remember that this manual is important to be shown to a client upfront and left on site as part of the documentation, here is an example.

2. The iterative approach really means that you are looking to the client as a member of your team. If that is so, then it is very essential to deploy each iteration as soon as it is complete. There are a number of good processes to aid this practice, such as standardising server infrastructure on each project. Ensure you have three types of servers; Production, Development and Staging. The staging server is used for UAT and thus won’t interfere with your next iteration, as with an Agile approach UAT does not necessarily need to be at the end of the programme. With the three server structure you have three “parallel streams” that can be independent and unreliant on other tasks that are being undertaken.

3. Developing some inhouse software solutions that enable you to deploy iteratives fast and efficiently can be of great value. These are usually developed by people with great experience and enable you to deploy each step as it is complete. With this method, it will also help you to decide which parts of each project are going to be undertaken in each step. Don’t be apprehensive about researching online to see if someone else has already developed the tools you need.

4. Make sure your toolbox includes as much uniform documentation as you possibly can. For example standard BRD’s, UAT Questionnaires and Technical Requirement Documents will save you hours of work for the BA’s and Developers when piecing the project together. Also, many of the projects you undertake will be of a similar nature so having pre-developed cubes for say GL projects (such as Capex, Look-up cubes and Revenue are good examples) and grouping them under different industries will be a godsend. They will help you do speedy and efficient Proof of Concepts and will save you mountains of time with your implementation. If you can find a way to enable  these models to be stored as an object and develop your own or buy tools that allow you to modify them (ie quickly change names), it will ensure you are way ahead of the curve.

The last thing is to ensure you have other tools to enable you to make your solution perfect. Tools that enable you to set-up email notifications, tools like Winmerge for comparing and detecting any changes that have been undertaken and also maybe something like SVN for version control.



As you can see a well equipped Toolbox will greatly assist your development team as they will always benefit from all the other projects and experience that the whole company have previously undertaken.