Transactional & Referential Data

One thing is certain: your data is not in the state you want it, nor is your reporting. That is the number one reason you want to integrate into one system. You are tired of the disparate set of data going, coming and not connecting. The integrity of your reporting has uncovered errors totaling into the millions.

The “busts” are there. You know they will come again. There is no panacea for data, it will always be an issue.

I continue to be amazed, though, at the state of data going into projects and how important it is to the success of any project.

The key to data is very simple. Tell me the reporting you need and the granularity, and you will know what data you need. Data, for clarity, that may need to be separated into 3 component parts in the new system. And, plenty of cases where data is just missing to meet the reporting needs.

If you can’t deliver, in the requirements phase, a full set of reports with every data column provided and defined, you are going to be ‘burning dollars’. A few will disagree with this concept. They will say it is contradictory, and that reporting is a design-phase project step. I hear ya. But all of us in consulting know reporting horror. Our mission isn’t to maintain the status quo; it is to change the game.

The two things that need to happen before a project can be successful, are that you need to know which reports you require, and then the data you’ll need to serve those reports, in that order.

Skip this step and you will find yourself spending millions more on your project than necessary. Spun another way, start with the end in mind, and the beginning will be much clearer.

Referential Data

This is the first data set that you must ensure is ready to go and load first. This data is simply the data that supports the transactions, trades, or assets you are going to load, or capture and process. Therefore, this data is essentially all of the drop-down choices one can insert on a trade, and many more data attributes that allows a transaction to process front-, middle- and back-office.

This is a key area in most projects that is not done well. Why?

It doesn’t seem important to you, and most consultants want nothing to do with it. Everyone wants to escape this because this is the area that, if not done properly, timely and well, will send your project off the rails. Most never realize this is the underlying reason projects get delayed.

What everyone wants to work on is the supposed ‘big stuff’. It is much safer to work on design documents, fiddle around, meet with clients, pass drafts back and forth, get coffee, and then hand the design specs to someone else to insert or develop for the project.

That is how many escape connecting the dots on a project. I have seen industry veterans not know their own system. Put double digit years on their resume and clients think they know everything about the system. Again, it is great to suggest expertise, but doing the basics well still has to be a criterion, along with ability to solve more complex project problems. It is a challenge, I realize, to find someone with those capabilities, but it will cost millions if we are not discerning.

Someone has to know how to connect the dots and do the basics well. Bring me a consultant who can speak book structures, balance sheets and price indexes to physical optimization, and you have someone who knows the basics are more important to get done right, and get done right first.

Throw in those great big designs and their related extensions, and the dots are not connected. I have seen projects go completely backwards, where high-level consultants are staring, deflecting onto others as if they were the client themselves. Where is my bingo card?

Basic data is the issue. We will address how and why we will never make this mistake on your project, in the chapters on project planning and project execution. Wisdom will prevail.

Transactional Data

For clarity, the contracts with each counterparty is referential data at this stage of a project. When I say transactional, these are the trades which include the transportation trades as well.

The same concept applies here. Start with small sets of trade types, insert them, run them and check for errors before just loading all kinds of trades hoping it will work out. I have 99% confidence the first pass on any data migration is not going to end well. When that data migration is being done by someone new to the industry, it will end even worse.

Here is one simple question to determine if someone knows trading: ask them what is the difference between a trade type and a trade