Turning Data into Decisions

*This article first appeared on 29 December 2016 in The Business Times.

We’ve arrived at the digital age where the difference between man and machine is often debated. While it’s true that our brains can pull information together automatically, we are unable to process ever-increasing volumes of data fast enough. It’s ridiculous that ‘gut instinct’ is still the way some decision-makers make final calls. Key players are faced with make or break decisions everyday, and Gartner’s statistics (also data) have revealed that one in four companies are experiencing a data crisis. It’s clear that relying on knowledge and experience gained on the job is not enough in today’s competitive, digitised world. Frankly, there’s just too much going on for anyone to know everything anymore so there’s no excuse for not being data driven. But being data driven can be challenging for some, and organisations are still learning how to manage and act on data. In a way, the data deluge crept up on us.

From Drip to Deluge

Data used to be friendly, simply because there was so little of it available, relatively speaking.

When we were working with ‘small data,’ spreadsheets were great. You could pore through them and see everything; it was ok to update your spreadsheet once a week. But as data grew, so did spreadsheets. Information overload crept in and rows and columns of data saw no end; data is meant to be a boon, not a bane. As a solution, spreadsheets gave way to the seemingly sexier dashboard. These showed high-level graphs and charts that were great for summarising data and trends. Despite this, dashboards did not tell the whole story. Personally, I could see what was going on, but when I asked, “Why is this happening?” I found myself painfully turning back to spreadsheets to manually hunt for the answers. Information overload persisted.

The nature of business today makes spreadsheets of raw data look obtuse and unscalable. Conversely, the bird’s eye view of a dashboard misses important details. So it comes as no surprise that the challenges today are threefold:

1. Volume - in the realm of 2,500 terabytes a day for several multinationals. In a day, that fills enough 1TB hard drives to line a football field. Relying on human capabilities alone to sift through this is obviously not going to cut it.
2. Velocity - people are expecting answers and they often want it now. If analysing and reporting a social post today still takes a week, the insights become as irrelevant as yesterday’s old news.
3. Clarity - this refers to accuracy of the data and the way it’s interpreted to make decisions. As I mentioned earlier, due to progress with data platforms, there’s just no reason why companies or brands should not be tapping into this and making full use of the tools available.

To solve these challenges, a range of solutions have been developed, but before blindly jumping on the latest analytics unicorn, let’s also look at the human element in all of this. Naturally, mindsets take time to adapt to the data wells springing up. Hence, change has to happen from bottom up as much as top down. As data sources grow, organisations migrate across platforms and people move, which results in information getting fragmented. So what’s a company to do?

Building a data process

In the words of Seth Godin, ‘If we live in a world where information drives what we do, the information we get becomes the most important thing. The person who chooses that information has power.’ Who then, should be privy to all this information? Based on the company’s objectives, brands should decide on the technology needed to build a complete ‘stack’ for data. The end result is a platform or set of platforms able to handle the full capacity of the company’s data, while being accessible and understandable by everyone who needs it.

The key is to be strategic and think about managing data as a process. On one side is the raw data. On the other, the reader gets information. Organisations can view it as a relatively simple 5-step process:

1. Collation - collecting all the bits of data across the various data pipelines
2. Cleaning - screening data for aberrations that can skew the analysis, and ensuring data harmonisation
3. Governance - Ensuring information is accessible, but only to those who should see it
4. Analysis - making sense of all the data collated and which to use
5. Presentation and Delivery - turning data into decision-ready insights, and then getting it to the right people

Looking ahead, data management will mirror the evolution of today’s consumer technology. We went from typing in command lines to touchscreens and swiping, so humans could get what they needed, faster. Instant gratification is key. The world of data will still be massively complex, but just like the latest tech gizmos from Apple or Samsung, the complexity will be hidden away. When marketers ask “Why is this happening?” the report can and should show the underlying reason, without the need to backtrack and look at raw data.

There is no Pause button

The volume of data is only going to increase. Everything collects data now - our phones, our wallets and even our watches. This new consumer approach to business intelligence doesn’t mean information should not be secure, and free for all. Accessibility needs to be managed and privacy should be a key pillar of this. But too often, privacy is an excuse many corporations hide behind for their inaction. They are happy to sit on the fence and not do anything because they are either clueless about what the policy should be, or are simply afraid of breaking rules or laws. While their concerns are understandable, it’s more important to gain clarity and be focused on finding a solution that works for the business quickly. A data privacy policy in 2017 is going to be different from one in 2010. If your data privacy policy is not clear and is blocking you from moving forward, you need to put a timeline on when that will be, and sort that out.

It’s an exciting time to be a marketer, but if you want to be a good one you need to get onboard with becoming one with data. Companies are moving from storing data with manual analysis in dark corners, to building intelligent workflows that spread the insights to everyone. It’s still nascent in much of Asia, but there’s growing interest from the top and these visions are starting to get put in place. This kind of stuff doesn’t just happen like magic though; you can’t pay more to get it faster, it’s a process that will have many iterations. There’s no such thing as hitting pause and playing catch up. You need to take control of the reins, now.

Of course, being data driven and master of your insights doesn’t come for free. Companies that want to get in control need to invest properly, and treat this investment as a profit centre for the entire business, not an overhead, the way IT is treated. This isn’t just another “expense” for a database system on the cloud; it’s an investment towards becoming the company that develops the most intelligent customer engagement programs and answers business questions with the greatest clarity. It’s an investment in future-proofing your business by becoming agile, and responding to opportunity efficiently.

Knowledge is power, and if you don’t start turning your data into the right decisions, your competitors will.