Stupid Claims About Big Data

If your company has big data or is considering it, you should be aware of these false statements and why they’re wrong.

1. Big data is nothing new.

No it’s not arrived out of a vacuum, but has been part of an evolution. But we can’t deny the fact that the data volumes are exploding and our ability to analyse the data is changing rapidly.

1. Big data will change everything.

Of course it won’t. There is a lot of hype out there and many promises will be broken, but at the same time it is changing our world every day and will continue to do so. Every success a company sees will be fodder for many more down the line.

1. The only cost for big data is hardware and software.

False. You can have the best hardware and software in the world, but unless you have a team to help maintain it, query it, and analyze it, it’s going to be a big expensive brick. You’ll also have costs in the future when you need to scale up (capacity) and scale out (applications). Companies would also be wise to budget for disaster recovery, a test environment, and staff training.

1. Big data applications require little or no performance optimization.

Big data applications are designed to analyze vast amounts of data quickly, but that doesn’t mean they do it perfectly. First, you have to have good data; if your data isn’t great, that will require optimization. Second, queries can always be optimized to return even more focused results. And finally, as your data input increases, your load capacity may also have to increase as well.

1. You need to hire one data scientist.

They call data scientists “unicorns” because a single person who can fill all the roles a company needs rarely exists. Instead, most companies need to hire a team comprised of people who have a cross-section of skills that can be combined to meet the company’s needs.

1. We need to implement machine learning.

In fact, probably about 85% of what people refer to as machine learning really comes down to statistics. A good statistician may be what you need instead.

1. Every problem is a big data problem.

Absolutely not. Big data is the shiny new tool that everyone wants to whip out in every situation, but sometimes it’s not required. Sometimes the analysis is simple enough to do without big data solutions. Sometimes you need small data. It’s not a panacea.

1. I don’t have enough data for big data.

More data does not equal better data. Many companies believe that because they have “small” data, big data solutions won’t work for them, which may or may not be the case. It often depends on the problems you’re looking to solve more than how much data you have. And, you may have more data than you think you do.

1. We need real-time data.

More granular data is not always better. The first quarter of a football match does not accurately predict the outcome. Sometimes real-time can be too close to the action, and we need to zoom out and examine the big picture. Taking that camera analogy a step further, zooming in on a digital photo is helpful up to a point, but continue zooming in and soon all you see is a blur of colors and pixels. The same is true of data; looking at data by the week might be useful whereas looking at it by the minute might not.

1. Data analysts are the “new gods of the Information Age.”

Yes and no. A good analyst can be worth his weight for a company, but I believe big data tools will start trending more towards self-service, bypassing the analyst or analytics team, and providing more tools to allow the marketer to make his or her own choices.

1. Big data has all the answers.

Yes and no. The biggest problem is that you have to ask the right questions. I’m reminded of the book, The Hitchhiker's Guide to the Galaxy, in which people build a supercomputer and ask it the meaning of life, the universe and everything. The computer spits out the answer, “42,” and when the people are baffled, the computer tells them it’s not powerful enough to calculate the actual question to which 42 is the answer. Same goes for big data; it has a lot of the answers, but you have to know which questions to ask.

1. Big data is an IT matter.

A big data program, run well, will cross borders throughout the company. It might affect sales and marketing, research and development, customer service, human resources, supply chain management, and any number of other departments. To relegate it to being an “IT issue” is to miss the point entirely.

1. Having so much data makes data flaws insignificant.

People often think that because they’re collecting so much data, individual data flaws don’t affect the overall outcome. And while it’s true that with a bigger data set, one individual flaw means less, having more data also means having more flaws. Plus, if you’re acquiring any of your data from outside sources, data quality becomes an even bigger issue than before.

1. Data warehouses aren’t needed for advanced analytics.

The argument is that advanced analytics use data sources beyond traditional warehouses, but warehouses are often needed to perform the analysis or refine data to make it useable for analysis. Data lakes won’t be replacing warehouses right away either, because they lack the maturity and breadth of features that warehouses have.

1. Big data is only for big companies.

In today’s internet-based economy, even small businesses are generating large quantities of data. Companies like SumAll are developing tools to help small and even micro businesses capitalize on and understand their data.

1. All of our competitors have already adopted big data.

Probably not. Big data adoption isn’t as high as you think. According to one report, only 29 percent of companies surveyed had implemented big data to make predictions. But you should see this as a reason to get started, not a reason to wait; big data implementation could give you a huge competitive advantage in your field.

1. Big data can overcome human bias.

Right now, machine learning isn’t anywhere near sophisticated enough to replace a human in many tasks. For now at least, analytics are a complement to human intuition and experience, not a replacement.

1. End users want flexibility, not guidance.

Loads of companies believe that the consumers of big data want more: more data, more metrics, more dimensions, more access. But the truth is that we as humans actually thrive with fewer choices. There’s a classic psychological study in which people were offered three flavors of jam to sample and buy or 12 flavors. Those offered three flavors made significantly more purchases; those offered 12 flavors got overwhelmed, couldn’t choose, and didn’t buy. End users often want to be guided to the best choices, not the most choices. 

1. No one has asked for it.

In all likelihood, people are asking for big data in your company — they’re just not using that term. If anyone in the company (or your clients) has ever wanted more reports, wanted to compare something to an industry average, or wanted more frequent access to numbers, that’s big data.