Data Science Isn't Just About Predictions

Attended a data science talk today in one of the country's financial institution. Not as a presenter though, just accompanying a colleague who is.

We were basically invited there to share our experience on how Seek have been implementing data science throughout our products, and what were the learnings. The audience were internal stakeholders who are in charge of crafting policies, regulations and etc. All are mostly new to data science.

The Wrong Answer

The Q&A was quite interesting, as there were many questions being thrown at us by the audience. Even had to answer one myself, since it's something out of my colleague's domain.

"What's your go-to algorithm?" 

Me: "Well we have none to be honest. It really depends on the kinds of data you have. If you don't have much of them, then perhaps linear regression is enough, or maybe SVM. But if you have a lot, then you could probably use neural network."

They seemed to accept the answer.

After the event though, it didn't really felt that I did enough justice to the question. Because my mind was too fixated on "classification" as a problem, it led me to a response that was less than satisfactory. The right answer should've been - it depends on the problem.

The thing is, you won't "just" get classification problems at ones organization.

Sometimes you'll need to find and understand relationships between multiple entities (ie. network analysis to find influencers in community).

Sometimes you'll need a way to segment or cluster your data into something that you can act upon (ie semi/unsupervised learning for customer segmentation).

Sometimes you'll need a way to simulate scenarios and estimate the what ifs (ie Monte Carlo simulations in default risk analysis).

Nurturing Talent

Limiting the scope of my answer was a mistake. I believe it was a wasted opportunity where I could've shared a better perspective on how data science relate to their core activity. It was a wasted opportunity because the event was really geared towards creating the awareness that those economists should really pickup coding and learn how to practice data science themselves.

Imagine having Richard Thaler, the Nobel Prize winner in behavioural economics; being able to run millions of simulations and predictions on how people might behave given a set of scenarios, and later recommends what are the set of policies that could be set in place to achieve the most optimal outcome. How about if it's in real time? Consider the impact that it could have on the target community, and society as a whole.

I've always believed that it's not actually that hard to find a good data scientist. Sure, you could always put up a job ad and wait for the applications to come in. Or maybe, just maybe, take that leap of faith and train the subject matter experts in your team to BE that data scientist. Not only will he then be able to do magic data science, but he'd also be aware of the hidden biases and idiosyncrasies of the data and know what generally would make sense when interpreting said data.

Which goes back to the meat of this article. Data science shouldn't just be about doing predictions.

It's all about understanding the problem, and later devising the right strategy, to solve them.