How I decide when and where to use analytics to build better products

During my career, I have been involved in building over 50+ products, including an iconic one – Motorola Razr. When I started, I seldom had a reason to use analytics. Now, I seldom have one to not.

History is the best predictor of our future – Mark Hopkins

This quote is at the heart of a key application of analytics: prediction. Analytics looks at historic data and predicts future events. Specifically, two:

1. Will something happen? (a classification problem)

2. How likely is it or how much of it would happen? (a regression problem)

For example, let’s say you are planning a get-together at your home for a winter day. Of the many things on your mind, two are these: Will it snow (classification)? And if so, how much (regression)?

Analytics can answer this for you by looking at historical data for that day. Specifically, it can tell you if it has historically snowed on that day and if so, on an average, how much. Granted, in real life, you’ll likely pull your weather app but hopefully you get the point about what a practical application of analytics could look like.

Now, imagine the power of analytics to answer business questions as diverse as: will my costs rise? If so, how much?

Will I hit my revenue/funding targets? If not, how much will I miss it by?

How many new customers/members will I acquire?

Or in our case, product specific questions, which I cover next.

To begin, let’s take the first question. When to use analytics to improve products?

When the following is true.

You rely on intuition to make decisions: As a leader of a product organization, these are some questions that you run into often. Should I add features to my product? Or remove? Is this my last quality issue? Do I need more resources?

You do your best to address such questions. But you do so, mostly from experience and intuition. But what if you could look at historical data to answer them?

You have existing data to inform such decisions: Say you are interested in adding product features. What if you have had historical product data that could help you decide which features to add? This could be data such as number of products built in the past, number of features per product along with information on how well these products performed. These are examples of data product that organizations usually have. But sits abandoned in siloed legacy systems.

You perform repetitive tasks: These are mundane tasks that you hate doing but have to.

Let’s take the second question now: Where to use analytics?

Here are the top 3 that I use often and find promising.

Product definition: What to make?

Product definition is a critical product development step. Get this part right, and you have a product that is on-point, under-budget and on-time. Mess it up? Your product is off-track, over-budget and late.

In the over 50+ products I have been involved in building, I have found at least one common theme.

Product definition is highly ad-hoc and intuitive.

Although market studies (e.g., research, surveys, etc.) are done to inform product definition, by the time product development ends, they are either ignored or influenced by ad-hoc and anecdotal inputs from various stakeholders. This makes product definition challenging.

And if you are in the business of making many products a year (say tens, hundreds or even thousands, as some companies like these do), imagine how challenging product definition can be. 

But what if you could minimize ad-hoc inputs? Even better, what if you could use historical data to predict which products will succeed? Which features will resonate with customers more? And why?

Analytics can do this for you in 3 steps. First, it can evaluate all your past (or other benchmark) products and break down their key attributes. Second, it can correlate the commercial success of these products back to these attributes. Finally, it can arm you with insights about which attributes are more likely to result in success of your future products.

You can then use these insights to inform your product definition and include only those attributes that have a higher likelihood of success. See this MIT link for how companies making on-demand products (e.g., Netflix) and consumer goods (e.g., P&G) use this technique.

Product execution: How to make it right?

In an article I wrote on LinkedIn earlier, I said this:

Products do not become great because of great features. They become great when great features do exactly what they should. Every. Single. Time.

It’s never fun when products don’t do what they should. You lose.

During product development, you lose time. During manufacturing, you lose money. During customer adoption, you lose trust. And with driverless cars around the corner, you could potentially lose lives.

The good news? Analytics can help. Consider this. What if you could predict where a quality issue or a bug will arise? And plan for it? Or even better, prevent it?

Analytics can look at quality issues from prior products including the factors that caused them (e.g., new technology, new IP, special use-cases, etc.). It can then determine if such factors are relevant to your new products. And if so, to what extent. Once done, it can then predict the occurrence of quality issues (or bugs) in your future products.

This information can help you identify quality hot-spots earlier and plan/prevent them. See details of work I did here at McKinsey where leading semiconductor companies are getting benefits of this approach.

Product delivery: How to make it on-time?

Of the 50+ products I was intimately involved in building, I can literally count on fingers, the ones which were delivered on-time. I am not proud of this. But this is hardly uncommon in the case of complex product development.

The reason products get delayed is less about lack of talent, process or leadership. Often, it is about uncontrollable external factors. Market needs change. Features get added. Deleted. Planning gets optimistic. And so on. And when such external factors kick in, real-time determination of their impact on product delivery becomes virtually impossible.

But what if you could predict, with reasonable accuracy, if your product will be late to market? And by how much? What if you could then take real-time remedial action to prevent it from being late (e.g., add resources, reallocate them, etc.)?

Analytics can do this for you. It can evaluate delivery slippages from past products including responsible factors (e.g., large size of product or code, high number of individual parts or modules, number of processors, etc.). Similar to its application in quality, it can then determine if such factors are relevant to future products. And predict how late you’ll be, if at all any.

You can then use this information to take necessary action to prevent slippage. See details of work my colleagues did here at McKinsey with leading tech companies.

These are just a handful of application of analytics to build better products that I am a huge fan of. I am especially curious to know what has worked well for you. Pls. share below in comments.

How have you used analytics to improve products?

Views are my own and not of my employer or other organizations I may be associated with. I would like to thank Ashley Lucchese and Greg Leffler for reviewing this article.

A few of my other posts that you may like:

3 design lessons I learned over a decade of building products

The real reason why our eyes are the next battleground for technology dominance

What I learned about building world class products by patenting my ideas

I am in McKinsey & Company's Strategy and Operations practice where I work with high-tech clients on topics of strategy, go-to-market and operations. If you like the topics I cover, please follow me on LinkedIn here to stay tuned.

#analytics #bigdata #product #productdevelopment #smartphone #iphone8 #machinelearning #ML #AI #ArtificialIntelligence