Ramblings of a Peripatetic Mind - Act IV: Data Driven Product Development

Being data driven is about building products, tools, most crucially: a culture that acts on data

There is so much going on in the world of data analytics beyond the mundane data transformations and BI dashboards. Fancy IOT pipelines, AI/ML algorithms, predictions that dazzle you with the intellectual horsepower and value propositions are the lofty expectations that seem to be the norm!

While most companies struggle with basic software development methodologies i.e., Agile vs Waterfall vs “somewhere in between” approaches, treading with trepidation into DevOps (dare I say CI/CD), imagining systems where data streams in, interacts with their application code seamlessly, learns on the fly to predict/prescribe responses is not something these companies can envisage near term, and this does not even factor in any level of ML Ops.

The topic of data analytics has been addressed in journals, books, online courses, white papers, case studies and specialized degrees. Top cloud vendors are making it easier to develop ML models with little to no experience in data science [Amazon, Google, IBM, Microsoft to name a few]. These providers have also made it easy to learn and build models via free access to educational materials and cloud resources to experiment with!

There are a number of examples of small companies being able to bring significant innovation by leveraging the affordability and ease of use of ML in the cloud leaving me to wonder if smaller companies have a competitive advantage in realizing the benefits of ML! While ML is cool, it is not a pre-requisite to embark on a data driven product development (DDPD) journey! Let us set the context:

The “Gartner” 4 step process to a DDPD enterprise is a great place to start. While there is "still" value with "descriptive analytics", innovation increases as enterprises move closer to "predictive" and "prescriptive" analytics.

From the table, analytic maturity correlates directly with ML incorporation. If you watched "Minority Report" and thought that was cool, remember: that is still only "predictive" - not fully mature in the analytics spectrum! We should all be disappointed if in 2054, we do not have a "pre-crime" unit that does it all without the need of "gifted pre-cogs" ... but I digress!

So, What does it mean to be a "data driven product development" (DDPD) organization? While this can easily become overwhelming, we will keep this brief and consider only 3 aspects:

1. Enterprise setup & Data IQ
2. What is usable data?
3. Specific skills?

We will defer the topic of what data cannot be used to a separate topic!

Part 1: Enterprise setup & Data IQ

The oft repeated question is: "Should the data analytics function be centralized or decentralized?" You can engage top consulting firms, spend a lot of money, go through "fact finding" interview sessions, tweak the enterprise with the newly gleaned insights. Or go with the simpler answer: .there is no such thing as a "correct setup".

• If you have a decentralized data function, you can make it work
• If you have a centralized data function, you can make it work
• If all you do is build data products - with outsourced data - you can make it work

What is NOT correct is: "to ignore leveraging the data" for bettering your products, improving customer experience, innovating and providing value to your stakeholders with the wealth of knowledge waiting to be unearthed via a data driven product management approach.

Depending on the size of data generated, it may make sense to have a team to manage it. With cloud managed services, this has become a relatively easy workload to manage, especially for smaller companies allowing them to focus on product innovation with the data taking center stage (and not worry about "am I setup correctly?") - a big advantage!

The graphic shows a decentralized setup: this can be physical or logical based on the the Data IQ of the enterprise.

What is the Data IQ of the company?

To leverage data, the enterprise needs to understand the data they collect, ensure the right data is being collected, and build data products (extracts, dashboards, streaming data, ML insights...) that create value i.e., data monetization, sell more products, bring in new customers, or improve existing customer stick-ability. This can be effected in a high data IQ company. While black-box ML models allow non-data scientists to generate powerful "predictions", these models cannot compensate for the user's lack of knowledge of their data: so, data IQ is simply the subject matter expertise (SME) of the business, data generated by the business, what the data represents, and how to use the data to improve the probability of expected outcomes.

Example #1: Online retail

Data can be leveraged to improve product performance in real time. This is by far the most common use-case for data analytics and is evidenced in almost any shopping sites these days. Data can be analyzed to understand what works in the product portfolio and what does not, high value from the low value, where we go things right, where we need improvement, and what we need to stop doing. This is especially time sensitive to avoid investments in bad bets.

In the graphic, data analytics is foundational to provide recommendations i.e., "frequently bought together". This is the easy part - the more difficult part: if probability of "Frequently bought together" goes up or down, the ability to monitor, and adjust the offers is critical. Assuming you can do this, the website will find the feedback insightful, and can invest more in similar books, categories, run experiments / promotions for slow selling (similar) books/authors, and offer products/services in parallel (non-book) categories. The ability to understand the "demand signals" based on data requires a high Data IQ, and to adjust on the fly via ML intelligence requires technical savvy: this is quickly becoming a much sought after skill, and will have a huge upside as even the most advanced companies are still on the lower end of the maturity spectrum,

Example #2: Travel Shopping

In the complex world for airline shopping, a response set can have as many as 1000 elements - not all of which are visible to the end consumer. While storage cost and compute costs need to be fine tuned, data IQ allows the enterprise to extract a distilled set of elements for downstream analytics by identifying data that adds value towards (1) improving offers, (2) building new products, (3) help carriers understand competitive landscape, (4) identify markets to expand or (5) contract services, (6) understand the size the market dynamically to respond quickly and appropriately ....

Some of the data elements that will allow for the above use-cases are what were the shopping qualifiers: one-way vs round-trip, number of connections, number of passengers, date (including day of week & time of travel, season) of travel, passenger type, date shopped vs date booked for), in some cases, even date/time of shop.

Shop response characteristics that can have profound impact on revenue/conversion are: (1) what itineraries are sold? (2) what itineraries do not sell? (3) is there a sweet spot for the price in the market? (4) how do my itineraries compare with my competitors? (not something that is available in real time); (5) for specific markets, what are popular connections? (6) Market by price i.e., which markets afford higher prices? (7) Can the carrier offer service from a cheaper airport in the same city? (8) Balance the market size with available equipment i.e., ply bigger equipment in bigger markets

In today's world saturated with data, external data sets also can bring tremendous value such as maybe there is a convention that drives travel demand in a market, or a major weather event that drives traffic through alternate routes. There are companies that offer data that allow carriers to think outside the box and eke out every bit of intelligence to drive up revenue. The more successful companies are the ones that can respond in real time with little to no manual intervention.

Part 2: What is usable data?

Over 95% of the companies fall into 1 of 2 buckets: (1) either store all their data, (2) store no data. When all the data is stored, the intent is usually to generate some value now, and a lot of value (unknown currently) later. In an on-premise situation, the cost of storing data is not usually transparent: this problem is a whole lot easier to solve in the cloud with significant opportunity to slash cost while still keeping the data (think near-line, cold-line options).

With no focused effort to generate value out of the data, saving on data cost is still a "cost" to the company. Research also suggests the value/revenue generated from the subset of data used does not compensate for the total cost of storing and processing the datasets.

Companies with Data IQ figure out ways to slash cost further by being smart even when storing the entire payload, making it easier when the time comes to realize value. A simple example is storing the payload as-is (an XML payload is 4 bytes vs 1 byte with just the data). An extrapolation of this example allows for a 75% reduction in storage cost + you now have allowed the citizen analyst to derive value relatively easier than to deal with XML or JSON payloads (Note: Cloud vendors provide native support for some of the data formats).

• In the air shopping example, a single transaction has over a 1000 elements. Identifying a smaller "critical" subset of data elements allows further cost savings in storing data for real time access + make it much easier for your "citizen analyst" to derive value!
• Finally, processing a more focused data set makes it cheaper, faster, and less complicated

Part 3: Specific Skills / Roles that can make DDPD a reality

There are 3 roles that can usher an era of data driven product development to the enterprise.

Product Managers:

Benefit from understanding how their products are performing - even with basic analytics:

• What works?
• What does not?
• What features are used?
• What features are not?
• Any insights to add new capabilities, products?
• Has usage improved with recent changes (user sentiment testing),
• Is response time meeting expectations
• bounce rates, exit rates to name a few useful KPIs

While these are basic, it is surprising that these may not be readily available - especially in the legacy enterprises. Assuming the needed data points are instrumented and captured, running reports require basic to advanced SQL skills - that are readily taught in most college curricula or can be picked up through any number of online learning portals. BI tools also make it easy to extract these insights via drag and drop capabilities. Cloud vendors take it 1 step further by providing integrated tools i.e., if you are on Google Cloud Platform (GCP), and the data is stored in (or accessible via) BigQuery, users can run reports via DataStudio - a simple intuitive tool.

Engineering Manager / Team:

There is a wealth of information in data that allows an Engineering team to write, maintain and improve their product performance. Limiting the benefits to software development, some obvious benefits from data allows teams to:

• Understand system bottlenecks
• Identify and resolve performance issues
• Troubleshoot problems
• Leverage system, storage resources more efficiently
• Monitor and optimize operational costs
• Identify problems before they occur (Yes!)

Engineering teams are usually adequately equipped (skills) to realize the advantages from data analytics. What is usually missing is the priority and focus in this area. Companies are racing against time to get features out the door. Anything that does not provide tangible, immediate ROI does is relegated to a "desired" state and does not get actioned. The silver lining is with the adoption of cloud technologies, a number of the advantages listed above comes out of the box, and more readily realizable than ever before.

Data Analytics & Engineering Function

This function/team typically exists in bigger enterprises with core competency being data analytics, and is usually associated with a P/L i.e., beyond just servicing the needs of the enterprise, they are expected to bring in revenue as well. The skills are very advanced consistent with the responsibilities. Kaggle has a nice graphic that lays it out with clarity

In summary,

• An enterprise can become data driven if data IQ is an inherent capability with the enterprise's product management and engineering functions. It is less dependent on how the enterprise is setup, and ready to leverage the IQ to improve product efficiency, and drive up revenue
• If the data IQ is lacking across the enterprise, having a dedicated group to understand, recommend, put systems in place, help Product and Engineering to instrument data collection, and integrate the analytics can reap rewards. In my experience, this is not a trivial task
• Whether your enterprise is a novice, an expert or somewhere in between, it is a journey that provides immense value each step of the way. No matter where you fall, there is empirical evidence that "if you can be 1% better today than yesterday", in data analytics (as with everything else), you will slowly but surely climb the ladder of DDPD maturity and realize tremendous value along the way! I hope you do an !

d share