Surviving Survivor Bias

Disclaimer: The views expressed here are my own and do not reflect those of my current/former employers or clients

The tome by Thomas J. Peters and Robert H. Waterman Jr. titled In Search of Excellence, Lessons from America's Best-Run Companies is a classic in management studies. Originally published by Harper & Row Publishers in 1982, its authors Peters and Waterman from the management consulting firm McKinsey & Co., launched a study of management techniques to answer some of the vexing questions around the secret to successful business management.

They picked a number of companies across various industry sectors to research what made them Excellent according to the criteria defined by the authors (described in detail in the book). The list of companies are given below:

The book lists eight traits of excellent companies as observed in the Peters and Waterman research and termed the Eight Basics. They are:

1. A bias for action: Excellent companies are quick and effective in decision making
2. Close to the customer: They listen to and learn from customers
3. Autonomy and entrepreneurship: Excellent companies foster innovation
4. Productivity through people: Empower rank and file employees to improve quality
5. Hands-on, value-driven: Excellent company leaders are in the trenches with their people demonstrating their commitment everyday
6. Stick to the knitting: They stay within the business that they know and do not digress
7. Simple form, lean staff: The best companies have lean corporate staff and adopt simple structures
8. Simultaneous loose-tight properties: Autonomy and shared values coexist in excellent companies

Companies in the survey

The companies were from 6 (six) different categories and the research on them was through interviews and review of literature by the McKinsey researchers. They are listed below in two sets under each category. The first set is:

• structured interviews plus 25-year literature review
• limited interviews plus 25-year literature review

High Technology

Allen-Bradley, Amdahl, Digital Equipment, Emerson Electric, Gould, Hewlett-Packard, IBM, NCR, Rockwell, Schlumberger, Texas Instruments, United Technologies, Western Electric, Westinghouse, Xerox

Data General, GE, Hughes Aircraft, Intel, Lockheed, National Semiconductor, Raychem, TRW, Wang Labs

Consumer Goods

Blue Bell, Eastman Kodak, Frito-Lay (PepsiCo), General Foods, Johnson & Johnson, Procter & Gamble

Atari (Warner Communications), Avon, Bristol-Myers, Chesebrough-Pond's, Levi Strauss, Mars, Maytag, Merck, Polaroid, Revlon, Tupperware

General Industrial

Caterpillar Tractor, Dana Corporation, Ingersoll-Rand, McDermott, 3M

GM

Services

Delta Airlines, Marriott, McDonald's

American Airlines, Disney Productions, K mart, Wal-Mart

Project Management

Bechtel, Boeing, Fluor

Resource-based

Exxon

Arco, Dow Chemical, Du Pont, Standard Oil (Indiana)/Amoco

What is striking when you go through this list is that most of the companies in this list no longer exist or have become a pale shadow of what they used to be. There is nothing inherently wrong with the "Eight Basics". It is common sense. Yet, why did so many of the companies that exhibited these critical behaviors fail in spite of them?

The answer may lie in what is called as Survivor Bias. The eight basics may have also been adopted by other companies that were filtered in the selection criteria. When we look at only the survivors and generalize results over the larger population, we get survivor bias. The characteristics of the survivors are useful for emulation only if we don't find exemplars in the graveyard that also share similar characteristics.

In 1994, another business writer Jim Collins published a management strategy research book titled Built to Last that adopted a similar approach like In Search of Excellence to find out what gave companies the traits that made them strong and survive the vagaries of the market forces and trends. Built to Last was a best seller that sold millions of copies.

As the new millennium arrived, a strategy book titled Good to Great written by Jim Collins, published by HarperCollins Publishers Inc., hit the stores in 2001 and again sold millions of copies. It tried to answer the question why some companies make the leap and others don't.

Jim Collins and his team of researchers picked 11 (eleven) companies that they labeled as 'Good to Great companies', 11 (eleven) companies that they labeled 'Direct comparisons' and 6 (six) companies that they labeled 'Unsustained comparisons'. The 'Direct comparisons' were companies in the same industries as the good-to-great companies with the same opportunities, resources at the time of the transition, but did not make the leap from good to great. The 'Unsustained comparisons' were companies that made short term shift from good to great but did not maintain it. The list of companies are given below:

The book lists seven characteristics of companies that made the Good to Great leap. They are:

1. Level-5 Leadership: Humble leaders driven to do what is right for the company
2. First Who, Then What: Get the right people on the bus and then figure out where to go
3. Confront Brutal Facts: Confront brutal facts with honesty without losing hope
4. Hedgehog Concept: The three overlapping circles of what we are passionate about, what we could be the best at and what can make us money
5. Culture of Discipline: Building a team that can be counted to do the right thing (ie., rinse the cottage cheese)
6. Technology Accelerators: Use technology to accelerate growth within the three circles of the hedgehog concept
7. The Flywheel: The additive effects of many small initiatives

Good-to-great vs. Direct comparisons companies

Abbott vs. Upjohn

Circuit City vs. Silo

Fannie Mae vs. Great Western

Gillette vs. Warner-Lambert

Kimberly-Clark vs. Scott Paper

Kroger vs. A&P

Nucor vs. Bethlehem Steel

Philip Morris vs. R. J. Reynolds

Pitney Bowes vs. Addressograph

Walgreens vs. Eckerd

Wells Fargo vs. Bank of America

Unsustained Comparisons

Burroughs, Chrysler, Harris, Hasbro, Rubbermaid, Teledyne

The historical context of this list is the year 2001 when the book was published. The Dot-com bust was underway leading to the culling of companies with stratospheric valuations built on quicksand, the world had seen its worst terrorist attack of 9/11 and the markets were filled with doom and gloom.

In the years that followed, some of the Good-to-great companies met disastrous ends. Circuit City became bankrupt. Fannie Mae was hit by the 2007 Financial Crisis and had to be rescued by the US Federal Government, Pitney Bowes share price plummeted from the $43+ per share it was trading at 2001 to its current price of $4.40 per share.

What is even more striking is that we do not see any of the current giants in this list!

No Apple. Steve Jobs had not launched his famous iPod that would change the course of Apple's history. Not until October 2001.

No Microsoft. Bill Gates and Microsoft were busy fighting an existential crisis due to the US Federal Antitrust suit between 1998 to 2002.

No Amazon. No one paid attention to Jeff Bezos selling books online with Amazon losing money. Remember it was after the dot-com bust and Bezos had just pivoted to retail 3rd party selling.

No Google. It was still a toddler.

No Facebook. Facebook wasn't founded until 2004.

Alas, Good to Great is another glaring example of Survivor Bias in management strategy books. Time is a great leveler.

So, what is the lesson from all of this and how does one survive survivor bias? The lesson is that we need to be extremely careful while extrapolating the characteristics of a few exemplars over a larger population. Our mind continuously works to see patterns when none may exist eg., optical illusions. This includes some of the current giants that I have listed above. They may have competition that is lurking in the sidelines or yet to be formed. Believing that these companies have achieved greatness and hence will remain great is another type of bias in thinking called The Halo Effect which is a separate topic in itself. The simplest way to check for survivor bias in your thinking is to see if there are companies in the corporate graveyard that have adopted similar strategies, tactics, culture, behavior etc., and failed.

Banner Photo Credit: Photo by Ryan Bruce from Burst