Common lore says that most product initiatives fail. Over the last 10 years, lean and product discovery practices have increased because they help us validate opportunities in the risky world of product development. From personal experience, I know that many initiatives fail, but I have not found a source that compiles evidence as a benchmark for this failure rate. I’ll try to put together a few references to come up with an “evidence-based” number.
Startups — 75% to 90%
“9 out of 10 startups fail” is what we usually hear. Let’s see some studies:
- After having researched over 2000 venture-backed startups, Shikhar Ghosh, a senior lecturer at Harvard Business School, concluded that three quarters fail to deliver a return to their investors.
- If you think that number is biased or exaggerated, the Startup Genome Report has studied over 3200 growth stage startups and claims that 90% fail.
But a high failure rate is not only for startups. Existing and experienced product development companies also report somewhere between 50% to 80% of failures in their new features and product initiatives.
Microsoft — 66%
Microsoft has shared a report where the narrator starts by saying that “it is humbling to see how bad experts are at estimating the value of features”. They close by saying that only 1/3 of initiatives successfully impact the key metric.
Booking.com — 70%
The European travel giant is known for world-class product development practices and lots of A/B testing. In a fascinating video about experimentation culture, Stuart Frisby shares that 70% of those experiments fail to produce the expected impact. What’s more interesting is that they admit (as we all should do) that we are usually wrong about what is “good” for the user.
VWO study — 85%
Visual Website Optimizer is a popular tool used by many companies to run A/B tests. They have information about many industries and an incredible data set of executed experiments. They have shared in an article that only 1 out of 7 tests result in a statistically significant improvement.
Amazon — ?
In his 2016’s letter to shareholders, Bezos said that Amazon was the “best place in the world to fail at”. Even when it is a bit of a cliche, Bezos continues to state a fundamental principle: “Given a 10 percent chance of a 100 times payoff, you should take that bet every time. But you’re still going to be wrong nine times out of ten”. He has also shared why they spend billions of dollars in failures: “If the size of your failures isn’t growing, you’re not going to be inventing at a size that can actually move the needle”.
So while we don’t get a failure rate from Amazon, we can say its founder willingly accepts (and understands the need for) a 90% failure rate.
Bezos on betting 1 out of 10 for a 100x success
Bezos on failures and the importance of experimentation.
Facebook — ?
I do not have a ratio for Facebook, but rather an interesting report of numerous significant failures. Zuckerberg is also famous for having said “move fast and break things”, embracing as Bezos the need for failure to create a positive impact. Reviewing failures like Gift’s (send real-life presents from Facebook), Deals (Facebook’s version of Groupon), or even their email service makes us think they also invested millions in failed expansion initiatives.
Conclusion
I know this doesn’t clarify a real benchmark for our product efforts. For example, A/B test failure rate is quite different from a startup failure or Facebook’s incursions in new businesses. But I feel we can confidently say that the failure risk is high, and having so many good companies and product managers failing should teach us that the most likely scenario is that we will have similar or worse rates ourselves.
The real question is, what should you do about it?
- The goal of Discovery is to minimize the risk of failure. It doesn’t mean that ideas won’t fail. It means that we will discard them before we build them. Instead of wasting 70% of our product development resources in initiatives that didn’t produce any impact, we will invest 10/20% on experimentation and learning to discover the right product to build.
- Product Strategy is another significant factor, with it’s potential to reduce failure usually disregarded. By making a proper diagnosis and using insights to solve the right problems, we direct our Discovery into the most profitable areas.
Note: my upcoming book, Product Direction, talks about the connection from Strategy to Discovery to reduce this failure rate. You can grab a free section here.
Can you help?
I want to improve this “study”. Do you have information about other failure rates to share? Feel free to contact me or fill this quick form to improve the benchmark for all of us!