When my wife and I were considering leaving London and moving back to Warsaw, we felt a lot of cognitive dissonance. On one hand, most of our friends were sharing clickbaity headlines of despair. Depending who was sharing it, either the whole country was turning into a totalitarian state or it was in a state of moral apocalypse driven by the EU. On the other hand, when we visited grandparents during the summer at their log cabin, the lazy afternoons on the porch felt very cozy, safe, and relaxed. The only thing that was clear: articles shared on social media were exaggerating on both ends of the political spectrum.

The Netflix special "The Social Dilemma" has tried to popularize what most of the tech industry has been aware of for a decade. We life in a world of social media bubbles. Each of us are subject to massive confirmation bias. Confirmation bias refers to discounting any facts which are inconsistent with our current beliefs. So we only believe facts which are already consistent with our current beliefs, accumulating unfounded confidence in how right we are. Social media amplifies this dynamic. If our friends keep repeating our own views back to us, we radicalize in our opinions.

This dynamic is similar to that of cults. Once you are on the inside, any information sourced from outside the cult is discounted as unreliable. So despite being able to think for yourself rationally, you end up believing that staying in the cult is the only reasonable choice you have–even as they serve pink Kool-Aid to thousands of naked people in the desert.

So why am I writing about confirmation bias on an innovation blog…of all places? Because it’s the underlying problem behind the most common reason startups fail: building a product that nobody wants. And despite Lean Startup and similar reasonable approaches, 40% of new product failures from startups and corporates have been occurring for this reason since studies the 1970s.

I would even say that the real problem isn’t rational. We need to go deeper to understand why they haven’t reduced the frequency of this problem. The founder is so convinced of their vision, they avoid gathering feedback or they discount any feedback inconsistent with their vision. Even if it comes from prospective customers. Just like believing every headline you read on Facebook.

Bias and false beliefs

The nature of early stage innovation is that small changes in trajectory can have massive implications years into the future. To borrow an aeronautics analogy, this is similar to the proverbial autopilot mechanism on most planes. For most of a long haul flight, a plane is a few degrees off course. But as long as it constantly corrects and readjust for the right destination, it will get there.

If the autopilot didn't correct in the early stages, it could miss the mark by a massive margin. This is why disconfirming evidence is useful (necessary in my opinion) in this context. If the autopilot mechanism doesn't know you are two degrees of course when flying from Europe across the Atlantic, you'll end up in Florida when you expect to land at JFK airport in New York. Let's say the autopilot mechanism breaks and the pilots fall asleep once the plane reaches a cruising altitude. Some of the planes will arrive at JFK regardless. Most won't; early diversions off course (even small ones) will have the largest impact on where the plane "arrives".

Essentially the purpose of experiments is to systematically test your beliefs, ideally turn it into a habit, to identify opportunities or threats early. Ones which you would miss otherwise. Beliefs themselves influence what we notice and see. And if you are starting a completely new venture, your own beliefs can get in your way. They are a major source of "unknown unknowns" of two varieties:

1. being absolutely convinced about something that isn't actually true (common wisdom and best practices)

2. not noticing evidence to the contrary because it would undermine a bias that you have, like your social media confirmation bias or a sunk cost (that's the way we do things around here).

To be clear, you don't need to do this when starting any business, just a high risk business model. If you are starting a dentist's office, the overall risk is pretty low. The service is easy to explain. And the variables that will influence your success are relatively well known. There are off-the-shelf books, courses, and advisers which can walk you through all of the important decisions, because all of the variables are known.

But what if you don't even know what the important variables are? This is the scenario with most high tech businesses. You are subject to more risks, many which you don't even see. And being hard-headed and opinionated is more of a liability, especially in this early stage. Because you avoid gathering disconfirming evidence, to expose your own biases. And you don't course correct, like the airplane on autopilot.

Later implications for your company

A really important implication here is the emotional contagion of being a leader. As a founder or a member of a founding team, your beliefs form the basis of your company's culture. And therefore, if you don't install the rigor of systematically questioning your beliefs into the culture, then it won't be there. One of the commonalities of the major tech companies is that they continue to foster this type of open debate and feedback.

Alex Kantrowitz (@kantrowitz), author of Always Day One, says Facebook is probably the most famous for this. On an upcoming episode of the Align Remotely podcast, he recalled the case of when Mark Zuckerberg was pushing strongly for a web only app, which would also be used on mobile. This clearly simplified the (already massive) technical challenge for his company; however, a web-only interface had a sub-par user experience. A native approach from within the mobile operating systems gave developers more tools and components to orchestrate a great experience. When one of his technical czars came to Zuckerberg with feedback like this, Zuckerberg suggested that they do a prototype of how this will look, both within their company and technically. They assembled a team and got on with it. And a few months later, when they created a mobile prototype, Zuckerberg saw that his vision was wrong. The user experience was so much better when Facebook was an app (also it allowed for gathering a lot more data about each user, thus making advertising more targeted). With this disconfirming evidence, they continued to pursue Mark's overall goal but changed the approach.

Stepping back, this is a great example of how irrationally holding on to a false belief could have cost Zuckerberg his company. It’s almost ironic that the social media giant is such a good example how to deal with confirmation bias, given its role in making it so widespread.

Ultimately, incorrect assumptions can be both risky and expensive. So rooting them out, as quickly as you can, lies in your best interest, even though it can be unpleasant and feel counterproductive (at first). The same holds regardless of company size. It's true if you are an early stage founder or riding a bucking growth bronco like Zuckerberg was at the time.

Wait, but isn't this too negative of a mindset to achieve your goals?

Think the danger here is that you only do experiments. If you keep questioning everything, you never feel like you are making progress. If you only focus on finding counterfactuals, isn't that like visualizing failure then expecting it to happen? Not if you get reliable counterfactual evidence. A useful variant of the thought process (especially at small sample sizes) is: assume your are wrong. Gather evidence to prove prove yourself wrong. And then hopefully be happily surprised.

The point here is not to be right. It's to be successful, as quickly as you can. And do that you have to prove the proverbial naysayers wrong, by taking their concerns seriously.

In this context, I find it useful to think of risks as blockers which you address with experiments. If you identify the major risks, and run experiments, you will unblock yourself. If you have addressed all major risks around the product, then customers will be clamoring for your product or service and happy with it when they use it. Once you address major risks around growth, you will have a repeatable and scale-able way to access more of your existing customers in a large market. In both cases, execution should be simple and rote. You know exactly what to do, and you just need to get on with it to make money as a company.

Case Study

My "signature failure" was starting a website with video courses for financial professionals about Microsoft Excel in 2008. In today's terms, the idea was something like a super-focussed Udemy on that particular topic for that audience. After spending 11 months building it (and a lot of personal savings), I realized nobody wanted in my market segment actually wanted it. Their day-to-day problems and challenges were quite different, and couldn't be addressed with a =VLOOKUP() tutorial.

A few years later, as a retrospective test done out of curiosity, I set up a landing page offering a similar product to the same audience. I promoted it on social media and bought ads. Unlike many similar tests which I have run, I had minimal signups and interest. So if I had sourced counterfactual evidence for a few days, I could have saved myself from a painful, long running, and expensive failure. And moved on to find an idea actually worth pursuing.

Next week, I’ll be writing about an unlikely tool I used to build a habit of counterfactual thinking from way outside the startup world. If you’d like to be notified, grab my hero canvas, and you’ll automatically get an email update when that post comes out.

Key Takeaways

• Your biases and beliefs affect the assumptions you make when starting a new business
• Unproven assumptions can be expensive and/or risky, especially if you are trying something new
• Common knowledge and hidden biases can easily sidetrack you through "unknown unknowns" or not realizing the full implications of "known unknowns"
• By using learning goals to seek out counterfactual evidence actively, you establish a solid baseline for building a groundbreaking new business