One of the really “hard things” about the startup game is getting on a growth trajectory that grows exponentially. VCs debate how fast you should be growing on a weekly basis. But one thing about growth is clear. Instead of having the occasional bump in sales or growth, high growth startups are systematic in how they pursue their results. In particular, the heavy focus in this deep dive is on channel testing.
Strategyzer’s Business Model Canvas: channel testing
The following is based on a discussion with Nemo Cerovac (@ncerovac), the cofounding team member responsible for the explosive growth of FishingBooker, a site that has been dubbed the AirBnB of fishing charters.
Nemo Cerovac
Meeting customers where they already are–at first
In our first year, we spent a lot of time on the captains’ side. Finding things that worked for us. We realized that initially we could do 10% growth weekly by sending emails to captains.
Targeting captains in the harbour
We tried to target a couple of Captains in the same Harbor, call them up, talk with them. Initially, reactions were mixed. “Why would I sign up on the platform that is getting a commission from this? Why would I register there? You didn’t exist? I don’t know who you are.”
What started happening eventually was they heard from potential customers and previous customers. We were heavily targeting consumers with ads.
Your riskiest assumptions are probably related to your prospects and customers. Establish empathy quickly with your target prospect, figure out what's valuable, and get your innovation into the market.
Once we got one or two captains from the Harbor, and we were killing it with Google ads, they start saying, “Oh, I got like two clients from this FishingBooker this month.” Moreover, the same Captains who were skeptical initially came back. They’re like, “Hey, you know you contacted me like three, four months ago. Let’s talk again.” It was a roller coaster.
Going and meeting everyone in person helped a lot. When we went to Florida, we were going from Harbor to Harbor. Just meeting people there. Just talking with them, showing them the platform, and just meeting every notable potential user.
In the end, I think it was the user drive that got us the captains. At first, it’s also no harm. You’re showing captains the approach: “Hey, this looks professional. You don’t need to do anything. If we get you some bookings, then we earn something from it.”
Growing the user base
There were a lot of chicken and egg problems. To be honest, Vukan’s mindset helped a lot. Vukan, the CEO, was super focussed. We were really pushing hardcore. We were figuring out each channel. For users, we figured out that the best channel for consumers at the start was paid advertising: Google ads. Finding them where they are, not where you want them to be.
Basically, most people search for “fishing charters in Maldives” when they’re preparing for the trip. Or they just reserved a hotel, and now they want to plan their activities. That’s the moment of strongest intent. The easiest moment to convert.
Once we figured that out, we then just focused on adding money. To convert as many as possible. We were trying to stick to this ratio where 30% of the revenue goes to paid advertising. We were pushing hard.
About to start a greenfield project?
Have Launch Tomorrow run an in-house "riskiest assumption workshop". Remote delivery options also available. Discover where to prioritize your validation efforts, to get to market fast.
Every Monday, we would have this white board in front of us. We listed our golden numbers. Golden numbers are the metrics that are the most important for your product–to make it or break it, at that stage. [Ed note: check out this KPI checklist to choose golden metrics for your team].
In our early days, our golden numbers were:
Create an explainer video for your complicated new product.
Make sure your audience understands it, without being overwhelmed by technical details.
number of captains that finished their profile or uploaded at least one of their boats there
number of people that booked this week or this month
the number of bookings started but not completed
Every Monday we would look at our numbers from the previous week. We would compare them to our targets from the previous week. Our goal was 10% growth per week. Especially in the start, we had a heavy focus on testing. As fast as possible. Find what’s working, and what’s not working.
Let’s say we wanted to have 10 Captains registered that week, and we hit seven. We then discussed why this happened, and tried to analyze why. Was it something that we can influence? or something that we can’t? So, sometimes it would be like, “Oh, I was sad. I was sick. Or I wasn’t focused.”
If it was a problem in our control, and a channel was getting lower and lower results week on week for 4 weeks. We then decided we should try something else, since there was a clear pattern over time.
These numbers and discussions forced us to think about immediate results. You want to act upon things that give immediate results first. You don’t have time to think about, “Oh, what would happen if we did this or that?” Should we go do a road show? a conference or not?
A failed channel experiment
After going to conferences, we realized it’s a waste of time. For us. At that time. We didn’t get anything. If you’re building the right thing, you don’t need the conference to continue building things.
I’m still maybe a bit harsh on conferences. People who attend conferences usually want to chill in a half-work environment. Some people go to build up their ego.
The upside of conferences, though, is if there are potential clients or users attending. If somebody can benefit from what you’re doing is there–and it’s not just two people– then it might be worth it. Like ship captains, in our case. Or people that are interested in fishing. Then you can try to invest your time in it.
But even then, be careful. Let’s say you convert 10 Captains at a conference. That’s the same amount that we get from sending cold emails to Captains and calling them on the phone in one week. During that one week, you can sit in the office and work on this for 20% of your time. Then you are free to do other stuff with the rest of your time. During a conference, you’re running around. You are trying to meet people. And you need to prepare beforehand to know who’s attending. If you have time, you introduce yourself earlier. Basically, you need to measure and compare the benefits of using each channel.
Conferences do have a place. They help us to connect. They help us meet new and interesting people. So, I definitely recommend people attend conferences, as long as they have this perspective.
The 10x Game
Something I call the 10x game came from this idea of focusing on immediate results only. Often when you start, you start thinking about a lot of things. A lot of concerns. It’s easy to get distracted.
You even start thinking of problems that are not immediate problems. Or immediate tasks that you need to complete now. For example, you are trying to think of how to optimize the process for 1000 users, but you only have five. It’s just not the best use of your time at this moment.
Sure, you need to think about that in the future. But at the moment, park that idea. Put it aside. Because it sucks time from what matters now.
Here’s how the 10x Game works
1. Pull out a box of multicolored post its with your team
2. Calculate what order of magitude you want to start with. This sounds complicated but actually it’s really simple. Take the number of clients you currently have. And then figure out the next power of 10 from that number:
10^1=10
10^2=100
10^3=1000
10^4=10000
Let’s say you have 33 customers. Then 100 would be your next power of 10. If you have 0, just use 10 as a baseline.
3. Assuming that is your next milestone, what marketing channels do you think will help get you to that number? So if you don’t have any customers yet, how do you want to get your first 10 customers. Take your stack of post-its, choose a color and brainstorm at least 10 different ways you can reach your target customers.
4. Then, repeat for the next power of 10 using a different color, and repeat again.
5. Finally, loop back to your first set of ideas. Priortize them. You can do this using dot-voting or any other mechanism that works for you & your team.
This results in a lot of ideas, clearly prioritized in terms of what you need to do first. It give you the space to think big, but also helps you drill down into what needs to happen right now.
Same channel today, different results
When you have 10 Captains and you want to grow 10 more Captains next week, then maybe a conference is not the best way to go. Or a hundred Captains in the next two months. The conference was more useful for branding purposes. Just not relevant to the golden numbers at the time.
Now, FishingBooker goes to a lot of conferences as a brand. You want to position and build a strong brand. You want to be in people’s minds. Once you have a budget for that, it becomes an opportunity. I saw the team a couple of months ago. They were speaking at a Google conference about their experiences.
building the brand is a good use case for conferences at a later stage
FishingBooker is roughly a hundred people today. It’s still one of the top Google ads spenders in Eastern Europe.
Conferences were just one of the channels we tried. And as you can see, different channels are appropriate at different levels of a startup’s growth trajectory.
Stepping back
This is Luke again…
This same insight can be applied to pretty much any channel you might consider. Everything from cold outreach, PR stunts, or any other marketing tactic which you might dream up. The numbers give them context. They are a filter to help you focus on what matters right now.
On a small scale of the first 10 customers, it’s possible for the co-founders to do the majority of this work, possibly with some help. As the startup being growing, there will be different requirements to hit 100 customers. And 1,000. And so on.
Key Takeaways
Use Golden Numbers to drive team conversations about inputs, tasks, and alternatives needed to achieve results
Focus on factors under your control and take action on trends over time
Channels that might be inappropriate initially can be very appropriate later when the company is larger
If you enjoyed this post, and would like to try out the 10x Game with Luke’s help, there will be an upcoming online & free 10x Game workshop you can join. Get notified when this happens!
Icarus and his father Daedalus, the architect of the Labirynth, attempt to escape by air from Crete. Daedalus had become a political refugee. He had helped Theseus, an enemy, break into the Labirynth to kill the Minotaur.
Icarus falls, assuming the plan so good he doesn’t need to watch himself implement it
Daedalus then constructed wings from feathers and wax. Icarus’ father warns him first of complacency and then of hubris. Daedalus asked that Icarus fly neither too low nor too high. The sea’s dampness could clog his wings. The sun’s heat could melt the wax. Icarus ultimately ignored his father’s latter instruction. He flew too close to the sun. When the wax melted, Icarus tumbled out of the sky, fell into the sea, and drowned.
How is this relevant when launching new products?
Entrepreneurs have a strong sense of vision, particularly when creating something new and magical like the wings in the legend above. That’s why we love them. Working alone or as a small team, they execute. They see something no one else does, and they pursue it relentlessly, until it happens. In one form or another.
Conversely, corporate innovators often have to plan a lot up front, to align across multiple departments and stakeholders. Which means that detailed planning results from significant negotiations. And that any change often requires re-negotiation. But there is a great deal of vision and planning required to create something out of nothing.
These two scenarios have more in common with one another. Vision and planning is crucial, but having too much of an attachment to any plan is counterproductive.
Your riskiest assumptions are probably related to your prospects and customers. Establish empathy quickly with your target prospect, figure out what's valuable, and get your innovation into the market.
Believing too much in the initial plan is clingy (and can feel unproductive)
For years, I was curious about whether there was any data to back that up. If you want to create a successful product, usually you want high growth as an ultimate metric of success. Recently, I found that data point.
In The Origin and Evolution of New Businesses, professor Amar Bhide of Harvard interviewed the founders of an entire year’s-worth of Inc 500 companies in the late 1980s, a ranking of high growth startups. This is independent of sectors and also not influenced by any of the more recent lean startup/small batch thinking. At the time, most startups needed to raise a significant amount of money to launch anything new. From a funding & risk perspective, the success criteria were close to that of corporate product development today.
Bhide reported that approximately 2/3 of the founders he spoke to pivoted away from their original concept:
More than one third of the Inc 500 founders we interviewed significantly altered their initial concepts, and another third reported moderate changes.
The majority of the founders shifted away from their original vision. Was it out of necessity? Or is being both willing and able to adapt is actually a requirement for new product success. One third of the founders were lucky to have guessed exactly what their customers wanted. But most needed to adapt and pivot to the best possible business model.
Plans are worthless but planning is everything –Eisenhower
In other words, the most helpful mindset & approach is to:
assume that you will be wrong about something when launching a new product
make sure you have the option to pivot
often, this happens once you collect data that challenges your assumptions…so collect data first!
This is how innovators work. This is also how “search” for a new business model differs from executing a known business model and aiming for efficiencies only.
About to start a greenfield project?
Have Launch Tomorrow run an in-house "riskiest assumption workshop". Remote delivery options also available. Discover where to prioritize your validation efforts, to get to market fast.
This post is part of my “7 Lies of Innovation” email course. Click here to sign up if you’d like to know when the common wisdom around innovation doesn’t actually hold up in the data.
Create an explainer video for your complicated new product.
Make sure your audience understands it, without being overwhelmed by technical details.
Richard Branson, the billionaire adventurer, has a complicated relationship with risk. On the one hand, he loves trying outdoorsy stunts and attempts to break world records.
Branson as an expert at managing the risk of getting punched in the face professionally
For example, he managed to cross the Atlantic, Pacific, and then circumnavigated the globe in a balloon. He even succeeds at doing so, despite the occasional crash:
On the other hand, he’s brilliant at managing his risk professionally, when starting new ventures. For example, thirty five years ago, he was stuck for hours on Peurto Rico. He wanted to fly to the British Virgin Isles to join his girlfriend, and came up with the following scheme:
Branson walked to the back of the plane and asked for a chalkboard and a writing implement. He figured out how much it would cost to charter a plane to BVI — and how much it would cost each passenger if the expense were pooled. And then he went for it, walking up and down the aisles of the grounded airplane selling tickets. On the chalkboard he wrote, “$39 one way to BVI.”
In one move, he confirmed that there was demand for an alternative. So he charted it. That got his appetite wet. Piggy-backing on his record store chain, Branson went on to sell plane tickets. He entered into a conditional lease of a plane from Boeing, where he insisted on having a one year break clause, to be able to return the plane if necessary. He aimed to fly it between the UK and the Falkland Iles. So Virgin Atlantic was born.
Your riskiest assumptions are probably related to your prospects and customers. Establish empathy quickly with your target prospect, figure out what's valuable, and get your innovation into the market.
In short, he made a couple of moves which significantly limited his downside while giving him the possibility of evaluating whether it was worth soldiering on.
With new products, your primary goal is to manage your risks
Denying the existence of risks only makes you more susceptible to them. Particularly when you are faced with a high risk of failure. Like 50-90%, depending on the study you look at. No joke.
“Jed’s dead. Dead dead.”
It’s really important to note that this is not about being pessimistic. Or questioning your vision. This is about figuring out your vision’s fastest possible path from your head and into reality.
The “magic pixie dust” resides in prioritizing what you test. By testing your riskiest assumptions first, you reduce the chance you go belly up. Regardless of where you are in your new product’s journey, you will always face unexplored or unmitigated risks.
Yet, identifying this riskiest assumption is hard. The key factor to consider when managing your resources (including your time) is risk. All of that requires you to have a solid grasp on what your biggest risk actually is at that moment. Which isn’t always easy, particularly if you are doing something new.
What makes this particularly hard: it’s likely there are some “unknown unknowns”. So even if you do deliberately try to identify problem areas, you’re likely to miss something.
About to start a greenfield project?
Have Launch Tomorrow run an in-house "riskiest assumption workshop". Remote delivery options also available. Discover where to prioritize your validation efforts, to get to market fast.
Essentially, the idea is that you want to generate a large number of ideas about what’s risky. And then prioritize to identify what is riskiest (and were you need to start).
Ways to get inputs on what’s riskiest:
Create an explainer video for your complicated new product.
Make sure your audience understands it, without being overwhelmed by technical details.
This is hard to describe. Sometimes you just have a gut feel. Pay attention to it. And run with it. Essentially, you’re looking for what feels like it could have really big & bad implications. And it is likely to be a problem.
Personally, I feel this with tension in my gut. So it’s just a question of tuning into that sufficiently, especially before making bigger decisions. And that’s often good enough. But it has taken some effort to be aware of this. Moreover, in a larger company, you may still need to convince others and create alignment, and “I feel it in my gut” is not a strong enough argument. Even if it’s true.
2. Who is the customer and does the customer actually have problem you think?
This is a great insight I got from Lean Startup Machine specifically. Running around London like a headless chicken with some friends, trying to interview young parents. (admittedly before I was one myself). We had a product idea which we though would be attractive for this group. We made some guesses about their biggest challenges. And then we hit the streets and looked for anyone pushing a buggy/pram.
This was the net result:
hitting a low point somewhere during 67 customer interviews in one weekend
After a few failed hypotheses, we had to acknowledge that we actually had no idea what parents of kids 0-2 years old struggled with. And while there were some patterns in what we were hearing, it was clear that it was still too wide of a market to have clear patterns. The tool on the wall helped us systematize testing:
who is the customer?
what is their biggest problem or challenge?
would the solution be an appropriate solution to that problem?
In the end, we had to acknowledge that we the pie chart we were drawing had too many possible answers.
survey of product managers
So we weren’t going small enough to have a clear pie chart, with a significant share of answers being the same:
looking for well ordered market segments like this
In other words, we needed a consistent easily identifiable & reachable group of people who share a particular challenge. That is a good market segment on which you can build a business.
This is almost always true. And often overlooked up front. Without it, you’ll have a hard time building something that will be relevant for enough people to make a dent. At least in a B2C business, where you need a lot of sales to get started.
In a B2B business, that breadth is less important, as much as depth. You need to be certain that whatever you do build and launch fully addresses the customer’s problem. And has positive and measureable side effects around:
increasing revenues
reducing costs
lowering risks
improving their customers’ experience.
At a high level, most businesses will only pay for those outcomes. But each industry and business is a bit different, so you will need to dig into the details of how to achieve it. But the starting point is really deeply understanding the problem from your prospects’ eyes.
3. Analogs
In short, ask people having similar success in different areas or industries doing things similar to what you want to achieve. This was an insight from Getting to Plan B by Komisar and Mullins.
For example, let’s say you discover a really powerful yeast which helps give red wine a unique and attractive flavor. Then you buy wine in bulk, add your magic dust, and then try to resell it to large supermarket chains.
Coming up with the yeast was the hard part. To figure out how to sell the supermarkets, you can probably speak to any salespeople who also sell to supermarkets in your country. It doesn’t have to be other wine merchants or wholesalers. You will learn a lot from a few conversations with people like that, enough to get your head around how this sales cycle works. So even though you don’t have in depth experience doing it yet, you’ll have a clear picture of how to proceed. In this case, you are using an analog to discover any hidden assumptions that could derail the use of direct sales as a distribution channel.
Channel testing example: selling superior wine into supermarkets
This is a good technique to turn “unknown unknowns” to “known unknowns”. Possibly even “known knowns”. Even though it’s less likely to have a company killer impact, it is still possible that you are making assumptions about that initial sales process which could derail you.
4. Antilogs (or Picnic in the Graveyard)
Some industries, especially software and internet, have a lot of public information about startups that have tried something. And failed. There are lots of writeups by previous founders of why they think their startup failed, such as:
If you are thinking of building something similar, you are just at the beginning. Those founders had been through years of trying to get a similar idea off the ground. So it’s quite likely you will learn something from them, if you can read about their failed attempts. Or even better, call them up and speak with them directly.
The idea here is to formulate a good reason why you think you will be successful this time around. It might be a different feature or market. It might be market timing. It might be any number of reasons. Because if you can’t articulate it, it questions whether it’s an idea worth pursuing. Not to mention, it will be hard to convince investors, too.
5. Risk Scorecard
This is kind of the left-brained alternative to #1 above. Basically, you can create a spreadsheet scorecard, where you:
list all of the risks
assign each a weighting from 1 to 5 for both likelihood and impact
multiply the weights together
reverse sort by the highest number going down
The outcome is that you have 1 specific risk you need to address and start focusing your experimentation on.
Here is a more detailed writeup of how to do this. Or here.
6. Pre-Mortem
Imagine your business failed 4 years into the future. You meet with your cofounders over beers or tea. As your tears flow, and drip on your Business Model Canvas (BMC), you discuss what you wish you would have known up front.
Which is the likeliest to have caused the failure?
Write down all possible additional reasons you come up with onto post-its.
After a general session, you can also think through any assumptions in each box of the BMC which might be relevant for this particular business.
Am I going after the right customer–for me or my company? Is it a strategic fit?
Can I reach my customer cost-effectively?
Is this customer type willing to do anything about the problem? Because if they aren’t, they aren’t likely to spend any money.
So for example, it’s a good idea to build a business around a group of people you can easily relate to and who you like. I tried building a business in the weight loss industry despite not having a medical background or being a middle-aged woman. So while I did have a signature success story in this context, my marketing would always be less effective than if either of those requirements were fulfilled. And I wasn’t keen on going back to school to become a doctor first.
All of these do matter in certain contexts. And they’re probably good “spot check” questions anyway, before you proceed with a specific idea.
8. Talk to industry experts
Once I get outside of the internet and software worlds, there is a lot I don’t know. Lately, I’ve been working with AgriTech and MedTech startups, for example. And while I know the concepts behind innovation, I honestly don’t know have that much experience in how they apply to those sectors.
For example, agriculture has a “crossing the chasm” like curve. However, it’s heavily skewed to the right. So basically, there are many more people in the late majority or even the laggards. In other words it takes a lot longer for a technology to diffuse. But when it does, it’s everywhere.
negatively skewed tech adoption curve
Anything related to medicine has to go through regulatory approval. In this case, you can’t just pooh pooh the regulators like various marketplace startups did (Uber, AirBnB). People’s lives and health are potentially at stake. In practice, for founder this means it takes a decade before you can go to market. Which means two things:
you need a lot more money up front
you have less attempts at building a business during a founder’s lifetime
In short, these industry specific details significantly change how you’d look to innovate in these areas. And I learned both of these points from experts with decades of expertise in the industries (thanks Huw and Anthony).
From a risk management perspective, expert’s ideas are simply ideas that require testing. You don’t want to ask experts for their opinions and treat it as gospel either. As that would mean you replicate their biases. Gather your own data and proof. But at least you know what to pay attention to.
9. Red team, Blue team
Finally, to stress test your business model, you can use this approach that the British military came up with. This is a strategic exercise, to try to identify any holes or oversights in a plan. The red team tries to penetrate a well designed defense plan.
Essentially nominate a few well-meaning friends.
As them to “red team” your current business model.
Looking together at the current BMC together with your “blue team” consisting of the co-founders or delivery team, ask the “red team” to identify any major assumptions you might have missed. Holes that need to be covered. Things which could easily go wrong and derail your product launch.
There was a good example of this in the HBO serial Newsroom. The news team was working on a high stakes story. Certain people were intentionally kept in the dark about the big scoop. When the team producing the story was ready to go live, they assembled the people who had been intentionally left out to form a Red Team. The Red Team’s job was to poke holes in the story, as the paper was risking its reputation by going live with this story. People kept in the dark are more objective. They aren’t invested in the story, so they see it from a different perspective.
What you are after
Basically the goal is to uncover as many assumptions as possible. And to prioritize them. So you have 1 specific assumption to start testing. This focuses your experimentation on company killers and existential risk, the most meaningful place to start managing risk. This is, in fact, what successful founders like Branson excel at–like when he put up a cardboard sign and started confirming there was enough of a market to create a challenger airline.
If you can take smart risks while being better at managing them, then you are much more likely to do better over the long run. Any particular risk might still go sour. But this ability to manage risk will ultimately define your career as a startup founder and business builder.
Branson as an expert at managing the risk of getting punched in the face professionally
“Jed’s dead. Dead dead.”
hitting a low point somewhere during 67 customer interviews in one weekend
survey of product managers
looking for well ordered market segments like this
Channel testing example: selling superior wine into supermarkets
negatively skewed tech adoption curve