Empty pocket testing by Buffer

buffer launch tomorrow

Tweet or Posts on a social media at a pre-defined schedule

The following are a number of Lean Startup validation case studies. Some will already be well known; some will be completely unknown. A lot of landing page testing has happened since The Lean Startup was being pieced together by Eric Reis. These are retrospective reconstructions of what happened using landing pages as vehicles for minimum viable products.

For example, Buffer did empty pocket testing with a landing page before building their product. Just for your context, Buffer is a social media sharing tool, allowing you to publish tweets or social media posts on a pre-defined schedule.

While you may have heard of some of the lean landing page case studies before, there is a lot of nuance in exactly what each test actually tested. They are typically not “traditional” A/B split tests, where they were testing whether a new variation of an ad or landing page beat the old one.

In order to help make it more explicit, I’ve tried reformatting the experiments to be lightweight. Lean Startup experiments are generally not about testing the landing page or the product, but the business ideas they represent.

Hypothesis: The target audience wants this product

Test Type: Value hypothesis, confirm the problem exists and people want a “hands off” way to tweet

Success Criteria: Emails gathered > 0

Traffic Source: Social media, word of mouth

[image:Buffer]

Result: Pass. A few people used it to give founder Joel Gascoigne their email. He used these to get some useful feedback and initiate a conversation with prospects.

About to start a greenfield project?

Have Launch Tomorrow run an in-house "riskiest assumption workshop". Remote delivery options also available. Discover where to prioritize your validation efforts, to get to market fast.

Contact Us

or call us now at:

US/Canada: +1 202 949 4478
UK: +44 773 952 7708
EU: +48 692 870 297

Step back: Potential users had left their email address at a random web page promising them help with this particular problem. This meant the idea itself was valuable, and there was potentially unmet demand for this particular idea. I would be careful to use only # of emails gathered as the primary metric in all cases.

For a consumer facing product, this is probably good enough, assuming you have enough traffic. It would be better to also include some kind of a target number of sessions, to make sure that you have enough “attempts to convert” to make your metric meaningful.

Create an explainer video for your complicated new product. Make sure your audience understands it, without being overwhelmed by technical details.

or call us now at:

US/Canada: +1 202 949 4478
UK: +44 773 952 7708
EU: +48 692 870 297

Hypothesis: The target audience is willing to pay for the product

Test Type: Value hypothesis, confirm declared willingness to pay for “a way to automate their tweeting”

Success Criteria: People would click-through the additional pricing page, and still leave their email.

Traffic Source: Social media, word of mouth

buffer mvp launch tomorrow

Pricing packages of this social media tool

[image:Buffer]

Result: Pass. People were still clicking through this additional step. Joel was able to gather useful information about the suggested pricing plans, in order to figure out pricing.

Step back: Potential users weren’t put off by the blatant pitch, and still kept leaving an email address at the far end. What Joel hadn’t tested was whether people would actually buy; however, he was able to complete a functional prototype within seven weeks, and tested this hypothesis with a functional system. He actually got his first paying customer 4 days after the “rough-around-the-edges” product launch.

I just wanted to thank Joel for contributing this fantastic case study to the Lean Startup community. It’s quite a well known one. As a result, I really wanted to cover it as an example of a line of thinking that’s worth following.

Empty Pocket Testing

At the core, the Buffer landing page MVP test was meant to address a major question for founders: will “they” buy it, if I create it? Before getting caught up in theoretical debates about what is and isn’t an MVP, Buffer just did an experiment. It just happened to be using a landing page to address a major risk factor for a new product business.

In this particular case, checking for whether early adopters

  1. had the budget and
  2. were willing to spend it

de-risked spending more time and money on the solution significantly. Even though they were asking theoretically, this helped to validate their sense that their target early adopters would be willing to pay.

Buffer landing page MVP launch tomorrow

People are willing to pay out of their pockets for this tool.

[image:Dan Moyle]

If you’d like to see a number of case studies like the above, grab Launch Tomorrow. I’m updating it in an upcoming version with a lot of in-depth experiments that have been run.

“Beam me some money, Scotty!”

Palm pilot launch tomorrow

Palm Pilot beams money from one to another via infrared

In a previous life, Elon Musk was one of the main creators of PayPal. He’s pretty open about how the initial idea didn’t actually catch on very well. When he was doing customer interviews, he’d get that glazed-over look. Yeah, that one.

At the time, the team was approaching consumers. They were saying they aimed to “replace the US dollar with a new internet currency”. Remember this was 1999, almost a decade before Bitcoin existed.

Their product allowed people to beam money from one PalmPilot to another via infrared. Journalists voted this as one of the worst business ideas of 1999.

If you’ve never heard of them, PalmPilots were like early smartphones. Except they weren’t phones.

But then the customer development led to an epiphany. Customers loved a small sub-feature the the team built just to get the PalmPilots to sync:

“Sending money to an email address!”

In 1999, “sending money to an email address” resonated with people. Not just with PalmPilot owners. At a deep level. Once the Paypal guys slapped on a viral engine of growth, that feature became what PayPal is known for.

All of this shows how critical it is to have the right value proposition, when first pitching a product. Regardless of whether you present your product in initial conversations with customers. Or on a landing page MVP.

[image cred: Rama & Musée Bolo]

About to start a greenfield project?

Have Launch Tomorrow run an in-house "riskiest assumption workshop". Remote delivery options also available. Discover where to prioritize your validation efforts, to get to market fast.

Contact Us

or call us now at:

US/Canada: +1 202 949 4478
UK: +44 773 952 7708
EU: +48 692 870 297