Are you making this common split testing snafu?

In case you didn’t know, SNAFU is a military slang acronym meaning “Situation Normal: All Fouled Up.”

More and more, I hear founders saying that they’re working on a product, who’re concerned whether or not there will be any demand when they launch. I’ve seen it come up in surveys, in youtube videos, even during personal conversations with founders.

After all, there’s more and more noise out there. We’re all getting overstimulated by hundreds of marketing messages and notifications per day. The more we do, the less relevant this messaging is.

How do you find unmet needs, to position your product effectively? A critical piece of building the right product is to address latent demand. Often, demand exists. It just isn’t immediately obvious where.

There is a hint in the book Lean Startup. First you need to prove what Eric Reis calls the “value hypothesis”, before you try to go after the “growth hypothesis”.

The value hypothesis claims that “the product delivers value to customers once they are using it.” This requires a deep understanding of:

  • who your customer is
  • what problems they have
  • how big of a niche they are
  • how much demand is growing
  • and most importantly….why they would buy from you specifically.

in addition to number of other factors, if you want to build a business and not just launch a product. If you aren’t sure that your customer needs what you plan to provide, then optimizing or scaling up sales is a waste of time and money.

So for example, split testing is a form of optimization of a message that already works. Running a split test before before you know that your product is attractive and valuable to your prospect is pointless. Yet many newbie founders have this belief from I don’t know where that split testing is what they should be doing.

Nowadays, to that I say…BS.

Here’s what you can and should do before you even reach for split testing or conversion optimization.

List out your assumptions, so that you can prioritize them in the order of riskiness. See if you can or want to de-risk the launch even at this stage:

About to start a greenfield project?

Have Launch Tomorrow run an in-house "riskiest assumption workshop". Remote delivery options also available. Discover where to prioritize your validation efforts, to get to market fast.

Contact Us

or call us now at:

US/Canada: +1 202 949 4478
UK: +44 773 952 7708
EU: +48 692 870 297

  • An easy way to identify riskiest this is to work backwards from an imagined failure of the project. You’re having a chat with a close friend or collegue over beers, and you say it would have worked if it hadn’t been for X.
  • Once you generate a list of these, say 20 different ones, then prioritize them in a google docs spreadsheet.
  • Once you have this, start with the riskiest one and think of how you can test this assumption. With advertising, it’s often quite easy and quick, because you have direct access to your audience.

Apologies if this is a bit pessimistic, but it’s a powerful method to both plan a product and learn about a new market. As a side benefit, once you’ve tested the assumptions, you confront and abolish any big worries you have. It’s also completely specific to your product idea. It doesn’t matter if it worked for anyone else in any market. You generate quantitative proof of your assumptions are valid or not before proceeding.

Oh, and by the way, be prepared that a few of your big assumptions won’t be right. That’s the usual experience of most entrepreneurs with this approach. In fairness, these false assumptions are the assumptions you want to learn about as soon as possible. You don’t want make decisions still thinking they’re true.

Create an explainer video for your complicated new product. Make sure your audience understands it, without being overwhelmed by technical details.

or call us now at:

US/Canada: +1 202 949 4478
UK: +44 773 952 7708
EU: +48 692 870 297

Once you’re done that, there’s another exercise worth going into. Testing the your value proposition in your ads and your landing page MVP, so that you know what speaks to your market. People get hung up on what value prop is, but it’s quite simple. It’s the reason someone buys from you. Their (nor your) reason why. Of course, only test value props that you’d be comfortable selling and building.

See, this is where the MVP naysayers have it wrong. A landing page MVP isn’t a smoke test. If you’re using it to validate yourself and tell you how smart you are, your mirror will be more effective. And cheaper. And you’ll learn just as much.

Test your value props.

The fastest path to identifying your value prop with advertising? I thought you’d never ask.

Why, Launch Tomorrow of course. My book Launch Tomorrow delves into exactly how to do that with paid advertising and a landing page MVP.

Is premature optimization a warranted concern?

Here’s a wonderfully geeky question from a forum I’m on, related to lean startup:

I have a product that I want to test. I’ve built the one-pager website and set up some Facebook advertising to figure out the messaging / features that makes people sign up to determine the direction I will eventually build towards.

I have three ads: A, B, A+B
and three website versions: A, B, A+B

My question is whether I should test the advertising AND the website versions at the same time. I want to test quickly, but I’m worried that if someone clicks advertisement A and then lands on website B, it won’t be relevant and it will skew my data.

Is this a warranted concern? Should I test the advertising first, then the website version? Or should I do both and just figure that, with enough users, the data will smooth itself out faster then if I test one at a time? Any insight would be greatly appreciated.

premature6

premature6

Couple of things:

  1. Yes it is a warranted concern. When you send traffic from ad A to landing page B, you lose them. You can set up ad A to point to landing page A, ad B to landing page B, and so on. Then by rotating through all ads, you’ll implicitly rotate through all of your landing pages.This “scent” which a prospect follows is really important. You may be losing people. They land on a LP which doesn’t correspond to their post-click expectations. This may skew your results.The goal here isn’t necessarily to convert, i.e. get a lot of signups. It’s to measure user behavior and preferences. Discover where you have latent/unmet demand.
  2. The right test depends on what you’re trying to learn and what your hypothesis is exactly. For example, why are you even split testing in the first place? If you start split testing too early, it ends up being premature optimization. Understanding why you are split testing will help you formulate a meaningful hypothesis. Split testing only tells you if option A is relatively better than option B.You’re better off first choosing a threshold for conversion, then running ads to see if you hit that threshold. It’s a different type of statistical test and way of thinking about the problem. Then you’re also learning more about demand.
  3. You will achieve statistical significance faster focusing only on user behavior around ads. Clicks. Based on that you can “test before you test”, and find out what larger/full tests are worth doing in a full environment.

As a rule of thumb, understanding your users better will usually be more profitable than understanding your product idea better.

If you want to go into greater depth about using landing pages rigorously to learn about your market, then check out Launch Tomorrow. It clears up a lot of the confusion around new products and landing page MVPs. Why blunder your way through? Launch Tomorrow‘ll take care of ya.

38 different ways to prove your case

While ideally you have some sort of proof direct in your headline and ad, your persuasiveness argument relies on how well you prove your point. You see, it’s ultimately about belief and feeling.

As heavy hitter Gary Bencivenga says:

Almost everyone in the world, in every field of human endeavor, is desperately searching for someone to believe in. Be that person and you can write your own ticket. Belief is today’s most overlooked yet most powerful key to boosting response to any ad, in any medium. Harness it and you unleash the core atomic power for exploding response.

Most prospects want to believe the claims you make in a landing page, yet the claims challenge their world-view and the status quo. You need proof, ideally proof that resonates emotionally, in order to get them to take action.

A landing page, or a salesletter, is like a one-to-one conversation between you and the prospect. You put various things on the landing page, designed to instill a particular reaction in the reader’s mind.

A good landing page is written in a conversational tone. Short words, short sentences, short paragraphs. In fact, you can read it out loud to ensure that the text “flows” well. prove your case

A good landing page is written in a conversational tone. Short words, short sentences, short paragraphs. In fact, you can read it out loud to ensure that the text “flows” well. prove your case

Imagine it as a phone conversation with a friend. They call you. They bring up a problem they’re struggling with. You say something surprising. You empathize with their pain. You talk about an approach you’ve used in the past or a product you can recommend to address it, as you know it will help them out. At the end, you help them buy the product or implement a solution in their lives. Empathize with your reader in the same way you’d empathize with that friend on the other end of the line.

Direct response progenitor Eugene Schwartz puts it well:

It is the facts that the prospect believes in and accepts, and the way that he passes that acceptance along from one fact to another, that determines the ad’s development, the arrangement of your claims and your images and your proofs, so that there is a step-by-step strengthening, not only of your prospect’s desire but of his conviction that the satisfaction of that desire will come true through your product.

You are building up the emotional weight of your argument as much as you can. You want the solution to become real in the prospect’s mind.

When you are making claims about the benefits your product has, your prospect is likely to not believe a claim that you make. It’s that “yeah, right” knee-jerk response. On the phone, you might be able to tell based on voice tone. Some prospects might tell you outright that they don’t believe you.

Proof counters that pushback. It’s your job, as a product creator or founder, to provide strong counter-arguments to this type of objection. In other words, your copy explicitly addresses the prospect’s objections. Show exactly how your solution can solve his problem. Or hers.

Well, the best type of proof is a poignant detail that knocks out a line of questioning or thinking. That’s why direct response copy that sells is clear.

About to start a greenfield project?

Have Launch Tomorrow run an in-house "riskiest assumption workshop". Remote delivery options also available. Discover where to prioritize your validation efforts, to get to market fast.

Contact Us

or call us now at:

US/Canada: +1 202 949 4478
UK: +44 773 952 7708
EU: +48 692 870 297

If you want to know what types of proof you can use, I’ve got your back. In its next update, Launch Tomorrow will include at least 38 different types of proof you can include on your landing page.

You can get a copy over here.

Create an explainer video for your complicated new product. Make sure your audience understands it, without being overwhelmed by technical details.

or call us now at:

US/Canada: +1 202 949 4478
UK: +44 773 952 7708
EU: +48 692 870 297

To be crystal clear, most of the 38 different types of proof don’t require you to even even have a customer, much less a success story.

Even on a landing page MVP, it all comes down to knowing how to present your product.

[image: typexnick]