Market Testing Velocity March 5, 2014 by LaunchTomorrow Leave a Comment Completing tests quickly will help you run multiple tests in succession Your success will be a function of the learning cycles you fit into your runway. — Tom Hulme, Ideo The best way to discover what makes your prospects tick is to run an experiment. If you are pressed for time, the best way to discover what makes them buy is to run many experiments over a very short time period. Assuming these two would cost exactly the same amount, which would be more valuable to you as a product manager? 1 test, which runs for one week, which gives you 99% certainty on your results 30 tests, which take one hour each, with 90% certainty each Clearly the second set of tests is much more valuable. When you finish the first test, you can form another hypothesis and test that. One set of results directly feeds into the rest. If you want to identify surprising results quickly, as this is most likely to give you outsized results, option B is a much better way to enter a market. To some extent, it’s reversing the iterative nature of agile, and applying it during pre-development when performing product research. Feedback loops are enormously powerful when dealing with prospects and customers too. “Market Testing Velocity” can mean a lot to you in a high-pressure situation if you want to get something going quickly. If you complete tests quickly, you can run multiple tests in succession. In this case, you are doing marketing research on your prospects or existing customers, in order to map out what they need. Your goal is to identify what they need most, a “bleeding neck”, as quickly as possible. You are trying to discover a cognitive and emotional map of their needs, in order to orient your own efforts most effectively. If this tickled your fancy, and you’d like to read more about market testing velocity, you’ll find out more in Launch Tomorrow. << Help Yo' Friends The Software Construction Metaphor is Broken July 23, 2013 by LaunchTomorrow July 23, 2013 by LaunchTomorrow DIY strategy in setting up an apartment. I dragged my feet up the cracked cement staircase. Next to the stairs, wild, thorny shrubs grew out of control. No one had bothered with them for years. A metallic pipe with black paint peeling off served as a fence. It was a snappy cold February day. As we got to the top of the staircase, two massive guys got out of a parked sedan with an engine humming to keep them warm. In black, furry sport jackets, they tumbled up the stairs, with stubby legs sticking out from underneath rather big beer bellies. We were meeting with a builder named Slawek. Slawek was the last entry in our “beauty contest” of building contractors. My wife and I had just bought our first apartment. We’d never made such a big purchase in our lives- ever. We had serious “skin in the game”. The housing market already started spectacularly nose-diving. We wrangled down the purchase price by quite a bit, relative to the area. The owner was desperate to sell an apartment he had rented out for over twenty years. It showed. Unfortunately, neither of us had any experience with DIY, other than hanging up a picture frame, much less construction work. At least, I knew “Project Management”. Moving in from remote parts, we had to have the renovations done well, and finished on time. Otherwise, we’d be paying a both a mortgage and rent. We also knew that we’d be living with any mistakes for a long, long time. Everything had to be redone (electricity, plumbing, floors, walls, ceilings-basically everything), in one month at most. The scope was staggering, at least for a small apartment. It was probably cheaper just to build a new one. We also wanted everything to be certified, and in accordance with the local regulations, since we know how important that would be when we’d be selling the place, based on recent experience as house-hunters. Most of Slawek’s peers had a sheer look of terror when we laid out the scope and timeline-they started losing nose hairs. In contrast, Slawek and his buddy just smiled. Slawek said, “Yes, sure, no problem. Three weeks should be enough, actually.” “Even with the regulatory certifications?” “Yes, sure” “Ok,” I said, “in that case, you won’t mind a penalty for lateness, if you run over one month,” I suggested. He flinched a bit. I thought giving Slawek a buffer was benevolent move. At minimum, I wouldn’t feel guilty about imposing a penalty. He had some space for risk-for the unexpected. At the same time, he had to know we were serious about not missing our deadline. “Yeah, one month. No problem,” he repeated. This is one cool cat, I thought. Deal! As the work progressed we visited every few days. Rather than actually being a builder, Slawek himself turned out to be a sales guy and an extremely talented project manager. While anything PC-related clearly overwhelmed him, as I had to help explain what an RTF file was, it didn’t stop him at all. He wouldn’t even install MS Project even if I paid him a few thousand extra. At the same time, he had an intuitive grasp on timing, i.e. when to curse out one of his hired hands to “motivate” him. In his next breath, he was back to dishing out highly technical advice to us, packaged as stellar customer service. I had severely underestimated how street smart the guy was. He was a real rainmaker. There was clear progress every single time we visited. This guy ate milestones for breakfast. After a few slips of the original schedule, the builders gave back the keys exactly one month after starting; taking full advantage of the buffer I had given them. The work was well done. Visually, the apartment was everything we had imagined it would be. We could move in. We were happy. All our boxes were ticked. A few months later, we were eating dinner at our dinky IKEA table, chatting away and…the room lights in the entire flat go out-all of the lights in every room, and many of the other lamps, including outdoor lighting. Dropping my fork mid-bite, I ran to the circuit breaker. One circuit had popped. I realized most of our lights were on one circuit, which I had missed during the handover from the builders. Not only that, the circuit breaker kept popping back when we tried to set it. There was a serious underlying design flaw in our electricity. I quickly realized the only way to fix it was to redo the wiring in the whole flat, but now-we actually lived in the apartment. It was painful thinking about it. I flipped out, and started looking for Slawek’s phone number. I couldn’t get through. At the time of the renovation, he had said he was planning on leaving the country soon. We only wanted to get the electrician’s phone number, but no luck. After scratching our heads, we realized we had received a certificate from the electrician. I desperately paged through old paperwork, and thrilled, I pulled out the flimsy-looking thing. All hope was not lost. To add insult to injury, I realized, only now, that the certificate looked photocopied-many times. There were some scribbles on the paper, and I could make out a phone number. A bit suspicious, I jumped online to find out more about the exact nature of the certificate. It looked like a forgery. Not only that, it definitely wasn’t the gold standard certificate we thought we had bought. He had given us a no-name certificate, and we hadn’t even noticed it. Of course, the electrician didn’t pick up the phone either, only responding to our queries with nasty text messages. To be honest, he had his cash, and he had no interest in providing us service “under warranty”, even if we paid for it. He knew exactly what the problem was, so he didn’t even want to try fixing it. While we could temporarily introduce a few floor lamps, we had expected the lights to work after a fresh rewiring of the entire flat. That was unacceptable. By my standards, the ultimate result was not functional. At the same time, I could only really blame myself, or to be more precise, the approach that I took. I had slyly slipped in a late penalty for delays. As a result, Slawek couldn’t let the electrician mess around any longer than the first week, if he wanted any chance of avoiding the penalty. I wanted to minimize my own risk, forcing them to take shortcuts. Those shortcuts ultimately “broke” the deliverable in the long term. I had no idea about the interdependencies among their tasks. I wanted them to manage it, and I just pressured them to do work faster. If the above was a software project, it would have been a massive success: • Everything in scope was delivered • Everything was delivered on time • To be honest, we actually got a pretty good price given the initially perceived quality • Didn’t need to use metrics • Didn’t need to use specialized project tracking and handoffs, as popularized by MS Project At the same time, the builders introduced a big, undetectable, architectural problem, which became a big, unpredictable, and nearly unfixable problem for the “users”. The main cause? The rush, the hurry-which I introduced as the moneybags. Of course I could be angry at this electrician, but I had insisted on finishing fast. As a result, when they had to cut corners, they did…because I had clearly defined the criteria they need to satisfy. When launching a new product, you are typically faced with the same problem. You want to deliver something quickly, in order to be first, in order to make a splash. You want first mover advantage. It has very concrete financial implications for you. In highly competitive markets, especially online markets, there are typically only a few large players. In the late nineties during the dot-com boom, as new business models were being introduced, many of the segments had three players, and a handful of also-rans which typically didn’t have more than 10% market share. Nowadays, it’s not much different. Whether you realize it or not, delay is usually the biggest cost you bear when launching a new product. It can mean the difference between a majority market share, and a minority market share for your product. You risk making fewer sales if your competition beats you to a market. Even if they have more bugs, they will already be earning cash. This is particularly true for startups. When there is big disruption, and you create a new segment, customers will only remember 3 three players in a segment at most. In most markets, according to Ries & Trout’s booklet the 22 Immutable Laws of Marketing, there is space for three players in your users’ mind. How many brands of toothpaste can you come up with, without checking a store? More poignantly, do you know who the second person to cross the Atlantic was, without checking Google or Wikipedia? (Hint, it wasn’t Charles Lindberg or Amelia Earhart). You are up against the limits of perception and memory, at least within your niche, and the clock’s started ticking. << Help Yo' Friends
The Software Construction Metaphor is Broken July 23, 2013 by LaunchTomorrow July 23, 2013 by LaunchTomorrow DIY strategy in setting up an apartment. I dragged my feet up the cracked cement staircase. Next to the stairs, wild, thorny shrubs grew out of control. No one had bothered with them for years. A metallic pipe with black paint peeling off served as a fence. It was a snappy cold February day. As we got to the top of the staircase, two massive guys got out of a parked sedan with an engine humming to keep them warm. In black, furry sport jackets, they tumbled up the stairs, with stubby legs sticking out from underneath rather big beer bellies. We were meeting with a builder named Slawek. Slawek was the last entry in our “beauty contest” of building contractors. My wife and I had just bought our first apartment. We’d never made such a big purchase in our lives- ever. We had serious “skin in the game”. The housing market already started spectacularly nose-diving. We wrangled down the purchase price by quite a bit, relative to the area. The owner was desperate to sell an apartment he had rented out for over twenty years. It showed. Unfortunately, neither of us had any experience with DIY, other than hanging up a picture frame, much less construction work. At least, I knew “Project Management”. Moving in from remote parts, we had to have the renovations done well, and finished on time. Otherwise, we’d be paying a both a mortgage and rent. We also knew that we’d be living with any mistakes for a long, long time. Everything had to be redone (electricity, plumbing, floors, walls, ceilings-basically everything), in one month at most. The scope was staggering, at least for a small apartment. It was probably cheaper just to build a new one. We also wanted everything to be certified, and in accordance with the local regulations, since we know how important that would be when we’d be selling the place, based on recent experience as house-hunters. Most of Slawek’s peers had a sheer look of terror when we laid out the scope and timeline-they started losing nose hairs. In contrast, Slawek and his buddy just smiled. Slawek said, “Yes, sure, no problem. Three weeks should be enough, actually.” “Even with the regulatory certifications?” “Yes, sure” “Ok,” I said, “in that case, you won’t mind a penalty for lateness, if you run over one month,” I suggested. He flinched a bit. I thought giving Slawek a buffer was benevolent move. At minimum, I wouldn’t feel guilty about imposing a penalty. He had some space for risk-for the unexpected. At the same time, he had to know we were serious about not missing our deadline. “Yeah, one month. No problem,” he repeated. This is one cool cat, I thought. Deal! As the work progressed we visited every few days. Rather than actually being a builder, Slawek himself turned out to be a sales guy and an extremely talented project manager. While anything PC-related clearly overwhelmed him, as I had to help explain what an RTF file was, it didn’t stop him at all. He wouldn’t even install MS Project even if I paid him a few thousand extra. At the same time, he had an intuitive grasp on timing, i.e. when to curse out one of his hired hands to “motivate” him. In his next breath, he was back to dishing out highly technical advice to us, packaged as stellar customer service. I had severely underestimated how street smart the guy was. He was a real rainmaker. There was clear progress every single time we visited. This guy ate milestones for breakfast. After a few slips of the original schedule, the builders gave back the keys exactly one month after starting; taking full advantage of the buffer I had given them. The work was well done. Visually, the apartment was everything we had imagined it would be. We could move in. We were happy. All our boxes were ticked. A few months later, we were eating dinner at our dinky IKEA table, chatting away and…the room lights in the entire flat go out-all of the lights in every room, and many of the other lamps, including outdoor lighting. Dropping my fork mid-bite, I ran to the circuit breaker. One circuit had popped. I realized most of our lights were on one circuit, which I had missed during the handover from the builders. Not only that, the circuit breaker kept popping back when we tried to set it. There was a serious underlying design flaw in our electricity. I quickly realized the only way to fix it was to redo the wiring in the whole flat, but now-we actually lived in the apartment. It was painful thinking about it. I flipped out, and started looking for Slawek’s phone number. I couldn’t get through. At the time of the renovation, he had said he was planning on leaving the country soon. We only wanted to get the electrician’s phone number, but no luck. After scratching our heads, we realized we had received a certificate from the electrician. I desperately paged through old paperwork, and thrilled, I pulled out the flimsy-looking thing. All hope was not lost. To add insult to injury, I realized, only now, that the certificate looked photocopied-many times. There were some scribbles on the paper, and I could make out a phone number. A bit suspicious, I jumped online to find out more about the exact nature of the certificate. It looked like a forgery. Not only that, it definitely wasn’t the gold standard certificate we thought we had bought. He had given us a no-name certificate, and we hadn’t even noticed it. Of course, the electrician didn’t pick up the phone either, only responding to our queries with nasty text messages. To be honest, he had his cash, and he had no interest in providing us service “under warranty”, even if we paid for it. He knew exactly what the problem was, so he didn’t even want to try fixing it. While we could temporarily introduce a few floor lamps, we had expected the lights to work after a fresh rewiring of the entire flat. That was unacceptable. By my standards, the ultimate result was not functional. At the same time, I could only really blame myself, or to be more precise, the approach that I took. I had slyly slipped in a late penalty for delays. As a result, Slawek couldn’t let the electrician mess around any longer than the first week, if he wanted any chance of avoiding the penalty. I wanted to minimize my own risk, forcing them to take shortcuts. Those shortcuts ultimately “broke” the deliverable in the long term. I had no idea about the interdependencies among their tasks. I wanted them to manage it, and I just pressured them to do work faster. If the above was a software project, it would have been a massive success: • Everything in scope was delivered • Everything was delivered on time • To be honest, we actually got a pretty good price given the initially perceived quality • Didn’t need to use metrics • Didn’t need to use specialized project tracking and handoffs, as popularized by MS Project At the same time, the builders introduced a big, undetectable, architectural problem, which became a big, unpredictable, and nearly unfixable problem for the “users”. The main cause? The rush, the hurry-which I introduced as the moneybags. Of course I could be angry at this electrician, but I had insisted on finishing fast. As a result, when they had to cut corners, they did…because I had clearly defined the criteria they need to satisfy. When launching a new product, you are typically faced with the same problem. You want to deliver something quickly, in order to be first, in order to make a splash. You want first mover advantage. It has very concrete financial implications for you. In highly competitive markets, especially online markets, there are typically only a few large players. In the late nineties during the dot-com boom, as new business models were being introduced, many of the segments had three players, and a handful of also-rans which typically didn’t have more than 10% market share. Nowadays, it’s not much different. Whether you realize it or not, delay is usually the biggest cost you bear when launching a new product. It can mean the difference between a majority market share, and a minority market share for your product. You risk making fewer sales if your competition beats you to a market. Even if they have more bugs, they will already be earning cash. This is particularly true for startups. When there is big disruption, and you create a new segment, customers will only remember 3 three players in a segment at most. In most markets, according to Ries & Trout’s booklet the 22 Immutable Laws of Marketing, there is space for three players in your users’ mind. How many brands of toothpaste can you come up with, without checking a store? More poignantly, do you know who the second person to cross the Atlantic was, without checking Google or Wikipedia? (Hint, it wasn’t Charles Lindberg or Amelia Earhart). You are up against the limits of perception and memory, at least within your niche, and the clock’s started ticking. << Help Yo' Friends
