test driven development | Launch Tomorrow

When developers start talking about test-driven development and automated testing, most product managers get antsy. They want to invest lots of time, without actually creating any new features. Oodles of cash disappearing, with seemingly nothing to show for it. Sounds like a terrible business idea.

Or is it?

By getting rid of waste in your product development, there are three main business reasons why automated acceptance testing will make you more money. Not only does acceptance test driven development (ATDD) help reduce really important structural costs and risks, automated tests bestow an existing product with a lot of sales and marketing mojo. A decent automated test suite helps the developers execute really fast on new ideas. Magic.

Enforcing well thought out specifications

First of all, a test can be used to enforce a specification. These specifications exist to be sure that the functionality isn’t changing, once it’s understood. Conversely, missing tests highlight missing understanding at the initial stage.

Historically, specifications have been written by reams of business analysts (BAs), negotiated, and nailed down as the ideal solution to a particular problem worth solving.

Your riskiest assumptions are probably related to your prospects and customers. Establish empathy quickly with your target prospect, figure out what's valuable, and get your innovation into the market.

The spec would be tossed over the wall. Developers then try to make sense of it all. They turn it into code. Then testers use those same specs to confirm (manually) that the specifications been hit. Sounds great in theory.

In practice, you play Chinese Whispers. What the client said isn’t quite what the BA’s heard, which isn’t quite what the developers heard, which then is not what the testers heard.

This is hard work, particularly if you aren’t really clear your big-picture business goals. If it’s not clear what the main goal was, then it’s very difficult to justify having the specification handed over in this way.

In contrast, if those initial specifications are turned into acceptance tests, you are forced to take apart this bigger vision into specific functions in the code. Once the developers write code which passes those tests, the tests are essentially enforcing the original scope as specified. By converting the specification into code, the original spec becomes immortal like the code itself. Acceptance tests are testing the code from the point of view of the client or end user, typically exercising code all the way through the system. Unlike unit tests, they aren’t testing individual methods or classes.

Whatever it is the customer wanted, automated acceptance tests confirm that that’s what they get. Each time the tests are run. There could be edge cases which were missed in the original specification, but if there wasn’t a test for it, then it wasn’t initially specified.

Alternatively, the acceptance tests can be modified or expanded, as you discover new requirements, but then that’s a very conscious decision. Chris Matts has previously pointed out that writing your specifications as automated acceptance tests increases the amount of communication on a team, which maximizes learning and the embedded real options in your development process. By minimizing the Chinese Whispers between BAs and the manual testers, you are much more likely to deliver what you expect, and everyone is clearer on what needs to be done.

About to start a greenfield project?

Have Launch Tomorrow run an in-house "riskiest assumption workshop". Remote delivery options also available. Discover where to prioritize your validation efforts, to get to market fast.

or call us now at:

US/Canada: +1 202 949 4478
UK: +44 773 952 7708
EU: +48 692 870 297

When you could use acceptance test driven development, you get a high level of certainty in the moment. You’re delivering what you expect. If you manage to build out your acceptance test suite, you genuinely verify the relevant edge cases. It’s much more likely that you’ll capture side effects of changes you didn’t consider earlier.

In the end, you also deliver a much higher quality product.

Create an explainer video for your complicated new product. Make sure your audience understands it, without being overwhelmed by technical details.

or call us now at:

US/Canada: +1 202 949 4478
UK: +44 773 952 7708
EU: +48 692 870 297

Tests reduce the cost of release

Second of all, releasing software, particularly when regression tests take a long time, can be quite problematic. Not only is the release itself quite a headache, it costs some extra time. But that’s not the real problem.

The real problem lies with unreleased features. If you have a big backlog of unreleased features, it’s like having a pile of a pile of boots without soles in a shoe factory. You can’t sell it, yet you’ve already bought the raw materials and started the work. Those boots have near zero value, because they aren’t done. The same thing happens with unreleased features. Any feature which is dev-done but not released is like a sole-less boot.

Typically, the cost of release is a big reason why companies don’t release half-finished software. What’s the fastest way to reduce that cost? Automated acceptance tests, and an automated release process. While it will take a lot of time and aspirin for the related headaches, having this in place can put an entire software business on a completely different footing (You got sole, baby!).

Moreover, with a regression test suite, you can have much great certainty that none of the changes in a release candidate have broken pre-existing functionality. With automated unit and acceptance testing, you can completely eliminate the regression test component of a release. As soon as the manual testing is done for a given new feature, and/or any new test for that feature, you’re ready to release. Your cost of release goes down significantly in terms of time, developer time and everything. And your turn around time is much, much faster.

Lowered lead times

Third of all, this big business benefit ties the above together. Typically, customers expect to wait a long time after requesting new software or a new feature. In many cases, they may be willing to pay for it. Even if not, it’s clear that this particular request would have a lot of value in their eyes.

There’s an easy way to measure and quantify this. Lead time is the amount of time between a customer’s request and when that customer actually gets what he wants. While a lot of effort goes into producing this, ultimately this is all the customer really cares about. They want to know what’s in it for them.

Yet, it’s inevitable that your lead time will grow as your code base gets larger. The more parts you have, the more interconnections you have. This grows exponentially, unless each part has it’s own set of tests.

Tests keep your lead time linear. They’re an insurance policy. If customers do request a particular feature, with relatively high level of confidence, you can give them what they want without breaking anything else. That new feature is contained behind a test suite, so you don’t need to worry your pretty little face about that.

Imagine a big company being as nimble as a startup. Most of the real life examples have famously large test suites. In addition to massive test suites, Google even wrote and open-sourced their own testing framework.

If you can minimize lead time, that tends to put a lot of upward pressure on revenues (that’s a good thing). At this point, we aren’t just talking about containing costs, we’re talking about earning more revenue. Generating new business. To be blunt, this is what all of your costs are subtracted from when calculating profits.

The investment that pays for itself

As with any business decision, deciding to build an automated test suite is an investment. Depending on the details, it will take your geeks some time to build this automated test thingy. There is a lot more to automated testing than well-intentioned “software craftsmanship”. At the same time, you can be sure to reap significant bottom line benefits in an existing software product business.

Even if users are clamoring for new features, dedicating some time to creating automated test suites will eventually make your product more profitable. The trick is to focus your testing efforts on areas where you expect to get high pay-offs. Prioritize these efforts rigorously, to make sure that this test suite pays for itself each time you release.