First test, then invest

During trainings and workshops, I often tell about the Zappos example to explain the concepts of hypotheses and cheap validation. How many pairs of shoes did Zappos have in stock when they first launched their web site? Zero.

One time, after the Zappos example, I got an interesting question, showing how hard it is to grasp the essence of hypothesis discovery and validation.

“I understand the example, but for Zappos, it’s easy: they were working with an existing product, they didn’t have to build the product anymore. In our case, the product was new, so we had to build it in order to test it. We were able to attract a lot of users, which resulted in thousands of app downloads. But already during setup of the service, many of them dropped out. It has been a big investment that in the end gave us almost no return. It has cost us a lot of money. How could the Zappos approach have prevented this?”

Identifying assumptions

Now, there are two important types of assumptions you want to identify and validate for any idea:

• Are we solving a relevant problem?
• Are people willing to use our solution?

So I check the first question: what is the key problem you are solving? They had been developing a personal financial management service that gave customers very specific advice on how they could save money, based on their transaction history. The problem was quite well defined and the solution had some specifics that indeed did not exist elsewhere. Customer interest had even been validated through focus groups and surveys, which had shown a genuine interest in the service. The interest, however, had not turned into sales.

Validating assumptions

Then I made a suggestion. What if we would invite some of these interested users and perform the service together with them? We would ask to receive their transaction history, perform some manual analysis off line, and, in a face to face conversation, we would explain our advices and learn from the customers’ reactions.

The answer held the punch line:

“People will not participate in the test because they are not willing to share their financial transactions with us.”

That’s your validation, without a single line of code.

Some observations

Do you really know your customer’s behaviour?

Lack of problem relevance is probably the most important reason for failure. In 42% of startup failures, the main reason for failure is they offer a solution for a problem that does not exist, or is not big enough for potential customers to worry about. Even if there is a genuine interest in solving the problem, that interest often does not turn into actual usage: willingness is not adoption. Reasons can be:

• The problem is not that important after all.
• Something in the solution blocks potential users from becoming real users. This was the case in the personal financial management example.

Facilitate internally, evaluate externally

When you are facilitating the hypotheses discovery and validation process, it’s important not to emphasise your opinion about the problem and solution. You are responsible for ensuring a very clear problem definition and statement of the key parts of the solution that will help the user solve his problem. Your opinion about whether it’s a good idea or not, is just that: your opinion. It’s one out of about 7.5 billion opinions, so be modest.

The real evaluation of the viability of an idea should be done outside, with real customers.

Identifying assumptions: find the unknown unknowns

Identifying assumptions is a key part of the validation process. With the right approach, we can come up with quite a few relevant assumptions up front, but the real contact with your potential customer will always bring new assumptions to the surface. This is where walking the process manually is such a valuable technique that usually does not cost a lot of money. When walking the process, don’t just validate the assumptions you stated up front, but make sure you are open to discover new assumptions as well. These hidden assumptions are often the most risky ones.

MVP: are you building a product or performing a test?

Once the first set of assumptions has been defined, people often start thinking about their first MVP. They quickly focus on the viable product, with minimal often being shabbily treated. At this early stage, you’re not trying to build a product, you’re trying to learn about your customer’s problem. So I always challenge people to come up with a validation for their assumption that requires no development at all. Can we validate for free? Of course, the validation still needs to be relevant, but it’s a good way to force people to be creative in their validation approach.

References

• This article is one of the cases in Johan Merckx‘ book “Value creation for the digital world“. The book offers both inspiration and practical advice on how to become better at value creation in your organisation, and includes several real life cases. You can attend Johan’s keynote on May 16 2017 to find out more.
• The Zappos example is from Eric Ries’ Lean startup book
• The Top 20 Reasons Startups Fail lists “No market need” as the number one reason.

Acknowledgements

• Photo by Negative Space on StockSnap.io.