Tuesday, April 14, 2009

Validated learning about customers

Would you rather have $30,000 or $1 million in revenues for your startup? Sounds like a no-brainer, but I’d like to try and convince you that it’s not. All things being equal, of course, you’d rather have more revenue rather than less. But all things are never equal. In an early-stage startup especially, revenue is not an important goal in and of itself.

This may sound crazy, coming as it does from an advocate of charging customers for your product from day one. I have counseled innumerable entrepreneurs to change their focus to revenue, and many companies who refuse this advice get themselves into trouble by running out of iterations. And yet revenue alone is not a sufficient goal. Focusing on it exclusively can lead to failure as surely as ignoring it altogether.

Let’s start with a simple question: why do early-stage startups want revenue? We all know why big companies want revenue – it’s one of two critical halves of the formula for profit. And big companies exist to maximize profit. Don’t startups exist for the same reason? I think such reasoning is an example of the “startup dollhouse fallacy” – that startups are just shrunken-down big companies. In fact, I don’t think revenue is in and of itself a goal for startups, and neither is profit. What matters is proving the viability of the company’s business model, what investors call “traction.” Demonstrating traction is the true purpose of revenue in an early growth company. (Of course this is not at all true of many profitable small businesses, but they are not what I mean by startups.) Before I explain what I mean, let me add an important caveat: traction is not just important for investors. It should be even more important to the founders themselves, because it demonstrates that their business hypothesis is grounded in reality. More on that in a moment.

Consider this company (as always, a fictionalized composite): they have a million dollars of revenue, and are showing growth quarter after quarter. And yet, their investors are frustrated. Every board meeting, the metrics of success change. Their product definition fluctuates wildly – one month, it’s a dessert topping, the next it’s a floor wax. Their product development team is hard at work on a next-generation product platform, which is designed to offer a new suite of products – but this effort is months behind schedule. In fact, this company hasn’t shipped any new products in months. And yet their numbers continue to grow, month after month. What’s going on?

In my consulting practice, I sometimes have the opportunity to work with companies like this. Diagnosis is easy: they are exceptionally gifted salesmen. This is an incredible skill, one that most engineers overlook. True salesmen are artists, able to hone in on just those key words, phrases, features, and benefits that will convince another human being to give up their hard-earned money in exchange for even an early product. For a startup, having great sales DNA is a wonderful asset. But in this kind of situation, it can devour the company’s future.

The problem stems from selling each customer a custom one-time product. This is the magic of sales: by learning about each customer in-depth, they can convince each of them that this product would solve serious problems. That leads to cashing many checks. Now, in some situations, this over-selling would lead to a secondary problem, namely, that customers would realize they had been duped and refuse to re-subscribe. But here’s where a truly great sales artist comes in. Customers don’t usually mind a bait-and-switch if the switched-to product really does solve an important problem for them. These salesmen used their insight into what their customers really needed to make the sale and then deliver something of even greater value. They are closing orders. They are gaining valuable customer data. They are close to breakeven. What’s the problem?

This approach is fundamentally non-scalable. These founders have not managed, to borrow a phrase from Steve Blank, to create a scalable and repeatable sales process. Every sale requires handholding and personal attention from the founders themselves. This process cannot be delegated, because it’s impossible to explain to a normal person what’s involved in making the sale. The founders have a lethal combination of insight about what potential customers want and in-depth knowledge about what their current product can really deliver. As a result, potential customers are being turned away; they can only afford to engage with the customers that are best qualified.

And what of the product development team? They are busy too, but they are not creating value for the company. They are trying to build a product to an ever-changing spec, based on intuitions from the founders about what might be able to sell itself. Worse, the founders are never around – they are too busy going out and selling! Without access to customer data, or even a clear product owner, the product development team keeps building feature after feature based on what they think might be useful. But since nobody in the company can clearly articulate what the product is, their efforts result in incoherence. Worst of all, their next-generation product is so bad they are not allowed to try it out on any customers. The team is thus completely starved of any form of external feedback.

Let me describe a different company, one with only $30,000 in revenue (again, pure fiction). This company has a large long-term vision, but their current product is only a fraction of what they hope to build. Compared to the million-dollar startup, they are operating at micro-scale. How does that stack up?

First of all, they are not selling their product by hand. Instead, each potential customer has to go through a self-serve process of signing up and paying money. Because they have no presence in the market, they have to find distribution channels to bring in customers. They can only afford those (like Google AdWords) that support buying in small volume.

Compensating for these limitations is the fact that they know each of their customers extremely well, and they are constantly experimenting with new product features and product marketing to increase the yield on each new crop of customers they bring in. Over time, they have found a formula for acquiring, qualifying, and selling customers in the market segments they have targeted. Most importantly, they have lots of data about the unit economics of their business. They know how much it costs to bring in a customer and they know how much money they can expect to make on each one.

In other words, they have learned to grow renewable audiences. Given the data they’ve collected about these early customers, they are also able to estimate with modest precision how big the market is for their product in its current form. They may be at micro-scale now, but they are in a very good position to raise venture money and engage in extremely rapid growth.

Our million-dollar startup, by contrast, is stuck in the mud.

Stories like these are what has led me to this definition of progress for a startup: validated learning about customers. (Steve calls this just Customer Validation, but I like to emphasize the learning aspect, so I accept a far more awkward phrase.)

This unit of progress is remarkable in several ways. First of all, it means that most aggregate measures of success, like total revenue, are not very useful. They don’t tell us the key things we need to know about the business: how profitable is it on a per-customer basis? What’s the total available market? What’s the ROI on acquiring new customers? And how do existing customers respond to our product over time?

Secondly, this definition locates progress firmly in the heads of the people inside the company, and not in any artifacts the company produces. That’s why none of dollars, milestones, products or code can count as progress. Given a choice between what a successful team has learned and the source code they have produced, I would take the learning every time. This is why companies often get out-competed by former employees (Palm vs Handspring to name just one), even though the upstart lacks all of the familiar resources, tools, processes, and support they used to have. (Incidentally, it’s also why these upstarts often get sued for bogus reasons. Companies can’t believe they didn’t steal any of their “precious” assets.)

But learning is a tricky thing to quantify, which is why the word “validated” is so important in this definition. Validation comes in the form of data that demonstrates that the key risks in the business have been addressed by the current product. That doesn’t always mean revenue, either. Some products have relatively obvious monetization mechanisms, and the real risks are in customer adoption. Products can find sources of validation with impressive stats along a number of dimensions, such as high engagement, viral coefficient, or long-term retention. What’s important is that the data tell a compelling story, one that demonstrates that the business is on a solid economic footing. (It being so easy to convince yourself that you’re in one of these “special case” businesses, I do recommend you give revenue a long, hard look first.)

For example, I’ve talked a few times about how IMVU raised its first venture round with monthly revenues of around $10,000. This wasn’t very impressive, but we had two things going for us:
  1. A hockey stick shaped growth curve. People often forget the most important part of the hockey stick: the long flat part. We had months of data that showed customers more-or-less uninterested in our product. We were limping along at a few hundred dollars a month in revenue. All this time, we were continuously changing our product, talking to customers, trying to improve on our AdWords spend. Eventually, these efforts bore fruit – and this was evident in the data. This lent our claims about learning and discovery credibility.

  2. Compelling per-customer economics. We had only a small number of customers – if memory serves, only a few thousand active users. But a little math will show that we were making over a dollar per-user per-month. Our cost to acquire a customer on AdWords was only a few cents. Our eventual VC’s were quick to grasp what this meant (in fact, they understood it better than we did): that if our product achieved significant scale, it would be wildly profitable.
These two aspects could be plotted on one simple graph, which tells this equally simple story: if there is a market out there for this kind of product, we are the team that will find it and profit from it. That turned out to be a compelling investment thesis, despite our micro-scale results.

Let’s return to my example of the million-dollar-revenue company. If you find yourself in this kind of situation, what can you do? I’d suggest a few things, each rooted in the idea of breaking down the wall between the two halves of this company.
  1. Go on an agile diet quickly. With a product development team that is not shipping, any agile methodology will surface major problems quickly. Force anyone who is in customer contact to take the role of the Product Owner and insist that they deliver something new on a short regular interval.

  2. Get product into customers’ hands. The sales strategy currently leaves many customers completely un-served (those that don’t qualify for the founders’ personal time). Start using some of those customers as guinea pigs for a self-serve version of the product. Even if the product is absolutely terrible, it will establish a baseline against which the product development team can try and improve.

  3. Build tools for the sales team that reduce the time investment required for each sale. Instead of devoting all product development efforts to building a full-blown product, try building just those parts of the product that would allow the current sales process to go a little faster. For example, could we develop a simple landing page that would allow customers to pre-qualify for sales time? Iterating on these kinds of features has two benefits: it frees up time for the founders and simultaneously starts getting building a feedback loop with the product development team. Pretty soon, the text on that landing page is going to become an effective explanation for what the product does, because if it’s not the salesman will have to spend time re-explaining the product to potential customers. Time-to-complete-a-sale is not a bad metric for validated learning at this stage.
This last point is especially important. Although this kind of team may understand their customers well, they don’t yet know how to talk to them in a standardized way. Without that, they probably won’t achieve significant scale. (For more on how this plays into the process of scaling up, see the Customer Creation stage of the customer development model.) Perhaps they’ll be able to hire someone especially skilled in the marketing skills needed to find this positioning. But in the meantime, by iterating on their product with customers, they have a chance to get there on their own.


  1. Great post!

    I've seen this dynamic in so many product-based companies.

    You do a great job articulating.

  2. Great post! I've definitely felt some of this 'pain' in our startup.
    We had reached a decent level of revenue but it was highly dependable on the founders selling and delivering it. We just couldn't scale, so recently we ended up downsizing the company in order to find the Product/Market fit. (We're still going through this learning period)
    Your last tip is going to be my takeaway from this post.

  3. extraneous SHIMMER reference spotted; you are dating yourself eric... that's SNL Jan 1976:

  4. Great Post on Start Ups. These are very important points. And will lead to longer term success.

  5. This is possibly the most insightful startup article I've read in a while. People talk about funding funding funding, bootstrapping, etc., but few step back and really examine the underlying assumptions of startup finance.

    So here's the question that I pose to you - our startup helps software companies develop a scalable revenue engine in the form of software-specific hardware devices. However, its not that the scalability problems disappear, its just that we take them on for the client. Since we're offering a complete solution, which involves the development of a new product, there is design, prototyping, etc., involved before we go into production.

    Do you have any thoughts on scalable customer acquisition in cases where a decent bit of knowledge work has to be invested into each customer? This is a puzzle I've been actively working on solving.

  6. Transitioning from founder sold bootstrapped revenue to a functioning sales team is certainly possible. The greatest trait for an entrepreneurial founder is adaptability, so this is fundamentally no different from many other challenges that they face.

  7. @dave - glad somebody noticed. It's called cultural literacy, not dating. Right.

    @Alex - that sounds like an especially good case for Customer Development. You really should read Steve's book: http://bit.ly/FourSteps

    The short version is, you need to build up an in-depth mental model of a prospective customer. If you're still very early stage, then you need to know what a true early adopter looks like. Regardless, this model should tell you at least three things:

    1. Within the companies that you sell, who is the decision maker for this kind of product? Who's the economic buyer, the end user, potential saboteurs?

    2. "A day in the life" of the customer - in particular the decision maker or economic buyer. How is that picture changed before and after your product.

    3. The selling order. What's the right set of materials, conversations, etc. that completes a sale for these customers? Steve likens this to a map: first go here, then go here, etc.

    The whole point of this exercise is that, once you have this level of understanding, you can use it to train full-time salespeople who can simply execute your sales strategy. That's quite scalable, assuming that the market is large and the revenue per sale is also high.


  8. Eric, I have Steve's book and wish I had it for 1 couple of my earlier startups where the framework would have helped our team get on the same page.

    In these examples I was selling enterprise software where the vocab matched perfectly. Now that I'm in a web based company I'm finding a difficult match the framework, culturally. I get it personally and can see how it applies cause I've lived the pain but today there's so much "just try it, its cheep to get started" mentality that its difficult to get people to think about this level of planing when your customers are less well defined. Perhaps thats a fallacy?

    In short: I'd love to read your take on a web version of customer development.

    Cheers, and wish I hit w2e this year. Whould have loved to grab a coffee/beer.

  9. This comment has been removed by the author.

  10. Eric,

    Thanks for a great article. It struck a chord when you talked about companies that go down the customization route in return for greater revenues and how they face future problems when it comes to scalability. Currently we are in that exact position; however, because we are a very early stage startup we decided to accept a temporary strategy of accepting customization requests from clients until we reach ramen profitability. Then we could focus on standardizing a product that could have an automated sales cycle online. Would like to hear your thoughts and if there are any pitfalls to such an approach.

  11. Fantastic post. At present would represent the $30,000 revenue company. We are going through those very processes you mention. They key point which relates to our current situation is understand customers extremely well. So far the we have learnt a hell of alot about our customers that we REALLY did not know before. Looking forward to more of your posts, this blog is a favorite of ours right now!

  12. Great post! The part about scattering efforts really struck a cord with me, since it seems people are trying to find new things to sell, instead of sticking with one thing and letting it grow and improve organically.

    Very thorough and informative post. I appreciate it!

  13. Great post!! Very well narrated with clear illustrations.

  14. The key to making progress as a startup is validated learning about customers; defining the customer both in terms of interests and also metrics. It is better to be well defined and small than large and unsure of your customer.

  15. For start-ups with friends and family funding, the pressure to use revenue as the most important measure of success can be almost overwhelming. Thank you for your simple, insightful explanation of why that is the wrong metric to use.

    Thanks to Twitter, I just discovered your blog. You'll be on my regular list now.