Tuesday, March 31, 2009
My overwhelming takeaway from the most recent Lessons Learned survey is that there is tremendous hunger out there for in-depth opportunities for entrepreneurs of all stripes to develop their lean startup skills. As you can tell from recent posts, I've been experimenting with different formats and channels to try and find the best way to help. And today I am excited to announce a new experiment: the Lean Startup workshop.
I don't want to go into too much detail in this post, because (as usual) I want to invite anyone who is interested to take a new survey specifically about the workshop. It's designed to take less than ten minutes, and I appreciate those who are willing to donate their time in this way. Thank you.
lean startup, you'll know that I am trying to apply customer development principles. And if you know those principles, you may notice that I am trying to pass from the "customer discovery" phase to the "customer validation" phase. I won't say any more, for fear of ruining the data. But look for a future post on the topic using the data you all help me collect.
See you at the Expo!
NB: if you're already sold on the workshop and would like to pre-register, click here.
Friday, March 27, 2009
The full formula works like this:
# iterations = runway / speed of each iteration
Very few successful companies ended up in the same exact business that the founders thought they'd be in (see Founders at Work for dozens of examples). Some aspects of the vision remain, but often in a significantly modified form. Unfortunately for the current crop of founders, there is intense pressure to engage in selective memory when companies tell the story of their founding. Journalists, PR firms, investors, and the public at large love a scrappy come-from-behind story about two guys in a garage who figured out how to take down Goliath. But the story usually features visionary protagonists who had it figured out from the start. Explaining that half of what these visionaries thought in their early years bordered on delusion tends to be a hard sell.
These successful startups managed to have enough tries to get it right. As a member of a startup, your incentive is to do everything possible to maximize the number of iterations you have left. What counts as an iteration? I believe it is a full, company-wide turn through the OODA loop (for a software business, see especially Ideas-Code-Learn). Minor experiments and variations don't count, although they are helpful. The key is to be able to refute as many major hypotheses as you can. We're talking PayPal-sized variations.
To increase the number of iterations you have left, you can either increase cash on hand (by raising money or increasing revenues), reduce burn rate, or increase the speed of each iteration. The most powerful changes you can make affect the speed of iteration. Even more powerful are costs that currently slow you down – by cutting those you can get the double-whammy benefit of lowering burn and increasing speed.
The key is to examine every cost through the lens of its effect on each stage of your learning feedback loop. Even if it helps optimize one stage, if it slows you down somewhere else, it might not be a net win.
The hardest costs to cut are those that are embodied in sacred cows. For example: we have to have strong documentation for our internal API's, else we might not be able to have third parties use them in the future; we can't afford to ship a low-quality product to our customers, because it might diminish our startup's brand image; we can't charge for our product until it contains Essential Feature X; we have to have a dozen managers sign off on every proposed feature, to protect the integrity of our product. Each of these might be good policies, at least for one stage of the feedback loop. Once they attain sacred status, they are rarely questioned. A crisis is sometimes needed to force that reexamination. In fact, every single lean transformation documented in books like Lean Thinking took place in the midst of serious external threats.
You cannot become a lean startup by willpower alone, any more than you can lose weight by going on a willpower-based diet. You need a process for systematically reviewing your costs and eliminating those that slow you down. In fact, the essence of many of the practices I have learned is to take systematic advantage of the power of crisis to spur creative thinking. This is why I constantly stress the need to set specific, actionable targets for new product initiatives or new feature split-tests. You cannot learn if you cannot be wrong, and vague goals are exceptionally easy to rationalize as success.
So how many iterations do you have left? What could you do to squeeze out one more, just in case your current plans don't pan out? For some specific suggestions, I recommend:
- The fundamental startup feedback loop: Implement/Measure/Learn (a version of the OODA Loop).
- Customer Development: a disciplined approach to finding out if there is a market for your product before it's too late.
- Customer Development Engineering: techniques for accelerating your product development.
- Getting started with split-testing and the one-line split-test.
- A checklist of practices in the new Joel Test for Startups.
- Work in small batches.
Wednesday, March 25, 2009
In keeping with my recent theme of blogging about upcoming events, I'm happy to announce that I'll be speaking at Agile Vancouver on April 21. (For those of you who subscribe to the blog for substantive content and whatnot, I promise to get back to more regular posts soon. And when did there get to be 3000 of you? Thank you so much!)
Agile Vancouver: Lean Development for Lean TimesI'll be speaking about the lean startup in what I hope will be a more interactive setting than the usual lecture-with-slides. It will probably be the first time I'll be able to take extensive Q&A on some of the new frameworks I'll be debuting at Web 2.0 Expo. As always, if you decide to attend, please come say hello. If you can't come but have a burning question anyway, please leave it as a comment. I'll do my best to respond.
On Tuesday April 21st, we are hosting an afternoon event on the application of Lean Principles to software development. This workshop brings together leading thinkers from Lean Production and Lean software.
Please register to attend this event as space will be limited.
Tuesday, March 24, 2009
The metrics and levers of engagement, presentation on Engagement Loops for Facebook Developer Garage SF
I'll be presenting a talk at the Facebook Developer Garage SF Wednesday evening. You can learn more about the event here. It's hosted by Kontagent and sponsored by Intel. Most of the content for my presentation is drawn from my original article on engagement loops, with new diagrams courtsey of my friends at Kontagent and a few new examples. Without further ado, here are the slides (feedback welcome!):
When I work with startups on improving engagement, I really try to emphasize the importance of their most powerful lever: positioning. In most applications, most of the time, customers return to use the application not in response to a notification or event, but under their own volition. These decisions are critical to the success of any high-engagement product, and they take place entirely inside the minds of customers. Companies have an opportunity to influence these decisions, but only if they act well in advance of the result they are trying to achieve. Unfortunately, it's easy to lose track of positioning effects when optimizing for a single metric.
This is a common problem that results from viral-loop optimization. By copying the exact same registration flow as every other successful viral app, many viral apps completely lose their positioning. Customers can't even remember what apps they've signed up for, and become entirely dependent on notifications to bring them back. This starts a downward spiral: as more and more apps become indistinguishable, they send out more and more notifications, which leads to increasing fatigue on the part of customers. As notification channels get stuffed full of these messages, customers tune them out (or platforms have to put in place dramatic limits on access).
The solution for app developers caught in this vicious cycle is to develop competency in positioning. Luckily, a great example of the power of positioning fell into my lap recently. Some friends of mine at EA tried to break my World of Warcraft addiction by walking a copy of Warhammer Online over to my house. I dutifully installed it and played with them for a little while. But pretty soon the lure of WoW dragged me back, even though some of my friends stayed behind on Warhammer.
A few days ago, I received a great email from Warhammer Online. It's an example of an excellent synthetic notification. Take a look at this screenshot:
This synthetic notification gets everything right: it has a compelling offer and a clear call to action, it addresses me by my character's name, it's from a well-known NPC inside the game, it even includes the names of several friends who are still playing, and calls me to act on behalf of my guild (yes, it's called GUID). It then proceeds to list a whole host of cool new features the Warhammer team has added since I last logged in. Impressive.
Unfortunately, it didn't work, at least not for me. That's because I have much too strong an attachment to World of Warcraft. It's the ultimate high-engagement product. And yet, WoW never sends me emails. It doesn't notify me of anything, not even when my friends are logged in and about to start a raid. If I want to know what's going on, I have to log in and find out. It's up to me to decide when I want to do that. And how do I make that decision? Somewhere buried in my brain is a list, called something like "Things to do when you want to zone out and still have a feeling of accomplishment and power." At the top of that list is WoW. I'd only ever consider going to #2 on the list if #1 failed me completely. That's how most of us are - we only ever consider the #1 provider of any given service if it is available. Getting customers to see your service as #1 for a given category is what positioning is all about. (And manufacturing a new cateogry that you can be number one in is what resegmentation is all about)
In WoW's case, its positioning is established by the gameplay itself. WoW is a fun and addictive experience, and once it's sucked you in it's pretty hard to stop. But that is not the only source of positioning: brand advertisers have been using packaging and TV ads to do this for years. And most web applications do their positioning right in the first few screens of the app. This is why the registration is so important. At IMVU, we would routinely find retention effects that would stem from registration changes and have impact days or weeks later. One example I like to use is this: we added a YouTube video about IMVU to some landing pages. It was not prominently featured, but it did auto-play. We split-test that change, and watched the effects on engagement. Customers who saw the video were materially more likely to be active customers of IMVU ten days later.
The impact on behavior was pronounced, even though the immediate effect of the change was subtle. If I had to guess, I would say that if we had interviewed customers in the experimental group, they would not have been able to consciously recall the video they had seen during registration. But, unconsciously, it had affected the positioning of IMVU in their minds. Mission accomplished.
Anyway, for those of you planning on attending the Garage event, please come say hi. And for everyone else, please consider leaving your feedback - positive or negative - about the form or content of the presentation as a comment to this post. Your help is always greatly appreciated.
Monday, March 23, 2009
Is "release early, release often" enough?
The issue there is, if you just follow the release early, release often mantra, you find yourself running around in circles, because you ship code, you get some feedback from people, you do a focus group.
Customers say,”Give me feature X,” “Give me feature Y,” and sometimes you do what they want, maybe sometimes you’re going to do what you want, and then they get mad at you. Pretty soon you’re chasing your own tail a little bit because you’re not operating against a clear, long-term vision of what you’re trying to accomplish.
The idea of minimum viable product is useful because you can basically say: our vision is to build a product that solves this core problem for customers and we think that for the people who are early adopters for this kind of solution, they will be the most forgiving. And they will fill in their minds the features that aren’t quite there if we give them the core, tent-pole features that point the direction of where we’re trying to go.
So, the minimum viable product is that product which has just those features (and no more) that allows you to ship a product that resonates with early adopters; some of whom will pay you money or give you feedback.Solving the chicken-and-egg platform problem
Developers don’t want to develop unless there are customers who are there to buy their products, and customers don’t want to come on the platform unless developers are there selling them something useful.
What we did is we took early adopter developers and we told them a story about how IMVU was going to take over the world and be this really powerful product for mainstream customers ... and we gave them an economic incentive that said, the earlier you get on board with the platform, the bigger your take is going to be for derivative products that get created down the road.
We shipped a product that had almost no customers — certainly no mainstream customers — but, because we had told that story effectively and we really understood those early adopter developers, we got a ton of them on the platform developing. They felt like they were in the middle of a gold rush, despite the fact that there was really no evidence to support that belief yet.
Starting with just a landing page
What we should have done, and what we did for a lot of features thereafter, is started with a landing page that promised people that product. Then we should have taken out the AdWords we were planning to take out, drive traffic to that landing page, and offer people to buy the experience that we are talking about.
What we would have found out if we were doing that experiment is 0% of people would have clicked through, which means it doesn’t matter what is on the second page.
The first page is so bad, not because it is badly designed, but because the features are wrong that you don’t need to go through the effort of building out the product. So we wished we had done that, and we did make that mistake really
Overcoming the fear of the false negative
As long as you’re not afraid of the false negative, that is, if you don’t get discouraged because you’ve built your first paper prototype of it and shown it to people and nobody wanted it. That can’t mean that like you give up because,”Oh, forget it, we’ll never make it.” You’ve got to say, "OK, well then let’s iterate some more.”
If you keep iterating at it, you keep making it a little bit more sophisticated, at a certain point after you’ve been through 10 iterations, that you still got no uptake whatsoever, and the feedback you’re getting from customers is still a yawn, you might say to yourself,”You know what? We’re not moving in the right direction. In fact, we’re past the point of minimum viable product. This just isn’t a viable product.”
Read the rest of the interview at Venture Hacks.
Friday, March 20, 2009
I'm working hard trying to figure out how to bring the lean startup message to different audiences. I think the O'Reilly audience is particularly important, as so many entrepreneurs got their start in technology (as I did) from their worn-down copy of Programming Perl. So if you're a reader of this blog and also of Radar, what kinds of information from this blog would you want to see explained over there? What concepts, techniques, or stories do you think will resonate? Please drop a note in the comments if you have a few minutes to spare. Thanks!
Read the stories of successful startups and, if the founders are willing to be honest, you will see this pattern over and over again. They started out as digital cash for PDAs, but evolved into online payments for eBay. They started building BASIC interpreters, but evolved into the world's largest operating systems monopoly. They were shocked to discover their online games company was actually a photo-sharing site.
Each of these companies were fortunate to have enough time, resources, and patience to endure the multiple iterations it took to find a successful product and market. The premise of the lean startup is simple: if we can reduce the time between these major iterations, we can increase the odds of success.
And here's where working on something that matters to you more than money is critical. When you're committed to something larger than yourself, every minute counts. Hype and transient success won't keep you going. But the simple process of finding out whether or not your vision is right will. Because people who are dedicated to the truth are more likely to fail fast, learn, and try again.
Tuesday, March 17, 2009
- Those of you at the Expo will be able to join in an open discussion session immediately following my talk on April 1st.
- Everyone else can register to come to both sessions for free, including the Lean Startup talk in the main conference.
You heard me right. You - yes, you - can enjoy a full 90 minutes of Lean Startup discussion plus all of the other cool sessions offered by the unconference, entirely for free. Plus your Web2Open pass will "give you access to Web2Open, Hybrid Sessions, Keynotes, Sponsored Sessions, the Expo Hall, and BoFs." To sign up, simply register for for the conference and use the code websf09opn during registration. If you're planning to register for the full conference (which includes Web2Open for free) and would like a discount, you can use the discount code websf09fsd. Check out a full schedule of the Web2Open unconference here.
Last, I know some people have written in with feedback (and in surveys) that they'd like to be able to come to events like this but cannot afford to make the trip. Especially in times like these, it's nice to be able to help out. If you or your company is interested in sponsoring someone with a scholarship to travel to the expo and contribute to the conversation, please leave a comment or let me know. Similarly, if you'd like to be able to come but cannot for financial reasons, please drop me a line. I'll do my best to match deserving people up with appropriate sponsors, and I'll be glad to recognize those that give on the blog and at the conference itself.
Thanks, and looking forward to a healthy dialogue. Bring your questions (or post them as comments). See you April Fool's Day (no pranks, please).
Monday, March 16, 2009
But startups sometimes have trouble applying agile successfully. Or, rather, they apply it successfully, but things don't turn out so well. Enter Jim's post...
Customer Development - The Missing Piece!
But, over the years I’ve realized that the toughest problem - the one that matters most and was consistently the most challenging - was figuring out what the product backlog should be.
The backlog is the answer to the question: “What is the most important work we should do right now?” it presumes that you could confidently make that list, and keep it up to date as things change - or at least articulate what you’re building and for whom. Embedded in that assumption is why startups fail. How do you really make the best backlog for your company?
XP and Scrum don’t have much to say - they punt. Its by far the hardest part of the puzzle of shipping successful products and both recommend that you get a customer in the room and ask them to clarify what they want as you go. Well, that’s fine as far as it goes but when you’re a startup and you don’t have customers yet you need a way to bootstrap and that can feel awfully chaotic and wasteful. What’s worse is that as you grow you’ve probably developed some pretty bad habits as far as setting priorities and strategy: like thinking you’re a genius - just because you got funded - and that genius is what allows you to *know* what the market wants.
I remember having this exact same "aha!" moment, auditing Steve Blank's class when we were first building IMVU. Ever since that time, I have struggled to explain how the feedback loop in customer development should interface with the feedback loop in product development.
If you look at the origins of most agile systems, including Scrum and XP, they come out of experiences in big companies. Consider the classic project that was essential to the creation of extreme programming, the Chrysler Comprehensive Compensation System. This was to be a new piece of software to run payroll for Chrysler. In a project like that there are lots of big questions that need to be answered in order to build a working product. But you don't generally have to ask "what problem are we trying to solve?" That's pretty clear. In the case of C3, that was to run payroll for 87,000 employees, who were presumably receiving payroll before the project began. What causes projects like this to fail in traditional software development is that the solution is unknown. Agile is one way to succeed, because pursuing unknowns iteratively is a good way to mitigate risk. What do you do if the problem itself is unknown?
In a startup, rather than think of ourselves as having a marketing department and an engineering department, I now believe it's better to think of ourselves as focusing our energies on unknown problems and unknown solutions. Approaching each of them iteratively is the right thing to do. But the biggest payoff of all can be found when we combine them into one large company-wide feedback loop.
Last year, I found myself back in Steve Blank's class at Haas, this time trying to teach the students about what it's like running engineering alongside customer development. Working with Steve, I came up with schematic diagrams that I hope illustrate this point. (You can see the full deck in my post on Customer Development Engineering or listen to audio from a more recent lecture)
I thought given Jim's prompting it might be useful to post this excerpt. Notice that the unit of progress changes as we move from waterfall to agile to the lean startup. For more on this latter point and why it's so important, consider taking a look at the posts Achieving a failure and Throwing away working code.
Anyway, thanks Jim for the great post. And credit once again goes to Nivi from Venture Hacks for sharing it with me.
Friday, March 13, 2009
First off, what does it mean to launch? Generally, we conflate two unrelated concepts into the term, which is important to clarify right up front.
- Announce a new product, start its PR campaign, and engage in buzz marketing activities. (Marketing launch)
- Make a new product available to customers in the general public. (Product launch)
Launching is a tactic, not a strategy. In the right situation, it's a very useful tactic, too. In particular, a marketing launch can help you do three things (courtesy, as is most of my marketing advice, of The Four Steps to the Epiphany):
- Drive customers into your sales pipeline. This is the usual reason given for a marketing launch, but for most early stage startups, it's a failure. That's because a marketing launch is a one-time event, and rarely translates into renewable audiences. Worse, if you are not geared up to make the best use of those customers when the launch sends them your way, it's a pretty big waste. And, as we'll talk about in a moment, you don't get a second chance.
Because this reason is so often used as an excuse, I recommend giving it extra scrutiny. Are you really choosing to engage in marketing in places where your potential customers pay attention? Do your customers really read TechCrunch? If not, do not launch there. Even if you must launch to your customers, avoid the urge to also launch in extra places, just because your PR firm can do it at the same time.
- Establish credibility with potential partners. In some businesses, especially in certain industries like traditional enterprise software, you simply cannot bring a new product to market on your own. You need to combine your product with others, and this requires partners like OEM's or system integrators. A marketing launch can help you get in the door with those partners, if you're having trouble getting their attention. Again, it's critical to focus your marketing launch on those publications, venues, and channels that your potential partners are paying attention to. If you don't know who the partners are, what they pay attention to, or what kind of message they are open to receiving - it's too early to launch. Do some Customer Development instead.
- Help you raise money. If you are having trouble raising money, sometimes a little PR can help. But don't be too sure. When VC's and other investors see PR activity, they are going to expect to see significant traction as a result. If you launch and see only mediocre results, it may actually make it harder to raise money. Sometimes, it can be easier to raise money pre-launch, if the launch is not imminent and there is some fear on the part of investors that they might lose the deal when the launch drives awareness of your company to all their peers.
- A marketing launch establishes your positioning. If you don't know what the right positioning is for your company, do not launch. Figuring this out takes time, and few entrepreneurs have the patience to wait it out, because the business plan does such a good job of explaining what customers are going to think. The problem is that customers don't read your business plan.
When you launch with the wrong positioning, you have to spend extra effort and money later cleaning it up. For example, we did some early press (in Wired, no less) for IMVU that called us the next generation of IM and compared us positively to AOL. At the time, we thought that was great. Now, I look back and cringe. Being compared to AOL isn't so great these days, and IM is considered a pretty weak form of socializing. When we finally launched for real, we had to compensate for that early blunder.
Of course, we didn't realize it was a blunder at all. We were actually really proud of the positive coverage. In fact, at that time we were auditing Steve Blank's class at Haas, since he was an early investor. Since we hadn't shown him much in the way of progress recently, we actually brought in the article to show off. I won't recount what happened next (although your can hear us recount it in audio). Suffice to say I can trace my understanding of what it means to launch to that day. We're lucky we had a mentor on board who could call us on the bad strategy before it was too late. Most startups aren't so fortunate.
- You have to know your business model. Most startups launch before they've figured out what business they're in. Pay attention to your fundamental driver of growth. If the product needs to be tweaked just a little bit in order to convert users into customers, you want to figure that out before the launch. If the viral coefficient is 0.9, keep iterating until it's 1.1 before you launch. And if your product doesn't retain customers, what's the point of driving a bunch of them to use it? Spend your time with renewable sources of customers and iterate.
- You never get a second chance to launch. Unlike a lot of other startup activities, PR is not one where you can try it, iterate, learn, and try again. It's a one-way event, so you'd better get it right. Remember the story about IMVU's early encounter with Wired? When we finally did launch the company, even though our product had grown and changed significantly, Wired didn't cover it.
Worse, we tricked ourselves into thinking that what the press said about our success was actually true. And even worse, we'd cranked up the burn rate in order to be ready to handle all those millions of mainstream customers we anticipated. When they failed to materialize, the company was in big trouble.
Why do startups synchronize marketing launch and product launch? I think it has mostly to do with psychology.
- Investors push for it. Many investors have a desire to see their companies lauded publicly. This actually makes a lot of sense, if you see the world from their point of view. Third-party validation is one of the few forms of feedback they have available to them. Most investors in startups have a 3, 5 or even 10 year horizon for liquidity. That means they don't really know if they made a good investment for a very long time. Seeing the press talk about what a great investor they are is a great form of feedback. As a bonus, it gives them something to show their partners and LP's.
This trend is so strong, this is actually a question I recommend to screen potential investors: "How do you know it's time to launch the company?" See if their answer is about tactics or strategy.
- Founders push for it. Who doesn't want to see their name in print? Investors aren't the only ones with ego invested in the company. In some ways, founders are even worse. How do they know they are making progress? They spend so much of their time trying to convince everyone around them that their idea is great and the company is doing well: employees, investors, partners, friends, family, significant others - it's a long list. But when they go to sleep at night, who's there to convince them that they are making progress? My experience is that many founders actually have a deep anxiety that maybe they are not succeeding. Sure, they are keeping everyone busy, but are they really working on the right things? A marketing launch is a temporary salve for these kinds of worries. Plus, it gives you something you can send home to mom (hi, mom!). Unfortunately, it's not a long-term solution, so it can become a bit of an addiction and, therefore, a huge distraction.
- There is also fear of the accidental launch. Companies that are thinking strategically sometimes reason like this: "if we do a product launch, members of the public will see our early product. They'll form their own opinions, maybe see our wrong positioning, and maybe talk to members of the press. By the time we're ready for a marketing launch, it will be too late. Better to launch now and get ahead of the story, or stay in closed beta until we're ready."
In most situations, this fear is misplaced. Here was our experience at IMVU, which I have seen replicated at many other consumer internet startups. We did alienate and mis-position to our early customers. Luckily, if your product isn't good enough to have traction, you simply cannot alienate very many customers - because you can't get them engaged with the product. When you finally do get traction, the millions who see the right positioning will dwarf the few who saw the wrong one. And you can get an astronomical amount of traction before anyone will write about your company of their own accord. IMVU was a top-1000 website in the world, with millions of customers and making millions of dollars without getting any significant press coverage.
In fact, we often felt frustrated when new startups with a fraction of our success got terrific write-ups in Silicon Valley-centric venues. We had to resist the urge to launch just to make that frustration stop. And, more often than not, we'd watch those companies flame out and die while we continued to grow steadily every month. If we'd wasted energy chasing their PR coverage, we'd probably have died too.
How do you know you're ready for marketing launch?
- When you have a strategy for the launch, which means knowing why you're doing it. Make sure it's solving a problem you actually have, and not one that you think you might have some day.
- Know what the success metrics are for the launch. If you know what the strategy is, you'll know how to tell it was a success. Write it down ahead of time, and hold yourself accountable for hitting those objectives.
- Know what your fundamental driver of growth is. Make sure the math for your model makes sense. That way, you'll be able to predict the future. When customers come in from your marketing launch, you'll know exactly what they are going to do and how that benefits your business.
- Know where, when, and how to launch. If you know what your strategy is, and you know your target well (customers, partners, investors) you will also know where they are paying attention, and what messages they are able to absorb. Hold yourself and your PR agency accountable for developing a high level of understanding of these questions ahead of time.
When you're ready, enjoy the launch. Until then, resist the urge.
Wednesday, March 4, 2009
Let's start with what I learned last time. Thanks to those of you who were willing to fill out the survey, I learned my net promoter score (about 25) as well as some clear other segmentation insights: about 80% of you are founders of or work at a startup, you read many of the same other blogs, and many of you would like to engage with Lessons Learned in formats and venues beyond this blog.
Before I go any further, here's the link to the new survey. If you are willing, please take a moment to fill it out before proceeding with the rest of the post; it'll mean less biased results, and you'll have a better idea what I'm talking about later.
Click Here to take survey
As always, I like to start with the NPS question. If this was a startup, I'd be asking it of a sample of customers even more often, because getting a regular tracking result is helpful for informing your decision loop. As I've written before, it is often a leading indicator of major problems. Another key benefit is what it enables in terms of analyzing responses to the survey. For example, those of you who classify yourselves as promoters are more likely than average to have found out about Lessons Learned from another blog. People who joined from sites like Reddit are more likely to be detractors. That's not an earth-shattering insight, but segmenting answers gets more interesting as the quality of the questions we're asking improves. In the first survey, most of the questions were open-ended because I had no idea what kinds of answers to expect, or even if anyone would answer at all.
That's a common pattern to gaining customer insight. Start with open-ended, subjective inquiry, like talking to customers on the phone. As you begin to see patterns, turn to more quantitative options like structured surveys or split-tests to confirm those patterns without as much observation bias.
In the new survey, you'll find more structured questions. They are centered around two themes. The first is asking after problems that you're experiencing. The challenge with these sorts of questions is to keep them open-ended, and to read the answers carefully. Every product ultimately is satisfying a need in the customers who buy it or use it. Products that don't solve satisfy any need for any customers tend to die out quickly. The more painful curse for startups is the product that satisfies a need or solves a problem - but that problem is not very important.
For startups, it is critical that the problem you are trying to solve is top of mind for at least some customers. To see if you're on the right track, consider this hierarchy of customer pain, courtesy of the Four Steps to the Epiphany:
- Customer has a problem.
- Customer knows they have a problem.
- Customer knows they have a problem and is actively trying to solve it.
- Customer has a budget allocated to the problem or is otherwise trying to spend money solving it. Ideally, they are trying to spend money but can't find a vendor to solve it for them.
- Customer is actively spending money, time or effort on solving the problem and failing. This is likely to be an early adopter, because they have the vision to see that the problem is important and they are already putting together a solution out of piece parts. But if that piece-parts solution is great, then it's unlikely they are going to need to buy any additional products.
Imagine what it must have been like for Harmony in their early days. Most customers they talked to have a problem: their coffee table is littered with tons of obscure remotes. Do they know they have a problem? Not in that many cases. Sure, people like to complain about their consumer electronics, but most people have way bigger problems in their lives. But there are some people who have this problem in a more severe form, and since we have the benefit of hindsight, we can quote them. Here's a representative view from an actual Amazon customer review:
We all know how to operate our own entertainment center, but what happens when you have to explain it to your babysitter, mother-in-law, or your wife? Before this remote, I had a Sony RM-AV3000. It was a decent remote, except that it took hours to set up the macros, and the instructions that I had to leave for the babysitter were longer than the instructions for my kid!That's an example of someone who had a problem, knew they had a problem, and had previously tried to solve it with a complex and inadequate solution. If Harmony had sat down with this person in their early days and asked them about their "remote control situation" I am confident they would have heard an earful. And if they had sketched out the concept for the activity-based remote they were building, I'm pretty sure he would have signed up on the spot. That's an early adopter.
This remote took 15 minutes to set up 6 different components (including lighting) on Windows Vista. Long lasting battery, great feel, a little pricey but anything to make it easier on me. After all, a happy wife is a happy husband. A happy mother-in-law...is a happy? Well, I wouldn't go that far!
If you are grappling with understanding what problem your product solves, try building a "problem presentation." This is way of talking to potential customers in which you do not mention your product at all. If it's appropriate, build a few slides. Or create an online survey. Or just talk to people in person. However you do it, focus on explaining the problem you think you will eventually solve and how important it is. If your customer is nodding along, keep getting more and more detailed about how the problem affects their life or their company. When they look confused, stop and ask them to explain, clarify, and correct you. When you get to the point where you can get three consecutive customers to nod through your whole presentation, congratulations. You've got an accurate read on the problem. Then, and only then, should you proceed to start trying to sell people your solution.
The second set of questions in the survey are focused on learning where you currently go to find solutions. In particular, I've focused on the two channels of communication that were most commonly suggested in the first survey: conferences and books. In a startup, these channels of communication are critical to understand for advertising, launch and PR purposes. But beyond the obvious, it's also important to know where customers go to get answers to pressing questions. If they are willing to pay big bucks, for example, to attend a conference on a certain topic, that bodes well for your ability to sell them a product related to that topic. And, for conferences, it also tells you where to go to talk to them. This is especially important for startups that have a small number of target customers, like those that sell only to large companies in specific industries.
I'll mention one last thing about knowing what channels of communication your customers engage with. It also gives you insight into their language. This is important, especially if you are a very early stage company. If you're used to pitching your company in a startup hub, like here in Silicon Valley, you may suffer from a debilitating disease: the inability to describe your product to normal people. That's no big deal if all you want to do is sell to people who live in startup hubs. But for most of us, that's devastating.
For example, when pitching IMVU, I used to say things like: IMVU is an 3D avatar IM platform with a micropayment economy powered by user-generated content. No customers understood what that meant. If we had bothered to ask them where they get their information, none of them would have said TechCrunch or Mashable. If we had spent even a tiny amount of time engaging them on this question in our early days, we could have saved a lot of time. Of course, we would have had to spend some time reading teen magazines. But isn't that a small price to pay to change the odds of your startup making it?
So thank you so much for subscribing. And an extra thank you to those of you who can spare a few minutes more to let me know what you're thinking. Have suggestions for other questions to ask in future surveys? Go ahead and leave your thoughts in the comments.
Tuesday, March 3, 2009
The suggestion is that you implement one single company-wide rule. The rule is simple: every employee is 100% responsible for how they spend their time. If it sounds simplistic, let me try and make the case that it is actually quite radical.
For example, let's consider job descriptions. When I was a manager, I was once leading a cycle post-mortem. We were talking about what went right and wrong in the company's development process, when one (relatively new) employee said something like "I noticed that problem, but I couldn't do anything about it." I asked why. He obviously thought I was being dense, and told me "it's not in my job description." I asked, "how do you know what your job description is?" Now he really thought I was nuts, replying "from the recruiter who hired me, of course!"
It was my turn to be dumbfounded. Of course, we use job descriptions for recruiting, but it had never occurred to me that somebody would take them literally. But once I was confronted with the situation, it seemed obvious. What else was he supposed to go on? I realized that, although I personally believed in the policy of trusting employees to know what they are supposed to do, I hadn't taken the necessary actions to build that trust into the culture of the company. Oops.
So we instituted a new policy on job descriptions, which I would take pains to reinforce whenever I could find an appropriate segue. It was, simply, that every person in the company has this job description: in any situation it is your responsibility, using your best judgement, to do what you think is in the best interests of the company. That's it. Everything else is only marketing.
This is not an easy policy to implement. Most people think it can't possibly work; it sounds like a recipe for anarchy. But it's not. If you're hiring creative, resourceful and smart people (and you are screening for that, right?) you don't need to worry about them suddenly going off the reservation and deliberately harming the company. And if you're giving them all of the information they need to understand the company's strategy, customers, and processes, you can be confident that they'll be able to apply that understanding to their current responsibilities.
What's that you say? You're not currently doing all of those things? That's too bad, because if you don't keep your smartest people fully-informed, you're going to wind up telling them what to do. And telling knowledge workers what to do is one of the biggest causes of mistakes, waste, and general low morale that I've ever seen.
Taking this radical view thus leads directly to other important process improvements. In order to have everyone understand the company strategy, we opened up our board and board of advisors meetings to the whole company, not just a select few. In order to give people the data they need to apply the strategy, we were very open with our company metrics, making all reports generally available and easy to run. And we presented the full company P&L every month, as soon as we got it from our controller. And it requires a formal process for on-boarding new employees (including root cause analysis when it goes wrong) to make sure everyone understands their job description and what it means day-to-day.
If you've never engaged in radical transparency before, you may be anticipating problems already. What about when you have to present bad news, doesn't that impair morale? Board meetings are frenetic, doesn't that lead to confusion about what the plan is or who's in charge? Those are valid concerns, and I've had to cope with them and many more. But the coping is actually extremely useful. By forcing these issues out into the open, transparency teaches you what your employees are really thinking. Over time, your whole team will become much more aligned, and able to handle the inevitable curveballs that circumstance delivers. Mutual trust is a prerequisite for rapid response.
Let's return to my initial promise, that giving employees 100% responsibility for their own time will lead to a culture of learning. Part of the way it does that is by creating transparency. But it has a corresponding impact on employee responsibility.
This policy doesn't leave much room for excuses. Take a failed product launch. At IMVU, these were quite common (after all, we're shipping code 50 times a day). Lots of those features would turn out to be a bad idea. I expected a certain amount of grumbling, since nobody likes a failure. But I didn't have any patience for one particular complaint: "I knew it wasn't going to work!" There is no room for this idea in a company that insists that every employee owns their own time. If you know something isn't going to work, it's your 100% responsibility to speak up and prevent it from happening. That may lead to a lot of arguments, but that's what you want. When smart people clash over prediction of the future, two critical things happen:
- They are forced to make their predictions specific. This is a prerequisite for institutional learning. When you think a certain feature will give a 50% boost to a given metric, and it only eeks out a 5% boost, you can't spin that as failure. But if you weren't specific about the goal, 5% can be rationalized as success.
- People share their assumptions and prior thinking. Often arguments result in one side being convinced, because the other side has real data to support their conclusion. Other times, it will be revealed that one party or the other is making empty assertions, or relying on credentials. You want those situations to be painful.
I honestly don't know how startups survive without this sense of responsibility on the part of the people doing the actual work that affects customers. It's just so easy to check out, follow the plan, and not rock the boat. That may make it easier to get everyone to follow the plan, but I also think it makes it much more likely that you'll achieve a failure instead.
No, I do not mean masters of their own domain. Don't even go there.