The visionary’s lament

“But customers don’t know what they want!”

It’s an anguished cry that I have heard often from startup founders. In a way, I don’t blame them. I’ve been there myself. If we’re not attempting something truly new and innovative – what’s the point? If we’re just going to conduct the world’s biggest focus group to decide what to do, why couldn’t any old idiot do it instead? Isn’t the whole point of devoting our life to this enterprise to show the world that we have a unique and visionary idea?

I remember one conversation with a visionary quite well. He had just come back to the office after a few days away, and he was filled with big news. “I have incredible data to share!” which was pretty unusual – a visionary with data? He carefully explained that he had conducted a number of one-on-one customer interviews, showing them an existing product and then documenting their reactions. His conclusions were well thought out, coherently based in the data he was presenting, and painted an alluring picture of a new way forward. His team almost exploded on the spot.

“That’s the same idea you’ve been pushing for months!” “What were the odds? Customers explained to you that we need to do exactly what you wanted to do anyway? Wow!” It was an ugly scene.

We all know that great companies are headed by great visionaries, right? And don’t some people just have a natural talent for seeing the world the way it might be, and convincing the people around them to believe in it as if it was real?

This talent is called the reality distortion field. It’s an essential attribute of great startup founders. The only problem is that it’s also an attribute of crazy people, sociopaths, and serial killers. The challenge, for  people who want to work with and for startups, is learning to tell the difference. Are you following a visionary to a brilliant new future? Or a crazy person off a cliff?

True visionaries spend considerable energy every day trying to maintain the reality distortion field. Try to see it from their point of view – none of the disruptive innovations in history were amenable to simple ROI calculations and standard linear thinking. In order to do something on that scale, you need to get people thinking, believing, and acting outside the box. Their greatest fear is categorically not that their vision is wrong. Their real fear is that the company will give up without ever really trying.

This is where data, focus groups, customer feedback, and collaborative decision-making get their bad rap. In many cases, these activities lead to bad outcomes: watered down vision, premature abandonment, and local maxima.

When visionaries say “but customers don’t know what they want!” they are right. That’s the problem with false dichotomies: each side has a kernel of truth within it. You cannot build a great product simply by obeying what customers say they want. First of all, how do you know which customers to listen to? And what do you do when they say contradictory things?

And yet, the people who resist visionaries also have a point. Isn’t a bit scary, maybe even suicidal, to risk everything on a guess – even if it is emotionally compelling?

Like all false dichotomies, if either side “wins” this argument, the whole enterprise loses. If we just follow the blind mantra of “release early, release often” and then become purely reactive, we’re as likely to be chasing our tail as to be making progress. Similarly, if we pursue our vision without regard to reality, we’re almost guaranteed to get some aspects of it wrong.

The solution is synthesis: to never compromise two essential principles. One, that we always have a vision that is clearly articulated, big enough to matter, and shared by the whole team. Second, that our goal is always to discover which aspects of this vision are grounded in reality, and to adapt those aspects that are not.

A vision is like a sculpture buried in a block of stone. When the excess is chipped away, it will become a work of art. But the challenge in the meantime is to discover which parts are essential, and which are extraneous. The only way to do this is to continuously test the vision against reality and see what happens.

So what should you do if you find yourself working with a visionary? Almost every successful visionary has found partners to work with that help them stay grounded in reality. To do this you have to find ways to be supportive of the vision at the same time as reporting the bad news about where the vision falls short. I recommend a mantra that I learned from Steve Blank: always consider your job to find out if there is a market for the product as currently specified. Don’t try and change the vision every time you get new data. Instead, get out of the building and look for customers for whom your product vision is a slam-dunk fit. If and only if, after exhaustive searching, you cannot find any customers that fit the profile, is it time to have a serious conversation about whether and how the vision should be modified (a pivot).

And what should a good visionary do to help find synthesis? Based on the successful visionaries I have had the opportunity to work with up close, I'd like to offer two suggestions for the role a visionary should take on:

1. Identify an acute pain point that others don’t see. It’s important to specify the vision as much as possible in terms of the problem we’re trying to solve, rather than a specific solution. (Or, to use Clay Christensen's formulation, of the "job" customers are hiring us to do.) Even though the visionary surely has some concrete ideas which are to be tried, he or she should always be asking, “would I rather solve the problem, or have this specific feature?”
   
2. Hold the team to high standards. Despite Steve Jobs' incredible talents, he doesn’t personally design and ship every Apple product. It’s much more likely that his main function is to hold everyone who works for him to the same high standard. Once they’ve agreed to try and solve a dramatic problem, it’s the visionary’s job to hold each provisional result up to the light of that vision, and help the team remember that although trade-offs and compromises are always necessary – the real payoff is in solving that acute pain. This can help avoid the trap of the false negative: even if the first few iterations don’t get it right, the vision inspires us to learn from our failures and keep trying.

Let me close with a specific story of a visionary at work. I’ve heard from several sources a story about Jeff Bezos and the invention of one-click shopping. It may be apocryphal, but it’s illustrative anyway. Amazon had tasked a team with building their new one-click shopping feature, which was designed to reduce the friction required to make an impulse purchase. The purpose of naming the feature “one-click” was to clearly communicate to everyone the vision of maximum simplicity. When Bezos was meeting with the team to review their first version of the feature, so the story goes, after he clicked to make his purchase, he was prompted with a confirmation dialog box. He had to click “yes” to continue. In other words, one-click shopping required two clicks!

Now, it’s really important to see this story from both sides. Bezos was surely infuriated that the team had missed so obvious a point about his vision. But see his team’s point of view: they were immersed in a culture of protecting the customer. It was probably considered too dangerous to let someone “shoot themselves in the foot” and make an unintended purchase that could have serious economic consequences.

But by actually building a version of this feature, and doing some simple testing with customers and with Bezos, this team surfaced an issue that probably wasn’t really clear in Bezos’ vision from the get-go. Namely, how are we going to handle the case of customers one-clicking by accident? The synthesis solution is so simple, I’m sure it seems obvious in retrospect (and I’m sure dozens of people, for all I know including Bezos himself, are now sure they came up with it on their own): since mistakes are the uncommon case, give the customer several opportunities to realize and correct them after the fact, rather than trying to prevent them with a confirmation dialog box.

Those are the attributes I admire in successful visionaries: a determination to see the vision through, holding their teams to high standards, and a commitment to iterate in order to get there.

The Superbowl ad test

I am a firm believer in the danger of vanity metrics, numbers that give the illusion of progress but often mask the true relationship between cause and effect. Since I first started writing about vanity metrics, I’ve met more and more entrepreneurs who are struggling with a simple question: how can I tell a vanity metric when I see it?

From the outside, vanity metrics are a lot easier to see than from the inside, precisely because of the psychology behind them. Everyone wants to believe that the work they are doing is making a difference. So it’s easy to read positive causes into noisy data, whether it’s really happening or not. (This is called “the illusion of cause” and is discussed at length in the extremely readable book The Invisible Gorilla). Even worse, entrepreneurs are faced with a constant barrage of vanity metrics from competitors and other companies engaged in PR. Vanity metrics are generally bigger. And everyone knows bigger is better, right?

News publications print vanity metrics because they want to give their readers information about the companies they cover. Companies want the coverage, but they don’t actually want to reveal anything useful about their operations. The solution? Vanity metrics. By only releasing vanity metrics, companies co-opt the press into helping them mislead others. Is that really news? I’ll leave that to professional journalists to sort out. For the general public, it’s probably OK to treat company updates as entertainment. But for entrepreneurs, investors, analysts and competitors, it’s quite dangerous.

Here’s my quick heuristic for telling if a given number, graph, or chart is a vanity metric: could it have been caused by the company secretly running a Superbowl ad and not disclosing it?

If yes, it’s very likely to be a vanity metric. Let’s take a look at an example, one of my favorites, the "billions of messages" claim.

Here's Mashable's coverage of Facebook chat reaching "a billion messages a day." Or take a recent TechCrunch article about a startup I won't name: “X billion messages sent since June 2009.” These articles treat this as a huge number, and it is. Probably, it represents tremendous success for the company in question. It’s side-by-side with a number of other vanity metrics. But notice what’s not listed: messages sent per person, churn rates of active users, or activation rate of new user. Even worse, we have no indication of how these numbers are moving over time. Is the company growing because of an amazing viral loop paired with a strong engagement loop? It’s possible, but the article doesn’t say. Most of the article is about the features – new and old – of the product. The unstated implication is that these features are what are leading to this tremendous growth. But is that true? Isn’t it equally possible that this company is spending more money on advertising or marketing than it’s competitors? Or that there is some other external factor at work?

I have no insight into these questions, and I don’t mean to pick on these companies in particular. My point is that this article does not contain the kind of information we’d need to draw reasonable inferences, which is by design. That’s what you pay PR firms for: to get an article written that is entirely factual and yet still provides positive spin for your company. (For context "some 740 billion text messages were sent in the first half of 2009." The PR firm helpfully left out that context.)

So, could these numbers have been generated by a Superbowl ad? Of course. We have no idea when the billions of messages were sent. They could all have been sent very recently. That’s the magic of vanity metrics – you never know what’s really going on. The trouble comes when companies and investors come to rely on these numbers to make consequential business decisions. How should a company like the one above prioritize their next set of features? Hopefully, they have internal reports that show the true correlation between their features and customer results. Are employees paying more attention to those reports than to the positive press coverage? I sure hope so.

Notice that cohort and conversion based metrics do not suffer from this problem. When we look at the same conversion percentage for cohort after cohort, we are effectively getting a new, independent, report card for our efforts each period. Each cohort is mostly unaffected by the behavior of earlier cohorts. And it is much more insulated from external effects, like an advertising or PR blitz, than your typical vanity metric.

It is not difficult to translate a gross metric like total messages sent into cohort terms. Since I’m picking on the TechCrunch example from above, we’re talking about more than a year’s worth of data. Let’s divide it into monthly cohorts. For each month, messages are sent by two kinds of people: new customers and returning customers. In order to make each cohort as meaningful as possible, let’s define them as follows:

New customer: someone who registered for the service in a given month
Returning customer: someone who used the service in the immediately preceding month.

I choose the “preceding month” definition in order to give us a sense for individual people’s behavior. A huge advertising blitz might cause a temporary winback effect by bringing in lots of old customers, but this is generally the kind of effect we want to ignore (unless we’re measuring the short-term effectiveness of the advertising).

Now, let’s plot a single number for each cohort, the percentage of customers in that cohort who sent at least one message in that time period. That makes our numbers denominated in people, not messages, which is much easier to understand. (remember, metrics are people, too). If we wanted to get fancy, we could also plot the average number of messages sent per person in each cohort.

If these numbers are flat month-to-month, then we can draw some strong conclusions about the product features we’re working on: they are basically having no effect on customer behavior. Hopefully, that’s not the case. Hopefully, the numbers are steadily improving month after month.

The data needed to generate this simple graph already exists: it’s the same basic data you’d need to get an accurate count of the total number of messages sent, just presented in a different form. For understanding what’s really going on with a product, this alternate form is far superior. Is it any wonder companies don’t want the press to have it?

It’s my hope that, in time, our industry will start to reject vanity metrics as a serious part of the discourse about customers. But this will take a long time. Investors and journalists have the most leverage to start making this change. Entrepreneurs have a part to play, too. Playing with vanity metrics is a dangerous game. Even if you intend to “only” give that sugar rush to publicists or investors, it’s all-too-easy to be taken in yourself. Your employees probably read the same press you are trying to influence. Your investors may be taken in today, but they will use those same vanity metrics to hold you accountable tomorrow. It’s much easier to rely on actionable metrics in the first place.

Lo, my 57692 subscribers, who are you?

Since this blog's earliest days, I have made a habit of surveying you, my subscribers. I did it originally as a demonstration of the advantages of having a pathetically small number of customers, but I found the actual info so incredibly helpful, I have done it several times since. Since the last time, your ranks have grown tremendously, and I thank you all for this incredible support.

So, to celebrate Labor Day here in the US, I've created another survey. If you're willing to take five minutes to fill it out, I would be most grateful:

Click here to take survey

As usual, I've added a small minimum viable product (I'm starting to think of this technique as the "survey MVP") at the end, as yet another customer validation exercise. I'll post about the results later; to say anything here would bias the survey. 

SXSW

Next year will officially mark one-hundred years since scientific management, the first great management paradigm, burst into the national consciousness. It invented many concepts we now take for granted: efficiency, productivity, and the idea of management itself. We owe that movement an incalculable debt of gratitude. Have you decide already how to celebrate this centennial? I have: I'm going to mark the occasion at SXSW Interactive in March, 2011. You're invited (details below).

I'm starting to experiment with new ways of talking about the Lean Startup movement and the impact you all are having on the practice of entrepreneurship across countries, industries, and even sizes of companies. We are collectively bringing a new level of scientific rigor to the act of innovation itself, and our revolution is just beginning. We still have much to learn. So take a look at the ideas below, and please leave your thoughts as a comment. As always, I welcome your feedback.

The seed of this idea was planted by many of you on Twitter over the past few weeks. In fact, both I and the SXSW organizers got many emails and tweets about the necessity of having Lean Startup be part of SXSWi. As a result, they were kind enough to allow a very late Lean Startup submission to their user-generated PanelPicker system. They use crowdsourcing to figure out which speakers to invite and what topics are of interest to their audience. Even if you've never been to SXSW or don't know what I'm talking about, you can still go vote - it takes less than five minutes. Because we're getting a very late start on the other panels, our submission is way behind. We only have a few days to catch up, as voting ends on August 27 - just three days from now. So please vote, comment, tweet, and help make this happen. Thank you.

Here's the submission itself, with my first attempt at a new framing for Lean Startup as a rebirth of scientific management. I'd love to know what you think:

The Lean Startup: innovation through experimentation

2011 will mark the one-hundredth anniversary of Frederick Winslow Taylor's "Principles of Scientific Management." The tremendous material abundance we enjoy today is the result of the productivity revolution he unleashed by bringing the tools of science to the study of work itself. Management today is rigorous, scientific, and effective -at the production of physical goods. 

In other areas the picture is bleak, especially for innovative new products. We fail spectacularly in startups and big companies alike. Too often we're building something nobody wants. There is a movement that is trying to eradicate this disease. 

We are at the beginning of a second scientific management revolution that will bring science, rigor, and discipline to the process of innovation itself. It has already begun to transform the way startups are built around the world. It is called the Lean Startup. 

All entrepreneurs face these challenges: 

How do we know if we’re making progress? 

How do we know if customers will want what we’re building? 

How do we know what kind of value we can create? 

Answering requires more than just disciplined thinking at the whiteboard. It requires the coordination of people. In other words, it requires management. The Lean Startup is a management science for entrepreneurs of all kinds. It enables rapid customer-centric iteration. It helps startups test their vision before it's too late. It is a tool for people who want to change the world.

Regardless of what the SXSW organizers decide, I intend to host an event in Austin to coincide with the conference. I'm hoping we'll be able to top last year's Lean Startup Smackdown, which was put on by the Austin Lean Startup Circle. I don't know if it will be more like a party, or more like a mini-conference. In fact, I encourage you to weigh in with a comment. Would you be interested in attending? Co-sponsoring or co-organizing? Or just getting drunk? Let me know.

Most importantly, I want to continue to send you all my thanks for your tremendous support and encouragement. I can honestly say this is something I would never have imagined attempting on my own, and it is the latest in a series of amazing experiences you've all made possible. Thank you. 

Case Study: SlideShare goes freemium

(Normally, I do not write about companies that are doing a marketing launch. But I have decided to make an exception today, for two reasons. First, SlideShare is a fantastic product (that I use on a regular basis) and an impressive company example of Lean Startup practices in action. Second, their story illustrates a key Lean Startup idea: proving the business in micro-scale. It requires separating the product launch from the marketing launch (see Don't Launch) as well as other staple Lean Startup tactics: minimum viable product, split-testing, customer development and the pivot. This story especially demonstrates that these techniques are not reserved only for tiny startups just starting out. When SlideShare began the journey you're about to read, they already had more than a million visitors a day. Because the stakes were high, they had to successfully use a technique I call Innovation inside the box which is important for entrepreneurs inside established companies of all sizes.


Once again, this case study is a collaboration with Sarah Milstein, who conducted the interviews and wrote the post itself, with some minor edits and commentary from me. As this is a new initiative for this blog, we especially welcome your feedback. Did you find this post useful? One recurring request I hear from Lean Startup practitioners around the world is a desire to see examples of the ideas in action. How are we doing?


In the meantime, take a look at how SlideShare performed a significant pivot while still moving at full speed. -Eric]

“The first user experience was actually terrible.” Rashmi Sinha, co-founder and CEO of SlideShare, describes an early version of the analytics package that’s part of the Pro accounts the company announced today.

If your company is using minimum viable product, you’ve probably said the same thing yourself. A lot. SlideShare, founded in 2007, started experimenting with MVPs and A/B testing this year. Those tools, combined with focused customer interviews, have turbo-charged the company’s ability to learn.

What prompted the process change? Early this year, SlideShare launched custom channels. Designed for large businesses, the channels let a company share several types of documents, brand the channel with their own design elements, and then include display advertising, contest promotions, blog aggregation, social media integration and metrics reporting. The idea seemed to SlideShare to be a natural direction. Except it didn’t take off. [I was an early adopter of this feature, and participated in the last marketing launch, as you can see here. Alas, even brilliant marketing adorned with a giant picture of me can't fix the wrong product. -Eric]

Big companies said they liked the idea, but SlideShare found it hard to close deals. Meanwhile, individuals and smaller companies emailed by the hundreds to say that they wanted the features of custom channels, but the sales model—arranged like a media buy—didn’t make sense to them.

SlideShare’s existing customers had needs that the company’s new product—along with its pricing and positioning—simply weren’t solving. Realizing it had taken a wrong turn, SlideShare rethought its approach to premium accounts and ultimately performed what we’d call a value capture pivot, one where the company changes the way it collects revenue from customers.

The process started with a few moving parts. First, the company began quietly testing subscription pricing plans, initially positing a basic plan and an enterprise version. Second, when an individual or small company signed up, Sinha would email them to ask if they’d be willing to hold a phone interview with her to discuss their experience of the product. Despite the fact that SlideShare's product is well-established with many customers, Sinha still took the critical step of (to use Steve Blank's famous phrase) getting out of the building, a particularly important job for founders. Third, SlideShare started holding sales calls with large companies to learn what would prompt them to buy the enterprise version.

“Individuals and small companies wanted analytics, they wanted to know what was happening in social media [for their content], they wanted ad removal and lead gen. Branding was less important to them,” says Sinha. Big companies had other needs. “We didn’t anticipate at all the control features. For instance, we worked with Pfizer, and they wanted the comments turned off. I hadn’t thought that would be a feature. But they’re regulated, so it makes sense.” SlideShare used the two streams of information to segment their market and come up with three plans that recombined the custom channel features in meaningful ways.

But that’s just part of the story. As SlideShare was pivoting, it was also trying out two processes to get better results: 1) A/B testing to refine the pricing plans and the page describing them; and 2) MVPs to hone the actual premium features. The combo helped SlideShare learn a lot in short order. [This is the essential approach to testing a big vision that avoids the "local maximum" trap. See Learning is better than optimization. -Eric]

The company ran landing page splits every two or three days (they initially used Unbounce to generate the pages) and measured them carefully with KISSmetrics. They also used SnapABug for live chat on their site. Between the metrics and the direct customer questions, SlideShare had what Sinha calls “minor learnings and then major shifts.”

For example, early iterations of their pricing page included the original, free version of SlideShare. “We realized that was really confusing people,” says Sinha. “We don’t give you all this Pro plan information right away when you join SlideShare. It’s more like, ‘If you’re already using SlideShare, you might want to try this.’” They removed the core plan, and conversions went up.

The A/B testing did have its challenges. Because SlideShare has more than a million visitors a day, the team is used to developing features that at least 100,000 people will use. “You get used to having a big impact,” says Sinha. With the split tests, maybe 500 people would see an iteration (SlideShare drove traffic with calls to action around their own site). “You have to get ready to deal with much smaller numbers.”

The MVPs were tricky to implement for emotional reasons, too. Because the SlideShare team was used to giving away a high-value product, engineers balked at charging for a clearly imperfect product. The analytics package, for instance, launched in what Sinha calls “a very crude version; we started off and sold it before we were comfortable with it.”

The saving grace was follow-up interviews. SlideShare asked customers what they had expected in the product; the responses were often literal descriptions. People consistently said they were dying for analytics and specifically that they wanted to track social media and understand the people visiting their content (SlideShare eventually discovered that showing visitors’ locations and timing satisfied people’s needs).

“Charging for something half-baked is really interesting,” says Sinha. “It makes the product team uncomfortable. At the same time, you make sure that you get honest feedback. If the product doesn’t meet customers’ expectations, they cancel. It’s a very honest relationship. On analytics, we got a lot of feedback that it was half-baked, that we sold it under false pretense. But we would just respond honestly and fast and say, ‘Tell us what you want.’ Then we’d get back to them when we had built it.” Customers appreciated the follow-up, and many bought again after the feature had evolved. In this regard, SlideShare used the early adopter feedback not only to improve the product, but too improve its understanding of what subsequent customers would want. [That is customer development. -Eric]

The marketing launch for SlideShares Pro accounts is today. But the product launch has been happening iteratively over the past months—which means the company is confident in its new offerings. “When we launched custom channels in February, a lot of people reached out and said, “We’d love to buy’,” recalls Sinha. “But it never happened.” [Alas, customers don't know what they want! -Eric] Since creating and refining its premium accounts, SlideShare has closed a number of deals, including high-profile accounts like Dell, Cisco and Pfizer.

Sinha notes that Eric Schmidt, in a recent interview, said that you find out whether people truly like a product in the second phase after launch. In the first phase, you get a lot of curious people. Only after the buzz has died down do you truly understand what’s going on. With careful and continuous learning processes, SlideShare is inverting that idea and going to market with a validated product. That is the essence of proving the business in micro-scale.

[We'll see if the marketing launch results follow the predictions of SlideShare's validated business model. We wish them the best of luck, and hope we can convince them to share their results - positive or negative - in the near future. In the meantime, good luck and thanks for letting us share your story. -Eric]

Case Study: kaChing, Anatomy of a Pivot

(The following guest post is a new experiment for this blog. It was written by Sarah Milstein in collaboration with kaChing CEO Andy Rachleff. kaChing has been very active in the Lean Startup movement. If you haven't seen it, Pascal's recent presentation on continuous deployment is a must-see; slides are here. In the interests of full disclosure: I am an advisor to kaChing but did not participate in the interviews that led to this case.

With case studies like this, we aim to illustrate specific Lean Startup techniques through the stories of current practitioners. It is written using the information that the company voluntarily shared, and therefore reflects their current thinking and recollections. I am particularly interested in feedback on this case study. Do you find it helpful? Please give us your feedback in the comments. Thanks, Eric)

You probably know that Flickr, the photo-sharing site, started out as an MMOG. And if you’re a regular reader of this blog, you may know that IMVU started out as an instant messaging add-on. It’s common, perhaps the norm, for startups to pivot like that—to discover that a product is catching on in unintended ways worth pursuing. Yet there’s a lot of mystery around pivots, and entrepreneurs ask all the time how you know it’s time to commit to a new direction.

To shed some light, I talked with kaChing, a destination that enables individual investors to find outstanding money managers to manage their money. The company’s audacious goal is to disrupt the $11 Trillion mutual fund industry. The startling part is that kaChing started out as a…Facebook game. That’s an epic pivot, like shifting from making solar calculators to powering the Space Shuttle. How’d it happen?

kaChing launched a virtual portfolio management game on Facebook in January 2008 and a similar version shortly thereafter on kaChing.com. The intent was to discover amateurs who could manage a portfolio as well if not better than professionals (think American Idol) and then facilitate individual investors giving them their real money to manage. In other words, the game would serve as a kind of minor league for the profession. Because kaChing prefers its portfolio managers to have a long track record, the marketplace launch (i.e., the version that would facilitate the investment of real money) was planned for late-2009.

kaChing deployed the game across a slew of platforms, including MySpace, the iPhone, and the Yahoo App Platform. The result? They attracted more than 450,000 portfolios—a decent number for a company that hoped a good percentage would prove out as capable managers. They also hoped a reasonable percentage would realize they were lousy money managers and would then convert to clients.

In the early fall of 2009, as kaChing prepared for its marketplace launch, the management team showed the app—which included real time market data, SEC-grade accounting, analytics, compliance and customer management tools—to a number of investment pros to get feedback and endorsements. One of those pros was John Powers, head of the Stanford endowment. He noted the platform would be good not only for amateurs who had proven themselves as outstanding portfolio managers in the game, but also for professional money managers —a group that had insufficient tools for managing and scaling their businesses.

The kaChing system was based on full transparency. A portfolio manager’s entire track record & holdings had to be disclosed. The company didn’t believe professionals would be willing to reveal that level of detail. But Powers’s reaction was intriguing enough to prompt Andy Rachleff, kaChing’s CEO, to call friends who were professional money managers and describe the idea. The response was surprisingly positive.

Andy Mathieson, a founder and managing member at Fairview Capital, was particularly supportive. He was unconcerned about transparency, noting the good have nothing to fear. Mathieson signed on to be a money manager in the marketplace launch, committing five years worth of prior transactional data. Mathieson’s firm has a minimum investment of $1 million dollars outside of kaChing. On kaChing, consumers could invest in Fairview Capital’s strategy with as little as $3k.

When the marketplace launched on October 19, it included seven amateurs who had risen through the game’s ranks and four professionals, including Mathieson. Within a month, kaChing observed several interesting things. First, because the amateurs weren’t SEC-registered, the site had to refer to them with awkward terms like “geniuses.” That was confusing for consumers, who already had to figure out what on kaChing.com was a game and what was real. Second, out of 450,000 gamers, only seven had qualified to become kaChing managers. Third, the company expected hundreds of amateurs who performed poorly in the game to realize they weren’t good at investing and therefore become customers. in fact only five people converted into paying customers. Finally, after launch, 30 professional money managers, having read articles on the company, contacted kaChing out of the blue. These managers weren’t concerned with transparency. They were interested in the tools and new distribution medium kaChing provided.

In November, kaChing held an all-hands meeting, circling up chairs in their small Palo Alto office, to discuss whether they should focus solely on professionals and abandon the systems for proving amateurs. “Some people weren’t comfortable because it wasn’t as fun, and one senior engineer thought we’d be losing the part of kaChing that was an enabler for anyone who wanted to make it as a pro,” Rachleff recalls. “But what we really wanted to change was not who manages the money, but who has access to the best possible talent. We’d originally thought we’d need to build a significant business with amateur managers to get professionals to come on board, but fortunately It turns out that wasn’t necessary.”

The staff agreed they could better fulfill their goals by working just with professional managers. In December, they removed the game from kaChing.com. In February, they held another all-hands meeting to talk about shutting down the legacy Facebook game, which still had 60,000 active users. “Everybody felt the burden of supporting all those transactions every day,” says Pascal-Louis Perez, kaChing’s CTO. “It took a ton of our time, and just wasn’t contributing to our long term vision.” That all-hands lasted five minutes.

Which is a nice story. But when kaChing actually shut down each game, hundreds of angry players spewed venom. “We had to ignore them, because they weren’t our target audience – and were never likely to become customers.” says Rachleff.

kaChing says they had the fortitude to make quick decisions and stay the course not just because they’d observed how people were using the marketplace, but also because they’d spoken with hundreds of potential and past customers. To acquire new money managers, the company makes traditional sales calls, which means they’ve interviewed many, many professionals and gotten a strong sense of their needs. At the same time, whenever a customer closes an account, kaChing contacts the person to find out why; most agree to a short phone interview. (The site has about 700 active paying customers.

Perez says this level of contact, synthesized with their own observations, has given them confidence to make bold decisions. Of the money managers they’ve interviewed, he notes, “The feedback is consistent; we solve big enough problems for people that we believe they’ll come on board.”

With 21 employees today, kaChing is devoted to recruiting professional managers and finding product/market fit, first for money managers, then for consumers. Thus far the results are encouraging. More than 30 qualified professional money managers have been attracted to the platform and more than $190 million of customer assets have been committed as well.

The kaChing team is quick to note that because they’re still closing-in on product/market fit, they’re less data-driven than they plan to be once they’re in optimizing mode. “We create hypotheses, and test them,” says Rachleff. “If something fails, we cut it off. If something seems to succeed, we pursue it aggressively. You have to have the courage of your convictions. With limited data, you have to make tough decisions.”

Special thanks to Pascal-Louis Perez for sharing information and making this post possible.

Some IPO speculation

Inspired by Steve Blank’s post today about the “lost decade” of IPO’s, I’d like to make some predictions. Let me be clear: Steve is the historian. His posts are born of tremendous research into the secret history of Silicon Valley, and if you haven’t read those essays, you should. By contrast, what I’m about to say is pure speculation.

The fact that IPO’s are disappearing makes intuitive sense to me. And the fact that the effects of this IPO vanishing act are being felt first and foremost in the software business also makes sense to me. In fact, I believe that the software business is the canary in the coal mine: increasingly, all businesses are going to look more and more like software businesses.

My belief is that the root cause of the IPO shortage is that successful startup companies cannot find productive ways to invest large amounts of money to scale anymore. For software companies especially, scaling distribution and development is comparatively cheap. The old ways aren’t working. Large capital investments simply don’t have the ROI they used to. The world is changing too fast. New products become commoditized too fast. Increasingly, the only profitable thing to invest in is innovation, which means investing in people. And we don’t yet know how to do that on a consistent, scalable, basis.

Ironically, the VC’s who depend on IPO’s and the CEO’s who are supposed to be creating them are struggling with the same basic problem. They do not have a comprehensive theory of entrepreneurship that allows them to consistently invest in innovation that can create long-term value.

The only way I can see to achieve sustained growth is to create an innovation factory. The modern CEO needs to build an organization that is truly diversified: it is continuously investing in successful sustaining innovation and disruptive innovation. Such an organization should be able to deploy large amounts of capital effectively, by investing in its people. But this is a very different kind of diversification from the old-school GE model. We can’t just diversify across industries or geographies. We can’t even rely on a suite of line-extension products. We have to continually invent new categories of products, new platforms, and new business models – all extremely risky bets. Oh, and by the way, we still have to execute flawlessly. (Even the smallest flaw with an antenna can derail the whole train.)

Today’s management reality is just plain harder than that of the past. General managers need to know how to manage the execution-oriented "twentieth century" general managers that work for them. But they also need to know how to manage the new entrepreneurial managers that, increasingly, are essential to their growth. In other words, the old “manager vs. entrepreneur” dichotomy is breaking down. You cannot be a competent general manager in today’s economy if you do not understand entrepreneurship.

We are living in a transitional moment. The last of the old-school IPO companies are behind us (at least in software), and yet we have not yet witnessed the new-style IPO companies. Despite Google’s reputation as an innovative company, they seem to me to be counted as one of the last of the old breed. Their entrepreneurial successes are mostly to be found outside the organization, in the form of ex-Googlers who became entrepreneurs. Their internal “startup” projects seem, at least to my eye, to have a success rate only marginally higher than Microsoft’s (perhaps with Android – an acqusition – as a major exception). Certainly a large proportion of them end in failure.

The new breed of company currently finds itself satisfied with private capital, and has no need for an IPO. I think Zynga and other games companies may be the earliest exemplars for us to look at. Game companies naturally lend themselves to a “studio” model, with semi-autonomous teams building out their own franchise of sequels and spin-offs. From public reports, it seems like Zynga has really mastered a formula for innovating repeatedly and relentlessly across segments, platforms, and genres. And there are plenty of imitators and fast followers on their way. Will that cohort of companies need an IPO?

When private capital is available in sufficient quantities to satisfy investor and founder liquidity needs, why go IPO? The only reason I can think of is when you need dramatically more capital to grow your business, at a magnitude only available on the public markets and therefore worth the loss of control that going public entails. Our leading crop of pre-IPO web companies apparently do not need that much capital – yet.

It’s a debatable proposition why they don’t need IPO levels of cash. I freely admit that I have no inside information, no unique insight into what they are thinking. But I am nonetheless confident in my prediction: they don’t yet have the ability to manage an innovation factory at that scale. That’s not a criticism; nobody knows how – yet.

We need a new generation of managers trained in a comprehensive management theory of entrepreneurship. Comprehensive means it has to address all aspects of a startups life: marketing and product development, especially. It has to address all stages of startup growth and development – especially including the evolution into a true innovation factory. Tomorrow’s managers will need to know how to build a learning organization (where progress is measured by validated learning) and an execution organization (where progress is measured according to traditional value streams). They will need to know how to combine those organizations into one coherent whole.

I believe that the Lean Startup is the first such comprehensive entrepreneurship theory. But these are still early days. We have much work to do. We face problems today that would have bewildered the earliest management theorists. Their struggle was to use management to create enough productivity to feed, clothe, and house the world. Our civilization has excess capacity everywhere. We can build anything we can imagine. But the ranks of highly educated unemployed and the abysmal failure rates of new products both speak to the same question that needs answering: not, “can it be built?” but rather, “should it be built?”

The sum total of all we know about entrepreneurship is just the tip of the iceberg. We need to be disciplined, to study what works scientifically, and – above all - to introduce scientific methods into the practice of entrepreneurship itself.

When we master that, I think we’ll know what to do with IPO’s again. At least, that's my speculation. In the meantime, it’s going to be fun.

Founder personalities and the “first-class man” theory of management

At any given time, something like four percent of the US population is engaged in some form of new-company-creation. And that narrow definition of entrepreneurship doesn’t count all of the managers inside established companies who are effectively engaged in the same process of building an internal startup (see What is a startup? for my more expansive definition).

What motivates all these entrepreneurs? Typical explanations tend to focus on the well-known anecdotes and larger than life archetypes we have in mind: the twenty-something college dropouts (men, of course) from Stanford inventing some radical new technology. The academic research tells a very different story.

What do entrepreneurs look like? Are they born or made? This is a hard question. I think the root cause of that difficulty is that we tend to conflate two different questions into one. First, what causes someone to attempt entrepreneurship instead of a more traditional career path? And second, what attributes make someone likely to be a successful entrepreneur?

The difficulty lies in this paradox: many of the attributes that increase the likelihood of becoming an entrepreneur actually impede startup success.

Let’s start with the startup personality attributes. The academic research here is extraordinary. Here are the personality traits that are positively correlated with likelihood to pursue entrepreneurship: extraversion, skepticism, need for achievement, risk taking, desire for independence, locus of control, self efficacy, overconfidence, representativeness (the tendency to over-generalize from small samples), and intuition.

I think most of those factors correspond to our shared image of what an entrepreneur is supposed to look like. But many other attributes (especially demographic realities) cut against that stereotype. For example, Vivek Wadhwa and others have shown that most entrepreneurs are much older than we expect. Career experience and industry expertise are both positively correlated with entrepreneurship: contrary to stereotype, most entrepreneurs are not young and inexperienced outsiders. And unlike some psychological factors, these experience-based factors also increase the odds of the subsequent venture being successful.

It is in the psychological factors that we find the most paradox. For example, consider the propensity for risk-taking. Research has demonstrated the obvious: that people who have greater tolerance for risk or ambiguity are more likely to attempt entrepreneurship. That’s not too surprising. But does a risk-taking attitude actually lead to more startup success? The studies that have looked at this question in particular have found a negative correlation between risk-taking behavior and startup success.

That doesn’t strike me as shocking. And, although this hasn’t been subjected to a great deal of study (yet), I believe this same pattern will be found in a variety of other entrepreneurial characteristics: overconfidence, determination to succeed, perseverance, and even the desire to be in control. All of these factors are helpful in getting people to take the plunge, but all of them cause serious impairment of decision-making down the road. Think of the startups you know who are caught in a reality distortion field, heading full-speed off a cliff. Most likely, you will find the above attributes in excess supply.

I believe this is also why breakthrough success stories in entrepreneurship often feature a “classic” zany entrepreneur paired with someone you wouldn’t expect to be taking those kinds of risks. We often talk about this as the “visionary” and “the quant” or the “leader” and the “manager.” But I’m not convinced those labels are right at all. I think it much more likely that we’re seeing the embodiment – in the form of personality - of the “problem team/solution team” organizational structure. One team is in charge of carrying out the vision as currently specified, and one team is constantly asking the skeptical questions: who is the customer? Are we solving the right problem?

Although we have historically viewed this structure in startups by focusing on the personalities of the founders, I think that reflects our current, relatively poor, understanding of how startups work. We can do better by focusing on process instead of personality. We can consciously organize startups to become much more resilient organizations. Otherwise, we risk having them degenerate into cults of personality.

In the early twentieth century, before the advent of scientific management, the overriding management philosophy was that of the first-class man (and they were always men). The idea was, for any job, if you can simply find an individual with just that right combination of virtues, talents, and experience, you could safely delegate all decisions to them. Sound familiar? This kind of reasoning is almost impossible to disprove. If you empower someone to make decisions and then something goes horribly wrong, does that disprove the first-class man theory? Probably not; it’s much easier to blame the particular person who made the mistakes. In fact, making mistakes is seen as “proof” of being second-class.

In management jobs related to operations – that is, the people tasked with actually making and distributing physical products – this kind of thinking is now considered ludicrous, thanks to a century of progress. Our modern philosophy of management has this core belief (taken straight from scientific management) at its heart: that the performance of companies is determined by the systems they create, not just the people they hire. No amount of individual superstardom can overcome a badly organized factory, because the weight of the system eventually overwhelms any well-intentioned but poorly organized resistance.

Yet we tolerate our modern version of the first-class man theory in the management of more “fuzzy” topics, especially innovation and entrepreneurship. When we look back on this period in history, it will seem just as ludicrous to future entrepreneurs as pre-scientific management looks to us.

I am determined to do everything I can to hasten the arrival of that day. If you’re part of the Lean Startup movement, then you’re actually making it happen. Thank you.


All of the academic research alluded to in this essay is drawn from Scott Shane’s General Theory of Entrepreneurship which is a fantastic and wide-ranging overview of the state of the art in academic research on entrepreneurship.

The Entrepreneur’s Guide to Customer Development

Brant Cooper and Patrick Vlaskovits have written a new book, The Entrepreneur’s Guide to Customer Development, which builds upon the foundational work of The Four Steps to the Epiphany, while improving accessibility, updating the ideas, and making it more actionable. I believe it is the best introduction to Customer Development you can buy.

As all of you know, Steve Blank is the progenitor of Customer Development and author of The Four Steps to the Epiphany. I have personally sold many copies of his book, and continue to recommend it as one of the most important books a startup founder can read. 

I used to give copies of Four Steps out to my employees, in the hopes that it would instantly indoctrinate them into the methodology of Customer Development. I just assumed that everybody would love the book as much as I did, and would instantly change their behavior based on what they read in a book. You can imagine how well that worked. Instead of that naive approach, I wish I'd had a book like this one, to help me figure out how to get started with customer development step-by-step. 

When I wrote a review of Four Steps on this blog in November, 2008, I did my best to be candid and warn of a few shortcomings:

And Steve is the first to admit that it's a "turgid" read, without a great deal of narrative flow. It's part workbook, part war story compendium, part theoretical treatise, and part manifesto. It's trying to do way too many things at once. On the plus side, that means it's a great deal. On the minus side, that has made it a wee bit hard to understand.

Brant and Patrick undertook a difficult challenge: to provide a generally accessible introduction to Customer Development, without diluting its impact or dumbing-down its principles. I think they've succeeded.

The Entrepreneur’s Guide is an easy read.  It is written in a conversational tone, doesn't take itself too seriously, and avoids extraneous fluff. It does a great job of laying out general principles and suggesting specific, highly actionable tactics. You can easily take from it whatever makes sense for your business, and leave the rest. And it's incredibly to-the-point: you can digest this book in a couple of hours.

While the customer development framework of Four Steps is universally relevant, The Entrepreneur’s Guide updates its practices for modern startups. Four Steps primarily centers its stories and case studies on B2B hardware and software startups. This new volume also tackles examples from the Internet and wireless startups of today, both B2B and B2C. And throughout, they maintain a thoroughly realistic take on the power - and limitations - of an entrepreneurship methodology:

Successful implementation of Customer Development, let alone simply believing in it, will not guarantee success for your business. Customer Development will help you – force you – to make better decisions based on tested hypotheses, rather than untested assumptions. The results of the Customer Development process may indicate that the assumptions about your product, your customers and your market are all wrong. In fact, they probably will. And then it is your responsibility, as the idea-generator (read: entrepreneur), to interpret the data you have elicited and modify your next set of assumptions to iterate upon.

Many “airport business books” urge entrepreneurs to never give in. They tell them to persist in their dream of building a great product and/or company, no matter what the odds are or what the market might be telling them – success is just around the corner. They tend to illustrate this sort of advice with inspiring stories of entrepreneurs who succeeded against all odds and simply refused to throw in the towel. While maintaining persistence and willpower is certainly good advice, Customer Development methodologies are designed to give you data and feedback you may not want to hear. It is incumbent upon you to listen.

The Entrepreneur’s Guide to Customer Development includes four powerful case studies/interviews with successful entrepreneurs who have taken iterative approaches to their respective startups that very much resemble the spirit of Lean Startups and Customer Development.  I found these to be particularly interesting and worthwhile.

At the heart of Brant and Patrick's interpretation of Customer Development is their belief that its fundamental teaching is to question assumptions. This gives them a hook with which to apply their ideas to a wide variety of situations. In other words, if particular examples in the book don’t apply to you directly, Brant and Patrick show you how to figure out what might work for you.  This is important, since every situation is different.  I'll give them the last word:

You are already skeptical of Customer Development and Lean Startups and the slew of emerging buzzwords and supple-to-the-point-of-meaningless terms. That’s great, more power to you; we applaud your skepticism. But be philosophically consistent: periodically take the time to question your own expertise and that of your friends, partners and investors. Make the effort to test your assumptions.

If there’s a shortcoming to this book, it’s that it focuses primarily on the Customer Discovery step in The Four Steps.  Here’s hoping they soon tackle Customer Validation. Well done, Brant and Patrick. I can't wait to see what's next. In the meantime, go buy a copy of The Entrepreneur´s Guide to Customer Development right now.

What is a startup?

I think most people have a fairly specific image that gets conjured up when they hear the word startup. Maybe it’s the “two guys in a garage” made famous by HP, or the idea of Jobs and Wozniack walking barefoot and shaggy through the Homebrew Computer Club. Maybe it’s the more recent wunderkinds like Zuckerberg or Brin and Page. What all of these pictures have in common is a narrative that goes something like this: scrappy outsiders, possessed of a unique genius, took outrageous risks and worked incomprehensible hours to beat the odds.

But this cinematic view of entrepreneurs is flawed in many ways. Let’s start with the most basic. It leads people to mistakenly believe that any time they see two guys in a garage attempting the impossible, that’s a startup. Wrong. It also causes them to miss the numerous other kinds of startups that appear in less-glamorous settings: inside enterprises, non-profits, and even governments. And because both small businesses and startups have a high mortality rate, sometimes these images lead us to believe that any small business is a startup. Wrong again.

So let’s begin with a definition of a startup that captures its essential nature, and tries to leave behind the specific associations of the most famous startups.

A startup is a human institution designed to deliver a new product or service under conditions of extreme uncertainty.

Let’s take each of these pieces in turn. First, I want to emphasize the human institution aspect, because this is completely lost in the “two guys in a garage” story. The word institution connotes bureaucracy, process, even lethargy. How can that be part of a startup? Yet, the real stories of successful startups are full of activities that can rightly be called institution-building: hiring creative employees, coordinating their activities, and creating a company culture that delivers results. Although some startups may approach these activities in radical ways, they are nonetheless key ingredients in their success.

Isn’t the word human redundant in this definition? What other kinds of institutions are there, anyway? And yet, we so often loose sight of the fact that startups are not their products, their technological breakthroughs, or even their data. Even for companies that essentially have only one product, the value the company creates is located not in the product itself but with the people and their organization who built it. To see proof of this, simply observe the results of the large majorities of corporate acquisitions of startups. In most cases, essential aspects of the startup are lost, even when the product, its brand, and even its employment contracts are preserved. A startup is greater than the sum of its parts; it is an acutely human enterprise.

And yet the newness of a startup’s product or service is also a key part of the definition. This is a tricky part of the definition, too. I prefer to take the most expansive possible definition of product, one that encompasses any source of value for a set of people who voluntarily choose to become customers. This is equally true of a packaged good in a grocery store, an ecommerce website, a non-profit social service or a variety of government programs. In every case, the organization is dedicated to uncovering a new source of value for customers, and cares about the actual impact of its work on those customers (by contrast, a monopoly or true bureaucracy generally doesn’t care and only seeks to perpetuate itself).

It’s also important that we’re talking innovation, but this should also be understood broadly. Even the most radical new inventions always build upon previous technology. Many startups don’t innovate at all in the product dimension, but use other kinds of innovation: repurposing an existing technology for a new use, devising a new business model that unlocks value that was previously hidden, or even simply bringing a product or service to a new location or set of customers previously underserved. In all of these cases, innovation is at the heart of the company’s success.

Because innovation is inherently risky, there may be outsized economic returns for startups that are able to harness the risk in a new way – but this is not an essential part of the startup character. The real question is: “what is the degree of innovation that this business proposes to accomplish?”

There is one last important part of this definition: the context in which the innovation happens. Most businesses – large and small alike – are typically excluded by this context. Startups are designed to confront situations of extreme uncertainty. To open up a new business that is an exact clone of an existing business, all the way down to the business model, pricing, target customer, and specific product may, under many circumstances, be an attractive economic investment. But it is not a startup, because its success depends only on decent execution – so much so that this success can be modeled with high accuracy. This is why so many small businesses can be financed with simple bank loans; the level of risk and uncertainty is well enough understood that a reasonably intelligent loan officer can assess its prospects.

Thus, the land of startups is a unique place, where the risks themselves are unknown. Contrast this with other high-risk situations, like buying a high-risk stock. Although the specific payoff of a specific risky stock is not known, investing in many such stocks can be modeled accurately. Thus a decent financial advisor can give you a reasonably accurate long-term expected return for a set of risky stocks. When the “risk premium” is known, we are not in startup land. In fact, when viewed in retrospect, most startups appear like no-brainers. Probably the most famous example today is Google: how did we ever live without it? Building that particular product was not nearly has risky as it seemed at the time; in fact, I think it is a reasonable inference to say that it was almost guaranteed to succeed. It just wasn’t possible for anyone to know that ahead of time.

Startups are designed for the situations that cannot be modeled, are not clear-cut, and where the risk is not necessarily large – it’s just not yet known. I emphasize this point because it is necessary to motivate large amounts of the theory of the lean startup. Fundamentally, the lean startup is a methodology for coping with uncertainty and unknowns with agility, poise, and ruthless efficiency. It is a completely different experience from the equally hard job of executing in a traditional kind of business, and my goal is not to disparage those other practitioners – after all, most startups aspire to become non-startups someday.

Still, these differences matter, because the “best practices” that are learned in other contexts do not transplant well into the startup soil. In fact the most spectacular startup failures result when people were in a startup situation but failed to recognize it, or failed to recognize what it meant for their behavior.

This definition is also important for what it excludes. Notice that it says nothing about the size of the company involved. Big companies often fail because they find themselves in a startup situation but are unable to reorient in time to cope with this situation; this specific pathology is explored in The Innovator’s Dilemma. This kind of crisis can be precipitated by many external factors: macroeconomic changes, trade policy, technological change, or even cultural shifts. But most often, the entrant of a startup into a previously calm market precipitates this kind of crisis. This has significant implications for general managers in enterprise, about which you can read more at HBR: Is Entrepreneurship a Management Science?

No departments

Big companies have departments. Startups are companies. Startups aspire to become big companies. Therefore, startups should have departments. Right?

Why do companies have departments? There are a lot of reasons: ladder of advancement, sharing of best practices, functional specialization. Each of these benefits also exists in startups, which is why most startups are also organized in departments. But I have come to believe that because of the unique context of startup land, the payoff is a lot smaller than it is for larger companies. Meanwhile the drawbacks of functional departments can cause real and lasting harm.

I once worked at a startup with an exceptional functional department system. The leaders of each department were world-class experts in their respective fields. The team hired only the best and the brightest. Looking back after a few years, it’s evident that many of the people who worked in these departments have gone on to do incredible things in industry. They are leaders, visionaries, founders and managers having tremendous success.

Yet talent organized improperly can lead to failure. I was an engineer on the engineering team. We had to work closely with artists on an art team. We sat in different parts of the building, ate lunch separately, spoke a different specialized jargon, and generally didn’t understand each other. According to the Waterfall methodology in which we worked, this shouldn’t have been a problem. After all, we rarely had to work on the same project at the same time. The art team would often be involved in the specification phase of a new feature, since they were responsible for the look-and-feel of the product. Then we’d build the feature, which would often include tools that were intended for the art team to use to build the parts of the product that they were responsible for (in video game parlance, this is the “art path” that allows artists to get their work into the production product).

Sure, some communication was necessary, especially as artists had to be trained on new tools periodically. But according to the theory, this should have been covered by the various specs and documentation we were rigorous about producing.

If anyone has ever worked in an environment like this, you’ll probably be able to imagine the things that can go wrong. For one, the engineers consider the artists stupid; the artists consider the engineers arrogant. Not a lot of trust builds up that can be used when real disagreements emerge. Instead, there’s a positive feedback loop of bad feelings. And like feedback on a simple microphone sound system, this would occasionally boil over into screeching.

I remember one such meeting vividly. I was the junior guy on a project team; I was called in to do some technical due diligence for reasons that were obscure to me, because the team already had much more senior engineers assigned to it. I was invited to a feature decision meeting, where the team was closing in on a detailed spec. The meeting was tense. The artists on the team had called in the big guns, and VP-level folks were there to explain the importance of certain aspects of the visual design that threatened to be cut. I eventually realized that I was there as part of the same plan – the art team has specifically requested someone technical but unimportant to be able to render opinions that might undermine their more senior opposition. Not to be outdone, the technologists on the team had also brought their big guns, and the meeting was packed with employees of every level – from VP’s all the way down to me.

As the meeting progressed, the temperature kept rising. At first, I couldn’t even follow the recriminations back-and-forth. Eventually, though, I realized what was at issue: the art team was insisting that the UI for this feature have rounded corners. Incredibly, they were willing to bring the company to a standstill to protest that this was an absolutely essential feature. Even more surprisingly, the engineering team was equally vocal about their contention that adding rounded corners would add weeks of development time to the project, which would have pushed it out way past its hard deadline, effectively killing it. On the surface, this was a ludicrous dispute – both sides were willing to kill the project rather than proceed with (or without) a minor UI tweak. Were they just crazy?

I don’t think so. This meeting was just the latest in a series of escalating skirmishes that had taken place over many months. The feedback loop looked something like this. The art team would create a spec for a feature, detailing the UI as best they could. The engineering team would then build that feature, mimicking the UI as close as they could using the current primitives supported by the system. When the art team would review the final product, they were inevitably outraged – it deviated from the spec in ways they considered major. So there would be a lot of scope negotiation at the very end, when it is most expensive. Sometimes, the art team would win the argument, and the engineers would pull a few all-nighters to make them happy, but feeling betrayed at the new additions to the spec. Sometimes the engineering team would win, and the art team would have to accept (and be held responsible) for a feature that didn’t really work the way they wanted, feeling betrayed at the violation of their agreement.

As an isolated incident, this wouldn’t be a big deal. But scope negotiations between departments are an example of an “iterated prisoner’s dilemma” situation, where the same parties repeatedly negotiate, and rely on their previous experience to inform their choices in the current round. Unfortunately, the equilibrium in this particular setup has one overriding outcome: longer and more detailed specs. Here’s why.

The art team feels burned that they didn’t get what they asked for. Last time, the engineers weaseled out of their commitments by point out areas where the spec didn’t specify what was really important to the artists. So this time, they are going to spell out what’s important in even greater detail, to leave less wiggle room.

The engineering team feels burned too, and feels that they were blamed for deficiencies in the spec as if it was their fault that the technology doesn’t really support what the artists want to do. So they react in two ways. First, they actively encourage a more detailed spec, and are more aggressive about pointing out possible inconsistencies. This forces the art team to make concrete decisions about stuff they don’t really care about. Second, the engineering team starts to pad their estimates, knowing that each feature in the spec is not really done when they think it’s done – there’s going to be inevitable scope creep from the art team when they finally see the final result.

What are the consequences of this more detailed spec? For one, it takes a lot longer to create, meaning that the projects themselves get larger in order to rationalize the increased investment in planning. Second, the extra detail obscures the artists’ original intent in specifying the feature, so the engineers are even more likely to miss the big picture and build the wrong things. And lastly, it removes the engineering team’s ability to find breakthrough solutions that might deliver most of the value at a fraction of the cost. They can’t use any discretion for fear of breaking the spec’s contract, even if the changes would probably go unnoticed or even be in the company’s best interests. The lack of trust (and the procedure of the Waterfall methodology) makes it very difficult to ask for clarification or changes in the spec while the implementation is underway.

This feedback is a nasty trap, and it’s just how this room full of otherwise rational adults wound up in a screaming match about rounded corners. It was painful to watch. Now, part of the reason I remember this particular meeting so well is that I wound up doing something considered really radical at the time. I suggested that we change the underlying architecture of our UI system so that the artists would be able to build their own UI pieces themselves and then integrate them into the product without requiring new code every time. It took an incredible amount of politicking and arm-twisted, but I did eventually get the teams to agree to that solution. I’m proud of that contribution, but the reason I tell that part of the story is not to show off, but rather to be able to tell you what happened next. Although both teams got something valuable about the new system, neither was very happy. I had successfully defused the situation, and by reducing the feedback loop between the artists spec and its implementation, I was able to help them realize their goals better. As a technical fix, it was brilliant. As a solution to the underlying problem, it was useless.

Neither side liked me very much for having "fixed" their problem. In particular, the artists felt like I had created a lot more work for them – they were used to having other people grapple with implementation details for them, and now they had to do it themselves. They either had to hire a developer onto the art team itself (unthinkable) or learn those development skills themselves (which was, to be fair, really hard). The engineering team wasn’t happy either. Creating this new architecture was a fair bit of work, and they couldn’t shake the feeling that I had basically sided with the enemy, giving them tools that would require a lot of engineering support but basically deprive the engineering team of any credit for the resulting features.

I’ve now come to believe that this confrontation was a direct result of the partition between the departments in this company, and that rather than seek technical solutions to the disagreement, I would have been better off working to break down those walls.

Let me tell one more story from that same company. I mentioned before the art path, the set of tools the engineering team was required to build and maintain for the art team to be able to create content. Because the art team was considered an internal customer (and “friendly” to boot), we didn’t waste a lot of time making the tools easy to use. Instead, we spent time making sure the exact behavior of the tools were well documented – by engineers, naturally.

This led to some pretty bizarre situations. One time I remember in particular, an engineer fixed a bug that was causing artists’ creations to render in our 3D environment with a skewed rotation. The details aren’t important, but it turned out that under certain relatively common conditions, the objects would come out completely upside-down.

I’m sure this engineer was expecting to be treated as a hero by the art team, since he had just fixed a major bug. But instead he was reviled. Why? The artists had known about this bug for ages, but had just assumed it was a natural part of the system. They had learned that the best way to solve problems is through trial-and-error, doing whatever it takes to get the object to look right. So sometimes the object would come out upside-down, sometimes not. When it did, they’d just invert their original work so that after the upside-down transformation, it looked right. Sure, there was a lot of randomness in whether they would need to take this extra step, but this was nothing unusual. From their point of view, the tools were full of meaningless jargon and bizarre incantations that resulted in nearly random behavior. This bug was not even considered a major one, since at least it had a deterministic fix.

But now you can see what happened when the bug was fixed. A large, but mostly random, sample of objects started rendering upside-down! This required either that we leave the bug in place, or that the art team go back and rework all those thousands of upside-down objects. Neither solution is particularly alluring.

Again, this led to lots of mutual recriminations. Why didn’t the art team come to the engineers and tell them of this bug as soon as it manifested? Why aren’t they smart enough to figure out what the tools actually do? On the other side, why don’t the engineers just deliver tools that work like they’re supposed to? Other more established companies have tools that are intuitive to use, why can’t we?

The solution to this problem is actually really simple. Just create an art path team, composed of some artists and engineers. Force them to live and work in the same physical space, force the engineers to actually do some art production, and force the artists to actually learn what the technical limits of the tools are. As the team gets traction, simply rotate members from both departments through this team, so that the knowledge they gain is eventually diffused through both organizations. And hold the leaders of that team – artists and engineers alike – accountable for a set of clear goals for the tools that are important to the company.

There’s nothing intrinsically difficult about this problem, just as their was nothing intrinsically difficult with the “rounded corners” problem I discussed earlier. The barrier to doing the right thing is the entrenched ideas originating from departments. Who would lead this team? Who would they report to? How would we ensure that the team was faithful to the best practices of both the art team and the engineering team? And plus, do we really want to be cross-training engineers in art and artists in engineering? Isn’t that a waste of time? After all, both teams are already too busy, how can we afford to pull people off and waste time?

This is another variation of the time/quality/money fallacy – the very quality problems that a team like this would address are currently wasting time and causing the team to be “too busy” to invest in the solution. So: enough with functional departments at startups. Let's start holding people accountable solely for their contribution to the only thing that matters: validated learning about customers.

The Five Whys for Startups (for Harvard Business Review)

I continue my series for Harvard Business Review with the Lean Startup technique called Five Whys. Five Whys has its origins in the Toyota Production System. I've written about this before in some detail, but this was an opportunity to try and frame it for a general business audience. After all, Five Whys is the most general, most transferable technique in the toolkit, because it can act as a natural speed regulator for any kind of work. (If you're curious about the theory behind this idea, see Work in small batches.)

The Five Whys for Start-Ups - The Conversation - Harvard Business Review

Root cause analysis and preventive maintenance are concepts we expect to see in a factory setting. Start-ups supposedly don't have time for detailed processes and procedures. And yet the key to startup speed is to maintain a disciplined approach to testing and evaluating new products, features, and ideas. As start-ups scale, this agility will be lost unless the founders maintain a consistent investment in that discipline. Techniques from lean manufacturing can be part of a startup's innovation culture.

One such technique is called Five Whys, which has its origins in the Toyota Production System, and posits that behind every supposedly technical problem is actually a human problem. Applied to a start-up, here's how it works....

Read the rest of The Five Whys for Start-Ups.

You can view previous essays in this series here:

• Is Entrepreneurship a Management Science?
• Two Ways to Hold Entrepreneurs Accountable
• Beware of Vanity Metrics 
• For Startups, How Much Process Is Too Much? 
• Speed up or slow down?