Thursday, March 11, 2010

For Startups, How Much Process Is Too Much? (for Harvard Business Review)

In the latest article for my series in HBR, I discuss the problem of how to figure out how much process startups should have. I often hear that what makes startups effective is their complete lack of process, but I don't think this is correct. Process is not the same as bureaucracy. In fact, I believe process is a form of discipline. When done right, it can help startups accelerate even as they scale.

You can view previous entries:

I'm a little late on the cross-post, but hope you'll enjoy it nonetheless.

For Startups, How Much Process Is Too Much? - The Conversation - Harvard Business Review:
Still, startups develop some kind of process — whether it's disciplined, haphazard, bureaucratic or empowering — because building a great product depends on it.

They just need to balance process with innovation. Companies that insist on building a world-class infrastructure before shipping a product are doomed to 'achieve failure,' because they're starved of feedback for too long. I learned this lesson first hand in a previous company (read the sad story here). On the other hand, companies that take a 'just do it' attitude without any process at all are also taking a major gamble. High-profile startup Friendster had first-mover advantage in the social networking space, but created openings for competitors when it could not scale to meet demand.

Finding the right balance requires an understanding of the fundamental feedback loop that powers all startups.

Read the rest: For Startups, How Much Process Is Too Much? - The Conversation - Harvard Business Review

Tuesday, March 9, 2010

Startup Lessons Learned - the Conference (April 23, 2010 in SF)

Today we are opening up registration for the first ever Startup Lessons Learned Conference on April 23, 2010 in San Francisco. I'm incredibly excited. The event is being produced in partnership with Charles Hudson, who puts on many of Silicon Valley's top events, including the forthcoming Freemium Summit.

The Startup Lessons Learned Conference is by-and-for entrepreneurs, and only entrepreneurs. We have a lineup of speakers who are primarily active practitioners of the lean startup methodology. They'll be speaking about their real-life experiences trying to put these ideas into practice. You'll also have a chance to hear from the leading lights of the lean startup movement, including Steve Blank, Sean Ellis, Dave McClure, and many more.

I have spent a lot of time over the past few years experimenting with what helps startup teams adopt new practices. One pattern has been overwhelmingly clear: it works best when a cross-functional team hears about new ideas all at the same time. As a result, we've built this event to be best experienced by teams, not just individuals. We have special team tickets; these teams will have special seating at the event and special access to mentors, who can answer questions and suggest ways to incorporate ideas from the speakers into each team's company.

This event is for present and future entrepreneurs only - not service vendors or investors. If you are working on new product development in any sized company, you are welcome to attend. Remember, a startup is "a human institution designed to create something new under conditions of extreme uncertainty." If that describes your job, you're welcome to attend. (We will reserve a limited number of spots for sponsors to attend; if you'd like to become a sponsor, please get in touch.)

For those that are able to make the trip to San Francisco, I hope you'll make the effort. I think this is going to be a one of a kind event, and you'll be glad you came. That said, for those who cannot make the trip, we're working to provide simulcast venues in cities around the world. We'll have more details shortly. In the meantime, if you'd like to attend remotely, or volunteer to host a viewing, please sign up as part of this survey. And for those of you who have already volunteered, please stay tuned for details.

A partial list of speakers and mentors is posted on the registration page. More will be posted as we're able to confirm them. If you'd like to nominate someone to be a speaker or mentor, please feel free to leave a comment on this post. We'll consider all nominees; when possible, please include a link to a video of them speaking or to a relevant blog post.

Last, I always try to have a scholarship program for paid events, and this one is no exception. If you'd like to sponsor a scholarship for a deserving startup that can't afford to make the trip, please let me know. At past events, the generosity of these scholarships has been amazing, and has meant a whole host of interesting people were able to benefit who would not have been able to otherwise. To those who can make a donation, you have my thanks. We'll post details of how to apply for scholarships after we get a sense for how many we will be able to award.

Saturday, March 6, 2010

Startup Visa update

As I write this, I am traveling at 30,000 feet on Virgin America's excellent IAD->SFO nonstop, following an intense few days of conversations in Washington, DC about the Startup Visa. This trip was designed to build momentum following last week's announcement that Senators Kerry and Lugar have introduced a Senate bill to make the Startup Visa a reality: the Startup Visa Act of 2010. This follows Congressman Jared Polis, who has introduced a similar bill as part of comprehensive immigration reform in the House.

Our trip was bipartisan, bicameral, and bi-branches-of-government. Everywhere we went, the Startup Visa received a fair hearing, despite the fact that we have no lobbyists, no PAC, and no organized presence in Washington. On the flight to DC, we had over 5000 registered voters join our "tweet hall" and generate a huge stack of paper which we delivered to members of Congress, courtesy of 2gov.



Grt Startup Visa team mtg with Senator Udall from Colorado. @... on TwitpicShare photos on twitter with Twitpic

Founders #startupvisa talking at the SBA!! on TwitpicWe are here!! #startupvisa on Twitpic


Your support has been critical to everything that's happened with Startup Visa. After so many of you stepped forward to show your support in last year's 2gov campaign, I got a call out of the blue from a member of Senator Lugar's staff. He gave me the opportunity to answer questions about our proposal, and share the stories of immigrant entrepreneurs who are creating job all over America, including the Senator's home state of Indiana. I firmly believe that without your grassroots organizing, this meeting would never have taken place, and who knows if, months later, the Startup Visa Act would have been introduced?
 
Having bills introduced in both chambers of Congress is a huge accomplishment. But it's still just the very beginning of the marathon required to get a bill to become law. For those who want to continue to press the case for the Startup Visa, here are five ways you can help today:
  1. Go to our campaign page and voice your support
  2. Write to your local newspapers, and let them know that you support the Startup Visa Bill
  3. Call your senators, and let them know about why you support the Startup Visa legislation
  4. Add the Startup Visa Widget to your blog or website
  5. Contribute to Startup Visa so that we can continue to spread the word!

And, on an unrelated note, we also had an entertaining night at the DC Lean Startup Meetup featuring me and Dave McClure, in what was billed as a "Lean Geek SMACKdown." You can judge for yourself if we lived up to that promise:


Lean Startup SmackDown from Frank Gruber on Vimeo.

Sunday, February 28, 2010

Kiwi lean startup + Australia next

Wrapping up a fabulous few weeks in New Zealand, where I had the privilege of attending some great events, like Kiwi Foo and Webstock; met some amazing entrepreneurs and inventors (yes, including a jet pack); and generally enjoyed a supportive and enthusiastic reception. I want to especially thank the dozens of Kiwis who acted as my guardians and escorts from place to place, even driving me around on the wrong side of the road

Every tourist will tell you that New Zealand is a beautiful country, and they are not kidding.
 


I would add that the people I met were extraordinarily welcoming, friendly, and humble. In fact, you have to learn to adjust to the humility. Quite a few Kiwis told me that they "didn't count" as entrepreneurs, even though they own their own business. Their supposed lack of ambition was belied by the many cool demos and startups I got to meet. If you're in Europe, for example, keep an eye out for the innovative YikeBike, a new kind of personal transport device. It was described to me as "what the Segway should have been." Overall, I came away from the experience optimistic about the potential of New Zealand to cultivate a significant startup hub. I look forward to seeing it happen. If anyone is interested, the Wellington Lean Startup Meetup is a good place to start.

Up next is an extremely brief stop in Australia. I'm particularly looking forward to inaugurating the Sydney Lean Startup meetup. We'll be kicking it off with an event featuring yours truly Monday March 1 at 6pm. If you're a Sydney entrepreneur, I hope you'll stop by. Last time I checked, there were still a few half-price tickets left. You can register here.

Most of the events I did here were private, so there aren't as many videos and slides available. For now, you'll have to make do with the slides from my keynote at Webstock:



Hopefully, video of that talk will be available soon. For a preview, you can check out this "backstage pass" interview, which was recorded at Kiwi Foo a few days before.

I also did a radio interview on Radio New Zealand's Nine to Noon program. You can hear me attempt to explain the ideas behind the lean startup to the general public in MP3 or Ogg.

Webstock generated a lot of follow-on commentary, including a number of reviews. For a good synopsis of my talk (and the other keynotes), check out Idealog's blog. I also recorded an interview with them, which should be in the next issue of their extremely cool-looking print magazine. Other write-ups: NZ Herald, gianouts, Te Ara, Public Address, Bibliophile, BIB. Also, if you're an event organizer, check out some of the innovative ways the Webstock team encouraged attendees to interact with each other and with the speakers. Highlights for me were the Webstock Game and Webstock Bingo.

Last, if you didn't get a chance to see it, be sure to check out this video of the closing performance for Webstock (which was also an award show called the ONYAs). It was mind-expanding:

Monday, February 22, 2010

Why diversity matters (the meritocracy business)

Diversity is the canary in the coal mine for meritocracy. As entrepreneurs, more than any other industry, we’re in the meritocracy business. The companies that make decisions based on merit, rather than title, politics, or hierarchy execute faster and learn faster than their competitors. For startups (and other innovators), that’s a decisive advantage.

So when a team lacks diversity, that’s a bad sign. What are the odds that the decisions that were made to create that team were really meritocratic? That’s why I care a lot about diversity: not for its own sake, but because it is a source of strength for teams that have it, and a symptom of dysfunction for those that don’t.

There’s been a lot of hand-wringing about gender equity in the high-tech and entrepreneurship worlds lately. This is a good thing. (I want to especially recognize Vivek Wadhwa and Brad Feld for their leadership.) We have a lot of introspection to do. By any objective standard, we ought to be diverse. Our industries are young, so there hasn’t been time yet to become encrusted with too many traditions that exclude outsiders. The work itself, especially in startups, depends primarily on intelligence, communication, creativity and empathy. Even the most radical Bell Curve-style thinkers have to concede that even if there are differences between men and women in the distribution of these traits on average, these curve have substantial overlap, and there should still be a lot more of them represented in high-tech startups.

For the record, I don’t think such biological differences are one-sided in favor of men. In fact, recent research suggests just the opposite. Vivek Wadhwa and his team continue their excellent work investigating the true nature of entrepreneurship. Their recent article suggests that startups led by women are actually more successful, on average, than those led by men. This doesn’t surprise me at all, and you don’t have to support a biological determinism theory to see why. If women face structural barriers to becoming entrepreneurs, then those few who are able to overcome those barriers are probably exceptional to begin with. Or, maybe it is true that women make better entrepreneurs than men. Either way, we’ll benefit if more women are welcomed into startups and other high-tech companies.

Diverse teams make better decisions than homogenous ones. I won’t recap the academic research that underlies this assertion; for that, you should read James Surowecki’s excellent Wisdom of Crowds. The most counter-intuitive part of this phenomenon, though, is that this diversity still improves outcomes even when the added opinions are less accurate than the previous consensus. The hypothesis, which makes sense to me based on the teams I’ve worked with, is that having someone with a wacky outlier-type experience causes everyone else to reexamine their own assumptions and find flaws in their own argument. So, even if you already know best, you’ll still benefit from having others on your team to challenge you.

Diversity benefits men, too. One of the most pernicious effects of groupthink is the sense of entitlement it breeds. Teams that are complacent are less likely to challenge their own assumptions, less likely to listen to feedback and, therefore, less likely to learn. This is especially important across functional lines. I’ve seen many times what happens when a single department get’s holed-up in its own space, like the terrifying “operations cave.” Outsiders are afraid to enter, let alone make a suggestion. The safety of the group becomes an impediment to dealing with reality. Problems are easily dismissed as PEBKAC ("Problem Exists Between Keyboard And Chair") rather than as opportunities for root cause analysis. Engineers are offenders in this category too, but so is any gender-segregated activity, like an all-female PR or marketing team.

Diversity is not the only requirement for making good group decisions. Two others – that each team member give their input independently and that the results be objectively aggregated – are also key parts of building a meritocracy. To be clear, though, this diversity refers only to diversity of opinion, not necessarily to demographic diversity. So why is demographic diversity important?

Demographic diversity is an indicator. It’s a reasonable inference that a group that is homogeneous in appearance was probably chosen by a biased selector. Even if men have an innate advantage at software development, the gap would have to be massive in order to explain why startup after startup has an all-male team.

Another explanation is that of compounding effects throughout the “pipeline” of candidates. In school, women are discouraged from pursuing math and science. Smart women are often stigmatized. Teachers aren’t encouraging. Women are paid less than men. And sexism is surely implicated in many career decisions. These effects compound, becoming larger over time. But this can’t possibly be the whole story. If it were, entrepreneurship would have a much larger proportion of women at every level compare to other industries – because tech entrepreneurs are younger than successful people in most other fields. There’s much less of a career ladder to climb, and therefore less time for these compounding effects to add up.

Plus, technical skills are easily learned – in my experience – by anyone with sufficient IQ and determination. I have personally taught many “non-technical” people to program – graphic designers, QA folks, even artists and animators. When those people have the chance to contribute in a meritocratic environment, they have many opportunities to learn and grow. So why aren’t more women finding themselves in the position to get started on their 10,000 hours?

With identity issues, there is no dichotomy between symbolic action and substantive action, thanks to a human tendency known as priming. In psychology research, when subjects are simply reminded of some aspect of their identity, and then measured performing a task, the pre-task activity “primes” their brain in ways that affects their performance. If the priming involves a negative stereotype, it has negative effects. So, for example, asking men and women about their gender before giving them a math test produces a significantly different result than asking other innocuous questions. (See this study among many others: http://www.nyu.edu/nyutoday/article/430)

Even the fact that a startup is all-male can make it less likely that a women would want to join. Even worse, it might even affect her performance in an interview. And just solving the gender imbalance might not be helpful, if the solution involves yet more negative stereotypes. Which is why affirmative action doesn’t help. A company with an all-female support staff sends, in some ways, an even worse signal.

Understanding these problems can make the solutions seem intractable. After all, if you already have an all-male startup, you’re already at a disadvantage when it comes to hiring the very women that could fix that imbalance. But priming cuts both ways: when we go out of our way to affirm meritocracy, it actually improves everyone’s performance. In fact, explicitly making meritocracy a value is actually better than rejecting stereotypes – by calling attention to the stereotype, you’re still engaging in priming.

Instead of focusing on programs designed to specifically benefit any one group, I think our focus should be on making our companies as meritocratic as possible. I want to start with the easiest suggestion I can think of, one that I’ve personally used with great success. I first tried it as an experiment after reading in Blink that after symphony orchestras instituted blind auditions (where conductors can’t see who is actually playing), gender equality soon followed. In the US, women’s participation went from about 5% to 50% over the course of two decades. What’s notable about this change is that it has nothing to do with gender per se, and probably also eliminated many other forms of unconscious bias.

Now, whenever I screen resumes, I ask the recruiter to black out any demographic information from the resume itself: name, age, gender, country of origin. The first time I did this experiment, I felt a strange feeling of vertigo while reading the resume. “Who is this guy?” I had a hard time forming a visual image, which made it harder to try and compare each candidate to the successful people I’d worked with in the past. It was an uncomfortable feeling, which instantly revealed just how much I’d been relying on surface qualities when screening resumes before – even when I thought I was being 100% meritocratic. And, much to my surprise (and embarrassment), the kinds of people I started phone-screening changed immediately.

And yet, when I suggest this practice to hiring managers and recruiters alike, they rarely do it. Hiring managers say, “the recruiter would never go for it” while recruiters say, “the hiring manager won’t accept it.” What I think we’re really saying is: “I don’t want to know if I am biased.” That's understandable - it's embarrassing! Even if our biases are only implicit and not consciously held, the systems we build can still contain bias. When we change a hiring policy, especially if we do it in a visible way, we reap two benefits. We actually improve our hiring process and also signal our commitment to meritocracy.

Now, I don’t know how to conduct blind interviews for startup or high-tech jobs. If I did, I would try it in a heartbeat. But even after the initial resume screen, I think there are things we can do to ensure our decision process is merit-based. I’ve covered these in some detail in two previous posts: one on Assessing fit and one on conducting a technical interview. In short, it’s essential to keep the interview focused on the substance of the work the candidate will be asked to do, and not on gotcha or brain-teaser questions. Then, in the evaluation process, it’s essential that the discussion focus only on pertinent aspects of the candidate’s performance, again obeying the three rules of group decision-making: diversity, independence, and objective aggregation.

One last suggestion, which is a technique I learned from my IMVU co-founder Will Harvey. When it’s possible, I always believe in giving a promising candidate who interviewed poorly a chance to demonstrate their skills with a real application exercise. At my last company, for programming jobs, we’d give some candidates a chance to prove themselves by writing a real working program in just a day or two (usually, to write a version of Tetris from scratch). We’d do the evaluations of that code blind – without the person in the room. In some cases, we’d dramatically revise an opinion formed during our live interview. The work product is a more realistic test, although it requires much more work on the part of the candidate.

Beyond hiring, our actual work process matters, too. I already advocate cross-functional teams as part of the lean startup methodology. Here I will only touch on one of their benefits, which is the opportunity for people to learn new skills. When teams are capacity constrained, their natural inclination is to let work pile up. This leads to a loss of quality and an increase in cycle time, both of which are negative. For more on this subject, take a look at The product manager's lament. Suffice to say that when teams are left on their own to innovate, they take on a “whatever it takes” attitude. This often leads to people discovering new skills they didn’t know they had.

By itself, that’s a pretty minor impact. But combined with a true merit-based decision making process, it becomes much bigger. If teams are rewarded for the results they achieve, and ideas can come from anywhere on the team, there’s a positive feedback loop available for people who are willing to learn: as they take on new skills, they become more successful at their job. Contrast that with a traditional siloed department. If you only get promoted for getting better at design, you’re unlikely to get positive reinforcement for picking up some programming skills (and vice-versa). And when people who started out as designers or marketers (or other supposedly "soft" disciplines) have the opportunity to learn programming, we effectively create new pathways to entrepreneurship that bypass the traditional pipeline problems. And, like everything else I've discussed, this also cuts both ways: programmers who learn design skills are much more likely to become successful entrepreneurs, too. I'll let Dave McClure explain why, if you'd like the detailed argument.

There’s a lot more we can do than just these ideas. All the usual ideas are good ones: I support inviting more women to speak at conferences, the creation of women-centric networking events like Women 2.0 and Girls in Tech, reforming the way we teach math and science in public schools, adopting more family-friendly public policy, the creation of TiE-like organizations. And, of course, the Startup Visa will be a tremendous help as well. None of these are impossible. Last year, I traveled to dozens of cities talking about lean startups and meeting people interested in entrepreneurship. Almost everywhere, I spoke to rooms full of (mostly) men. There was one notable exception: Sweden. It’s not a coincidence; rather, it’s the result of proactive public policy. (And, before we get to that old tired argument: no, I do not believe you need to become a socialist country in order to achieve those results. I haven’t seen any evidence to support that assertion, although I hear it all the time.)

I think we could all do some serious thinking about how we can be part of the solution in our own lives. I’ve been in many rooms – and at countless events – with entrepreneurs and VC’s that I think would make women uncomfortable. The sexual jokes, the crude comments, even just the usual "success theater" chest thumping – all that macho BS. Personally, I have a moral objection to that behavior. But I recognize that there are plenty of people who disagree (and I’d welcome a chance to hear why they think it’s OK; I honestly don’t get it). We still shouldn’t tolerate it. We laugh at people who think software patents are awesome, or want to give the RIAA more power, or think big companies should have a veto power over new technologies that are “too disruptive.” Those are all positions that are coherent, understandable, and anti-meritocratic. We need to recognize that supporting a homogeneous status quo is just as dumb, and just as bad for our industry.


After all, we're in the meritocracy business.


Update: Some of the commenters both here and over at Hacker News seem to be struggling with the math part of the argument, which is that even if there are biological differences in ability, they are not large enough to explain the observed outcome. I'm not much of a statistician, so I am indebted to @hypatiadotca who shared this excellent and brief presentation by Terri Oda. It is so much better than my own attempts to argue the point that I am including it here. Enjoy:


Monday, February 8, 2010

Beware of Vanity Metrics (for Harvard Business Review)

The next article in my series on entrepreneurship for Harvard Business Review is live today. Once again, we revisit the topic of Actionable metrics and their nemesis: Vanity metrics. Any entrepreneur with a decent reality distortion field can find metrics that make it look like they're being successful. This "success theater" is occasionally useful for PR or getting through a tough board meeting. But it's lethal when we start to run the company using vanity metrics as a guide. But how do we know which metrics to look at? How do we know how much energy to invest in analytics vs. doing real work? And how do we get the whole team to take action that benefits real people when all they're looking at is abstract numbers on a screen? Remember "metrics are people, too." For more, read on...
Entrepreneurs: Beware of Vanity Metrics - The Conversation - Harvard Business Review:

The idea is simple. Establish baseline metrics by building the minimum viable product — the minimum required to measure the response of early adopters. Then, in each development cycle, use the insights gained by studying customers to make improvements. This is the source of validated learning — proof that the customer insights translate into tangible metrics improvements.

But this leaves a very difficult problem still to be solved: How do we know that these changes are what actually effect change in the metrics that we're observing?

This is the curse of vanity metrics, numbers which look good on paper but aren't action oriented: website hits, message volume, or 'billions and billions served.' They look great in a press release, but what do they accomplish?
Consider a scenario where a team makes a product change, and the very next month page views go up. As humans, we're hard-wired to infer causality from correlation: when the numbers go up, we tend to take credit. But when the numbers go down, we tend to blame someone or something else. Worse yet, different team members tend to attribute positive changes to whatever project they were working on at the time (but not negative changes, of course). As a result, different parts of the team are constantly "learning" in their own private reality. When those teams face difficult choices, it's incredibly hard for them to come together and make an informed, fact-based decision.

To avoid falling into this trap, I recommend you follow the three A's of metrics. All metrics should be actionable, accessible, and auditable...
Read the rest of Entrepreneurs: Beware of Vanity Metrics at The Conversation on Harvard Business Review. Don't forget to leave a comment.

Sunday, February 7, 2010

Tell your Startup Visa story

I've been promising big Startup Visa-related news on Twitter for a few weeks. There is news, it's very exciting, and I still can't share it. Soon, I promise. But in order for this idea to progress into legislation,  we need your help. Whether you're a US citizen or an immigrant, entrepreneur or investor, founder or employee, there's something you can do. This is especially true if you live outside the echo chamber of Silicon Valley. There are four ways to get involved below. Even if none apply to you, see if you know someone who could help. Thanks!

  1. If you are an immigrant founder who has helped build a company that has created jobs in the US, we need you to tell your story. If we receive your story by February 27th, it will become part of our Geeks on a Plane DC delegation (including me) talking to lawmakers. We've created a place where you can tell your Startup Visa story anonymously if you'd like. Or, if you'd prefer to do it by video, you can upload to YouTube - just use the "startup visa" tag.

  2. If you're a startup investor, and you support the goals of the Startup Visa proposal, we'd like you lend your name to our efforts. You can read about the latest iteration of the proposal at StartupVisa.com. If you're willing to publicly sign on to a letter of support, please get in touch. If you'd be willing to talk to the press about your support, please leave a comment here as well.

  3. If you are a US citizen that is employed at a company with at least one immigrant founder, we'd love to hear your stories, too. Part of our belief in advancing this legislation is that more startup founders means more jobs and economic growth for everyone. The fact of Americans standing up for our values - of openness, meritocracy, and entrepreneurship - is especially powerful.

  4. If you're registered to vote in the United States, and you'd like your elected representatives to know that you support the Startup Visa, you can register your support in just two minutes at http://2gov.org/visa. Although signing up is about as much work as a tweet, the impact is much larger. 2gov does the work to produce hard-copy reports of constituent sentiment and delivers them to the appropriate officials. So by signing up, you're sending a message directly to the decision-makers who depend on your votes for their jobs.
I hope you'll choose one of the steps above to help out. If you have other ideas, please feel free to leave them in a comment.

I know that sometimes campaigns like this can seem overwhelming. But your support has already had tremendous impact. The good news I hope we'll get to announce in a few days is the direct result of the twitter campaign you helped launch last year. At some point, I hope I'll be able to tell you more. For now, take my word for it. You are making this happen. Thank you.

Saturday, February 6, 2010

Speaking 2010: Webstock, GDC, Web 2.0, and more

I've been trying hard to cut back on travel in 2010 so as to have more time for writing. I started to feel like my blog became too much of a travel diary last year - and I still have many videos and presentations from last fall that I haven't shared yet. If you'd like to see me speak this year, there are only a few opportunities scheduled.

Next week, I'll be in New Zealand for Webstock 2010. I'll be giving a day-long workshop as well as a keynote address.They've got a great programme; my workshop will be on Monday, February 15 and my keynote will be Friday, February 19. I'll also be stopping by Kiwi Foo. If you're at either event, please do come say hello.

In March, I'll be speaking at the Game Developers Conference in San Francisco. Since we'll have a game-oriented crowd, I'll be talking more about my "virtual worlds" background than I normally do. I often get asked how lean startup ideas can work for the video game industry - which is, of course, where I originally started working on them. The talk itself will be Tuesday, March 9 at 11:15am.

In April, stay tuned for word on the Startup Lessons Learned Conference, which will also be held in San Francisco. Rather than make a premature announcement, I'll invite you to take the survey and help us make the event better.

In May, I'll be giving a keynote address at the Web 2.0 Expo in San Francisco. I'm especially excited about this, because last year's Expo was the very first time I'd given the lean startup presentation to a large audience. The enthusiastic reception that day was part of what gave me the confidence to leap into this job full-time. I'm quite grateful to everyone who attended, and to the organizers who made it possible.

This year, lean startups will be a big part of the Web 2.0 Expo. In addition to my keynote, Steve Blank will be speaking. There's also a one-day Lean Startup Intensive on the first day of the conference (May 3). This will be a kind of "lean startup all-stars" event featuring a number of speakers and panels. Watch this blog for details. You can register for the intensive here. Thanks to the support of TechWeb, we'll be organizing a scholarship program for this event (stay tuned). Last, there are five large-scale sponsorship spots open for the event. If you've ever wanted to be a sponsor of a big event like the Web 2.0 Expo, but don't want to have your logo lost in the sea of sponsors, perhaps this would be a good choice. To get more info about sponsorship, you can contact Susan Young.

And, if you can't make it to any events this year, you can still catch the video. For example, here's the video of my talk last month at Twiistup in Los Angeles (slides are posted here):




I try to post event-related updates to Twitter, but if you want to subscribe to my event-specific plans directly, I'm trying Plancast (a new startup founded by a friend). You can subscribe to my plans here. Last, for those who are following the Startup Visa movement, we're planning a trip to DC in early March. If you'd like to participate, you can subscribe for updates on our Plancast page.

Sunday, January 24, 2010

Lo, my 18891 subscribers, who are you?

Way back when I first started blogging, I was amazed to have any subscribers at all. In fact, when I crossed the five subscriber threshold, I was moved to write about my excitement and about the advantages of having a pathetically small number of customers. And, in the survey that was attached, I learned a lot of useful information about how to make this blog better. Since then, I've tried to make it a regular habit to collect real data about what you think. Welcome to the latest installment.

Since the last time, your support has been overwhelming: your ranks have grown almost a full order of magnitude. To all of you who've taken the time to subscribe, who have convinced a friend to subscribe, or even talked your co-workers into it, let me say thank you. Your support makes everything I do possible, and I won't ever get tired thanking you for it.

For those who have been readers for some time, the survey below will seem familiar. I sometimes use these surveys to test new product ideas and even to engage in some customer validation. Today is no exception. If you're curious, please take the survey to learn more. I don't want to say anything here that would bias the results.

As usual, I'll share what I learn and how it affects what I'm up to next. Thanks again for stopping by. Without further ado:
Click here to take survey

Monday, January 18, 2010

Case Study: Continuous deployment makes releases non-events

The following is a case study of one entrepreneur's transition from a traditional development cycle to continuous deployment. Many people still find this idea challenging, even for companies that operate solely on the web. This case presents a further complication: desktop software. Without being able transparently modify the software in situ, is it still possible to deploy on a continuous basis? Read on to find out.


Ash Maurya is the founder of WiredReach, a bootstrapped startup that he has been running for seven years. Recently, he was bitten by the lean startup bug and has started writing about his experiences attempting to apply lean startup and customer development principles. I've previously named his post Achieving Flow in a Lean Startup as one of my favorite blog posts of 2009. 

What follows is his own account of the challenges he faced as well as the solutions he adopted, lightly edited for style. If you're interested in contributing a case study for publication here, consider getting started by adding it to the Lean Startup Wiki case study section. -Eric

Of all the Lean Startup techniques, Continuous Deployment is by far the most controversial. Continuous Deployment is a process by which software is released several times throughout the day – in minutes versus days, weeks, or months. Continuous Flow Manufacturing is a Lean technique that boosts productivity by rearranging manufacturing processes so products are built end-to-end, one at a time (using singe-piece flow), versus the more prevalent batch and queue approach.

Continuous Deployment is Continuous Flow applied to software. The goal of both is to eliminate waste. The biggest waste in manufacturing is created from having to transport products from one place to another. The biggest waste in software is created from waiting for software as it moves from one state to another: Waiting to code, waiting to test, waiting to deploy. Reducing or eliminating these waits leads to faster iterations which is the key to success.



My transition to Continuous Deployment

Prior to adopting continuous deployment, I used to release software on a weekly schedule (come rain or shine) which I viewed as pretty agile, disciplined, and aggressive. I identified the must-have code updates on Monday, official code cutoff was on Thursday, and Friday was slated for the big release event. The release process took at least half a day and sometimes the whole day. Dedicating up to 20% of the week on releasing software is incredibly wasteful for a small team. This is not counting the ongoing coordination effort also needed in prioritizing the ever-changing release content for the week as new critical issues are discovered. Despite these challenges, I fought the temptation to move to a longer bi-weekly or monthly release cycle because I wanted to stay highly responsive to customers (something our customers repeatedly appreciate). Managing weekly releases got a lot harder once I started doing customer development. Spending more time outside the building, meant less time for coding, testing, and deploying. Things started to slip. That is when I devised a set of work hacks to manage my schedule (described here) and what drove me to adopting Continuous Deployment.

My transition from staged releases to continuous deployment took roughly 2 weeks. I read Eric Ries' 5 step primer to getting started with Continuous Deployment and found that I already had a lot of the necessary pieces. Continuous integration, deployment scripts, monitoring, and alerting are all best practices for any release process - staged or continuous.

The fundamental challenge with Continuous Deployment is getting comfortable with releasing all the time.
Continuous deployment makes releases non-events and checking in code is synonymous with triggering a release. On the one hand, this is the ultimate in customer responsiveness. On the other hand, it is scary as hell. With staged releases, time provides a (somewhat illusory) safety net. There is also comfort in sharing test responsibility with someone else (the QA team). No one wants to be solely responsible for bringing a production system down. For me neither was a consideration. I didn't have time or a QA team.

I took things easy at first - made small changes and audited the release process maniacally. I started relying heavily on functional tests (over unit tests) which allowed me to test changes as a user would. I also identified a set of events that would indicate something terribly going wrong (e.g. no users on the system) and built real-time alerting around them (using nagios/ganglia). As we built confidence, we started committing bigger and multi-part changes, each time building up our suite of testing and monitoring scripts. After a few iterations, our fear level was actually lower than how we used to feel after a staged release. Because we were committing less code per release, we could correlate issues to a release with certainty.

These days, we never wonder if unexpected errors could have been introduced as a result of a large code merge (since there is no branching. We also rely on more testing and monitoring automation, which is way more robust and consistent than what we were doing before.

All that said, mistakes are still made and we commit bad code now and then. None that have taken the system down (not yet anyway). Rather than seeing these as a shortcoming of the process, we view it as an opportunity to build up our Cluster Immune System. We try and follow a Five Whys approach to keep these errors from recurring. There is always some action to take: writing more tests, more monitoring, more alerts, more code, or more process.

Looking back, struggled to balance the opposing pulls of "outside the building" versus "inside the building" activities. Adopting Continuous Deployment has allowed me to build "flow" into my day which allows me to do both. But easier releases are not the only benefit of Continuous Deployment. Smaller releases lead to faster build/measure/learn loops. I've used these faster build/measure/learn loops to optimize my User Activation flow, delight customers with "near-instant" fixes to issues, and even eliminate features that no one was using.

While it is somewhat easier to continuously deploy web based software, with a little discipline, desktop based software too can be built to flow. Here's how I am implement continuous deployment for my desktop-based application (CloudFire).

My Continuous Deployment process


Don't push features

If you've followed a customer discovery process, identified a problem worth solving, and built out your minimum viable product, DON'T keep adding features until you've validated the MVP, or more specifically the unique value proposition of the MVP. Unneeded features are waste and not only create more work but can needlessly complicate the product and prolong the "customer validation" phase.

Every new feature should ideally be pulled by more than one customer before showing up in a release.
Build in response to a signal from the "customer", and otherwise rest or improve.
As a technologist, I too love to measure progress based on how much stuff I build. But instead of channeling all my energy towards building new features, I channel roughly 80% of it towards measuring and optimizing existing features. I am not advocating adding no features at all. Users will naturally ask for more stuff and your MVP by definition is minimal and needs more love. Just don't push it.

Code in small batches

I've previously described my 2 hour blocks of maker time for maximizing my work "flow". Prior to starting any maker activity, I clearly identify what needs to get done (the goal) and sketch out how it needs to get done (the design).

It is important to point out that the goal of the maker activity need not be a user facing feature or even a complete feature. There is inherent value in committing incremental work into production to diffuse future integration surprises. During the maker activity, I code, unit test, and create or update functional tests, as needed. At the end of the maker activity, I check-in code which automatically triggers a build on a continuous integration server that is then run through a battery of unit and functional tests. The artifacts created at the end of the build are installers for mac and windows (for new users) along with an Eclipse P2 repository (OSGI) for automatic software updates (for current users). The release process takes ~15 minutes and runs in the background.

Prefer functional tests over unit tests whenever possible

I don't believe in blindly writing unit tests just to achieve 100% code coverage as reported by some tool. To do that I would have to mock (simulate) too many critical components. I deem excessive unit testing a form of waste. Whenever possible, I rely on functional tests that verify user actions. I use Selenium, which lets me control the application on multiple browsers and OS platforms, just as a user would. One thing to be wary of is that functional tests are longer running than unit tests and will gradually increase the release-cyle-time. Parallelization of tests with multiple test machines is a way to address this. I am not at that point yet but Selenium Grid looks like a good option. So does Go Test It.

Always test the User Activation flow

After the integration tests are run and the software packaged, I always verify my User Activation flow before going live. The user activation flow is the most critical path towards achieving initial user gratification or product/market fit. My user activation flow is automatically tested on both a mac and windows machine.

Utilize automagic software updates

A major challenge with desktop-based (versus web-based) software is propagating software updates. Studies have shown that users find traditional software update dialogs annoying. To overcome this, I am using a software update strategy that works silently without ever interrupting the user, much like an appliance. Google Chrome utilizes a similar update process. The biggest risk with this approach is that users will find it Orwellian. So far no one has complained, and many users like the auto-update feature. It helps that CloudFire, being a p2web app, runs headlessly with a browser-based UI.

This is how the software update process currently works:
  1. At the end of each build, we push an Eclipse P2 repository (OSGI) which is a set of versioned plug-ins that make up the application. Because the application is composed of many small plug-ins, coupled with the fact that we commit small code batches, the size of each software update can be downloaded quickly.
  2. Every time the user starts up the application, it checks for a new update, downloads and installs one if available. Depending on the type of update, it could take effect immediately or require an application restart. If an application restart is required, we wait until the next user initiated relaunch of the application or trigger one silently when the system is idle.
  3. If the application is already running, it periodically polls for new updates. If an update is found, it is also downloaded and installed in the background (as above) without interrupting the user.
Alerts and monitoring

I use nagios and ganglia to implement both system and application level monitoring and alerting on the overall health of the production cluster. Examples of things I monitor are the numbers of user activations, active users, and aggregate page hits to user galleries. Any out-of-the-norm dip in these numbers immediately alerts us (via twitter/SMS) to a potential issue.

Application level diagnostics

Despite the best testing, defects still happen. More testing is not always the answer as some defects are intermittent and a function of the end-user's environment. It is virtually impossible to test all combinations of hardware, OS, browsers, and third party apps (e.g. Norton anti-virus, Zone Alarm, etc.).

Relying on users to report errors doesn't always work in practice. To compensate, we've had to build  basic diagnostics right into the application itself. They can notify both the user and us of unexpected errors, and allow us to pull configuration information and logs remotely. We can also do remote rollbacks this way.

Tolerate unexpected errors exactly once

Unexpected errors provide the opportunity to learn and bullet-proof a system early. Ignoring them or implementing quick-and-dirty patches inevitably lead to repeat errors which are another form of waste. I try and follow a formalized Five Why's process (using our internal wiki) for every error. This forces us to really stop, think, and fix the right problem(s).

My continuous deployment process is summarized below:


So why is Continuous Deployment so controversial?
Eric has addressed a lot of the objections already on his blog. One that I hear a lot is the belief that you need a massive team to pull off continuous deployment. I would argue that the earlier in the development cycle and the smaller the team, the easier it is to implement a continuous deployment process. If you are a start-up with a MVP, there is no better time to adopt a continuous deployment process than the present. You don't yet have hundreds of customers, dozens of peers, or dozens of features. It is a lot easier to lay the groundwork now with time on your side.

If you enjoyed Ash's writing in this case study, I suggest you subscribe to his blog. -Eric

Friday, January 15, 2010

Two Ways to Hold Entrepreneurs Accountable (for Harvard Business Review)

The next part in the series I am writing for Harvard Business Review is online. This time, I'm discussing the challenge for corporate CFO's and VC's alike in holding entrepreneurs accountable. Of course, the method I recommend, that of using quantified learning as a yardstick, is of equal interest to disciplined entrepreneurs. After all, the hardest question to answer in entrepreneurship is: are we making progress?

As usual, if you would take an extra two minutes of your day to click through and join the discussion on HBR's site, I would appreciate it. Leave a comment or reply to someone else. Fostering that dialog is important.
Two Ways to Hold Entrepreneurs Accountable - The Conversation - Harvard Business Review
Way back when the money was doled out, the team made a compelling pitch about the large market that was going to adopt their new innovative product or service. The rules of a fundraising pitch are straightforward: pitch the largest "total available market" that you can credibly claim to be capturing. Behind this analysis is a spreadsheet model, complete with detailed metrics for a set of customer behaviors that show just how valuable the new product will be.
And when companies pursue sustaining innovations — like a product line extension or a new technology designed to serve an existing customer segment — this procedure makes complete sense. A good general manager is expected to use all the tools at their disposal — market research, competitive analysis, customer focus groups — to figure out a plan that will work. The logic of the CFO when presented with such a plan is also straightforward: once the plan is approved, either the team will deliver the promised results, or the project can be safely shut down. Safely because it is clear that the manager in question didn't do his homework.

This whole framework breaks down when teams confront entrepreneurial situations in which they're trying to build something new under conditions of extreme uncertainty. This is the domain of disruptive innovation, projects which are inherently low-ROI in the short term. They are long-term bets on the development of a new line of business, a new technology platform, or the creation of a new market.

Let's return to our team that's failing to hit their targets. They are on-schedule and on-budget, but their gross metrics are way off. Usually, they are delivering only a fraction of the revenue they promised. For a little while, the team can resort to the last defense of entrepreneurs in trouble: the promised hockey-stick.

One thing that is often overlooked about the hockey-stick growth shape: its most distinctive characteristic is the long, flat part.

 Read the rest here... Two Ways to Hold Entrepreneurs Accountable

Tuesday, January 12, 2010

Amazing lean startup resources

A year ago, there was no lean startup movement. The term was known by maybe a few dozen people. Being an evangelist for these ideas earned me a regular diet of funny looks and patronizing comments. At that time, my wildest dreams did not include even a fraction of what's happened since.

I continue to believe that the explosion of interest in the lean startup has very little to do with me. Rather, I see it as a reflection of a worldwide openness to new ideas about entrepreneurship. Recent economic events, technological change, and the rapid diffusion of information about the old models have combined to help us all realize just how important entrepreneurship is - and just how little we really know about it.

Beyond my own efforts on this blog (and more), there is now an amazing variety of resources for lean startup practitioners. For those that are following me on Twitter, you've probably seen many of these. But I wanted to create a centralized directory. If you are attempting to apply lean startup ideas in your own business - you are not alone.

The Lean Startup Wiki (hosted by long-time lean startup pbWorks) contains a number of resources, including links to case studies, meetups, tools, and people who have volunteered to help. It's a wiki, feel free to consume and enhance it.

Rich Collins organized the Lean Startup Circle mailing list, the most robust discussion forum for lean startup ideas anywhere, with over two thousands members already. For the many entrepreneurs that send me cold emails asking for me to review a business plan or answer a strategic dilemma: I'm much more likely to answer if you've already tried getting an answer on the mailing list. You'll probably get a better result, anyway.

The #leanstartup hashtag on twitter has become a firehose of information, but seems to feature great new links almost every day. I also use it to collect feedback from all my talks and presentations, so it's a great place to find out what real people think about the ideas.

Rich also organized the first Lean Startup Meetup right here in San Francisco. The SF Meetup now has over five hundred members, and has spawned many other meetup events. There's probably one in a city near you. Some of the most active: SF, New York, DC, Chicago - but there are many more listed here. I also recommend SKMurphy's excellent Bootstrappers Breakfast series, which is now in multiple cities. Not coincidentally, they were one of the first organizations to host me as a speaker.

If you want to create or join a meetup near you, just leave a note in the appropriate section of the meetup directory. There are efforts underway in more than a dozen cities to start a regular meeting. If you need a first speaker, I'm happy to join by skype.

There are also an impressive array of bloggers writing about these ideas. When I first started blogging, the startup blogging mafia immediately came to meet me and find out who I was: Dave McClure, Andrew Chen, Sean Ellis and Venture Hacks. My success is due in no small part to their early and enthusiastic endorsement. And Steve Blank (a long-time mentor) has made the leap to blogging in impressive fashion.

Yet beyond these usual suspects are a huge number of up-and-coming bloggers, most of whom are practicing entrepreneurs willing to share their lean startup stories. Some of my favorites: Ash Maurya (whose Achieving Flow in a Lean Startup was one of my favorite posts of 2009), Brant Cooper, CindyAlvarez, Laura Klein, Kevin Dewalt, Giff Constable - and I must be missing many more. Have a favorite who I overlooked? Please share in a comment.

Last, I've tried to keep this blog updated with events, slides, audio, video and books that are helpful as well. 


Did I miss anything? Please feel free to use the comments on this post to share any and all resources you've found helpful. Most of all, thank you for your continuing support. Together we can change our  industry for the better.