Rapid Iteration for Mobile App Design

Guest post by Lisa Regan, writer for The Lean Startup Conference. 

As we’ve mentioned before, this year’s Lean Startup Conference features a lot of speakers who have incredible expertise to share but are new to our event. Mariya Yao  is one such speaker. She’s the founder and Creative Director at Xanadu, a mobile strategy and design consultancy helping to guide app developers to success in a rapidly-changing, often chaotic mobile ecosystem.

We asked her a few questions about how mobile developers can measure and address their product’s performance in an environment that is both incredibly competitive and rapidly changing. She provided some basic answers for us here and will go into more depth at the conference.

LSC: You've spoken before about strategic failures--where people build the wrong product--versus tactical fails, where people build the product wrong. This is a great distinction; so how can a mobile app developer know which of these is their particular problem? In other words, are there dead giveaways that the problem with an app is strategic rather than tactical?

Mariya: A strategic failure occurs when--as Paul Graham is fond of saying--you build a product no one wants. This means that you can't easily get users through the door despite solid marketing efforts, they aren't proactively inviting their friends and colleagues, or no one is paying for your product. A tactical failure occurs when you do grow quickly or easily attract passionate users, but see major drop-offs at key points in product usage due to poor implementation and user experience.

When you build a product that is clearly performing poorly from the get-go and you've ruled out basic technical, marketing, or executive issues, it's very likely the product is a strategic fail. However, what often happens is a startup builds a product people like but don't love. They'll typically appear to do well early on, but won't have enough of a passionate following to achieve meaningful growth or revenues.

There are two questions that I recommend startups use to differentiate between being liked versus being loved. First is the question Sean Ellis popularized, where you ask your users, "How disappointed would you be if you could no longer use our product?" and have them answer with either, "Very Disappointed," "Somewhat Disappointed," "Not Disappointed," or "I no longer use the product." Sean did research across hundreds of startups and discovered that companies that had fewer than 40% of their users answer "Very Disappointed" tended to struggle with building a successful and sustainable business.

The second question is known as the Net Promoter Score, where you ask your users, "On a scale from 0-10, how likely are you to recommend us to your friends?" You mark those who answer 0-6 as Detractors, 9-10 as Promoters, and 7-8 as Neutral. Your Net Promoter score is the percent of Promoters minus your percentage of Detractors, which should be a number between -100 and +100. The world's most successful companies typically score around +50, and top performing tech companies like Apple, Google, and Amazon regularly score over +70.

LSC: You've also spoken before about the fact that mobile apps suffer a major dropoff in engagement between opening the app and registering it. When that happens, what has a developer typically failed to validate before this step? How can they test for this in the app development?

Mariya:  The drop-off between opening the app and registering tends to occur because an app developer doesn't clearly communicate the value of their app before demanding that a user put in work to register an account. This is a violation of the "give before you take" principle that governs social interactions.

For example, you'll often see apps where the very first screen is a Facebook-only login screen. Most of the time, all you see here is the title of the app, some vague background image or tagline, and this big Facebook Connect button. While social registration can be easier than regular registration, you're also asking users to give you access to their social data before you've clearly shown them WHAT your app does and communicated clearly WHY they should hand over sensitive information.

Imagine if a random stranger comes up to, someone you know nothing about, and immediately demands to know your birthday, your relationship status, and all your friend's email addresses. Obviously that'd be wildly off-putting and you'd refuse his request. That behavior is socially awkward for people AND socially awkward for apps, and the numbers show this. The typical drop-off rate at these kinds of Facebook-only login screens is about 30% and I've even seen cases where it is over 50%.

My advice for developers who want to combat this immediate drop-off is to test different kinds of onboarding flows for brand new users and try to delay registration until user data is absolutely needed. There are many apps that deliver plenty of utility and value without mandating that a user create an account up front. Great examples include Yelp and Flipboard. Others like Airbnb allow you to browse listings to your heart's content and only require registration when you are at the last step of completing a booking. That said, there will always be categories of apps — such as social networks or messaging apps — that require a user's identity in order to deliver value. In those cases, I'd recommend testing very short "Learn more" overviews prior to registration and optimizing your social invite flows, as they will often be the most compelling ways to get new users over the registration hurdle.

If a developer has a live product with sufficient usage already in the market, I'd recommend running several split tests with delayed registration if he or she hasn't already. For developers who are still in early ideation phases and are building utility apps that don't require user identification, one quick way to get early feedback is to create a multitude of paper prototypes on index cards that test different opening flows and show them to potential users in the app's intended context. For apps that are social or require a user's identity to be useful, a prototype needs to be more fully fleshed out to give meaningful test results. Here I'd recommend developers build as minimal as possible of an HTML5 app, hook up all the requisite analytics, and test as early as possible for retention on the core action loop they want their users to take. For less technical developers, I'll be covering some methods and tools to get functional prototypes built with less dependency on engineering know-how.

LSC: You do a lot of work in helping app developers create longterm engagement. Do you have examples of app-specific measures that developers really should pay attention to (and maybe generally don't) in order to validate customers' engagement?

Mariya: Compared to desktop usage patterns, mobile apps tend to see more frequent sessions but significantly lower session lengths. For example, a product that has both a desktop and a mobile presence might see desktop users visit 10-20 times a month for session lengths of over 10 minutes on average, whereas on mobile they might see users visit 30-50 times a month for less than 60 seconds at a time.

Another difference you'll see is that people will visit hundreds of websites in a month on desktop, but their bandwidth for apps is much more limited. On mobile, despite the fact that there are millions of offerings in the app stores, the average consumer only uses about 15-20 different apps per week on a regular basis. There's a limit on both the real estate on a mobile user's home screen and their capacity for adopting new apps for habitual use.

Thus for many types of mobile apps, the holy grail is to become a daily habit for users. For your app category, you want to be the "go-to" app that users depend on. Aim to get your users to come back every day, maybe even multiple times a day, in order to have a shot at broad long-term retention. A popular metric for measuring retention in the mobile games industry is DAU / MAU, or daily active users divided by monthly active users, and I highly recommend that consumer-facing mobile app developers keep track of that metric as well.

LSC: How can app developers, particularly those working in a cross-platform environment, quickly test and validate new features and processes?
 
Mariya: Moving quickly across multiple platforms is tough because development and testing are both so much slower and more bug-prone than on desktop or a single platform. Generally speaking, I'd advise developers to focus on nailing the product experience on a single platform first before becoming too ambitious on the cross-platform front, but occasionally you come across apps whose value comes from being ubiquitous.

Regardless of what app or feature you want to test, I'd recommend you first follow Eric's advice in The Lean Startup and clearly identify your hypotheses and unanswered questions. Then you should decide effective ways to test your assumptions and pre-determine what your metrics of success should be in order for you to make a go or no-go decision to build. Much of this is the same whether you are building for mobile or web, though on mobile there are some specific tactics and tools you can use to prototype aspects of your new products or features quickly that I'll share in my talk at the Lean Startup Conference. I shamelessly encourage all of you to attend my session on "Rapid Iteration on Mobile" if you'd like to learn more.

LSC: Let's say an app has 2,000 monthly active users and a simple function those people like—but the developer has done some testing and thinks there's a much bigger market in a related but different product. How would you recommend that the developer pivot to the new idea without losing all of the existing customers?

Mariya: My advice would heavily depend on the resources--time, money, and engineering prowess--that the app developer has available and what the growth metrics and business model look like for this existing app with 2,000 MAU. For the vast majority of social games or consumer-facing mobile products, 2,000 MAU is probably too low of a user base to sustain a real business model as typically only 1%-5% of your users will convert to paying customers and advertisers aren't usually enticed into partnerships unless your numbers are well into the millions. If there aren't real drivers of long-term growth behind this app, it may be the right (albeit incredibly tough) strategic decision to pursue a higher potential market even if it means abandoning some early wins.

That said, there are many ways to test new products and markets relatively cheaply so any major pivoting decision can and should be vetted thoroughly. If the new app idea is closely related to the existing one, the app developer should try cross-promoting the new product to his existing user base. 2,000 MAU is a ripe field for recruiting potential users and conducting user research and usability studies. He or she may even choose to launch the product in parallel with the existing one if the company can manage to do this without sacrificing too much momentum or morale. By comparing the live performance of both products in the market, you'll get the most accurate data to inform your strategic product decisions.

For an existing product on mobile, there are many ways to segment your audience to test new features. One of the most popular is to release an app in a limited number of countries, such as Canada or New Zealand, prior to a global launch. Another is to "white-label" your app and release parallel apps in the same market that test different value propositions. Yet another is to test with mobile web apps or Android apps first prior to officially launching. For example, pushing new changes out on Android is typically much faster than with iOS so it's popular, especially with mobile game developers, to fine-tune apps on Android rather than starting with iOS.

--
Learn more at The Lean Startup Conference, December 9 - 11 in San Francisco. Register today.

Lean Startup at Scale

Guest post by Lisa Regan, writer for The Lean Startup Conference.

As Lean Startup methods have been used now for a number of years, we’ve become increasingly interested in how companies use them to sustain growth. That is, once you’re no longer a small company and you have some success, how do you execute and continue to grow through innovation? Next Tuesday, October 22 at 10a PT, we’ll take a look at this advanced entrepreneurship question. In a webcast conversation, Lean Startup for Growing Companies, Eric Ries will talk with two Lean Startup Conference speakers: Wyatt Jenkins, VP of Product at Shutterstock, a stock photo site that has become one of the world’s largest two-sided marketplaces, and has expanded since its founding in 2003 from a single development team to 12 cross-functional teams spanning all phases of product development; and Ari Gesher, a senior engineer at Palantir Technologies, which specializes in data-mining software for a diverse set of problems across verticals, including disaster recovery work recognized by the Clinton Global Initiative, and has grown since its founding in 2004 to over 1,000 employees and undisclosed revenues reported to be approaching $1B today. The webcast is free with registration and will include a live Q&A with attendees.

Below are excerpts from conversations we had with Ari and Wyatt about their growing companies and about some of their suggestions for staying with Lean Startup as you expand:

When we asked Ari to talk about how he addressed some of the challenges of growth at Palantir, he gave the example of developing iterative cycles that could accommodate a scaling company:

Ari: At Palantir, we've had to tailor our software development process over time to deal with the scale of the team and the scale of our core software products. Each plateau of scale has required adjustments--changing or dropping an old process that wasn't working or creating a new process to deal with novel challenges. One good example is the way in which we've adjusted the length of different phases of our agile sprints. We don't follow a set agile methodology, but rather follow a more home-grown, minimal version of various approaches. We work in prototypically four-week iterations, with quality engineers and software developers working in close collaboration.

It wasn’t always this way. Palantir is a deep technical play and we had a lot of code to write just to fill out the product vision that we had already validated with potential customers; it took us two straight years of development to go from early prototypes to software that could be used in production. In these halcyon days, we weren't using iterative cycles as there was often nothing that could be tested for long periods of time (aside from unit testing, of course). During this period, the Palantir Gotham team grew from five developers to around 35.

This finally bit us after a four month stint of development blew through its testing schedule by a factor of four: two scheduled weeks turned into two months before the product reached stability.
So what was going on? Software development is all about managing complexity and the bigger and more mature the codebase gets, the more complex it gets. The interconnectedness had come to a head, such that new code was much more likely to disrupt existing code than ever before. As we started looking at bug counts something else became clear: we could now create bugs faster than we could find them.

We had finally hit the point where we needed to impose clear process - process whose main goal was to ensure that software stayed stable. But we couldn't have identified this without having clear metrics (that high bug count) to assess our development process. The result was a new process of four-week iterative cycles all about throttling new code. Here's the simplest form of that cycle:

• Week -1 - Planning/End-of-Cycle - Software engineers are planning: writing specifications, doing light prototyping, and experimentation. During this week, quality engineers are attending planning meetings but also doing end-of-cycle testing of the previous iteration - the final testing on an internal release before it's deployed to our dog-fooding systems.
• Week 0 - New Feature 1 - Software engineers are busy building brand new features. Quality engineers are writing up test plans based on the specifications and creating test data (if necessary). By mid-week, the software engineers have to deliver something testable so the quality engineers can start testing and (of course) filing bugs.
• Week 1 - New Feature 2 - Development and testing continue together. By the end of the week, the new code should be stable, with all filed bugs fixed.
• Week 2 - Regression - Software engineers start paying off technical debt by attacking a queue of existing bugs. Quality engineers make full regression passes through the software, looking for old functionality that may have been impacted by the new changes this iteration.
• Week 3 - Planning/End-of-Cycle - See Week -1.

Software is an interconnected system and thus a change can reverberate through the codebase with a super-linear, potentially quadratic complexity. A good analogue is air resistance, which has a quadratic relationship to velocity. If you've ever ridden a motorcycle, you know that at low speeds you hardly feel the wind. As you hit 45 mph, it starts pushing on you. At 65 mph, it's like having a large animal sitting on your chest.

The new process let us manage the velocity of change in the codebase low, keeping the resistance manageable.

More important, the iteration framework gave us something like a meta-process: we could try new ideas about how to manage development process and measure them against historical data to see what could further optimize the process.

Note that the iteration template above is our prototypical iteration. What started as a four-week cycle has since expanded to five-week cycle — adding a second week of regression to pay down technical debt. For the final iteration that turns into an external release, we push out to a six-week cycle, adding an additional week of testing: end-of-release testing.

We asked Wyatt about his experience with testing at Shutterstock, and the best ways to approach it in a growing company:
 
Wyatt: The concept that your "big idea" is nothing but a hypothesis until we test it is a part of the Shutterstock culture. We learned some hard lessons via A/B testing, but realized quickly that the pace at which we could perform tests and the systems for reading tests had to be both excellent and easily accessible by many different groups in the organization. Because of this (and because we believe our data is a competitive advantage that we don't want to share with third parties), we chose to build many tools ourselves. At this point, we have our own open-sourced click tracking "lil brother," an internal data visualization tool built on Rickshaw, as well as an A/B testing platform called Absinthe. We pride ourselves on the speed of testing hypotheses and reading those tests. Shutterstock is in a competitive market, but we have the most traffic and the most usage, meaning that we can run more tests (achieve significance sooner) and learn faster than our competitors.

The upshot of this is that speed matters if you hope to build, measure and learn faster than your competition. As Shutterstock has grown, there are a few key elements to our continued development speed:

• Small, autonomous teams: The more a team can do on their own, the faster they can go. The hand-offs between teams are (mostly) eliminated, and close-working autonomy creates a good startup vibe as well.
  
• Continuous deployment: A key component of speed is to keep pushing out work. This has kept us lean as well—we don't have release trains, and code generally goes live to the site every day of the week.
• Don't get religious about process – just continuously improve. You need to strike a balance between process and problem-solving. You don’t want to get so committed to a particular process that you can’t adapt to problems as they actually present themselves. So, depending on which team you are talking to at Shutterstock, we may be Lean Startup or Agile or Kanban or some other method depending on the type of problem that team is designed to solve. But that doesn’t mean that we don’t take these ideas seriously. You want to be flexible enough to change your process across teams as you scale--but as a rule of thumb we question every new bit of process someone tries to add because process is easy to add and very difficult to remove once it's in place. For that reason, it’s important to go with methods, like Lean Startup, that have proven results for the kind of problem that team is trying to address.

On Customer Development in a growing company, Wyatt offered the following advice: 

Wyatt: We've employed a number of systems in the organization that keep all of us close to the customer. As we've grown, we now have a great qualitative research team dedicated helping us stay close. There are 5-10 customers in our office (or remote) per week for developers, product owners and marketers to speak to and validate learning. The important aspect of scaling customer development is to build it into the process and make it so easy to put your idea in front of a customer that everybody does it. Skip the focus groups--they don't work and they take too long to set up. Create a steady stream of customer input that anyone can dip into.

Ari also talked about the changes in the information flow as a company grows, in his case thinking of it moving in a variety of directions--between the company and its customers, certainly, but also between the company and its own internal teams, or between parts of the company itself:
 
Ari: One of the biggest effects of scale has to do with internal information flows. For example, a small team of four people starting work on a product has it easy. By just sitting in one room, they can have amazing shared situational awareness. There's two things at work here: as new information comes into that room, it's as easy as an offhand comment or a lunch conversation to share. The second thing is that it has no history - everyone on the team is starting from a place of zero knowledge and accumulating context as it arrives.

I joined Palantir when it was one room and fifteen people--the above model was still pretty functional. We all knew a lot about what was going on. Things like shared meals helped us stay in sync such that the knowledge about everything from the state of the product to the outcome of our last meeting with potential customers was pervasive.
Another interesting feature of those early years: most of what we needed to know was outside the company.

Growth changed all that. We've been building the company for almost ten years now and we have three major locations in the United States, as well as about half-a-dozen overseas offices. Over a thousand people work here now.

Most of the information that most of the people need to do their jobs is actually generated inside the company now. We have a long history and so new employees have to spend a long time getting up to speed on the why, what, and how of everything that we do. As a result, we've had to designs process, protocols, and infrastructure to make sure that critical information flows to the right people in a timely manner. We've had to design and implement training programs to help on-board people to our culture and technology. We have a blog that's internal-only to capture important stories for prosperity. We have a dizzying array of email lists and careful protocols about which lists get copied to make sure we can maintain a shared situational awareness that can only hope to approach what we had when we were in one room. We have an internal directory that lets people tag themselves with the things they know about--often learning is about finding who knows the answer and can explain the full why (not just the what) of something.

And none of that addresses exactly how the company as a whole learns from the world. There is now an immense flow of information coming in from the world, about how our product is working (and not working), about how our software is solving customer problems.
There are two teams that handle the bulk of the learning that comes in from the field: Support and Product Navigation. Both of these teams are collating, distilling, and turning into knowledge the information flowing from the field.

Unlike most support teams, the [Palantir] Support Team is not contacted by end users but instead by our people in the field. Our model is to place Forward Deployed Engineers (FDEs) on customer sites to add with integration of data, customization of our platforms and applications to a specific task, and training of the customer's analysts who will be the end-users of the system. If an FDE runs into trouble with the product or a novel situation that we did not anticipate, they contact the Support Team, who handles both resolving the situation, communication with the product team and documenting the knowledge gained in this situation so it can be avoided in the future.

The Product Navigators are responsible for understanding the use cases that are covered by our products--the gaps, what new features we need, what's working, what's not. This is not bug tracking but more along the lines of customer development and guidance to the product team on what to build next. They collate, distill, and prioritize information coming in from the FDEs and product instrumentation about how well the product is actually solving customer problems, what features are in use (or that users don't understand). This is a part of what many other organizations would call product management, but we decouple the learning portion of that discipline from the design portion (handled by product designers and engineers based on the knowledge created by the Product Navigation team) of product management.

Both of these teams were created to handle the sheer scale of information coming in from the field as our customer base as grown from zero to where it is today.
--

Register today for our free webcast to join Eric, Ari, and on October 22 at 10a PT. For even more Lean Startup learning, register for The Lean Startup Conference, December 9 – 11 in San Francisco.

The Entrepreneurial Enterprise

Guest post by Lisa Regan, writer for The Lean Startup Conference.

How can established companies benefit from implementing Lean Startup? To answer that question, we hosted a webcast conversation earlier this week with Eric Ries, Brant Cooper, and Patrick Vlaskovits. We’d like to share a few highlights and invite you to join the ongoing conversation by posing questions for Eric, Patrick, and Brant in the comments to this post. They’ll jump in to answer in the coming days. (Of course, you can also continue the conversation by attending The Lean Startup Conference. Our latest batch of discounted tickets is about to sell out, so register today.)

A lot of people think the “startup” part of “Lean Startup” means the ideas apply only in young companies. But, in fact, they can be crucial in large, established companies that need to find growth in new products and new markets. As Eric puts it in the webcast, “I’ve met now the CEOs of some of the biggest companies in the world, and I still spend time with the CEOs of high-growth Silicon Valley companies from the garage on up, and what all those people have in common is that they are seeking out sources of sustainable growth…. What we care about in the innovation community is growth that is driven by customers and creating value for them. And we just so happen to think that the best way to create sustainable growth in our highly disruptive world is through continuous innovation…. And that challenge I have seen to be identical no matter the size of the company.”

Patrick and Brant understand these problems intimately. They are the co-authors of The Lean Entrepreneur, and the co-founders of the Moves the Needle Group, which advises the innovation practices of Fortune 100 companies. Their webcast conversation with Eric began with a discussion of the ways that large companies come around to realizing they need to implement Lean Startup. Among other themes they covered: the conditions under which established companies implement Lean Startup; the hindrances and incentives they face; what individual employees can do to implement the methodology; and how to protect a company’s core business from the potential impact of experimentation. Though we’re posting a few highlights below, we encourage you to watch the video in its entirety, as Eric, Patrick and Brant dig into technical details and case studies that will be helpful for people engaged with this topic.

--

The webcast conversation examined corporate versus startup structure, and the limitations on innovation that corporations create for themselves. Problems arise when companies isolate areas of the business in what Eric refers to as “functional silos”—independently operating teams (design, engineering, marketing, legal, etc.) that discretely address specific areas of the production chain. Each silo may be innovative in itself, but at some point it considers its work complete and hands off to the next silo, with which it has had little to no collaboration and for whose tradeoffs it has not accounted. What can Lean Startup do about this?

Eric:  I’ve now worked with a number of companies where once they adopt Lean Startup, it gives them a common vocabulary for all the different teams to use the same vocabulary, the same business-oriented, results set of concepts to wrap all these techniques in…. We knock the silos down and get everybody on a single, cross-functional team to say to everybody, look, you are a startup, you are not a set of different functions, so you, team…go experiment and learn how to make this happen. When teams are organized that way they’re so much more productive, so much more energized, the creativity you unlock is incredible.

Brant:  We’re still taught in MBA school is these silos, right? And if people can just imagine, if the way we motivate people, the way we do performance reviews, and incentivize people within silos—those measures cannot be drawn in a direct line to corporate objectives. You cannot draw a direct line to reducing waste or improving revenues or cutting costs…whereas the cross-functional teams, you can tie it to a performance metric that has a direct result in the corporate objectives.

Patrick, with a bit of highly practical advice: One of the things Brant and I counsel large organizations, particularly ones that have already embraced Agile, is to extend the Agile metaphor into the funnel, for example, and show the benefit of Lean Startup for the sales and marketing teams, and then show the benefits of Lean Startup to the HR folks…. With sales and marketing experiments, often you pick low-hanging fruit in the millions of dollars very, very quickly. We’ve seen this time and time and time again, and that’s how you get other parts of the organization and other functional roles excited about starting LS methods.

--

A variety of audience questions came in effectively asking how an established company, with an existing product, customer, and brand image, can make use of a minimum viable product (MVP) for experimentation purposes without damaging the larger company’s standing.

Brant: The core business has to be protected from the startup. We separate out these startups. They need to exist in a different place for the time being, so that the startup is protected from the questions around return on investment, and the core business is protected from the startup…. So you’re not launching a minimum viable product to all of your core business customers--that’s a no-no. You have to keep these separate. And then the Lean Startup acts like a new startup. You have to go find your own customers to experiment with, you have your own brand, so you have to think like a Lean Startup.  You’ve got your validated learning with the customer development you’ve been doing with your own market segment. But you’re not going to your core business.

Patrick: The startup “can’t run to the core business and have all its problems solved for it. Because then you get something like the children of helicopter parents, that can’t fend for themselves. And ultimately if it’s a real innovation, that has real value and creates real value for customers, it has to fend for itself. It has to be a stand-alone business case that makes sense.

Eric, on making lemonade out of an MVP lemon: The great thing about an MVP is that if customers don’t like it, but they care enough to complain, that’s actually great news. Most of the time when you do an MVP, no one even notices. Zero customers show up. You have “launch day” and then nothing happens, because your value proposition is so wrong that no one cares. So that doesn’t harm the brand. If people complain but you kept the scale of the product small--and MVP is about containing the scope of the experimentation so that the cost of failure is low--then you can make it right for those customers that complain. Like, you can send them a hand-engraved letter press apology. Every single one. Personally delivered to their house. By you. If that ever happens.

--

So how do you start a Lean Startup practice where you actually work? Patrick, Eric, and Brant all talked, in response to audience questions, about the importance of individual actors within corporations deciding to, as Patrick put it, “be subversive.”

Brant: Changing the culture is what’s needed and it’s hard and it takes time. And so I think if you’re a person inside of an organization that is ready for this type of stuff, the first thing you should do is go find like-minded people, and that’s actually how you start the process…. I’m not saying go do something that’s going to get you in trouble, but you can’t wait for some magic to happen, for some external force that says ok, now this company is prepared for Lean Startup culture. You actually have to go and make the change yourself.

Patrick:  Actually, let me go as far as to say you should go get in trouble, you should actually be subversive. I’m half-joking here, but I think we’ve all seen this, that where Lean Startup has managed to take root and flower is where initially you had some aggressive early-adopters act a little subversively. If you have a mortgage and a family, I’m not telling you to risk your career on adopting Lean Startup, but if you’re passionate about making change, you can’t wait for permission to do this stuff, you’ve got to start doing it and you’ve got to start doing it intelligently, and part of that is hacking the actual internal political system. That’s very difficult, obviously, but it’s part of that journey.

Eric:  Think of all the managers who worked at Kodak, or Nokia, or Blackberry--pick your favorite company that has had a total collapse in living memory. And say, “How would it feel to be the manager who was there, who saw the disruption coming and did nothing about it?” First of all, is that actually a good path to having a longterm career? If you have to feed a family and make your mortgage, and your company collapses while you’re there, is that really going to help you? I feel like it’s almost irresponsible to put your head down and say, “Whatever, I work for a great institution, and I’m going to let it crumble under my stewardship….” The future leaders of companies I think are going to be the people who today started learning with these techniques, because what we’re talking about here is nothing less than a full-scale paradigm change in the management culture and management philosophy of modern companies. So would you rather be an early adopter of that, or have someone else take that lead?

--

Interested in more in-depth discussion of Lean Startup in the enterprise? We invite you to check out the entire video, including a fascinating conversation about why revenue is not a good growth metric. For even more insight, register today to join us at The Lean Startup Conference in December. If you bring eight or more of your employee, we’ll give you a substantial break on the price. For more info on the benefits of team registration, see our post on fostering innovation in established companies. For pricing details just email our executive producer Melissa Tinitigan and use the subject header “group discount.”

Key Questions for Bringing Lean Startup to Established Companies

Guest post by Lisa Regan, writer for The Lean Startup Conference.

Our next webcast tackles a particularly challenging topic. On October 8 at 10a PT, we’ll be talking about bringing Lean Startup methods to established companies, precisely the kinds of businesses that are most resistant to experimentation, even as they desperately need it. Eric Ries will be joined by Patrick Vlaskovits and Brant Cooper, co-authors of the NYT-bestselling book “The Lean Entrepreneur” and frequent advisers to Fortune 100 companies like Hewlett-Packard, Qualcomm and Pitney Bowes. The webcast is free with advance registration and — the best part — features a live audience Q&A where you can submit your own questions for Brant, Patrick and Eric.

To set up the themes of the webcast, we asked both Brant and Patrick a few questions about their experience with enterprise companies. They didn’t pull any punches in telling us how difficult — and yet important — it can be to get leadership on board.

LSC:  What do you see as the most common reasons that large companies give for why they don't apply Lean Startup?

Brant:  In my experience, the conversations don’t ever go that way. Large companies don’t have to, and in fact don’t, justify choosing not to shift to a different methodology. In fact, most of our work is inbound, meaning that companies come to us already seeking to do Lean Startup; they’re already on board. I’m not currently in the business of – nor sure I ever want to be in the business of – convincing organizations to do Lean Startup. That's hard. That’s Eric Ries’s job.

Patrick:  Last week I had an interesting conversation with the CIO of a major European company operating in a mature, very profitable industry. His team was quite well-versed in Lean Startup, and asked hard and insightful questions. One theme that emerged a few times amongst the comments was essentially, “Yes, we think that Lean Startup could be hugely beneficial, but we’re worried about the changes in culture, organization and process required to pull it off.” Essentially, they have a “the cure might kill the patient” anxiety.

It would be too easy to shrug off their thinking as short-sighted, but I think their concern is valid. At the end of the day, it will be how they view the future of their industry, and their role in it, that will determine whether they see the coming disruption as something they choose to participate in, or remain willfully ignorant of.

LSC: What opportunities do large or established companies typically fail to see or take advantage of when they do attempt Lean Startup methods?

Brant: Companies take on Lean Startup for reasons that are all over the map, and they have experiences with it that cover a similar range. This is mostly because they’re not implementing the methodology in its entirety; rather, they have some concrete set of outcomes in mind. So I’ll see internal development groups trying to apply Lean Startup methods. And, companies that are modifying their design thinking processes are similarly interested. I also see employees attending Lean Startup Machine in essentially ad hoc efforts to try something new. Overall, this is part of a tendency to see Lean Startup as simply an evolution of Agile, and to mistakenly think that it thus “belongs” in Engineering, or that it’s only useful for new product endeavors. That kind of compartmentalization ignores existing products and processes, and encourages people to think that the methodology doesn’t apply to “their stage” of product development. 

In general, I think organizations are prone to seeing Lean Startup as a “tack-on,” a benefit to supplement the usual way of doing things. And while Lean Startup can certainly be useful from that perspective, the real missed opportunity for the company as a whole is in organizational transformation. In other words, why only make changes to a particular product or product line, when you could teach the entire organization, at all levels, to move faster, get closer to customers, and learn to continuously innovate?

Patrick: There is one area of low-hanging fruit that I do see for companies that institute Lean Startup as a “tack-on,” as Brant put it. That’s when the sales and marketing functions adopt Lean Startup for customer acquisition. By executing best practices, they can often realize huge gains in conversion from hitherto invisible opportunities around the funnel and the product.

LSC: Lastly, how can reform-minded individuals or teams address these resistances from within — that is, how can people interested in the methodology get buy-in up the hierarchy?

Brant: In my experience this cannot be only a grass-roots push. Somewhere in the C-Suite, there has to be an executive who has realized that a change in philosophy and in practices must occur if the company is to take “innovation” out of the realm of being just a marketing buzzword. This may be a maverick CEO, or a new CEO brought in for change, or one who simply has had an epiphany and is out looking for a new approach. We’re still in early-adopter territory here.

From that perspective, I’d say that the best path for reform-minded individuals is not to seek buy-in, but rather to attempt to foster an epiphany. That isn’t going to happen, at this stage, by seeking approval from your boss. But hey, we’re approaching the holidays. Buy the CEO or Chief Innovation Officer a copy of Eric’s book. With any luck, he or she will read it.

Patrick: I share Brant’s emphasis on leadership, but I think you gotta try the sandwich approach -- grassroots from the bottom and an aggressive leader from the top. Ideally, this kind of organizational change covers the entire spectrum of the business.

--

Our webcast with Brant, Patrick and Eric is October 8 at 10a PT. Register today to join hundreds of people attending, and bring your questions for the live Q&A.

If you’re thinking of bringing Lean Startup to your organization, we encourage you to take advantage of our team discount for The Lean Startup Conference, Dec 9 – 11 in SF: send eight or more of your employees and get a substantial break on the price. For more info on the benefits of team registration, see our post on fostering innovation in established companies. For pricing details just email our executive producer Melissa Tinitigan, and use the subject header “group discount.”

Fostering Innovation in Established Companies

Guest post by Lisa Regan, writer for The Lean Startup Conference.

Does this sound familiar? You want to apply Lean Startup methods at your company, but your boss is used to being the decision-maker and is resistant to the concept of testing ideas. Or, you are the boss, and your team is afraid of or unwilling to try new methods. This kind of gap between leaders and teams is common at all sorts of companies. But it’s particularly prevalent at established companies, where hierarchy tends to inform more of the action.

On October 8 at 10a PT, join us for a webcast, Bringing Lean Startup to Established Companies. Eric will speak with Brant Cooper and Patrick Vlaskovits, both authors and entrepreneurs who have served as advisers to large companies, including many on the Fortune 100. This webcast is free with registration, and Eric, Patrick and Brant will open it up to Q&A with attendees, so come with questions ready. In addition, we’re offering a Lean Startup Conference discount for corporate teams; details below.

We explored Lean Startup in the enterprise through a number of talks at last year’s conference. One of our favorites was a panel led by Intuit co-founder Scott Cook, who spoke about the change in leadership required at his company when he decided to implement Lean Startup methods after meeting Eric in 2009. His challenge was, as he put it, “Converting Intuit to run on Lean Startup principles by converting Intuit to be a network of lean startups.” This transition required changing both how decisions were made in the company and how leaders behaved, moving away from what Scott vividly described as decision-making by “politics, PowerPoint and persuasion,” and “the conventional, the consensus, and the commonplace.”



In this discussion, Scott identified four things leaders have to do in an established company in order to “enable decision-making based on the best ideas that can be validated in the marketplace.” These were: setting the grand challenge; installing the systems and culture that enable people, even junior people, to be able to run fast experiments to achieve that challenge; pulling insights from those experiments, no matter whether they succeeded or failed; and living by the same rules and the same discipline as the rest of the team.

Each of these four elements of what we might term Lean Startup Leadership was addressed by a different Intuit team leader (the video here shows them all). To take just one example: Carol Howe, Intuit’s Senior Product Manager, spoke about the way her team created TurboTax SnapTax. A mobile app that lets people use their phones to prepare and file taxes, SnapTax lets users snap photos of their tax documents. As Carol described, the app was a huge success — but, “this was not where we started nor what we intended to build.” The original plan was for an app to allow photographing tax documents that users would then need to enter into their standard, web-based version of TurboTax. Customers disliked this two-step process in early testing, and so Carol’s team built a version of the app that ran entirely on mobile, and was a one-stop-shop for tax preparation. Even then, they were careful to test the new app in a limited context, launching it first only in California, and only on a limited category of tax filers.

Carol: “We heard from all those other customers — customers in other states, and customers with more complex returns. They complained to us and said that they too wanted to file their return on their phone. And that’s when we knew we were on to something special.” They had used methods characteristic of a Lean Startup to validate customer demand and create a truly innovative product — even though this product would compete with Intuit’s own flagship TurboTax software.

Obviously, any company, whether large or small, wants to create products that consumers nationwide clamor for. But creating and empowering teams to both identify and fill the needs of those customers via a process of rigorous testing and rapid iteration requires buy-in at all levels of the company. Corporate leadership, as well as team leaders, need to be on board. On October 8, Eric, Brant and Patrick will investigate the hurdles large corporations face in implementing Lean Startup and the unique opportunities there are for companies that are willing to do so. Register today to join them.

To help support teams applying Lean Startup methods, we’re offering a discount for any company that wants to send a group to the 2013 Lean Startup Conference. If you send eight or more employees, we’ll give you a substantial break. Simply email our executive producer Melissa Tinitigan for details, and use the subject header “group discount.” Here are few of the benefits to sending a team:

•    Get out of a rut. The Lean Startup Conference provides fresh ideas even for teams that have been working together, or hammering on a single intractable problem, seemingly forever. It’s not just the on-stage presentations that help get your team members thinking — it’s also the networking events and startup site visits that expose you to the sharpest entrepreneurial talent. And, professional development improves staff retention and satisfaction. Your team will come back energized.

•    Save time. Teams thinking about instituting more testing, or exploring the Customer Development model, will get direct information about how to implement these methods. You’ll gain understanding and confidence in the process, thus reducing your fear of making mistakes — a barrier that Scott Cook cites as a major hindrance to embracing Lean Startup, especially at the lower levels of a company.

•    Look into the future. The next best minds in corporate innovation are speaking at the Lean Startup Conference. They’ll give your team a sense of what’s ahead and can help you better perceive where to go next. Site visits to highly successful startups, including those successfully scaling, will provide inspiration and key insights to how those teams approach their work.

•    See results. Have your team present to a larger group — the division or the entire company— about what you learned at the conference. We’re preparing a kit of resources that you can take back to your organization and share ways you can immediately apply Lean methods to ongoing projects.   

Don’t forget to register for the October 8 webcast and prepare your questions for the live Q&A. Register separately to attend the conference, and email Melissa for group discount info.

Applying Lean Startup Beyond Silicon Valley

Guest post by Lisa Regan, writer for The Lean Startup Conference. 

Entrepreneurs beyond Silicon Valley, including those working abroad, often have to retool Lean Startup methods to apply them in places with very different business cultures. On September 24 in the U.S. (September 25 in Asia), international Lean Startup experts Kevin Dewalt, Takashi Tsutsumi and Justin Wilcox will meet for a webcast to compare notes and hash out solutions. This event is free, and their conversation will be followed by a live Q&A for webcast attendees; register today to join them.

To introduce you to some themes of this webcast, we spoke to Kevin, Takashi and Justin last week about the biggest challenges they see in implementing Lean Startup methods outside the U.S. Here’s what they had to say.

Kevin Dewalt—a speaker at this year's Lean Startup Conference—is an entrepreneur, investor and adviser who has served as an investor for a strategic U.S. government fund and as Entrepreneur-In-Residence for the National Science Foundation. Two years ago, he moved to Beijing, where he founded Lean Startup Meetup Beijing, as well as his current venture, SoHelpful.me, a platform for leveraging one-to-one relationships to build reputation and word of mouth.

“Two years ago I moved to Beijing, where, as the founder of SoHelpful.me, I take calls from all over the world, talking entrepreneurship and innovation with people in 30-minute intervals. Those conversations can often circle around the difficulties of entrepreneurship outside Silicon Valley. I think one challenge in trying to take Lean Startup ideas outside Silicon Valley, and especially to places that are highly culturally different from the U.S., like Asia, is that the methods most of us are familiar with are actually pretty culturally specific. If you ask people in Asia about Lean Startup methods, they’ll often say, ‘I’m not sure that would work here,’ and in a sense they’re right–many of the familiar methods won’t work if applied unmodified. That’s why I recommend that people focus on ideas rather than tactics. Lean Startup ideas will work even in places as different from Silicon Valley as Asia–the specific tactics will need modifying, though.

“Think, for example, about the way we talk about sales and customer development and the idea of ‘getting out of the building.’ In Silicon Valley, you can go to people and ask them what their problems are, and what solutions they would value, and they’ll be happy to answer you. People in Silicon Valley are accustomed to openly discussing change, and to talking about what’s wrong or needs fixin —it’s culturally accepted there and you get a lot of practice at it. In most of the world, that’s just not the case. If you walk into a manager’s office almost anywhere in Asia and say, ‘I want to talk to you about your problems,’ he’ll tell you that everything’s fine, that he has no problems. He’ll probably suspect that his boss sent you. Right away, by talking in terms of problems and change, you’ve lost that person; they’ll just shut down.

“This is not to say that you can’t get out of the building in Asia, too. But you’ll have to go about it differently. Relationships are very important, and introductions are valued, even expected. You’ll need to do the legwork to get introduced, and to become known to people before you ask them for help or information.

“Another resistance or challenge faced by people starting businesses in Asia comes from within the startup itself, around getting support from co-founders and investors. There’s often a practice in Asia of locking onto the first idea as ‘the idea.’ Impatient investors and team members give little support to a founder trying to do Customer Development to verify or modify that idea. They often look on this as a waste of time. And then, when customers are not buying the product, the blame will tend to focus inward — on the founder for perceived shortcomings in the product, rather than examining the question of whether the product itself is actually solving a problem.

“What can you do in the face of such challenges? As I mentioned above, think ideas. When you find yourself getting overwhelmed with tactics, remember that the big idea of Lean Startup is search – it’s about searching for opportunity. People search differently in different cultures. To find out how they search, you need local advisers, people who understand how to apply Lean Startup methods where you are, and to do customer development in ways that are relevant to that culture.

“The good news is that those people make themselves known. The Lean Startup community is very tight and well-organized. There are Facebook pages, and Meetups, both of which are good filters for smart, passionate people around the globe. (I’ve observed that people don’t organize Lean Startup Meetups unless they’re serious.) Those organizers will in turn value the Silicon Valley perspective that you bring – you just want to make sure you let them know that you’d like them to show you how they do it where you are now.” 

Many of the challenges Kevin describe resonate with Takashi Tsutsumi. Takashi has been a venture capitalist for fourteen years, investing in technology startups both in Japan and in the United States. Enthusiastic about the scientific approach for a startup, he personally translated both The Four Steps to The Epiphany and The Startup Owner's Manual into Japanese. On weekends, he evangelizes Customer Development and runs a Lean LaunchPad class nationwide in Japan. Takashi spoke to us specifically about what it’s like to try to bring Lean Startup methodologies to a business culture as conservative as Japan’s. He described two challenges, and two pieces of good news. [Ed note: Takashi emphasized to us that his ideas here are his own and not affiliated with any companies that he works for or is involved with.]

“Japan is known for its conservatism and the norm of lifetime employment, both of which result in a lack of entrepreneurship. In this environment, I have been practicing and evangelizing Customer Development/Lean Startup for the last seven years (with great help from my teachers Steve Blank and Bob Dorf and my friend Masato Iino), facing challenges and delighted by the unexpected good effects for entrepreneurship. The following are a few examples.

“Challenge #1:  Perfectionism and detail-oriented culture

“Japanese are known for their perfectionism and the Japanese culture is highly detail-oriented. This culture particularly contradicts with minimum viable products. Entrepreneurs worry that they will lose their trust and reputation with customers if their products compromise features, UI/UX, quality, etc. In addition, although entrepreneurs come up with good MVPs, they gradually add more features as customers say that A, B, and Z are missing, resulting in a ‘maximum’ viable product instead. Therefore, one of the keys for success to practicing Lean Startup methodology in Japan is to encourage entrepreneurs to be patient in minimizing their products. I sometimes refer the nice rule of thumb from Eric Ries, ‘Take what you think is right now and cut it in half and do that two more times and ship it back.’

“Challenge #2:  Pivot is failure?

“Pivoting is a key Lean Startup concept, but in Japan, pivot mostly means failure. The Japanese perfectionism affects this thinking in that people consider it right to complete a plan once it’s developed. First, this is true for entrepreneurs. They stick to the initial idea (i.e., the hypothesis) even if facts tell them it’s wrong. They just hate to admit being wrong, or they believe themselves too much to change their mind. Second, and more important, stakeholders, such as investors and management, think this, too. Even when entrepreneurs get used to the principle of Lean Startup, in which a pivot is not necessarily a failure but is progress, their stakeholders don’t share the sensibility. Investors and management use an original (therefore unproven) plan as a yardstick, so I often hear funny conversations that effectively go like this:

VP Sales: I talked to many customers to see whether they have the problem we hypothesized and whether our solution properly fixed their problem—but none of them said yes. We should pivot.

CEO: I know, but we can’t pivot as I got approval from the board by committing to our business plan. We have to stick to our original plan untapped and carry on, even if we know we are wrong.

“I see this as the largest challenge for Lean Startup practitioners, especially in Japan. Therefore, ‘Step Zero of the Four Steps,’ or buy-in, is more important in Japan than in the Valley.

“Good news #1: Perfectionism and detail-oriented culture

“Perfectionism and detail orientation inhibit adapting Lean Startup methodology in Japan, but they turn out to be strengths once people buy in. Once they buy in, entrepreneurs in Japan follow and execute Lean Startup exhaustively.

“I see exhaustive execution of Lean Startup methodology often in the mobile/internet space. One good example is Cookpad, Japan’s top recipe site. Cookpad develops new services and features on a strict Lean Startup method. They test selling an MVP first to their colleagues, and they have to pivot if their colleagues do not pay for it. In fact, Eric Ries’s Lean Startup is mandatory reading for new hires prior to their first day. Cookpad also has their homegrown tools for hypothesis building and cohort analysis for effective execution of Lean Startup methodology. Yes, as Lean Manufacturing was the heart of 20th century Japanese manufacturing, Lean Startup’s process-oriented quality nicely fits the Japanese mind. I hope more startups and corporations in Japan execute the Lean Startup methodology in the Japanese ‘relentless’ manner.

“Good news #2: Customer Discovery nurtures entrepreneurship

“Customer Discovery is never easy in Japan. Ordinary people do not talk to strangers, knowing they hate unsolicited inquiry. However, the more customers an entrepreneur talks to, the more they learn. What surprises me, however, is that talking to customers also turns non-entrepreneurs to entrepreneurs because they feel a sense of fun and confidence in their idea. As many of you saw in our final presentation for the 2020 Olympic game city, Japanese inherently love ‘O-MO-TE-NA-SHI,’ meaning total empathy. Here’s an instance. A group of graduate students at Tokyo Institute of Technology whose goal was to be researchers, found that prospects loved their product during a Lean Launchpad class. They decided to pursue their startup idea. Three out of seven teams in this Lean Launchpad class in Osaka decided to found a company to pursue the business idea they had tested by talking to a bunch of customers. This may be an effect from the perfectionism in Japan, in that that they execute exhaustively once they believe and feel confident after Customer Discovery. I find this phenomenon an unexpected effect of Lean Startup methodology, but very interesting, particularly given that Japan must increase entrepreneurship and new startups for the revitalization of its economy.”

Finally, we talked to Justin Wilcox, another Lean Startup Conference speaker this year. Justin has started several companies that have not succeeded. This process has led him to investigate the science of entrepreneurship, a topic he spoke about at last year’s Lean Startup Conference in a talk on crowdfunding MVPs. He leads Lean Startup workshops around the world, recently spending a lot of time in Singapore and Puerto Rico. He is a mentor for both Founder’s Institute and Startup Weekend and he blogs about Lean Startup matters at CustomerDevLabs.com. He spoke about a carrot-and-stick approach to encouraging Lean Startup methods in areas where they may run up against entrenched practices.

“My experience outside the mainland U.S. has largely been in Puerto Rico and Singapore, where I've noticed some real differences from Silicon Valley—some that are challenges, some advantageous. One major challenge in smaller countries is that an acute sense of market size has a negative influence on people's sense of how to talk to customers. Founders in these smaller countries sometimes think of their home country as their market, when their opportunity could be much larger. And so when we tell people that they have to get out of the building and interview X number of people in a large market (U.S., Europe, etc.), it’s naturally daunting. It’s uncomfortable enough interviewing customers—complicate that with different accents, time zones and cultural references and ‘getting out of the building’ gets harder.

“Market size plays into a broader under-confidence outside the U.S.—or perhaps it’s actually an over-confidence inside the U.S. In Singapore, for example, the first question I'm often asked is, ‘So what do you think of Singapore?’ And of course I say that it's great, it's extraordinary — because it is. But I notice that in the U.S we don't ask foreigners what they think of our country because there's a basic assumption that the U.S. is amazing. Puerto Rico and Singapore were both colonized, and I sense their histories of ceding power to some force outside and greater than themselves (foreign governments, large corporations, etc.) is reflected in their entrepreneurial culture. In that context, several Lean Startup leaders in these communities have to empower founders to take risks and embrace failure in front of friends and family—harrowing work at best, inconceivable at worst.

“My approach to confronting these issues outside Silicon Valley —really, outside the U.S.—is similar to the way I teach Lean Startup in general, which is to accept that everything about the methodology is hard, because people have to be willing and able to overcome their own basic psychological defenses. I try to encourage people to think about what they want out of life, and the ways this set of practices can help them get there, instead of only focusing on the barriers. If I can connect with an entrepreneur about what they want to accomplish, and what they are passionate about, I have an easier time persuading them to get outside themselves and beyond their usual limits and defenses.

“Of course this is difficult abroad—it is anywhere. And in my experience, people in Silicon Valley don't have it figured out any better than people elsewhere. This gets to the advantage that people outside the U.S. can have in implementing Lean Startup. I've been struck by instances where I've been to other countries and spoken with people who really get Lean Startup, and who are shocked that all people in the Valley aren't already doing these things. In fact, the culture of Silicon Valley, flush with success, can create its own kind of complacency, as though simply being in the right place at the right time will be enough. People working in environments that are not flooded with success will use anything they can find that will step up their game. So while the broader culture may resist entrepreneurship, individual entrepreneurs abroad can be open to trying anything, because they have to be. For instance—I would put more money on JFDI in Singapore, based on the way I see them pushing themselves, than almost any incubator in the U.S.”

Join Kevin, Takashi and Justin on September 24 (September 25 in Asia) for a free webcast to discuss these and many other issues around implementing Lean Startup ideas beyond Silicon Valley. Don’t forget to register to attend, and then prepare your questions for the live Q&A to follow the discussion!

PS. For The Lean Startup Conference, we sell tickets in blocks. When one block sells out, the price goes up. We have just a few tickets left in the current discounted block. Register today for the best prices. 

Introducing New Lean Startup Practitioners

Guest post by Lisa Regan, writer for The Lean Startup Conference.

In building the program for this year’s Lean Startup Conference, we’ve tried to strike a balance. We’ve invited some deservedly well-known experts—people like Steve Blank, Laura Klein, Cindy Alvarez, Brant Cooper, Patrick Vlaskovits, Janice Fraser and Ash Maurya—to share their insights. But we’ve also sought out new voices. Indeed, the majority of our speakers this year are people who, though they’ve been doing compelling Lean Startup work and have great stories to share, are not on the conference circuit.

To give you a sense of these practitioners, we asked some our new speakers to talk about how they use Lean Startup methods. Below are three of their takes on  MVPs. To hear more from them, register for The Lean Startup Conference today. We’ve just  put a new batch of tickets on sale, and when this one sells out, the price goes up, so sign up now.

Steven Hodas will be speaking at the conference about  applying Lean Startup methods within extreme bureaucracy—in his case, the New York City Department of Education. Steven came to the NYCDOE via NASA, where he built the U.S. government’s first public website. After starting two companies of his own, he now heads the NYCDOE’s Markets initiative, which works on fostering smart demand for innovative solutions to the most urgent problems in New York’s public schools. Steven framed the value of MVP in terms of the enormous bureaucratic hurdles he has to cross:

“The NYCDOE is by far the largest school district in the country, with 1,800 schools, 1.1 million kids, and 135,000 employees. Our annual budget is $25 billion. As a life-long edtech entrepreneur I was excited and wary when I was asked to run Innovate NYC Schools, a program within the Office of Innovation that's intended to make the DOE more flexible, responsive, and better able to function. I knew that nearly everything I'd want to do would be anathema to standard operating procedure. After all, that is the point of the work.

For example, the basic notion of an MVP goes against the core tenets (whether spoken or unspoken) of large, bureaucratic organizations. Those tenets are:

1) We don't have customers, we have subjects. (Unspoken)

2) We are the experts in both the problem and the solution. (Spoken, loud and proud)

3) Iteration is a sign of failure, because (2). (Spoken, as if to a child)

4) The way to avoid (3) is to design by committee and then to ignore feedback, because anyway (1).

Still, because my job is to create bridges that don't yet exist between people inside the DOE (see 1-4) and those in the entrepreneur and maker communities, the MVP model provides the best template. In my case, the scarce resources, rather than time and money, are time and political capital. And the urgency stems not from an agile competitor, a first-mover market, or a power-law distribution of reward, but rather from the need to achieve enough velocity to overcome the tremendous gravitational pull of business-as-usual.

Further, my products are actually processes. My deliverable is not software per se (though sometimes there is that too), but alternative models for operating and engaging, a different way for DOE employees to view and go about their own work. An MVP strategy is the only way to generate the early wins that build credibility with both internal and external audiences, thus allowing me the space to learn, improve, deploy again, learn, improve, and so on. Each MVP is like a ratchet tooth or belay anchor that keeps the work from going backward, while enabling me to generate greater momentum for the next, more ambitious reach.”

Khalid Smith, co-founder of LessonCast, will be speaking at the conference about concrete things small startups can do to reach product-market fit with B2C customers. LessonCast builds tools that help teachers, schools, districts and teacher’s colleges improve the way that educators are prepared to teach. Khalid, who has a background in coaching, has talked about the challenge of really confronting your own decisions:

“When I speak at startup competitions, my talk usually starts with ‘I’m not as interested in how much of your business you can build this weekend as I am in seeing you execute the part of your business that creates value. So don’t build your business today. Today, do your business; then building it is just a function of time and money.’ But it’s really hard to take your own advice. This was true of our path to our current concierge model [Ed note: the concierge model is the practice of building a front end or offering a service that appears automated--and then behind the scenes, doing the work by hand]. My wife and co-founder Nicole pioneered the core idea behind LessonCast as she led the turnaround of a middle school struggling to raise its test scores. Using her experiences as a guide, we built a suite of tools that she wished she’d had during those tough two years. We thought we had our MVP and the next step was to scale our solution. But as we tried to sell it—crickets.

It took us a while to realize we’d committed the cardinal sin of using our own opinions and experiences in place of customer validation. I heard the words as I said them to teams in competitions: ‘You are not the customer!’ ‘What experiment did you run to invalidate your hypothesis?’ ‘How do you know what features are must-haves and should be part of your MVP?’ I realized we didn’t have answers to these questions ourselves. All we knew for certain was that the process a school leader would follow, and the software a school leader would use because he or she built it himself or herself, were both different from what other school leaders would actually pay for.   

What we had was an embarrassingly difficult-to-use, first-generation version of our software for school leaders, plus an accompanying process that had worked in an exceptionally skilled and passionate user, Nicole. Nowhere in there was anything that qualified as an MVP. So we sold our software the only way we knew how—by not telling the clients about it. We call our technique the concierge-coaching model. This is a hybrid model where we worked with clients as consultants, but gradually asked the clients to perform more and more tasks themselves using our software. Their ability to complete the ‘assignments,’ with us there or on their own, told us what new features we needed to develop, pinpointed where we needed to improve the user experience, and helped us develop better tutorials.

We focused on three customers: one K-12 school, one teacher’s college and one online course. It was slow and tedious, but the discipline of using the concierge model is paying off. We are on generation 3.0 of our minimum viable product. We are beginning to show the innovation economics of a healthy business and, for now, we can look ourselves in the mirror after a coaching session and be confident that we’re taking our own good advice.” 

Valerie Gofman will be speaking at the conference about how her company, Sharethrough, has scaled Lean Startup methods as it’s grown—a considerable challenge for any maturing organization. Valerie is a product manager, and she works with the business and development teams to guide strategy, testing and communication for both internal- and external-facing products. Her experience of Lean Startup methods has been shaped by the environment in which she works—as she says, “In the native advertising industry, which relies heavily on platforms that are mobile and provide scale, it is paramount that we are able to provide new product features quickly and efficiently.”

Here’s Valerie on how MVPs are woven into Sharethrough’s work:

“The nascent digital native advertising space, which Sharethrough created nearly two years ago, is rapidly evolving. In brief, native advertising is a form of paid media where the ad experience follows the natural form and function of the user experience in which it is placed. From the beginning, our product teams have been focused on iterating and learning as quickly as possible, with MVPs in common use.

MVP is a frequently-used acronym on the product and engineering teams and pervasive around Sharethrough as a whole. From early-stage efforts on new products to iterating on existing features, thinking in terms of the MVP allows us to focus on shipping-to-learn rather than shipping-to-ship. For example, with Vine's seemingly overnight success, brands began requesting it as a means to engage with their audiences. By limiting scope to a base set of functionality that we needed to get in front of those customers, we were able to turn around an MVP within two weeks.

Our MVPs commonly involve Wizard-of-Oz-style manual workarounds [Ed: this is akin to the concierge model Khalid mentioned] that provide efficiency and rapid learning for product and engineering teams and help us demonstrate early success with customers. For example, when thinking about bridging two applications that would eventually converge, rather than initially going through the multiple iterations necessary to  deliver a full-fledged integration, we shipped the most critical (and risky) components first, allowing us to have real users on the system give us a green light. Then, after some not-so-mild course correction, we were able to proceed with certainty.”

Steven, Khalid and Valerie will each focus on important Lean Startup lessons in their conference talks, and they are just three of dozens of new speakers this year. Sign up today for the conference to hear from them all.