Friday, November 7, 2008

Using AdWords to assess demand for your new online service, step-by-step

Google Analytics ベンチマークIf you want to build an online service, and you don't test it with a fake AdWords campaign ahead of time, you're crazy. That's the conclusion I've come to after watching tons of online products fail for a complete lack of customers. So I thought I would walk you through exactly how to run a "fake landing page" test using cheap tools that require no technical skills whatsoever.

Our goal is to find out whether customers are interested in your product by offering to give (or even sell) it to them, and then failing to deliver on that promise. If you're worried about disappointing some potential customers - don't be. Most of the time, the experiments you run will have a zero percent conversion rate - meaning no customers were harmed during the making of this experiment. And if you do get a handful of people taking you up on the offer, you'll be able to send them a nice personal apology. And if you get tons of people trying to take you up on your offer - congratulations. You probably have a business. Hopefully that will take some of the sting out of the fact that you had to engage in a little trickery.

To motivate you to give this a try, let me tell you a story from the early days of IMVU. It was fall 2004, and the presidential election was in full swing. One day, we became convinced that a killer app for IMVU would be to sell a presidential debate bundle, where our customers could put on a Bush or Kerry avatar, and then engage in mock debates with each other. It was one of those brilliant startup brainstorms that comes to the team in a flash, with a giant thunderclap. We spent weeks working on this new product, racing the clock so it would be done in time for the real presidential debates. We had endless arguments internally about what features it should include, how the avatars should look, and how much it should cost. We finally settled on a $1.99 price point, figuring that we wouldn't make much money, but at least we wouldn't get in the way of achieving scale. Finally the day came, we unleashed the landing page, emailed our existing customers, and started advertising online.

The net result: we sold exactly zero presidential debate avatars. None. Nada. We tried different price points, different ad copy, different landing pages. Nothing made any difference. Turns out, there was aboslutely no demand whatsoever for that particular product. And we could have found it out quite easily, if we'd used the simple five step process below. Oops - there went several precious weeks of development effort down the drain.

So, if you're interested in helping avoid mistakes like that, here are the steps:
  1. Get a domain name. It doesn't have to be the world's catchiest name, just pick something reasonably descriptive. If you're concerned about sullying your eventual brand name, don't use your "really good" name, pick a code name. Make sure your domain registrar offers free "website forwarding" if you don't use a hosting service that lets you use a custom domain name in step 2.

  2. Setup a simple website. I recommend using a hosted service like SnapPages. You basically want to create two pages: a landing page that says what your product does, and a signup page that people can use to register for it. If you're feeling charitable, you can add a third page that lets people know that the product isn't available right now, and that you'll get back to them when it is.

  3. Enable Google Analytics tracking. The nice thing about services like SnapPages is that they offer this built-in. You just have to sign up for Google Analytics, get your account number, and plug it into your site.

  4. Start an AdWords campaign. Google AdWords has no minimum buy required, so you can easily run a campaign for five dollars a day, or even less. Just put in your credit card. I recommend using their Keyword Tool to setup your initial list of ad targets. Don't worry about selecting particularly good keywords, if you're new to SEM. Just load them all in and choose a low cost-per-click. I used to use $.05, but you might want to go as high as $.25 or $.50. Just make sure you choose a maximum daily budget that is affordable for you to run the campaign for a few weeks with. I would aim to get no more than 1o0 clicks per day - over the course of a week or two, you'll get pretty good conversion data.

  5. Measure conversion rates. Use Google's built-in Analytics/AdWords integration, to track the effectiveness of each ad you run. Then set up "goal tracking" in Analytics to see how many people actually sign up using your registration page. Here are the stats you want to pay particular attention to: the overall conversion % of customers from landing page to completed registration, the click-through-rate for your ads on different keywords, and the bounce rate of your landing page for different keywords.
Armed with that data, you will know a lot about what your business will look like when you finally do build the product you're imagining. At the very least, you can plug those assumptions into your financial model, now that you have a sense for what the cost of acquiring new customers might look like.

Even more importantly, you can start to experiment with feature set, positioning, and marketing - all without building a product. Use Google Optimizer to try different landing pages (even radically different landing pages) to see if any particular way of talking about your product makes a difference to your conversion rates. And if you're getting conversion rates that you feel good about, try asking for a credit card or other method of payment. If that takes your conversion rate to zero, that doesn't necessarily mean you don't have a business, but I'd give it some serious thought. Products that truly solve a severe pain for early adopters can usually find some visionary customers who will pre-order, just based on the vision that you're selling. If you can't find any, maybe that means you haven't figured out who your customer is yet.

And if you don't know who your customer is, perhaps some customer development is in order?
Reblog this post [with Zemanta]


  1. Eric -- This is a pretty interesting idea. Seems like a fairly low-cost method of gauging customer interest for a would-be product. Have you tried this or is it just a promising theory at this point? It seems like if it worked, you'd have great data at low cost, but that there's a real chance that it would give false negatives. SEM only gets at one segment of users, and for a brand new product, you presumably need some skill at marketing in general and adsense in particular to get any useful info. ... Any specific experiences to share?

  2. This comment has been removed by the author.

  3. your post inspired me -

    i also bought some adwords but it's not driving a lot of traffic to the landing page. any suggestions on how to increase traffic to the landing page?

  4. False negatives would be an issue, but combining ad words with other forms of advertising, such as guerrilla marketing, might yield a more robust answer.

    Depending on the type of product, postings on forums, classifieds and emails could give enough information to help gauge potential response.

  5. Adword is a PPC method to promote your website. Another way to promote the website is doing organize search. For example, make sure your website is in the first page of Google search for the given keyword. This is one of the blog we posted about how some company is doing SEO to promote the website.

  6. A very interesting strategy. I too would be concerned about false negatives, but perhaps this strategy could be integrated into a broader market research strategy that included user engagement, viral marketing, etc which would all be quantifiable under the Google analytics.

    I'm defiantly going to give it a shot on my next project! Thanks for the insight!

  7. Eric, Excellent article about an essential technique for startups.... I respectfully disagree with your recommendation, "don't worry about selecting particularly good keywords" because I've found it's important to test with the full set of keywords that are relevant to the concept or business. To help find the right keywords, I've written an article on my blog Web Startup Help that details How to Do Keyword Research for startups. I've linked to your article from mine.

  8. This comment has been removed by a blog administrator.

  9. It's my understanding that at one point (if not currently) Time-Life books did essentially this for their book series.

    They would make commercials about a new book series, and then run the commercials at times when the TV rates were low (late night, UHF channels, etc)

    If they got enough orders, they then decided to print the series. Otherwise they just canceled the orders with some excuse.

  10. Great post - I think I'll try this. One concern I might have with doing this is that you are competing for ad space with your competitors. By putting yourself directly in their space you are alerting them to your emergence as a threat. Now we're not exactly in stealth-mode - we've pitched at Techcrunch - but I'd rather not go dancing in my competitor's faces.

    Do you think, therefore, that you /should/ use a specifically set-up URL, with no connection to your real company, just so that you are not alerting competitors to exactly who you are?

  11. What a great post. We run a web development company and we start by creating a splash page very similar to what is describe here. If we don't get a certain amount of signups on that splash page, we don't develop the rest of the site becuase they will not have any customer.

  12. Great idea, which I've tried twice, and I would do again.

    One thought and one question:

    - this also gives you a great idea of the keyword volume and potential cpc for your site
    - I imagine google quality scoring would be an issue in some cases, and you may not get volume (especially if your campaign/business is limited to a geography). Any thoughts on how to make sure your setup doesn't have this issue, so that you're not discouraged unnecessarily?

  13. I too think this is a very effective way of testing for actual demand of a product, but I'm worried this trickery (accepting orders for products that are just concepts) would violate AdWords terms and conditions. A customer that actually thinks they are ordering will obviously give the most realistic data, but how far can you push it and not violate terms? Anyone have thoughts on this?

  14. I would love to get much greater detail on this type of testing. 1. How do you get Google to not think this is a sham since these days the quality score is important when buying Adwords 2. What is a significant amount of testing per keyword (100 page views)? 3. Won't many users try to click around and see that's its not a real site and then not register? 4. With the many permutations of keyword versus testing many site messaging and calls to action how does one really know when sufficient volume of testing has been conducted? I realize no responses is an easy conclusion but what about the mid-case scenarios...5. Also, to get to this level of keyword/messaging optimization, I think the keywords would get expensive and this whole campaign would get expensive.

    All of that said, this makes 100% sense, but I'd love you to address these questions :).

  15. Great how to, Eric. In the standard talk I give on go-to-market for startups (, I always recommend this blog (when I started giving it two years ago, fewer people knew about the Lean Startup!). Now when I recommend using AdWords/Craigslist/eBay to test their market, I'll send them to this post.

  16. Thanks for the post. I've actually made the mistake you mentioned of investing time/money on building a product that not a lot of people wanted. I have yet to use Google Adwords but we are using Facebook ads and getting a few signups each day on our splash page for a new project. Looking to do Google Adwords next.