Customer Understanding - Surveys

by Scott Austin March 29, 2012

I'm a believer in the Lean Startup way of thinking. One of the things that Lean Startup stresses is a true understanding of the customer.  By true understanding, I mean actually going out and talking to your prospective customers.  As my company is very new, I've been spending a lot of time trying to understand the customer.  I'm doing lots of phone calls.  I go to networking events.  And I talk to them as much as possible.

One other tool I've been using is surveys.  While surveys won't get you the depth of information (qualitative) that a face-to-face conversation will.  Surveys do a great job of providing the aggregated opinions (quantitative) of the population.

I'm not going to discuss how to create the questions for a survey or how to analyze the results as those are longer discussions.  I will explain what tools I use to create/serve a survey and gather users for the survey.

Survey Tool

These days, there are many survey tools available.  They are easy to use.  Many have a freemium model where you get lots of capabilities even without paying.  They have fees for having higher capabilities, more respondents, etc.  The tool I use is Survey Monkey.  I use it because I have used it before and it meets my needs.  I have not looked into the other options.  I have a paid subscription which allows me to collect more responses.  The subscription also allows me unlimited questions and I can put my logo on my surveys.

Getting Users

As a startup, the hardest part is getting enough qualified people to make the results statistically relevant.  I do some of the usual tactics like posting to my social network and sending out to the company's opt-in list.  But that doesn't get me enough completed surveys.  So I have taken to using Amazon's MTurk to get respondents.  Here's AMZN's explanation of MTurk.

"Amazon Mechanical Turk is a marketplace for work that requires human intelligence. The Mechanical Turk service gives businesses access to a diverse, on-demand, scalable workforce and gives Workers a selection of thousands of tasks to complete whenever it's convenient.

Amazon Mechanical Turk is based on the idea that there are still many things that human beings can do much more effectively than computers, such as identifying objects in a photo or video, performing data de-duplication, transcribing audio recordings, or researching data details. Traditionally, tasks like this have been accomplished by hiring a large temporary workforce (which is time-consuming, expensive, and difficult to scale) or have gone undone."

So, the first step is to create a survey on Survey Monkey.  In the last page of the survey, I insert a completion code.  Then I create a job on MTurk.  It's a simple job that says go complete this survey.  I point to the Survey Monkey survey.  The MTurk user has to come back to MTurk when they are done with the survey and insert the completion code.

On MTurk, I restrict my job so that only people in the US can complete it.  I find that if I pay $.10 per completed survey, I can easily get 50 completed surveys.  Some surveys will get up to 100.  It can take a week or two to get that many completed. I've found that $.03-.05 will get you hundreds of completed surveys in a couple of days if you open it up to anyone on the planet.

A bit of caution

With surveys, it important to find a representative sample of people for your target audience.  It is also important to get a statistically relevant sample size.  Well, my process above has some flaws.  First, how representative are the people I'm getting through MTurk?  Well, their demographics are surprisingly relevant.  But the average MTurk user is more connected and tech savvy than the population in general.  Second, the best practice for completion size to get statistical relevance is 400.  I'm not getting that many.  I'd have to go to international or go with a higher price.  Neither, I want to do.

So my results must be interpreted through that lens.  I see them as directional.  Better than opinions or no data.  But any data that is key, I'll have to go back and refresh the data in the future.

Good luck in understanding your customers!

Scott Austin
Scott Austin