1 Topic 5 Blogs: DIY surveys suck or save the day #MRX


G’day all and welcome to this months issue of 1 topic, 5 blogs. Todays topic is DIY surveys. Links to my fellow bloggers Bernie Malinoff, Joel Rubinson, Josh Mendelsohn and Brandon Bertelsen can be found below.

————————————————————————————————————–

I will admit. From a client point of view, DIY surveys have a ton of advantages. You can get the survey out quickly without any middle-man holding you up. Without a middle-man, the cost can be greatly reduced. And, you get exactly what you want without pleading and whining and complaining. If you have some solid experience or training in survey design, I am quite in favour of DIY for short 5 minute surveys.

But, when it comes to substantial surveys, I have many, many issues. The issues revolves mostly around “getting exactly what you want.” Any time I review survey drafts from survey newbies, and EVEN from people experienced in survey design, I find many of these terrible problems.

1) Leading questions: The writer usually has no intention of doing this, but they almost always give away the answer they want to receive. And whether consciously or unconsciously, the responder recognizes that and happily provides it.

2) Missing options: The writer is so focused on the options they are interested in that they forget there are options that are far more popular. They make sure that Brand A and Brand B are represented, but completely forget about Brand C, Store brand, Never buy, and Don’t know. This is another way to get the exact answer you want purely by bad survey design.

3) I can usually find a bunch of category jargon including words I don’t even recognize. On numerous occasions, I have seen surveys ask something like “Do you plan to purchase the X6000?” I am left trying to remember if the previous question was about televisions or motorcycles in hopes that X6000 will suddenly mean something.

4) I can usually find survey jargon.  People don’t purchase, they buy. People don’t use gum, they chew gum. People don’t purchase the 75g bar soap package, they buy the two pack.

Even if you much prefer DIY, I 100% believe that everyone needs a middle man. Whether that middle-man is a reputable survey company or another person in a completely different department, you absolutely must have fresh eyes. Fresh eyes find the annoying mistakes that you can no longer see because you’re tired of reading your survey. Fresh eyes find logic errors, spelling mistakes, and unclear questions. Market researchers will criticize far more harshly resulting in a far better survey, but fresh eyes of any sort are always in your favour.

In the end, it doesn’t matter how much money you saved by going the DIY route if your survey results are useless.


—————————————————————————————–
Links to the other 4 blogs coming shortly:

Bernie Malinoff of element 54
Joel Rubinson of the ARF
Josh Mendelsohn of Chadwick Martin Bailey
Brandon Bertelsen

About these ads

4 responses

  1. Hey Annie… great post as usual. The one thing I’d add about DIY is that we really have no idea how good or how poor it is. Personally, I’ve seen very few examples of DIY surveys in action and can’t judge the quality of the writing overall. I do know that there are some pretty crappy surveys written by the so-called experts. So is the crap percentage greater or lesser with DIY? All we really know is that there are horrible examples on both sides of the aisle.

  2. This is a great post. I especially agree with the point you made about the need for a middle man. If you’re interested, here’s a link to a blog post I wrote a while back that dealt with many of the same issues:

    http://rmsbunkerblog.wordpress.com/2010/06/16/five-questions-you-should-ask-before-doing-a-market-research-project-in-house-survey-focus-group-syracuse-central-new-york-ny-upstate/

  3. Neil Wilkinson | Reply

    Don’t remember where I read this but it rings true:

    SURVEY GENERAL’S WARNING: DIY Surveys Done Poorly Cause Erroneous Conclusions, Bad Decisions, And May Cost You Customers.

  4. [...] you’re designing a survey Annie at LoveStats has sage advice: Even if you much prefer DIY, I 100% believe that everyone needs a middle man. [...]

Agree? Disagree? Share your thoughts!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Buy The Listen Lady Book

Annie Pettit Buy Book listen lady amazon kindle ibookstore
Now in the iBookstore!

Learn about social media the quick and easy way. Buy your copy on Amazon.com the iBookstore or smashwords.

Enter your email address to subscribe to this blog.

Join 9,231 other followers

LoveStats on Twitter

All Top

Featured in Alltop
Follow

Get every new post delivered to your Inbox.

Join 9,231 other followers

%d bloggers like this: