[tweetmeme source=”lovestats” only_single=false]
G’day all and welcome to this months issue of 1 topic, 5 blogs. Todays topic is DIY surveys. Links to my fellow bloggers Bernie Malinoff, Joel Rubinson, Josh Mendelsohn and Brandon Bertelsen can be found below.
I will admit. From a client point of view, DIY surveys have a ton of advantages. You can get the survey out quickly without any middle-man holding you up. Without a middle-man, the cost can be greatly reduced. And, you get exactly what you want without pleading and whining and complaining. If you have some solid experience or training in survey design, I am quite in favour of DIY for short 5 minute surveys.
But, when it comes to substantial surveys, I have many, many issues. The issues revolves mostly around “getting exactly what you want.” Any time I review survey drafts from survey newbies, and EVEN from people experienced in survey design, I find many of these terrible problems.
1) Leading questions: The writer usually has no intention of doing this, but they almost always give away the answer they want to receive. And whether consciously or unconsciously, the responder recognizes that and happily provides it.
2) Missing options: The writer is so focused on the options they are interested in that they forget there are options that are far more popular. They make sure that Brand A and Brand B are represented, but completely forget about Brand C, Store brand, Never buy, and Don’t know. This is another way to get the exact answer you want purely by bad survey design.
3) I can usually find a bunch of category jargon including words I don’t even recognize. On numerous occasions, I have seen surveys ask something like “Do you plan to purchase the X6000?” I am left trying to remember if the previous question was about televisions or motorcycles in hopes that X6000 will suddenly mean something.
4) I can usually find survey jargon. People don’t purchase, they buy. People don’t use gum, they chew gum. People don’t purchase the 75g bar soap package, they buy the two pack.
Even if you much prefer DIY, I 100% believe that everyone needs a middle man. Whether that middle-man is a reputable survey company or another person in a completely different department, you absolutely must have fresh eyes. Fresh eyes find the annoying mistakes that you can no longer see because you’re tired of reading your survey. Fresh eyes find logic errors, spelling mistakes, and unclear questions. Market researchers will criticize far more harshly resulting in a far better survey, but fresh eyes of any sort are always in your favour.
In the end, it doesn’t matter how much money you saved by going the DIY route if your survey results are useless.
Links to the other 4 blogs coming shortly: