1) Bad survey design
This is the one that is most annoying. Let’s imagine you started answering a survey. You gave your age and gender, possibly even other details like your level of education or your income. Then, you get horrible message, “Sorry but you do not qualify for this survey, but here is another survey for you.” And then you are presented with the age and gender question again. And possibly again. Any possibly again. What is going on here? Why do you have to answer it so many times? Couldn’t they just remember the question from the first time?
Well, i wish that was how it worked. Unfortunately, the problem comes down to pure business. Every client has their own unique survey. They may have spent weeks or months building, designing, creating a survey that fits their very specific needs. Even the age and gender questions, as common as they are, have been carefully crafted just for them. (Let’s forget about the quality of the crafting for right now.) When the client actually fields the survey, the survey company doesn’t actually own the survey. They can’t “steal” the age and gender data from one survey and apply it to another survey. Even when it appears that the questions are identical. It’s much like copyright rules for music and art. Just because I can see it and hear it, doesn’t mean i can have it.
There is a solution though. Survey companies should have one or two standard demographic questions that clients must choose from. Clients must also agree that they do not own the data from those questions, but they may have access to them. Given all of the challenges faced by survey clients and companies, this is an easy problem to solve. I hope they do it soon.
2) Good survey design
You’ve also seen those long questions that seem to ask the same thing over and over again. “Do you like product A?” “Do you love product A?” Do you hate product A?” “Do you dislike product A?” For most people, it’s just so darn repetitive. If you think about it carefully though, these questions are not the same. Think about how YOU interpret each of the words. You probably have slightly different definitions than everyone else. Just slightly enough that across a series of questions, you would probably answer them differently from someone else who said they also liked the product. And, across ten questions, all of which seem identical, your total score is probably different from their total score, perhaps even different by a point or two. This is why researchers bore you with so many questions.
Again, though, there is a solution. In many cases, surveys include far more ‘repetitive’ questions than are needed. Usually, 8 to 10 questions are more than enough to create a wide range of scores among people. Thirty questions? Well, that researcher was just lazy and didn’t have to worry about field costs.
There are some other very legitimate reasons for asking the same question 8 billion times, but you’ve probably got the point now. Now go answer a survey! Take a survey now!
- Pie Charts – Our Evil Friend
- Why market researchers can never be marketers
- Survey Design Tip #3: Do You Encourage Straightlining?
- Laugh at yourself and then cry at our flailing industry
- In Honor of Infographics. #MRX
- This is why Twitter will die