Why do surveys ask the same question 8 billion times? #MRX

1) Bad survey design

This is the one that is most annoying. Let’s imagine you started answering a survey. You gave your age and gender, possibly even other details like your level of education or your income. Then, you get horrible message, “Sorry but you do not qualify for this survey, but here is another survey for you.” And then you are presented with the age and gender question again. And possibly again. Any possibly again. What is going on here? Why do you have to answer it so many times? Couldn’t they just remember the question from the first time?

Well, i wish that was how it worked. Unfortunately, the problem comes down to pure business. Every client has their own unique survey. They may have spent weeks or months building, designing, creating a survey that fits their very specific needs. Even the age and gender questions, as common as they are, have been carefully crafted just for them. (Let’s forget about the quality of the crafting for right now.) When the client actually fields the survey, the survey company doesn’t actually own the survey. They can’t “steal” the age and gender data from one survey and apply it to another survey. Even when it appears that the questions are identical. It’s much like copyright rules for music and art. Just because I can see it and hear it, doesn’t mean i can have it.

There is a solution though. Survey companies should have one or two standard demographic questions that clients must choose from. Clients must also agree that they do not own the data from those questions, but they may have access to them. Given all of the challenges faced by survey clients and companies, this is an easy problem to solve. I hope they do it soon.


2) Good survey design

You’ve also seen those long questions that seem to ask the same thing over and over again. “Do you like product A?” “Do you love product A?” Do you hate product A?” “Do you dislike product A?” For most people, it’s just so darn repetitive. If you think about it carefully though, these questions are not the same. Think about how YOU interpret each of the words. You probably have slightly different definitions than everyone else. Just slightly enough that across a series of questions, you would probably answer them differently from someone else who said they also liked the product. And, across ten questions, all of which seem identical, your total score is probably different from their total score, perhaps even different by a point or two. This is why researchers bore you with so many questions.

Again, though, there is a solution. In many cases, surveys include far more ‘repetitive’ questions than are needed. Usually, 8 to 10 questions are more than enough to create a wide range of scores among people. Thirty questions? Well, that researcher was just lazy and didn’t have to worry about field costs.

There are some other very legitimate reasons for asking the same question 8 billion times, but you’ve probably got the point now. Now go answer a survey! Take a survey now!

Related Articles


4 responses

  1. Lovestats has every right in the world to say anything she likes. Welcome to the internet, where you are allowed to voice your opinions about anything….and unfortunately, we also have to allow comments like yours to exist in this free and open medium no matter how ass-backwards it may be.

    Good day.

    1. That’s right, Anonymous, don’t you wish you could shut me up! Well guess what, Annie here approves comments before letting them show on her blog, so if she approved my not-so-nice comment, then she knows more about the internet than people like you.

      Annie, I was just trying to point out that, IMHO, posting a link to the I-Say landing page under “Take a survey now!” does nothing good for the credibility of your blog.

  2. Why do you keep pitching Ipsos surveys? I thought you were not with them anymore… And even if you still are, I don’t want to see this on your blog…please.

    1. You’re right, I am no longer with them. However, I still have a fondness for them as they were very good to me. I still have a fondness for Canadian Tire and ICOM but they don’t fit so well into the research and survey experience so they rarely come up. I guess one of the perils of reading somebody else’s is you get a taste of who they are AND what they know.

%d bloggers like this: