In Search of Horribly Low Response Rates #MRX

Ask anyone what the response rate to their last research project was and they’ll hold their head in shame if the answer is a number under 10%. As researchers, we work really hard to generate response rates that are as high as we can possibly get them. In the competitive world of market research, the survey panel or focus group recruiter with the highest response rate just might win the job.

But wait. Why do some sources have higher response rates than others?

  1. Active rules: Sources that only invite people to research if they have completed a research study in the last month have much higher response rates.
  2. Incentives: Sources that provide more valuable incentives have higher response rates.
  3. Recruitment: Sources that recruit participants from research sources have higher response rates (e.g., “Thank you for answering our Purchase Satisfaction Survey. Would you like to join our panel?”

skinner box

In each of these three situations, the research panels have essentially pre-selected people based on their propensity to participate in research. And, as we all know, the propensity to participate in research is not a randomly distributed characteristic. Certain personality types are just more or less likely to want to participate in research. And this brings me to my point.

Shouldn’t we actually be seeking out the lowest response rate possible?  Instead of focusing on gathering opinions from people who are MOST likely to want incentives or who always participate in research, shouldn’t we keep the pipe lines open to accept opinions from research keeners as well as those who hardly ever want to participate in research and who couldn’t care less about incentives? Wouldn’t a really low response rate reflect a research participant pool that is awash with both keeners and frequent abstainers, a pool that is more reflective of the real population?

Perhaps we should actually be seeking out low response rates. Perhaps we shouldn’t judge sample providers simply on response rates.  Perhaps we should consider that the quality of a research sample goes far beyond response rates. What a strange thought.

4 responses

  1. Great perspective and intriguing point of view. we need to really get a much better grip on who participates, who doesn’t, and what this means.

    1. It’s certainly an odd perspective! I think sometimes we focus too much on high response rates and not enough on quality responses.

  2. Good thinking. Although even in a low response rate pool, presumably people who don’t like doing surveys/incentives will be under-represented in the responses? And then how do we control for that? How do we know how many people don’t do surveys and what their attitudes are, if they won’t tell us?

    I’m not saying it’s a bad idea – I work for a company which is proud of its at least once-unique approach to streaming respondents from all over the place, not being restricted to panels. I just don’t know what the answer is. I don’t know how to weight for the absence of people whose attitudes, opinions and often even demographics I don’t know. But then, if we all thought like that then the concept of survey based research is…mmmm….

    1. 🙂 Oh, the humanity! I guess we just need to always be open-minded. We can’t focus purely on response rates or incentive rules or panel rules. We need to understand and appreciate the entire picture. THAT is the sign of good quality research.

%d bloggers like this: