Screener keeners or rejection correction?

How much is too much? How much is too little? There are lots of things in market research that require a healthy balance between doing the right thing and conducting business. Deciding how many screeners to offer to potential survey responders is one of them.

Most survey panels recognize that screening people out of surveys, no matter why, is bad for two reasons. First, and completely justified, it ticks off panelists who feel their time has been wasted and their opinions ignored. Second, it’s a waste for panels that just used up one survey in their data quality rule of “one survey invite per week” and they didn’t even get a complete in return.

For both of these reasons, many panels strive to handle the problem by offering up a number of surveys in a row to panelists. Panelists receive an invite and then proceed through one or more consecutive screeners until they qualify for a survey. (Let’s not consider what this means for probability sampling.)

But what is the right number of screeners? Is it ok to send someone through ten minutes of screeners? Is it ok to give them two or three screeners?

Photo credit: xenia from morguefile.com

I just spoke with someone who said their company takes people through up to five screeners before they say enough is enough. Panelists are even compensated for each screener they complete. I worry that even though they are being compensated, it is annoying to panelists. Screeners are obviously not surveys. Panelists can tell that they’ve been rejected once, twice, three, four, and five times. Imagine being rejected by five screeners every time you try to participate. It’s just one more source of rejection, something none of us need now or ever.

In fact, I even wonder if there is a rejection effect for which I have a two tailed hypothesis. Does increased rejection cause decreased survey scores due to the annoyance or does increased rejection cause increased survey scores due to the satisfaction of finally getting a survey to answer. I’d love an answer to that!

So what does your experience tell you? Are responders keeners for screeners?

Read these too

  • #MRA_FOC #MRX Karaoke by Jim Longo. I dare you.
  • If Frozen Meat Isn’t a Bribe, What About Raw Meat?
  • It’s Here! Twitterversity is Here! #MRXU
  • Proof that sampling social media data is critical, but you already knew that
  • If you DARE – the chart test of death
  • About these ads

    4 responses

    1. Does increased rejection cause decreased survey scores due to the annoyance or does increased rejection cause increased survey scores due to the satisfaction of finally getting a survey to answer.

    2. In my opinion, there is a 3rd reason why MR should try to limit online screeners when adressing panelists:
      Facing a frustrating moment ( = being screened out for the Xth time ) panelists could engage in an ovestatement phase=> ticking as many items one can to be qualified for the survey !

    3. The order effects of screeners are a pretty interesting area of study too, as they introduce what OTX calls “routing bias”:

    4. In my opinion, there is a 3rd reason why MR should try to limit online screeners when adressing panelists:
      Facing a frustrating moment ( = being screened out for the Xth time ) panelists could engage in an ovestatement phase=> ticking as many items one can to be qualified for the survey !

    Agree? Disagree? Share your thoughts!

    Fill in your details below or click an icon to log in:

    WordPress.com Logo

    You are commenting using your WordPress.com account. Log Out / Change )

    Twitter picture

    You are commenting using your Twitter account. Log Out / Change )

    Facebook photo

    You are commenting using your Facebook account. Log Out / Change )

    Google+ photo

    You are commenting using your Google+ account. Log Out / Change )

    Connecting to %s

    Buy The Listen Lady Book

    Annie Pettit Buy Book listen lady amazon kindle ibookstore
    Now in the iBookstore!

    Learn about social media the quick and easy way. Buy your copy on Amazon.com the iBookstore or smashwords.

    Enter your email address to subscribe to this blog.

    Join 8,881 other followers

    LoveStats on Twitter

    All Top

    Featured in Alltop
    Follow

    Get every new post delivered to your Inbox.

    Join 8,881 other followers

    %d bloggers like this: