How much is too much? How much is too little? There are lots of things in market research that require a healthy balance between doing the right thing and conducting business. Deciding how many screeners to offer to potential survey responders is one of them.
Most survey panels recognize that screening people out of surveys, no matter why, is bad for two reasons. First, and completely justified, it ticks off panelists who feel their time has been wasted and their opinions ignored. Second, it’s a waste for panels that just used up one survey in their data quality rule of “one survey invite per week” and they didn’t even get a complete in return.
For both of these reasons, many panels strive to handle the problem by offering up a number of surveys in a row to panelists. Panelists receive an invite and then proceed through one or more consecutive screeners until they qualify for a survey. (Let’s not consider what this means for probability sampling.)
I just spoke with someone who said their company takes people through up to five screeners before they say enough is enough. Panelists are even compensated for each screener they complete. I worry that even though they are being compensated, it is annoying to panelists. Screeners are obviously not surveys. Panelists can tell that they’ve been rejected once, twice, three, four, and five times. Imagine being rejected by five screeners every time you try to participate. It’s just one more source of rejection, something none of us need now or ever.
In fact, I even wonder if there is a rejection effect for which I have a two tailed hypothesis. Does increased rejection cause decreased survey scores due to the annoyance or does increased rejection cause increased survey scores due to the satisfaction of finally getting a survey to answer. I’d love an answer to that!
So what does your experience tell you? Are responders keeners for screeners?
Read these too