Tag Archives: speeding

Data quality standards in mixed mode studies by John Bremer #CASRO #MRX

Live blogging from Nashville. Any error or bad jokes are my own.

– the most boring thing you can do with mobile is take a survey on it [HA! very true]
– it makes boring surveys more convenient than ever before
– dramatic growth in people starting surveys on mobile
– not all survey modes produce the same experience. there is differential completion rates. higher drop out rate on mobile particularly when the surveys get longer. different demographic set on mobile so quotas may overpopulate quickly.
– completion rates differ on mobile by country
– many people take surveys on multiple modes and this happens in every country. In the US, 60% of people ONLY take surveys on mobile [did i hear that right?]
– how do we treat quality in mixed mode studies. how should quality techniques be applied?
– why don’t we put quotas on mobile, should we?
– where 8% of people suspended a survey on tablet or computer, 20% of mobile phone starts were suspends
– tablets end up looking a lot of like computers
– think about your speeding metric. survey lengths differ greatly by mode. so if you’re including mobile phones times in you calculation, that raises the median time and raises the speeding time so that you’re cutting out more computer people than you ought to. you might need to use device specific speeder rules. [tell me now, who does this! we ought to! Love this 🙂 ]
– one you remove speeders, you can use a generic rule for random responding and straightlining
– they have a dataset of people who have taken an omnibus on a computer and on a mobile. it’s a matched dataset. [and John wonders, is it omnibii? 🙂 ]
– mobile responders always take longer and it gets worse the longer the survey gets. it’s not as far off for a shorter survey.
– we know there is a true mode effect
– must test quality at the mode level, must adjust speeding at mode level
– they recommend 5 to 10 minute survey though people still do 45 minutes on a mobile.
– you cut surveys into modules, they will take all the modules in a row.

[thanks for presenting data and tables John. i like that you don’t dumb things down. we need more of this because researchers KNOW NUMBERS even if people think its funny to say they don’t]

Radical Market Research Idea #2: Ban surveys longer than 15 minutes #MRX

An arrangement of psychoactive drugs

I know this is the right thing to do. You know this is the right thing to do. Market research suppliers, panel companies, sample companies all know this is the right thing to do. Clients know this is the right thing to do.


We continue to write surveys longer than 60 minutes. We continue to program surveys longer than 60 minutes. We continue to say yes to surveys longer than 60 minutes. We continue to worry about straightlining and speeding and random responding and declining engagement. We are as addicted to long surveys as some people are to smoking and I am to sugar.

Here’s an idea.

Everyone – MRIA, MRA, CASRO, ESOMAR, MRS, AMSRS – You – right now – Let’s get together and work as one team. Draw a line in the sand. Say no to long surveys. Say no to drugs.

How to encourage speeding in your surveys #MRX

As surveys grow longer and longer, market researchers have an urgent need to help responders save time and answer those surveys more quickly. To promote this cause, I have prepared this list with you in mind.speeding surveys

  1. Use lots of grids. Grids are an easy way to gather opinions about many questions in a very short space. Make it easy for responders to simply click click click click down a long column.
  2. Don’t reverse key items in grids. This only requires responders to actually read all the questions and this will really slow them down. Make every grid question positively worded and you’ll save everyone a lot of time.
  3. Include lots of Don’t Know and None of the Above options. It’s much quicker to choose those options than to think about your opinion.
  4. Don’t include screeners. Hey, everyone’s opinion is important so there’s no need to exclude anyone. Even if your survey is about lawn mowers and they don’t have a lawn. They’ve heard their friends talk about lawns so they know what they’re talking about.
  5. Don’t include open ended questions. People don’t really want to share opinions about things you didn’t ask and it just slows them down to have to verbalize their thoughts.

I hope you’ve enjoyed these tidbits of wisdom. Now go forth and survey!

%d bloggers like this: