Data quality standards in mixed mode studies by John Bremer #CASRO #MRX


Live blogging from Nashville. Any error or bad jokes are my own.

– the most boring thing you can do with mobile is take a survey on it [HA! very true]
– it makes boring surveys more convenient than ever before
– dramatic growth in people starting surveys on mobile
– not all survey modes produce the same experience. there is differential completion rates. higher drop out rate on mobile particularly when the surveys get longer. different demographic set on mobile so quotas may overpopulate quickly.
– completion rates differ on mobile by country
– many people take surveys on multiple modes and this happens in every country. In the US, 60% of people ONLY take surveys on mobile [did i hear that right?]
– how do we treat quality in mixed mode studies. how should quality techniques be applied?
– why don’t we put quotas on mobile, should we?
– where 8% of people suspended a survey on tablet or computer, 20% of mobile phone starts were suspends
– tablets end up looking a lot of like computers
– think about your speeding metric. survey lengths differ greatly by mode. so if you’re including mobile phones times in you calculation, that raises the median time and raises the speeding time so that you’re cutting out more computer people than you ought to. you might need to use device specific speeder rules. [tell me now, who does this! we ought to! Love this🙂 ]
– one you remove speeders, you can use a generic rule for random responding and straightlining
– they have a dataset of people who have taken an omnibus on a computer and on a mobile. it’s a matched dataset. [and John wonders, is it omnibii?🙂 ]
– mobile responders always take longer and it gets worse the longer the survey gets. it’s not as far off for a shorter survey.
– we know there is a true mode effect
– must test quality at the mode level, must adjust speeding at mode level
– they recommend 5 to 10 minute survey though people still do 45 minutes on a mobile.
– you cut surveys into modules, they will take all the modules in a row.

[thanks for presenting data and tables John. i like that you don’t dumb things down. we need more of this because researchers KNOW NUMBERS even if people think its funny to say they don’t]

One response

  1. Annie, just to clarify, the 60% number is that 60% of those that take surveys on their mobile device, only take surveys on their mobile device. So the base is not all survey takers but survey takers who are active on mobile.

%d bloggers like this: