Do I have your attention? By Pete Cape #CASRO #MRX


casrobanner
… Live blogging from beautiful San Francisco…

 

“Do I Have Your Full Attention?”

By Pete Cape, Global Knowledge Director, SSI

  • Why can’t people pay attention to surveys? why can’t they “choose 2” when we ask them to?
  • about a quarter of people got through an 80 item grid properly without skipping any items, Some people do a a bunch and then go back to the beginning again. 7% of people don’t start on the first item, 67% finish on the last item
  • More people fail the “Choose a 2” question if it’s in the middle of a grid rather than at the beginning
  • Can we understand the nature of attention?
  • “Task Unrelated Thoughts” – Ask people randomly throughout the day what are they doing. 30% of the time there is a mismatch between what are they doing and what are they thinking about.
  • “sustained attention to response task” – Ask people to click their mouse when they see a specific  number show up – 3 to 4% of the time we make mistakes
  • People make mistakes filling in forms all the time, e.g., wrong date on a passport form, wrong age on survey
  • 5% of people said “No” when asked if they birthdate they provided is correct
  • These are not deliberate errors, they are inattention
  • A grid is a decision making process, an application rules within a context
  • If an input is missing , you use heuristics or decision making stops and the schema is re-evaluated
  • “Please select the answer labeled 2” does not match up with the task required on a grid, if it’s the first item in a grid, they pass it because they haven’t created a mental schema for that question
  • Traps take longer to answer when they are at the beginning of a grid because we are working through our schema. When traps are in the middle, they are answered with the same speed as other questions.
  • A lot of people answer the trap question the same as the question immediately before – recency bias
  • Other people answer the trap with the answer they gave most often – frequency bias
  • Almost everyone would pass the question if it wasn’t in a grid – 4% failure rate
  • People use only half a second per item in a grid but 8 to 10 seconds per regular question
  • We aren’t seeing inattention, we’re seeing normal human behaviour. IT is the grid on trial, not the panelist.
  • Our attention traps catch good guys and bad guys. Grid encourage bad behaviour.
  • You can only say there is a problem with your survey if inattention is more than 3%.
  • [too many details for me to describe here, find the paper and read it]
%d bloggers like this: