WAPOR Day 2: People don’t lie on government surveys #AAPOR #MRX


Day two of WAPOR has come and is nearly gone, but my brain continues to ponder and debate all that I heard today. I hope you enjoy a few of the ramblings from my macaron infested brain.

  • People don’t lie on government surveys. Wow. That’s news to me! My presentation focused on how people don’t always provide exactly correct answers to surveys for various reasons – the answer isn’t there, they misread something, they deliberately gave a false answer. But, while people may feel more incentive to answer government surveys honestly, those surveys are certainly not immune to errors. Even the most carefully worded and carefully pre-tested survey will be misread and misinterpreted. And, some people will choose to answer incorrectly for a variety of reasons – privacy, anti-government sentiment, etc. There is no such thing as “immune to errors.” Don’t fool yourself.
  • How do you measure non-internet users? Well, this was a fun one! One speaker described setting up a probability panel (i know, i know, those don’t really exist). In order to ensure that internet usage was not a confounding variable, they provided a 3G tablet to every single person on the panel. This would ensure that everyone used the same browser, had the same screen size, had the same internet connection, and more. Of course, as soon as you give a tablet to a non-internet user, they suddenly become….. an internet user. So how do you understand perceptions and opinions from non-internet users. Chicken and egg! Back to paper you go!
  • Stop back translating. I don’t work much in non-English languages so it was interesting to hear this one. The authors are suggesting a few ideas:  questionnaire writers should write definitions of each question, preliminary draft translations should be provided by skilled translators, and finally, those two sets of information should go to the final translator. This is how you avoid “military rule” being translated as “role of the military” or “rules the military has” or “leadership of the military.” Interesting concept, and I’d love to know whether it’s efficient in practice.
  • Great presenter or great researcher: Pick one. I was reminded on many occasions today that, as a group, researchers are not great presenters. We face the screen instead of the audience,  we mumble, we read slides, and we speak too quietly. We focus on sharing equations instead of sharing learnings, we spend two thirds of the time explaining the method instead of sharing our insights. Let’s make it a priority to become better speakers. I know it won’t happen over night but I’ve progressed from being absolutely terrible to reasonably ok in a short matter of just 15 years. You can do it too.
%d bloggers like this: