Minimizing Nonresponse Bias (GREAT session) #AAPOR #MRX

AAPOR… Live blogging from beautiful Boston, any errors are my own…

AAPOR Concurrent Session A

Thursday, May 16, 1:30 p.m. – 3:00 p.m.
5 papers on Minimizing Nonresponse Bias (MANY speakers, check the website

First paper

  • Evaluation and Use of Commercial Data for Nonresponse Bias Adjustment
  • Using commercially available data to estimate non-responders, there is a lack of this info
  • RDD is showing increased non-response
  • Census aggregate data is uninformative of nonresponse bias due to error in matching, and low associations with the survey variables
  • Purchased commercial data is better, e.g., filling out your toaster registration card [this is why i NEVER register anything. registering products is ONLY for marketing purposes.]. There can still be lots of missing data, weighting may not be possible, can be high rates of error.
  • Allow the use of incomplete auxiliary data, use multiple imputation
  • They had success matching adults of the same age/gender in the same household

Second paper

  • Interviewer Observations vs. Commercial Data:

    Which is Better for Nonresponse Bias Correction?

  • Should be we budget for commercial data or interviewer observations?
  • How does the quality and usefulness compare?
  • Observer can make a record of a non-responder as they person says “i don’t want to participate”
  • Employment benefits study for social status, family type, house type, foreigners, young people
  • Study tested for observations and commercial data separately and together
  • [Wondering how important this information is given the recent miss on polling in BC Canada]
  • Results show that observer evaluations for unemployment benefits were pretty good, income not so much; Commercial data not so good, worked better for a general population not an unemployed population
  • Can’t say consistently which method was best for predicting employment benefits but commercial data was the worst
  • Make sure the observations are related to the survey topic
  • [cool paper!]
  • Similar costs for both methods

Third paper

  • Assessing the Reliability of Unit Level Auxiliary Data in RDD Surveys: NHTSA Distracted Driving Survey
  • [Why are there different standards of research for gvt research and private research?]
  • Purchased demographic data from a panel company, did a distracted driving survey
  • 18 minute interview [how many responders were just nodding their heads in agreement by the end?]
  • [Funny how we assume there is RIGHT data. Just because you bought or created data doesn’t mean it’s true.]
  • Landline and mobile surveys differ by 10% in terms of response by the youngest people (20% vs 30%)
  • Across 4 methodologies, demographic frequencies differ by 10% to 40%; match rates between those on multiple datasources as low as 20%
  •  Auxiliary data tend to be household characteristics while interview data tends to be individual characteristics
  • Agreement rates are too low for most measures to facilitate non-response analysis
  • But, auxiliary data might be useful for underrepresented segments

Fifth paper

  • [Awesome ten dollar word title]
  • Comparative Ethnographic Evaluations of Enumeration Methods Across Race/Ethnic Groups in the 2010 Census Nonresponse Follow-up and Update Enumerate Operations
  • Ethnographics were to accompany an interviewer and observe live interviews and tape them if they had permission; to identify issues with enumeration
  • Study targeted a number of race/ethnic groups as well as a control group
  • “No data source is truth”  [ha! that was my side remark further up this page 🙂  ]
  • Sources of inconsistency are in order are interviewer error, mobility/tenuoousness, respondent concealment/refusal, addressed missed, not in the census, respondent confusion, language barrier
  • 37% of questions were read correctly or with appropriate corrections — two thirds were changed by the interviewer!!!
  • How do you get into apartment buildings? What about occupancy hotels? No buzzer boxes? Unlabeled units? In the middle of nowhere requiring a helicopter to get there?
%d bloggers like this: