AAPOR Concurrent Session A
Thursday, May 16, 1:30 p.m. – 3:00 p.m.
5 papers on Minimizing Nonresponse Bias (MANY speakers, check the website
- Evaluation and Use of Commercial Data for Nonresponse Bias Adjustment
- Using commercially available data to estimate non-responders, there is a lack of this info
- RDD is showing increased non-response
- Census aggregate data is uninformative of nonresponse bias due to error in matching, and low associations with the survey variables
- Purchased commercial data is better, e.g., filling out your toaster registration card [this is why i NEVER register anything. registering products is ONLY for marketing purposes.]. There can still be lots of missing data, weighting may not be possible, can be high rates of error.
- Allow the use of incomplete auxiliary data, use multiple imputation
- They had success matching adults of the same age/gender in the same household
- Interviewer Observations vs. Commercial Data:
Which is Better for Nonresponse Bias Correction?
- Should be we budget for commercial data or interviewer observations?
- How does the quality and usefulness compare?
- Observer can make a record of a non-responder as they person says “i don’t want to participate”
- Employment benefits study for social status, family type, house type, foreigners, young people
- Study tested for observations and commercial data separately and together
- [Wondering how important this information is given the recent miss on polling in BC Canada]
- Results show that observer evaluations for unemployment benefits were pretty good, income not so much; Commercial data not so good, worked better for a general population not an unemployed population
- Can’t say consistently which method was best for predicting employment benefits but commercial data was the worst
- Make sure the observations are related to the survey topic
- [cool paper!]
- Similar costs for both methods
- Assessing the Reliability of Unit Level Auxiliary Data in RDD Surveys: NHTSA Distracted Driving Survey
- [Why are there different standards of research for gvt research and private research?]
- Purchased demographic data from a panel company, did a distracted driving survey
- 18 minute interview [how many responders were just nodding their heads in agreement by the end?]
- [Funny how we assume there is RIGHT data. Just because you bought or created data doesn’t mean it’s true.]
- Landline and mobile surveys differ by 10% in terms of response by the youngest people (20% vs 30%)
- Across 4 methodologies, demographic frequencies differ by 10% to 40%; match rates between those on multiple datasources as low as 20%
- Auxiliary data tend to be household characteristics while interview data tends to be individual characteristics
- Agreement rates are too low for most measures to facilitate non-response analysis
- But, auxiliary data might be useful for underrepresented segments
- [Awesome ten dollar word title]
- Comparative Ethnographic Evaluations of Enumeration Methods Across Race/Ethnic Groups in the 2010 Census Nonresponse Follow-up and Update Enumerate Operations
- Ethnographics were to accompany an interviewer and observe live interviews and tape them if they had permission; to identify issues with enumeration
- Study targeted a number of race/ethnic groups as well as a control group
- “No data source is truth” [ha! that was my side remark further up this page 🙂 ]
- Sources of inconsistency are in order are interviewer error, mobility/tenuoousness, respondent concealment/refusal, addressed missed, not in the census, respondent confusion, language barrier
- 37% of questions were read correctly or with appropriate corrections — two thirds were changed by the interviewer!!!
- How do you get into apartment buildings? What about occupancy hotels? No buzzer boxes? Unlabeled units? In the middle of nowhere requiring a helicopter to get there?
- Esomar Best of Bulgaria: Brought to you by BAMOR #MRX (lovestats.wordpress.com)
- Thoughts on the CMRP designation #MRX #NewMR (mriablog.wordpress.com)
- No spouses were harmed in this experiment #MRX (lovestats.wordpress.com)