Respondent Identity Verification with Non-Panel, Real-time Samples: Is There Cause for Concern?”
|As the research industry evolves toward non-panel sample sourcing and real-time sampling, questions have arisen about the quality of these respondents, especially in the area of respondent identity verification. This research addresses two key questions: Are fraudulent identities a concern with non-panel samples, and what are the research implications of assessing identity validation with these samples? This work examines identity verification and survey outcomes among five different levels of Personally Identifiable Information (PII) collection. In addition to the presenters, this paper was authored by Jason Fuller (Ipsos Interactive Services, Ipsos).
- Do people whose validity cannot be confirmed providing bad data? Should we be concerned?
- What do we know about non-panel people? Maybe they don’t want to give PII to just take one survey. Will they abandon surveys if we ask for PII? [I don’t see answering “none” as a garbage question. It’s a question of trust and people realizing you do NOT need my name to ask me my opinions.]
- Is it viable to assess identify validation with non-panel sources?
- In the study, PII was asked at the beginning of the survey [would be great to test end of survey after people have invested all that time in their responses]
- Five conditions asking for combination of name, email, address
- Used a third party validator to check PII
- 25% of people abandoned at this point
- Only 4 out of 2640 respondents gave garbage information at this point, 12 tried to bypass without filling it out and then abandoned. It’s so few people that this is hard to trust. [Hey people, let’s replicate]
- Name and address caused 6% of abandonment, name and email caused only 3% abandonment
- Did people get mad that we asked this? can we see anger in concept test? no.
- didn’t lead to poor quality survey behaviours – used a 13 minute survey
- when given a choice, people prefer to give less information – most people will choose to give name and email, low some people will give all information
- Simply collecting PII didn’t appear to influence other aspects
- Did their non-panel source give lower quality data? no. 82% passed the validation test across all conditions. Those who provide the most comprehensive data validate better but that’s likely because it’s more possible to validate them.
- Real-time sample gives just as good data quality, same pass rates, no data differences
- Conclude the screening question is necessary, heads up that PII question will be coming
- Younger ages abandoned more across all test conditions
- This study only looked at the general population, not hard to reach groups like hispanics, or different modes like mobile browsers, or in-app respondents
- Peanut Labs Ask-Me-Anything with special guest Tom Ewing
- Peanut Labs Ask-Me-Anything with special guest Kristin Luck
- What is a convenience sample?
- What does plus or minus 3% 19 times out of 20 mean?
- Short answer lists inflate endorsement rates
- What is Vue magazine? #MRX
- CASRO in San Antonio: The fun so far #MRX #CASRO