Respondent Identity Verification with Non-Panel, Real-time Samples: Is There Cause for Concern by Nancy Brigham and James Karr #CASRO #MRX


Live blogging from the CASRO Digital conference in San Antonio, Texas. Any errors or bad jokes are my own.CasroDigital

Respondent Identity Verification with Non-Panel, Real-time Samples: Is There Cause for Concern?”


Nancy Brigham
As the research industry evolves toward non-panel sample sourcing and real-time sampling, questions have arisen about the quality of these respondents, especially in the area of respondent identity verification. This research addresses two key questions: Are fraudulent identities a concern with non-panel samples, and what are the research implications of assessing identity validation with these samples? This work examines identity verification and survey outcomes among five different levels of Personally Identifiable Information (PII) collection. In addition to the presenters, this paper was authored by Jason Fuller (Ipsos Interactive Services, Ipsos).

  • Nancy Brigham, Vice President, IIS Research-on-Research, Ipsos
  • James Karr, Director & Head of Analytics, Ipsos Interactive Services

James Karr
  • Do people whose validity cannot be confirmed providing bad data? Should we be concerned?
  •  What do we know about non-panel people? Maybe they don’t want to give PII to just take one survey. Will they abandon surveys if we ask for PII?  [I don’t see answering “none” as a garbage question. It’s a question of trust and people realizing you do NOT need my name to ask me my opinions.]
  • Is it viable to assess identify validation with non-panel sources?
  • In the study, PII was asked at the beginning of the survey [would be great to test end of survey after people have invested all that time in their responses]
  • Five conditions asking for combination of name, email, address
  • Used a third party validator to check PIIEmbedded image permalink
  • 25% of people abandoned at this point
  • Only 4 out of 2640 respondents gave garbage information at this point, 12 tried to bypass without filling it out and then abandoned. It’s so few people that this is hard to trust. [Hey people, let’s replicate]
  • Name and address caused 6% of abandonment, name and email caused only 3% abandonment
  • Did people get mad that we asked this? can we see anger in concept test? no.
  • didn’t lead to poor quality survey behaviours – used a 13 minute survey
  • when given a choice, people prefer to give less information – most people will choose to give name and email, low some people will give all information
  • Simply collecting PII didn’t appear to influence other aspects
  • Did their non-panel source give lower quality data? no. 82% passed the validation test across all conditions. Those who provide the most comprehensive data validate better but that’s likely because it’s more possible to validate them.
  • Real-time sample gives just as good data quality, same pass rates, no data differences
  • Conclude the screening question is necessary, heads up that PII question will be coming
  • Younger ages abandoned more across all test conditions
  • This study only looked at the general population, not hard to reach groups like hispanics, or different modes like mobile browsers, or in-app respondents

Other Posts

%d bloggers like this: