The myth of the Total Survey Error approach

Total Survey Error is a relatively recent approach to understanding the errors that occur during the survey, or research, process. It incorporates both sampling errors, non-sampling errors, and measurement errors, including such issues as specification error, coverage errors, non-response errors, instrument error, respondent error, and pretty much every single other error that could possibly exist. It’s an approach focused on ensuring that the research we conduct is as valid and reliable as it can possibly be. That is a good thing.

Here’s the problem. Total Survey Error is simply a list. A list of research errors. A long list, yes, but a list of every error that every researcher has been trained to recognize and account for in every research project they conduct.

We have been trained to recognize a bad sample, improve a weak survey, conduct statistics properly, generalize appropriately, and not promise more than we can deliver. Is conducting research the old name of ‘total survey error?’ It is not a new, unique approach. It does not require new study nor new books.

Perhaps I’m missing something, but isn’t total survey error how highly skilled, top notch researchers have been trained to do their job?


2 responses

  1. I think, Annie, you may be oversimplifying a bit. In his 1989 book, Survey Errors and Survey Costs, Groves brought together a diverse set of concepts of error, mostly from statistics, psychology, and survey methods, into a holistic view of how to evaluate an estimate from a survey in terms of likely error. You and I know dozens of people (at least) who can’t really do that in a systematic way, let alone with a TSE framework. If they could, a lot of what gets passed off today as serious research would never had seen the light of day.

    1. Perhaps simplifying but I stick to my point. And I wouldn’t dare call much of market research “serious” research. It is enlightening, interesting, and hypothesis generating analyses based on innumerable skews and biases. But as smart researchers, we’re supposed to recognize and compensate for them. Supposed.

%d bloggers like this: