So you think you have a probability sample

A random sampling of 300 random digits from A ...

Image via Wikipedia

[tweetmeme source=”lovestats” only_single=false]Ok, I give. You’ve got a study that actually uses probability sampling. By some magical slight of hand, you’ve identified every person in your desired population. Perhaps your population is the immediate members of your family, cancer doctors in your medical clinic or survey panel members.

You’ve managed to apply a random sampling method that gives each person an equal opportunity to participate. Maybe you picked numbers out of a hat. Maybe you used one of those books with 500 pages of random numbers.

You’ve managed to apply a process that gives every person an independent opportunity to participate. For this argument, let’s just assume that survey panels don’t kick certain people out because a housemate has also been selected.

Fine. You have a probability sample. You have covered off random error.

But folks, we aren’t in the business of hypothetical research. We make money from actual marketing research. Real people, real studies, real every day work. In my world, we just don’t do many of those one in a million studies that are capable of employing a reasonable semblance of probability sampling. Random error is not the whole picture.

Why does it seem like we always forget about non-random error? What about the vast majority of research that has 90% opt-out rates? Do we decide that those people weren’t part of the population to begin with? Does the lack of random error make non-random error ok?

I’m just having a hard time understanding the ongoing push to prove we are using probability samples when there remain other uneaten slices of the error pie.

Mmmm pie….

Read these too

  • Consumer Segmentation Gone Wrong
  • Rats, Respondents, and Rutabega
  • 3 Simplistic Requirements to Leverage Transparent Relationships for Synergistic Returns
  • Arriving Yesterday: A New Era of Research
  • Research, MeSearch, WeSearch, YouSearch by Dr. Seuss
  • 4 responses

    1. I figure the ‘making money’ bit is part of the cause actually. Everyone ‘knows’ that non-probability sampling has some issues, (And maybe the knowledge about what those issues are and what they mean to your research ends there.) so it’s no wonder that others will try to prove they’re using a probability sample.

      All in all it quite often seems like the market is more eager to jump on some specifics we all know about than on a lot of other actual problems, just because it just sounds great.

    2. love ur style annie and yes… mmmmm pie

    3. On the money. Here here.

    %d bloggers like this: