Probability Sampling – Proof that only telephone samples are quality samples

Histogram of sepal widths for Iris versicolor ...

Image via Wikipedia

I completely disagree with my own title!
My world is online. I started out writing online surveys in 1996 when I bugged the computer helpdesk at my graduate school to set me up with an online database. No one else at the university had ever done such a thing and i confused the heck out of them. I wrote my own html code which allowed me to specify font sizes, font colours, page colours, radio buttons, check boxes and text boxes. ooooooo….. so sophisticated. I’d be embarrassed to tell a scripter now that “I write my own code.”
Online research has never tried to say it uses probability sampling but, other methods of research have. There has been a debate over the last year specifically directed at online panels. Well, not really a debate. Some folks have been outraged that online panels do not use probability sampling and therefore they do not qualify to use statistics. To go even further, they suggest that telephone samples do use probability sampling and so results from that type of research are the most valid.
Let me offer up some ideas…
Telephone research – Do you always answer your phone? Is your phone number unlisted? Do you return phone calls? Do you politely tell telephone interviewers that you are busy when in fact you are nursing a bag of cheetos?
Mail research – Do you just throw out all the junk you get in the mail? Do you fill out surveys AND mail them?
Online research – Are you signed up for an online survey panel? Do you click on the survey banners that appear after you run a search and then finish every survey?
It seems to me that no matter how hard you try to use probability sampling, human beings just cannot cooperate. We’re not worms or mice or molecules. People choose when they wish to pay attention or participate. It’s not online panels. It’s research with human participants.

Probability sampling of people? No such thing.

Related Articles

%d bloggers like this: