As promised on Twitter, here is the final chapter of my short book on probability sampling. :)
For those of us in the real world, where probability sampling is the impossible dream (see previous posts if you’re puzzled why I say this), what are we to do? What’s the point of doing any serious marketing research if we can’t even meet the most basic statistical requirements required for predictive analytics?
Let’s see. We’ve been doing heavy duty market research for at least 50 years now. We’ve been dang good at predicting the success of products and the failure of politicians. And, acknowledging the 5% of predictions we got wrong because we know there is always unpredictable error, we’ve been doing a dang good job WITHOUT absolutely perfect probability samples.
What? How is that possible? Predicting the future without using probability sampling? Of course it’s possible. Even if researchers can’t sample perfectly, we know how to sample really well with what we’ve got. We’ve been learning this skill for decades. We learn the idealized theory in school, we learn the practical theory on the job, and we tinker around with all the tidbits at our disposal to put together a pretty fine sampling job at the end.
We know how to identify flaws in data. We know how to clean data. We know how to write quality surveys (whether we always do is another story). We know how to interpret ambiguous data. We know our stuff.
We know our stuff so well that we are able to predict the future even when glitches creep into our research process.
So tell me again. Why all the failed efforts to prove we use probability sampling when we’ve taught ourselves to work smart with what we’ve actually got?
This rant has been brought to you by the letter P and the number 5.
Read these too