I recently wrote a blog post citing ten of the biggest complaints about social media research. Today I address complaint #6.
Social media research is not accurate.
Accuracy is something we address in every research methodology. We talk about probability samples, representative samples, appropriate research design, well written questions, margin of error, and more.
We know 99.9% of surveys do not use probability samples. And yet we still run surveys. We know we manipulate and weight research data so that it appears to be representative of the population we’re interested in. We do our best to choose and create a research methodology that is best suited to answer our research objective but have difficulty finding the one that does it all. We try to write quality questions but always find a leading question, a biased question, or a misleading question somewhere in our surveys. We know that margin of error shouldn’t be reported on most surveys and yet margin of error is everywhere.
So here’s the deal. As researchers, we understand that error is a part of every research project we conduct. We know when we start a survey or focus group project that multiple sources of error will be introduced at every step. We’ve learned how to work around these error so that our research results are still meaningful and useful. That’s how we’ve been trained.
Why do we expect anything different from social media research? No, it’s not a probability sample nor a representative sample. Sentiment analysis is far from perfect and content analysis isn’t perfect either. So in the end, if you decide that the error rate of your social media research is simply too great, then choose a better vendor. Quality does not depend on the method. Quality depends on the vendor.
- The Top 10 Things We Love About Social Media Research #MRX (lovestats.wordpress.com)
- I hate social media research because: there are no demographics #1 #MRX(lovestats.wordpress.com)
- I hate social media research because: It’s not a rep sample #2 #MRX (lovestats.wordpress.com)
- I love social media research because: You get historical data #1 #MRX(lovestats.wordpress.com)
- I love social media research because: You can track results daily #3 #MRX(lovestats.wordpress.com)
- I hate social media research because: It doesn’t measure awareness #3 #MRX(lovestats.wordpress.com)
- I love social media research because: You get instant results #2 #MRX(lovestats.wordpress.com)