I hate social media research because: It’s not accurate #6 #MRX

I recently wrote a blog post citing ten of the biggest complaints about social media research. Today I address complaint #6.

Social media research is not accurate.

Accuracy is something we address in every research methodology. We talk about probability samples, representative samples, appropriate research design, well written questions, margin of error, and more.

We know 99.9% of surveys do not use probability samples. And yet we still run surveys. We know we manipulate and weight research data so that it appears to be representative of the population we’re interested in. We do our best to choose and create a research methodology that is best suited to answer our research objective but have difficulty finding the one that does it all. We try to write quality questions but always find a leading question, a biased question, or a misleading question somewhere in our surveys. We know that margin of error shouldn’t be reported on most surveys and yet margin of error is everywhere.

So here’s the deal. As researchers, we understand that error is a part of every research project we conduct. We know when we start a survey or focus group project that multiple sources of error will be introduced at every step. We’ve learned how to work around these error so that our research results are still meaningful and  useful. That’s how we’ve been trained.

Why do we expect anything different from social media research? No, it’s not a probability sample nor a representative sample. Sentiment analysis is far from perfect and content analysis isn’t perfect either. So in the end, if you decide that the error rate of your social media research is simply too great, then choose a better vendor. Quality does not depend on the method. Quality depends on the vendor.

Related articles

One response

  1. […] I hate social media research because: It’s not accurate – Annie Pettit of the social media market research firm Conversition is tackling common complaints about social media research. She argues that errors in social media research are no different in kind from errors in surveys. […]

%d bloggers like this: