I got in a fight at the Esomar 3D conference, a near death, weapons drawn, lay your stats on the line fight. About what? About DIY research. I was accused of bashing DIY on the stage in front of the entire Esomar audience. In my effort to outline the serious consequences of failing to sample correctly, clean data correctly, analyze data correctly, I bashed research that is conducted with inappropriate and insufficient tools. So to be clear, here is my stance regarding DIY research.
Image via Wikipedia
I am against researchers doing their own plumbing, doctors doing their own wiring, lawyers doing their own carpet laying, locksmiths doing their own lawyering. I am against inexperienced, unqualified people doing any task that endangers the quality of the work and the reputation of the industry. I am against Joe Smith writing bad questions, applying statistics incorrectly, and generalizing results improperly.
I’m not against DIY. I’m against poor quality. How about you?
Related articles
- A Decade of Netnography #Eso3D #esomar (lovestats.wordpress.com)
- The Lives of Cool Digital Natives #Eso3D #esomar (lovestats.wordpress.com)
- The Researchification of Games by Peter Harrison #Eso3D #esomar (lovestats.wordpress.com)
- How Far is Too Far by Bernie Malinoff #Eso3D #esomar (lovestats.wordpress.com)
- Facial Imaging #Eso3D #esomar (lovestats.wordpress.com)
- Inspiring Online Dialogue by Rodriguez and Wong #Eso3D #MRX (lovestats.wordpress.com)
- Digital Matrimonies that Transform the Face of Research by Kristin Luck #Eso3D #MRX (lovestats.wordpress.com)
- The Earthquake and Tsunami in Japan #Eso3D #esomar (lovestats.wordpress.com)
- How to Engage Consumers in Multinational Communities by Austin and Lerman #Eso3D #MRX (lovestats.wordpress.com)
Totally agree with Kathryn, Annie and Sman – I’ve seen some pretty shoddy surveys and witnessed some really poor field execution over the years – done by actual professional firms, but I think it is best not to openly criticize them by name. I would prefer to say they’re a good company (except for the DIYers Sman referred to), although we can do a much better job.
Let’s keep up the good. Anti-poor quality no matter where it originates.
Unfortunately, I think many of us just get caught up in semantics. We don’t mean to bash in-house research done poorly any more than we bash agency-done research done poorly. “DIY’ implies, to many, research done on the client-side–when really it means anyone doing research who doesn’t have actual training. Our campaign should be against any bad research–and I see plenty of it that comes from “professionals”.
Agreed. It doesn’t matter to me who does research as long as they are trained to do it well. Professionals, who are very skilled at writing 60 minute surveys with 14 grid questions, need to open a psychology of surveys textbook.
I sincerely pity who are not getting your point now but will get it when research industry gets to highest point of commodization. Why you think market research is the first one under axe during budget cuts. During pressing times, shouldn’t company’s be spending more on research than less? They are not doing so because they are not getting value of research, especially people who DIY or researchers who just believe in providing data dumps regardless of realizing that research is a powerful tool to craft brand pathways. No wonder the recent Kantar health research on research yielded that 50% of clients are unsatisfied with current ATU reporting (see http://www.manmitonresearch.com/?p=179). Not only I am against DIY but also to those researchers who does not craft and execute each element of research (questionnaire, sampling, data analysis etc) scientifically. Just my two cents