At the CASRO online research conference, one of the panels focused on DIY research and included a couple panel providers and a couple DIY companies. This panel was of particular interest to me because I’ve watched how DIY has taken quite a pounding in the last few years but for the wrong reasons.
I asked the panel about when qualified researchers should use Do It Yourself research and the answers included when you need results better, faster, and cheaper. Ok, a very generic and unhelpful response.
Then, another audience member asked when should DIY research NOT be used. Sadly, the panel could not offer a single idea about when DIY research was not appropriate. Given that one of my mantras is “Every research method has pros and cons,” this was a completely disatisfactory answer. And misleading.
So here is my opinion on when DIY should and should NOT be used.
DO use DIY research when:
- A qualified researcher has written the survey and designed the methodology
- The survey is very simple, short, and contains no complicated skip patterns
- You need results extremely quickly
- You have the population of target responders and ‘random’ sampling is not necessarily required
- You understand statistics well enough to know when sample sizes are too small, when to use a t-test or a chi-sqare, and why MOE is under hot debate
- You need to test a simple methodological issue prior to launching the full study (e.g., will the distribution of responses be better served with a 5 point or 3 point scale)
Do NOT use DIY research when:
- You don’t need anyone to proof read your survey because you never make mistakes
- An expert in survey design has not created the survey
- An expert in sampling/weighting has not developed and implemented the sampling plan
- An expert in data analysis will not be analyzing the results
- (so assuming that a qualified researcher is managing all aspects of the research…)
- You are running a complicated tracker – weekly/monthly, complex balancing, multi-country
- Your survey incorporates multiple and varied skip and logic patterns
- You require complicated census/target balancing and weighting
It’s a pretty easy answer.
- DIY Panel: Gardlen, Ribeiro, Smith, Terhandian, Thomas #CASRO #MRX (lovestats.wordpress.com)
- Bringing Colour into our Digital Lives: Piet Hein van Dam #CASRO #MRX (lovestats.wordpress.com)
- Cyborgs vs Monsters in modularizing surveys: Edward Paul Johnson and Lynn Siluk #CASRO #MRX (lovestats.wordpress.com)