Wait, was that a typo? Quantity over quality? Well, I meant what I said.
Question #1: What was the sample size of your last tracker? 30 per time frame? 50 per time frame? What about your last custom study? 300? 500?
Question #2: How many pages of questions and demos and cross-tabs did you flip through searching for any chi-square or t-test that was statistically significant? 100? 200?
Here’s the problem. We run ridiculously long surveys with far too few participants per test cell and we are ok with searching through far too many Type 2 errors.
Here’s the solution. Put your money into large sample sizes and not into question topic after question topic. Focus on sample sizes within demographic groups rather than questions with 4 or 8 people per cell. Trade variety of questions for reliability of results. Trade overly long surveys for properly sampled cells. Trade breadth of topics for validity of individual questions. Take money away from more and more questions and put it directly into more and more validity and reliability. Radical.
Please comment below. What was the sample size of your last study and what was the sample size within many of the cells?
- How to encourage speeding in your surveys
- Merry Christmas to all and to all a good sample size #MRX (lovestats.wordpress.com)
- Big Data? Big Deal. #MRX (lovestats.wordpress.com)
- New book! The Listen Lady: A novel and social media research guide baked into one #MRX (lovestats.wordpress.com)