Does Sample Size still matter? By David Bakken #CASRO #MRX

… Live blogging from beautiful San Francisco…


“Does Sample Size Still Matter?”

David Bakken, Chief Operating Officer, KJT Group, Inc. and Megan Bond

  • Healthcare research is often forced into very small sample sizes
  • How do you determine how many interviews should be conducted for a health product?
    • Level of precision
    • Best guess of current market share
    • Budget
  • Sampling error is the primary source of uncertainty, as you collect more data, error approaches zero; uncertainty is expressed as long-term frequencies
  • [whoop! missed an entire section here because I was paying close attention 🙂 ]
  • People routinely use samples that are too small even though they know they shouldn’t
  • 30 used to be the rule of thumb for subsample sizes
  • We normally draw probability samples from non-probability frames
  • Classic paradigm
    • start with a population with known parameter values
    • draw samples
    • estimate sample parameters
    • compare samples to population
    • (you can even do this with fake monte carlo populations)
  • With a population of 800, they tested samples of 25 to 450.
    • 25 had unusually higher variance but seemed to level off around samples of 250 [but 250 is a huge sample of 800]
    • Smaller samples have more extreme errors when it comes to measures like max and min values
  • Sample size does still matter
  • Things get a lot better when samples are greater than 100
  • Bayes rules help reduce uncertainty in small samples it requires a change in our thinking
%d bloggers like this: