Training for survey research: who, where, how #AAPOR #MRX 


moderated by Frauke Kreuter

Prezzie #1: training needs in survey methods

  • started a program in the 1970s with 4 courses, two in statistics and 2 in sampling, that was pretty good at the time, it covered the basics well
  • in 1993, 3 courses data collection, 3 in sampling, 2 practicums, 4 statistics, 3 design classes, 1 on federal statistical system
  • many journals have started since then, survey methodology, journal of official statistics, POQ and aapor publications journal of survey statistics and methodology, international conferences, now entire conference on total survey error
  • statisticians need to know basic theory, sampling theory of complex designs and weighting and imputation, small area estimation, disclosure control, record linkage, paradata, responsive design, panel survey methods, survey management, ethics; it’s impossible to know about and training for everything now
  • in early days, treating humans nicely was never mentioned, it wasn’t important; now we realize it’s important [yet we still don’t treat people as nicely as we ought too. isn’t a long, poorly designed, poorly written survey disrespectful?]
  • a masters degree can no longer cover everything we need to know as survey researchers, can run summer programs for training, can do online training, can do advanced certificate programs
  • the world is training so fast so how can training keep up with everything, punch cards are history and whatever we’re doing now will be history soon enough
  • we need to train more people but undergrads don’t know about our field

Prezzie #2: training for modern survey statisticians

  • survey practice journal special issue – February 2015
  • might be 147 federal positions per year advertising for statisicians, we are training only about a quarter of what’s needed
  • we need core statistical skills but also communication and presentation skills
  • training gap right now is most grad courses only have one course in sampling
  • most courses use R (55%)
  • only 40% of courses are taught by faculty who work specifically in statistics
  • weighting is a major gap, don’t talk about non-response adjustments
  • big aining gap in design trade offs – discrete parameters, continuous parameters, split sample randomization
  • most training happens on the job
  • [this session is so popular I can’t put my feet up on the chair in front of me! The room is full!]

Prezzie #3:  assessing the present

  • our science draws on many other disciplines
  • trained in the methods and how to evaluate those methods, trained in qual and quant like ethnography and psychmetric analysis
  • there are five university based programs, mostly at the graduate level, plus professional conferences, short courses and seminars
  • current programs do the core well, increasing focus on hybrid training, trainers are also practitioners which is invaluable
  • training gap on full survey life cycle experience, not enough practical experience in the training, not enough multi-cultural training and the population has a large and growing non-native english speaking base
  • quant dominates most survey programs [well of course, a survey program is surveys, why not develop a RESEARCH program]
  • you can have a productive career with little statistical knowlege, you can be a qual researcher [well that’s just offensive! why shouldn’t qual researchers also know statistics?]
  • ideal program still needs the core classes but it also needs more qual and user experience, more specialized courses, more practicums, more internships, more major projects like a thesis

Prezzie #4:  on the job training

  • she did interviews with people for her talk – she’s qualitative🙂
  • the workplace is interdisciplinary with many skill sets and various roles
  • know your role – are you a jack of all trades or filling a niche
  • in private business, everyone knows a bit about everything
  • at the census bureau, it’s massively specialized – she works on pre-testing of non-english surveys
  • you need to create opportunities for yourself – request stretch tasks, seek mentors, volunteer to help with new projects, shadow experienced people – screen sharing is a wonderful thing
  • take short courses, pursue graduate degrees, read and present – you are responsible for your future growth
  • as management you can – promote learning by doing, share the big picture, encourage networking, establish a culture of ongoing learning
  • you can learn on the job without money

Prezzie #5: future of training

  • we are in the midst of a paradigmatic shift in our industry
  • survey informatics – computer science, math and stats, and cognitive and social spcyhology – this is the new reality
  • resistance to online surveys is the same as the emergence of the telephone survey – skeptical, resistant
  • the speaker was a heretic when he first started talking about online surveys
  • we need technology and computers for higher quality data, increase quantitative in data collection
  • we now have paradata and metadata and auxiliary data – page level data, question level data, personal level data, day level data
  • data is no longer just the answers to the questions, methodologists need to understand all these types of data
  • concerned we’re not keep up and preparing the next generation
  • [discussion of how panels can be good, like people have never heard of panels, sadly some people do need to hear this]
  • computer science must be our new partner [go home and learn R and Python right now]
  • we won’t have to ask are you watching TV, the TV will know who’s the room, who’s looking at the TV, who left to get a snack
  • least powerful low level professors who know the new tech have no power to do anything about it and they have no funding

2 responses

  1. when measuring the quality of an estimate we typically compare the estimated value to the actual value.

    let’s look at the last two Census directors and last two AAPOR presidents, how many of them meet this criteria?

    how about Wayne Smith in Canada, Dan Gaylin at NORC, do they have the training to be an effective leader?

    Joel Benenson has a degree in theater, is he qualified to lead his organization?

    i think when some folks talk about training for they future, they mean the future of academic experimental survey methodologist, they are certainly not talking about real world program managers and decision makers.

%d bloggers like this: