Laugh at yourself and then cry at our flailing industry

[tweetmeme source=”lovestats” only_single=false]

Well, once you manage to catch your breath after laughing solid for 4 minutes, let’s really think about all the people involved in this little prank.

1: Interviewer: First of all, this interviewer deserves a raise, a bonus, and a promotion for going through this interview without laughing, getting upset, or antagonizing the survey responder. I’m sure he deals with this sort of thing, whether real or fake, all day long every day. And yet, the utmost professionalism on his part. Kudos for a great job.

2: Responder: How did our industry get to such a state where surveys are written so poorly that people leave a tape recorder at their telephone waiting for researchers to call in order to make fun of them? This is nothing for us to be proud of.

3: Data Analyst: How exactly is the data analyst going to handle data which is clearly horrible quality? Will the analyst think of checking for outliers in each question? Will the analyst review the entire set of responses to recognize that it is an across the board outlier and probably a troublemaker? Will these responses lead to completely invalid analysis and conclusions?

4: Survey Author: Of course, we understand the need to use standardized questions in surveys. But, no matter how convinced you are, the world does not consist of people who know how surveys work. There are absolutely people out there who need to be taken through a survey with far more care than what we

permit when writing surveys. Telephone surveys need to be written so that interviewers can speak naturally and help those people who actually need some help. That’s where good data comes from. I’m really curious if the survey author left a place for the interviewer to indicate that this instance was possibly an outlier.

So, enjoy. But the next time you write a survey, keep this in mind. Are you antagonizing yet another survey responder or are you responsible for creating a more positive market research experience?

4 responses

  1. On the one hand, I’m in the MR industry and so I feel that I should participate if asked. On the other hand, many surveys are very poorly designed. The questions often indicate that even minimal pretesting isn’t done.

    (I used to pretest using my mother; hardly a random sample but at least someone who would tell me if I had a bad question. Are there no more mothers?)

    The one I found particularly irritating recently was a reader survey for Scientific American. This has been a great magazine for >150 years, but the survey was so poorly designed I used it as a bad example in my research class.

  2. WOW!!! I hope Todd gets married too and his wife can have his kids that suckle too…

  3. I don’t think one person trying to be funny (and failing!) while answering a market research call means the death of our industry. I’m sure you can find satirical pieces for all professions on youtube somewhere. I agree that the interviewer was excellent in the circumstances which reflects well on our industry but I don’t think this one example backs up your other points (I’m a quant researcher and require more proof!).

    1. I agree that n=1 never does the job, unless of course you’re a qualie doing an n of 1 study!

      Unfortunately, there is a ton of research out there showing just how disappointed survey responders are with the quality of surveys. It’s a tough up hill battle we fight.

%d bloggers like this: