Laugh at yourself and then cry at our flailing industry

Well, once you manage to catch your breath after laughing solid for 4 minutes, let’s really think about all the people involved in this little prank.

1: Interviewer: First of all, this interviewer deserves a raise, a bonus, and a promotion for going through this interview without laughing, getting upset, or antagonizing the survey responder. I’m sure he deals with this sort of thing, whether real or fake, all day long every day. And yet, the utmost professionalism on his part. Kudos for a great job.

2: Responder: How did our industry get to such a state where surveys are written so poorly that people leave a tape recorder at their telephone waiting for researchers to call in order to make fun of them? This is nothing for us to be proud of.

3: Data Analyst: How exactly is the data analyst going to handle data which is clearly horrible quality? Will the analyst think of checking for outliers in each question? Will the analyst review the entire set of responses to recognize that it is an across the board outlier and probably a troublemaker? Will these responses lead to completely invalid analysis and conclusions?

4: Survey Author: Of course, we understand the need to use standardized questions in surveys. But, no matter how convinced you are, the world does not consist of people who know how surveys work. There are absolutely people out there who need to be taken through a survey with far more care than what we

permit when writing surveys. Telephone surveys need to be written so that interviewers can speak naturally and help those people who actually need some help. That’s where good data comes from. I’m really curious if the survey author left a place for the interviewer to indicate that this instance was possibly an outlier.

So, enjoy. But the next time you write a survey, keep this in mind. Are you antagonizing yet another survey responder or are you responsible for creating a more positive market research experience?

Related Articles

• DADT survey results won’t be accurate (bilerico.com)
• Exploring Mode Effects in Establishment Surveys (kauffman.org)

From academia to a job: Free advice from a Phd

Image via Wikipedia

I’ve been asked on a number of occasions how I found a job in market research after completing a Phd in psychology. You’d think the opportunities are endless but, like any career path, there are always obstacles.

Interviewers have told me to my face that they refuse to hire or they don’t like Phds. They’ve even given me strange tests to determine whether I’m human or robot, like “What does this abstract painting on my wall mean to you?” (Honestly, that piece of crap is major ugly.)

So here is my advice. It’s free advice so the confidence intervals are wide. Please do ask questions and I promise I’ll answer all of them.

1. The fact that you have a Phd means you know research and statistics. Don’t waste your cover letter proving this.
2. The fact that you have a Phd means employers think you don’t know the real world and that you can’t speak casual english. Prove this wrong in your cover letter. Write in business dialect not in dissertation dialect. This is one case where fancy words do NOT impress.
3. Forget the stats speak. When they ask you what a t-test is, don’t tell them it’s an analysis of mean scores and confidence intervals of a quantitative variable for a second qualitative variable. Speak english. Say it’s a way to determine if two groups of people differ on a measure like height or weight.
4. Join a few online survey panels so you can get a lay of the land. What questions get asked? How are they asked? Do you want to shoot yourself during the survey because it’s so horrid? This will give you insight about the business you think you want to get into and…
5. … something to talk about during interviews. In which you will speak like a human being not a professor.
6. Go to a used bookstore and buy a Market Research 101 textbook. Learn it.
7. If your field isn’t psychology, you would do well to take a course in social psychology or personality psychology. It will give you great insight into survey question design.
8. Learn either SAS or SQL. Not the menu driven kind, the syntax programming type. Even if you don’t end up using it on the job, you will be better able to talk to the statisticians and get what you need in the time you need.
9. Accept that in the business world, projects take 14 days not 14 months, with sample sizes of 200 not 2000, and conclusions that are final not proposals for 8 more years of research. Now is not the time to try to convince your gracious hosts otherwise. They aren’t stupid.

You are already qualified. You just need the right vocabulary and the right perspective on research for business. Any questions?

Read these too

Kama Sutra Statistics

What’s Your IQ?

It’s your turn to discover the nuances of social media research

The Joy of Stats with Hans Rosling #MRX

My Theory on Declining Response Rate

What Ethnographers Do: Interview Guide (via Patterns of Play)

This is the first time I’ve reblogged via wordpress so who knows what will happen!

Anyways, I thought this was an interesting way to remind people what social media research is. It’s not monitoring or reading or counting or answering questions. It’s formulating a plan around structured questions in order to come up with answers and possibly more questions. It’s “good old fashioned” ethnography brought into the world of social media.

Read and appreciate!

Part of the documentation we assemble before every project is an interview and observation guide. It’s supposed to serve as a checklist for supplies, as well as a guideline for how to conduct the interview. It includes the information we are trying to get at, as well as the questions we’re using to do so. Here is our interview guide for our current project. If you would like a copy of our interview guide, you can download it here as a Word docume … Read More

via Patterns of Play

Read these too

1 topic, 5 blogs: Rich Media in Surveys

3 Reasons Why Researchers Hate Focus Groups #MRX

1 Topic 5 Blogs: DIY surveys suck or save the day

Paul the Octopus, Phd in Statistics, Lettered in football

Pie Charts – Our Evil Friend

If you DARE – the chart test of death

Image via Wikipedia

After asking someone what a t-test is and whether they watch American Idol, I have another question I like to ask interviewees to see if they know their stuff. I ask them to create a chart that they would present to a client. Ahhh, the variety of answers this reveals.

1) Some folks tell me that this version of Excel doesn’t do charts but they can do it on their version. (Honestly. This one astounds me.)
2) Others create a chart. That’s it. Nothing else. When I re-prompt that it’s to give to a client, I still get a default chart in default colors with a default no title and default lack of zero on the axis.
3) A few create a chart and try to pick the right chart, try to put a title, and try to fix the axis.

Right away, I can see that Person #1 has weak problem solving skills and great excuse skills. There are multiple ways to chart in excel and if you can’t get one of them to work, I can’t help you. Thanks for coming out.

Person #2 pays no attention to detail and barely wants to get the job done. Thanks for coming out.

Person #3 is my person. Whether they choose the right options, I know they are curious and want to do the right thing. If they don’t know what the right thing is, I can teach them and they can learn.

If I ever get to ask you this question, please make sure your chart has a non-misleading scale, a good title, non-florescent colours, and isn’t 3D. My sincere thanks.

Read these too

Art, History, and Culture in Chicago Two Balloons and a Great Song

Survey Panel Questions – Enough Already! Word Cloud of my Resume – Wicked Cool!

Survey Design Tip #4: Brands are people too

Oh oh, I Have Another Favourite Statistic!

TV Shows From When I Was Short

Are you a scientist, an artist, or a magician?

Is research an art or a science? Personally, I think it’s both. Actually, I think it’s a magical combination of the two. So, when I’m hiring a new researcher, I have a few standard tests that help me determine whether I’ve got an artist, a scientist, or a magician in front of me. Let me share one of my tests with you.

I like to ask people to consider that I know nothing about research and statistics and then describe to me what a t-test is. I tend to get one of three answers.

1) A 500 word technically stunning and accurate monologue incorporating as many statistical terms that can be thought of in the time allotted.
2) A series of ums and awws followed by an admission that they didn’t so well in statistics but they think it has something to do with numbers.
3) A round about discussion about how the average guy is taller than the average girl.

The first answer is the scientist. Even after I prod and give examples, they still can’t find their voice. I know this person knows the answer, but I wouldn’t dare ask them to write a report, answer an email, or talk to a client. Imagine the motion sickness that client would experience!

The second answer is the artist. Even though the jobs ad clearly said “knows statistics,” this person has creatively managed to work around such an unimportant detail. They attempt to stun me with a selection of phrasing in hopes that some combination of words will mean something.

The third answer is the magician. They have been gifted with that rare skill of actually doing what they’ve been asked to do. They have proven to me that they know the answer AND I know they will be able to talk to clients. They are a magical combination of scientist and artist. And then, only then, I ask my real interview question, “Are you addicted to American Idol?”

Would you pass my test?

Read these too

 

So you think you have a probability sample

Cronbach’s Alpha, My Favourite Statistic

Permissible pie charts and 1 bar chart

Esomar Chicago: Day 1

Tagxedo for colorful shapeable saveable word clouds