Guest Post: Jam, introspection & decision making

By Richard Shaw, @richshaw

Last Thursday I stumbled across an article: “Thinking Too Much: Introspection Can Reduce the Quality of Preferences and Decisions“. The article described two experiments, the first experiment concerned itself with jam (or more specifically how introspection effects our jam related decision making). During the jam experiment, participants were given the tough task of tasting different brands of strawberry jam. The experimental condition then filled out a questionnaire about why they liked or disliked the jams whilst the control condition filled out a non jam related questionnaire. Both conditions then rated how much they liked/disliked each of the jams. Finally the ratings given by the participants in each condition were compared to the ratings given by a panel of jam experts.

It turns out that those in the experimental condition (who wrote down why they liked or disliked the jams) were more likely to disagree with jam experts than the control condition. The conclusion being that asking people to inspect their own thinking can impede the quality of subsequent decisions. In a market research context, this sort of behaviour could have led to the conclusions which supported the launch of the much maligned New Coke in 1985.

The jam experiment isn’t the only one to look at this effect, there are a number of studies that have investigated the effect of introspection on decision making (Tordesillas, 1999; Wilson, Kraft & Lisle,1990; Wilson, Kraft & Dunn, 1988). These studies throw up some interesting challenges for market researchers investigating how or why people make decisions. The main concern is that asking people to reflect on how they come to a particular decision effects their subsequent answers to our questions. This is not to say we should stop asking people questions which cause them to analyse why they have come to a particular decision, indeed the changes caused by introspection highlighted above will be more prominent when people attempt to analyse decisions which would normally be made intuitively (Hardman, 2009). However it does highlight that we need to be careful, because the way we conduct our research can often effect our results in subtle ways which we may not be aware of.

So the jam experiment has given me food for thought, I’ll think twice when I’m next tempted to ask a “and why do you….” type question in a survey or interview. There are also some rather broad unanswered questions which require some brain time (or more likely the brain of someone who knows a lot more about the psychology of decision making than me): what is the best way of investigating why intuitive decisions are made, if we subscribe to a two process model of intuitive vs analytic decision making how do we distinguish which process is used and most importantly, is there any jam in the fridge, this all writing about jam has led to a huge craving for jam sandwiches.

Vote on the 2010 Topic of the Year: Cell phones or Social media

I’ve seen the predictions for 2010 go many ways, but they always seem to come down to two options. So, given your own particular biases, and the amazing scientific value of this poll, where would you place your bet?

Fer cryin’ out loud, use the MASTERSLIDES

Excuse my screaming, but I would like to introduce people to a brand new concept: Masterslides.

Masterslides exist in powerpoint to serve several purposes
1) Every slide has your logo in the exact same spot
2) Every title is in the exact same color
3) Every text box is the exact same font

Masterslides mean that
1) You don’t have chart jiggle (try flipping through your slides really fast) 2) The audience isn’t doing a where’s waldo on every slide
3) Presentations are cohesive and easy to look at

Please, if you don’t know how to use them, learn right now. You will save yourself hours of frustration and people will think you are smart.

Today’s public service announcement is brought to you by the number 17 and letter Y.

That’s not my job… or is it? Someone please tell me!

Where does the line fall?

As a researcher, my skills are carefully slotted into the design, analysis, and interpretation of research. And by interpretation, I mean explaining what the numbers on a chart are, explaining if/why they are significantly different, explaining how they relate to each other.

But where does explaining the consequences of the numbers come into play? Who is the right person to draw conclusions about implications and action items? Is it the researcher? Is it the client? Is it a collaboration?

I’m just asking.

A Box Score Lesson for Psychology Students

I was in school for a bunch of years, and took a bunch of research design courses and a bunch of statistical analysis courses. Easy ones, hard ones, and a few really interesting ones. Surprisingly, one thing I never learned about was box scores, a statistical staple in the market research world.

Box scores are a way of talking about and working with Likert scales or other types of categorical scales so that everyone knows whether you are talking about the positive end of the scale (top box, top 2 box), the middle of the scale (middle/neutral box), or the negative end of the scale (bottom box, bottom 2 box).

Instead of calculating average scores from the Likert scale responses, box scores are reported as the percentage out of the total number of people who answered the question.(If 10 out of 50 people chose strongly agree, top box score is 20%) Box scores let you clearly identify how many people fall into a subgroup – people who are happy, unhappy, or just don’t care about your product.

Why do box scores matter? In a sense, they do report the same type of information as average scores. But, unless standard deviations are near and dear to you, average scores often appear very similar between groups. It’s hard to explain to a client why scores of 3.6 and 3.9 are very different because there is no intuitive difference between those numbers.

But, let’s think about box scores now. Can you intuitively understand the difference between 30% of people liking your brand and 40% of people liking your brand? I’m pretty sure you can. And you don’t need to understand what a standard deviation is either. I’m not in favour of dumbing down statistics but I am in favour of people understanding them.

Here’s another reason box scores are good. The average score calculated for a result that is 10% top box, 10% bottom box and 80% middle box is exactly the same average score you would get for a result that is 40% top box, 40% bottom box, and 20% middle box. I’d certainly like to know if 10% or 40% of people hated my product. That’s a pretty important difference to be aware of and I wouldn’t want it getting lost because someone had a weak understanding of what a SD is.

So now, psychology/sociology/geography majors, go forth and prosper as market researchers!