If a survey asks a bad question, does anybody give a valid answer? #MRX

Alaska forest - trees

Would you buy a brand because a friend likes it?

Would you buy a brand because you saw a commercial for it on TV?

Would you buy a brand because you found out they endorsed an important event?

Just about every survey about a product or service (as opposed to a person or idea) includes a question about purchase. In some ways, purchase questions are pretty easy to answer. The soap is in your cupboard because you purchased it. The cheese is in your fridge because you purchased it. Sometimes, we even know why we have purchased something (though in many many cases we have absolutely no clue).

But when you switch the question style away from past purchase to future purchase, you introduce massive error. Why? Because you’ve just asked your survey responder to guess about something they haven’t yet done and might never have thought of doing until you brought it up. You’ve just introduced an irreparable confound.

So please. Stop asking people to hypothesize about what they will do and ask them about something where they have a reasonable chance of knowing at least some of the answer. Ask if they like the product, ask if they like their friend who recommended it, ask if they like the event, ask if they like the concept.

Just don’t ask them to attribute a reason to it.


4 responses

  1. This is interesting, Annie! Are you saying that people can’t predict their future behavior?

    1. Precisely. Unless you always buy the exact same product every single monday without ever thinking about it, people don’t really know why or what they will do. The unconscious mind is far more powerful than we give it credit for.

  2. Annie,
    If I called you a bad name, do you think you would be angry? If you were drowning and I threw you a life jacket would you take it? If you went shopping for groceries would you buy bacon? Do you think you would buy the lowest priced bacon or would you buy the new super bacon flavoured bacon?

    1. I would buy the most bacon I could possibly buy given the money I have. In this case, it’s quantity over quality. 🙂

%d bloggers like this: