You know what a leading question is, right? #MRX

survey designWelcome to Really Simple Surveys (RSS), the younger sibling of Really Simple Statistics. There are lots of places online where you can ponder over the minute details of complicated survey designs but very few places that make survey design quickly understandable to everyone. I won’t explain exceptions to the rule or special cases here. Let’s just get comfortable with the fundamentals.

~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~

Sometimes, leading questions are easy to spot. You’d recognize one if you saw one, right? Like in the title of this post, right? Or like the following question, right?

What is your opinion about the current government?
a) They are doing a good job
b) They are doing pretty well
c) They are disgustingly horrid

In most cases though, leading questions slip into theThe little pony was teasing his big friend ...... mix with little notice.  The questions and answers are written in such a way that people almost unconsciously focus on and choose one particular answer. It’s like leading a horse to water.

Leading questions cause responders to choose answers that don’t necessarily reflect their true feelings and for that reason they must be avoided at all costs. They come in many different formats and I’ve given examples of just of few of them below. See if you can spot the answer I’m leading you to and then figure out why you focused on that answer.

1. What is your opinion about the current government?
a) They are doing a good job
b) They need more resources
c) They are doing fairly well but they need to spend more time improving healthcare options

2. How much money has our government wasted?
a) 1 billion dollars
b) 2 billion dollars
c) 47.38 billion dollars

3. What does our government need to work harder on?
a) handling the country’s debt
b) passing the bill to save babies from being killed
c) dealing with various important healthcare issues

4. Complete the following sentence. I really wish my government would:
a) decrease my tax rate so I can afford better housing
b) debt issues that keep the country from growing
c) healthcare issues and other similar topics

5. Should the government work harder to improve the lives of the people?
a) yes
b) no

And the answers are…

1) Answers that are much longer or shorter than others draw extra attention
2) Unusually precise answers draw extra attention
3) Unusually loaded emotive answers draw extra attention
4) Answers that follow grammatically correct rules are easier understand
5) Some questions have no logical answer other than to agree

Now that you know, you can keep your eyes peeled for other formats. No more leading questions!

2 responses

  1. Great info on leading questions…You can couple this issue with queuing, or the process of presenting one question that gets a respondent thinking about a topic, then asking the question you are really interested in. I have seen a number of instruments where respondents are lead into a particular issue through queuing questions. The effect can be the same as the leading questions…so as you construct an instrument, be aware that the topical flow of questions can influence responses as well.

    1. I suspect that is the most common form of leading questions. It’s hard to see those but they are far more dangerous because just showing the question doesn’t reveal the bias that led to it.

%d bloggers like this: