Welcome to Really Simple Surveys (RSS), the younger sibling of Really Simple Statistics. There are lots of places online where you can ponder over the minute details of complicated survey designs but very few places that make survey design quickly understandable to everyone. I won’t explain exceptions to the rule or special cases here. Let’s just get comfortable with the fundamentals.
~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~ ~~
Sometimes, leading questions are easy to spot. You’d recognize one if you saw one, right? Like in the title of this post, right? Or like the following question, right?
What is your opinion about the current government?
a) They are doing a good job
b) They are doing pretty well
c) They are disgustingly horrid
In most cases though, leading questions slip into the mix with little notice. The questions and answers are written in such a way that people almost unconsciously focus on and choose one particular answer. It’s like leading a horse to water.
Leading questions cause responders to choose answers that don’t necessarily reflect their true feelings and for that reason they must be avoided at all costs. They come in many different formats and I’ve given examples of just of few of them below. See if you can spot the answer I’m leading you to and then figure out why you focused on that answer.
1. What is your opinion about the current government?
a) They are doing a good job
b) They need more resources
c) They are doing fairly well but they need to spend more time improving healthcare options
2. How much money has our government wasted?
a) 1 billion dollars
b) 2 billion dollars
c) 47.38 billion dollars
3. What does our government need to work harder on?
a) handling the country’s debt
b) passing the bill to save babies from being killed
c) dealing with various important healthcare issues
4. Complete the following sentence. I really wish my government would:
a) decrease my tax rate so I can afford better housing
b) debt issues that keep the country from growing
c) healthcare issues and other similar topics
5. Should the government work harder to improve the lives of the people?
And the answers are…
1) Answers that are much longer or shorter than others draw extra attention
2) Unusually precise answers draw extra attention
3) Unusually loaded emotive answers draw extra attention
4) Answers that follow grammatically correct rules are easier understand
5) Some questions have no logical answer other than to agree
Now that you know, you can keep your eyes peeled for other formats. No more leading questions!
- Radical Market Research Idea #5: Drop the decimals
- Radical Market Research Idea #4: Make your spouse take your survey
- Radical Market Research Idea #7: Participate in an untrusted methodology
- Radical Market Research Idea #6: Don’t calculate p-values