Interpreting Polls: Fiona Isaacson #MRIA #MRX

mria 2012Welcome to this series of live blogs from the MRIA Sample the Edge conference in St. John’s Newfoundland. All posts appear within minutes after the speaker has finished. Any errors, omissions, or silly side comments are my own.


  • The Alberta election is a great example of this
  • April 23, 2012 was Alberta’s general election and it was predicted that the Wildrose party would dethrone the PC party. assumed it would simply be whether it was a majority or minority govt.
  • The real excitement was the tweets about the “failure” of the prediction
  • Reporters had to file their story early, reporters figured they were fine with their story because the prediction was so clear
  • In the example, a reporters story was scrubbed from the internet but still appeared in print
  • Pollsters just criticized the outcome, they had planned for Wildrose to win easily so obviously something was wrong
  • …. Polls are not statements of fact. They are polls…
  • People had no problem using the polls to write their stories but afterwards the polls were trashed
  • Media will ALWAYS report based on polls. They are easy stories. Sometimes, the horse race is the only thing to report on when there are no real issues.
  • There is always a battle to be first in social media and this encourages poor reporting
  • Lots of places have quick easy unscientific polls on their websites which lowers the value of accurate polling
  • Many journalists boast they are bad at math. [great. that’s something to really be proud of 😦 ]
  • DON’T create sexy headlines that over inflate the results [wow, isn’t that every headline?]
  • DO put everything vital in the release, not in the attachment
  • DON’T bury the results of undecided voters
  • DO include method, MOE if applicable, sample size, time period, list of exact questions, who paid for it, and put this at the top of the release
  • MOE – most journalists have no idea what this is, they figure it’s a “cut and paste” line that just needs to be there for some reason or another. They may cut it if they run out of space.
  • DON’T bury the margin of error but demonstrate what it would mean by showing the range
  • 29.7 vs 28.2 with a MOE of 2.8 means the numbers are EQUAL
  • It’s easy to “plug the numbers in” without paying attention to what they really mean
  • Showing the poll questions without the results helps them see whether it’s even a good poll to begin with
  • Sometimes the story is not the poll results, but how biased or leading the poll question is, especially when written by one political party. e.g., Given how ridiculously horrible that plan worked, do you think this plan will work?
  • DO have more local polls
  • DO run more polls on issues so media has something to grab to other than who is in the lead
  • DO run fewer polls to stop over-reporting when really there is no change in the numbers
  • DO leave the twitter bickering to the media and the public. Pollsters should not be arguing there unless they are simply discussing methodology.
  • DO send background information during off-season so it’s in their election folder ahead of time
  • DO erase the myth that cell phone users aren’t being polled
  • DO offer to teach a 2-day course at journalism schools — pro bono [someone take this up!]
  • What happened in Alberta? Voters make the final decision and they did it at the last minute.
  • MRIA must take the lead in educating about polling


One response

  1. MRIA Ottawa is going Beyond the Headlines on June 14 with an An Open Forum with Leading Researchers and Journalists.

    Researchers have been busy discussing the merits of different methodologies while journalists are working to make sense of an unfolding and shifting world.

    Journalists work in the pursuit of fact and truth so that Canadians have the information they need to make choices for themselves and understand the world in which we live. Researchers like to think that “polls strengthen Canada’s democracy by giving voice and influence to Canadians on products, services and issues that directly affect their lives.”

    Hear from some of Canada’s leading voices through presentations and an interactive panel discussion. This forum will also give participants ample opportunity to ask their own questions and discuss implications.

%d bloggers like this: