Tag Archives: education

Voxpopme 5: How can you best educate, develop, and improve yourself in market research?

Along with a group of market resevoxpopme logoarchers from around the world, I was asked to participate in Voxpopme Perspectives – an initiative wherein insights industry experts share ideas about a variety of topics via video. You can read more about it here or watch the videos here. Viewers can then reach out over Twitter or upload their own video response. I’m more of a writer so you’ll catch me blogging rather than vlogging. 🙂

Episode 5: How can you best educate, develop, and improve yourself in market research?

Our industry is really lucky to be one that is focused on learning, educating, and sharing knowledge. We have an abundance of mentors who love to help people grow. No matter your budget, you have great options for learning more about market research.

But given that not everyone has the funds to attend conferences, buy books, or take classes, I’ll share three of my favourite free options.

Blogs: Literally hundreds of market researchers around the world keep personal blogs. They share their unique opinions based on their unique experiences about a plethora of topics from focus groups and surveys, to chatbots, artificial intelligence, and augmented reality. You don’t need to know anything about the topics and you don’t even need to agree with a single thing they say. All you need to do is have an open mind to consider the ideas they’re sharing and you’ll quickly become a more informed and better decision maker.

I keep a list of hundreds of blogs in my RSS reader but here are some of my absolute favourites:

  • Affectiva: If you’re ready to see what else is out there in market research land besides questionnaires, this is a great place to start. Facial coding, emotion, AI, biometrics, yeah baby!
  • FlowingData (Nathan Yau): Lovely collection of beautiful charts, maps, infographics, and visualizations. Good place to stay on top of new and beautiful data.
  • Pew Research Center: For the absolute best production of research about digital, social, there is no alternative. Every post could be considered a how-to guide on analyzing and presenting data. The topics are usually from the USA but the methodologies are universal.
  • Lexalytics: Really nice posts on applying social media research to real life problems.
  • Math with Bad Drawings (Ben Orlin): Lots of people hate math. Lots of people can’t draw. This blog uses bad drawing to help readers better understand math. It makes me smile!
  • Not awful and boring examples for teaching statistics and research methods (Jess Hartnett): If you recall back to your statistics classes in university, most of them were pretty darn boring and made it really easy to tune out (Yup, I think 9 out of 10 of mine!). This blog has so many great ideas on how to teach statistics in ways that people enjoy. If you need a refresher, start here!
  • System 1 Research Blog: Extremely well written thought pieces about marketing and research in all aspects of products, brands, life, and culture

Webinars: Ideas that start in personal blogs often make their way into companies where they turn into full-fledged research projects and then webinars. Sure, sometimes webinars feel like sales pitches but if you read between the lines, you can collect many tidbits of knowledge. Besides, learning about competitors’ products will help you in many ways – to improve your own products, point an existing client to a product that might work for them, or even partner with them on a project. In addition, Ray Poynter regularly runs webinars on a variety of topics with guests from around the world. In both cases, you can likely view them after the fact. Just register for the original webinar so you can the link afterwards.

Online conferences: I have two absolute favourite online conferences. You’ve probably heard of the NewMR festival that Ray Poynter runs. I’m also a big fan of VizFest which is run by Keen as Mustard marketing and E-Tabs. These two events run once per year over several days and they host speakers from around the world on a variety of topics. Some of the top speakers from other conferences speak here so if you have no budget to travel because you’re a solopreneur or a tiny company, you can still enjoy some of the best and brightest in the industry!

So there you go! Free and informed!


The next generation of market research and insights creation #IIeX 

Live note taking at the #IIeX conference in Atlanta. Any errors are my own.

Panel: The Next Generation of Market Research & Insights Creation
Moderated by Leonard Murphy (GreenBook) with panelists Chris Enger (Periscope by McKinsey), Tamara Char (Periscope by McKinsey), & Simon Chadwick (Cambiar)

  • Periscope by McKinsey is a suite of tools for collecting learnings, analytics
  • Our entire industry is fragmented, over half of companies that source data did not exist ten years ago and they may not exist ten years form now
  • Technology is not the driver of change, client needs and circumstances are the drivers of change, they are being asked to do far more with budgets lower than they used to be, they much get creative
  • Behavioural data and analytics techniques to analyze that data is suddenly easily available and analyzable, this changes everything about being able to identify insights and work in an agile way, can get to 80/20 answers more quickly, we don’t need the 100% answer, we need to make progress on problem solving
  • Are analytics pushing the business forward, are the ‘researchers’ falling behind and failing to get seat at the table?
  • Need to elevate the quality and consistency of data so that the leadership is never getting three answers to the same question nor are employees hearing diverging answers
  • You must have a c-suite leader and hopefully the chief financial officer who has a longer tenure in a company, not the chief marketing officer
  • The CMO needs to spend time developing strategies not waiting to get data, let the machines do the heavy lifting so the team can spend their time strategizing
  • What is the role of the methodologist, understanding fit for purpose of all the tools, this is why we’re seeing so much fragmentation, 
  • In the USA, people are attracted by tools. In the EU, they are more focused on ideas and creativity, and try to be creative all through the entire process. Need to be less technologically focused in the USA. 
  • Try assigning various people on th c-suite to BE a person in a segment, have them go shopping for her, experience her, all to get them to empathize more clearly, because c-suite lives are so completely different from their segments
  • Is automation a dirty word? Machine learning templates and speeds everything up, may eliminate bias of an individual person although it will perpetuate bias that exists within the data
  • We need to present data for ten minutes and then discuss the oilers and solutions for the remaining 50 minutes

Panel: The GRIT Report & Future Impacts
; Moderated by Leonard Murphy (GreenBook) with panelists Aaron Reid, Ph.D (Sentient Decision Science), Patricia Chapin-Bayley (Toluna), Rick Kelly (Fuel Cycle) & Isaac Rogers (20|20 Research)

  • Automation is mostly used for analysis of surveys data, charting and infographics, analysis of text data, analysis of social media, sampling
  • “My clients aren’t asking me for social media data” no they aren’t, they’re asking someone else
  • Automation frees up time to expand capacity and do more, many things will soon be automated. We must adapt to this or fall by the wayside.
  • Buyers are slow to adopt automation, automation is a dirty word because they think it is DIY and it will be more work. It will actually free up resources and allow you to do more once you are trained and moving forward.
  • Do you want to be at a data collection conference in five years or at an insights conferences? Your business must adopt automation.
  • People don’t CARE if you automate, they want better research insights and thinking. You must have automation to get there.
  • Automation may not cut your budget but it allows you to move your budget into higher value endeavours.
  • What should samplers do? Advise on representativity, enforce length of interview limits, consult on questionnaire design, restrict to mobile only, forbid mobile-unfriendly. it is an absutive relationship – clients don’t want to pay for consumer friendly and respectful questionnaires.
  • There is no such thing as a non-mobile study. Every device must work and work well. You cannot run a survey without mobile respondents or you are guaranteed a nonrepresentative sample. Why is this even a conversation?
  • If you aren’t thinking mobile first, you are being stupid. We spend half of our time on our devices.  It is a data quality issue. [Cannot agree with this comment enough]
  • Educating the researcher of the future – they need critical thinking and storytelling skills. We all need to be critical thinking experts, you shouldn’t in the business without that.  We need to train the current workforce on how to do this. We’ve trained people on how to run cross-tabs but they need training on storytelling and turning insights into action.
  • Quick research doesn’t have to be quick and dirty or poor quality
  • The technology doesn’t matter, the platform doesn’t matter, we need to stop talking about the technology and focus on consultation, understanding the problem 

Plus c’est la même chose, The Future of Market Research Education Reg Baker, MRII #NetGain2015 #MRX

Netgain 2015Live blogging from the Net Gain 2015 conference in Toronto, Canada. Any errors or bad jokes are my own.

Plus c’est la même chose, The Future of Market Research Education

Reg Baker, Executive Director of MRII

  • People don’t trust us with their data, we need to have this conversation with them
  • Most of us are in the business by accident
  • If we knew what we were doing, it wouldn’t be called research
  • forces shaping research – from data scarcity to abundance, from asking to listening, from analyzing to synthesizing
  • The argument – there is a set of principles that distinguish good from bad regardless of method – science
  • Clients have expectations of accuracy, how bad they are willing to accept, how good to use for predictions
  • MR education is way too focused on training to do a task and not enough on teaching principles to apply across technologies
  • Most training is “how to write a survey” and “how to run a focus group”
  • Training is not education. Training is acquiring knowledge for specific competencies. This is bringing new people into our organizations. People learn a tracking project and they populate the graphs.
  • Education is knowledge, skills, habits, beliefs, formative beliefs, ideals
  • How do people become educated in market research? There are 5 programs in the USA, a couple dozen people per year. Associations help with webinars that are sometimes sales pitches of one method, Continuing education people like Reg’s organization. Employers do the bulk of the training to make people productive – the smart ones will figure it out over time.
  • What should a program teach its students? What about social listening, predictive analytics, management consulting, big data, consumer research?
  • The firm of the future needs – specialists (data scientists, survey researchers, neuroscientists), business consultants, polymaths (he who knows much, generalists who understand how all the methods work together and assign the right method to the right objective)
  • Joan Lewis – we need to be methodology agnostic. The answer to every business problem is NOT a survey [WHAT!?!?!]
  • We need to teach people how to see noise.
  • Market research is easy. There are just a few easy steps. Understand the business problem. Know the full range of methods and data sources that might be used. Gather the right set of data. Understands the strengths and weaknesses of the data and resolve the inconsistencies. Create an actionable narrative.
  • We need to have an open mind about methods and learn when to use each one.
  • We need to focus on principles not ways of doing things. What makes “good” research? reliable, credible, can bet the farm on it.
  • We need to teach people the art of synthesis.

Why do kids (and you) need to learn statistics?

Statistics are boring. They’re hard. They’re useless. You’ll never use them in real life.

p value statistics memeOh, how wrong that is. I’ll agree that if you aren’t blessed with the genes that make math and statistics a piece of pie (mmmm, pie), then yes statistics are hard. But there are innumerable real-life examples to show just how important it is to be comfortable with statistics.

Sports: If you’re a fan of sports, you no doubt are bombarded with statistics throughout the season just like these interpretations of statistics shared by  on the Pensburgh. The headlines are exciting but the reality of each headline is simple – they mislead and even outright lie. If you understood statistics, you’d immediately see for yourself what the numbers really said.

HEADLINE! Pittsburgh’s penalty kill is going to be an Achilles’ Heel this season!
(The team has killed 18 straight chances over its last five games.)

BREAKING! The team is going to walk away with the Metropolitan Division again!<
(They’re in second place, and half the division is within a point of catching them.)

THIS JUST IN! The Penguins just can’t put away teams late in the game!
(They’ve outscored their opponents 5-0 in the third period of two straight games, both wins.)

via On the Penguins, Sample Size and Apocalypse Narratives (Tuesday Slew) – PensBurgh.

Medicine: How many commercials on TV and ads in magazines extol the virtues of amazing new drugs, perhaps even drugs that you are desperate to try to alleviate your own health issues? If you understood statistics, you would know right away when the ads were misleading. You’d spot when the sample sizes were too small to be reliable, when the effect size was too small to be meaningful, or when the lack of a test-retest design suggested insufficient testing.

Sometimes companies egregiously exaggerate how well their drugs work. In a brochure given to doctors and nurses last year, the Japanese drug company Eisai claimed that its Dacogen drug helped 38% of patients with a rare blood cell disorder in a clinical study. This figure was false, the FDA said in a November 2009 warning letter. In fact, the figure was taken from a tiny subgroup of patients who responded well to the drug. When all patients in the study were included, the real response rate was a much less impressive 20%, the FDA noted.

Read more on this and other misleading advertisements here.

Politics: Political polling is becoming more and more prominent in the news. If you had a better understanding of statistics, you would know when to trust the polls. You would know why percentages don’t always add to 100, why polls ‘weight’ data, or why the margin of error is ridiculously important (even if you don’t have a random sample).

Seven hundred randomly selected New York likely voters were interviewed by landline and cell telephone between October 1 and November 1, 2014. The margin of sampling error is +/- 3.6 percent. The data have been weighted to adjust for numbers of adults and telephone lines within households, sex, age, and region. Due to rounding, percentages may not sum to 100%. Responder numbers in each demographic may not equal the total respondent number due to respondents choosing not to answer some questions.

No matter how you look at it, statistics are among the most important classes you can take. It’s in your best interest to sign up for a class now.

Applying a Corporate Reputation Model to the Higher Education Sector by Sean Simpson #MRIA14 #MRX

Live blogging from the #MRIA national conference in Saskatoon, Saskatchewan. Any errors or bad jokes are my own.saskatoon

Applying a Corporate Reputation Model to the Higher Education Sector
Sean Simpson, Vice President, Ipsos Reid Public Affairs

  • when someone trusts your company, they are more likely to believe your communications
  • are people aware, familiar, favourable, trusting, advocating your company, you need to get to the top of that, when people trust you enough they become your advocate
  • trust and marketing efficiencies are related,
  • higher education may not be trying to selling anything but the model works for education in addition to corporate
  • ROI of reputation – apply to a school, attend a school, graduate from school, attract professors, influence government, create advocates for your school
  • do trust scores get more donations? no, because donations are linked to the school you attended not really any other school
  • more likely to donate to your own school if you have stronger trust with it
  • drivers of trust include quality of programs, care of wellbeing of students, community involvement, job readiness
  • two key drivers for schools overall – quality and wellbeing of students, but there are differences by size of school
  • for large schools – quality of the program is the number one driver
  • for small schools – care about wellbeing is the number one driver, quality of program is important but only via other routes
  • rankings are produced but they aren’t the full picture, ranks don’t match up to macleans because they split their list into groups; colleges rankings show differences as well
  • challenges – is trust the right proxy for reputation? it’s not an index because those are too intercorrelated that the index is too complicated
  • model tells schools how to improve their reputation
  • drivers are customizable

Other Posts

It’s RAPP time #MRX

Georgian Rapp

Welcome to me! It seems that I am the newest member of the Georgian College Research Analyst program advisory committee. I’m not completely sure yet what my role will entail but at least a portion of it will be to advise the college on the types of skills and knowledge their students should acquire as part of the program.

In my first meeting, I learned a number of interested things.

  • Only about 20% of applicants are accepted into the program. Wow! That’s tougher than most university programs and many graduate degree programs!
  • Their major research projects are often conducted as ‘freesearch.’ In other words, businesses and government offices take advantage of their students to conduct research for free. Given that the research projects I reviewed as part of the Education issue of Vue magazine (September 2013) were on par with a lot of paid-for work I’ve seen, whatever Georgian college is doing is top-notch and worth far more than free.
  • Employers hiring RAPP students often write in the student evaluations that the students were productive on the very first day. I saw the quotes. I was impressed!
  • While there are a number of awards for students conducting outstanding work during the program, there are currently no entrance scholarships for students who may be deserving of the program but simply cannot afford to apply to attend.

What did I take from this? The RAPP program finds great people and turns them into great researchers. It is to our advantage as market research employers to provide the students with internships as many of those interns will likely become our next awesome new hire. And think about whether your company can provide an entrance scholarship to a deserving student. There are a lot of organizations out there that can could easily make this their good deed of the day.

The Worst Teacher I Ever Had #PostADay

In grade 10, my math teacher showed up to class to snooze and babble. He took naps while we took tests, and he used chalk to write out problems on the board while we used chalk to whip at his head. Students enjoyed discreetly dropping inappropriate items into his pockets as he walked past them and they took full advantage of the test scoring keys that he left open on his desk. I didn’t learn much that year. In fact, the classroom was so distracting and intimidating that I managed to achieve my worst math grade ever. Worst teacher ever.

The next year, having realized just how far I had fallen behind in math, I took what, at the time, was an embarrassing step. I chose to enrol myself in the general math program. Not the advanced, headed off to university program. I prepared myself to join a class of losers being taught by a woefully inadequate football coach.

Find me in this photo!

Mr Verhoeven did not employ any discipline in his classroom, even though the students had a reputation as being unruly troublemakers. Truthfully, he didn’t need to. He was quiet, conscientious, and respectful of the students, and the students behaved the same way towards him. He paid attention to who was succeeding and gave special care to those who struggled. It was easy to learn in his classroom because he kept everyone relaxed and comfortable. I caught up quickly and ended up at the top of the class. And when I returned to the advanced math stream, I remained at the top of the class with a new-found respect for the teacher and the students who struggled in the class I left behind.

I fast forward once again to university, to the dreaded first year statistics course where the professor says, “two thirds of you will fail my course.” The first test came along and I managed to fail it along with nearly everyone else in the class. When I attempted to argue my unfair and undeserved grade with the professor, he told me that he knew I knew the answers but I wasn’t expressing myself clearly. For example, the median is not the middle item in a list, but rather the middle item in an ordered list. He was firm about not changing anyone’s grade. He worked hard to teach us that hard work and careful attention to detail would allow us to recover from failure.

I went into the final exam with a terrible grade but we were told if we achieved a better grade on that exam, it would become our final grade. I remember handing in my exam paper early. The professor took it, smiled at me, and asked if I had double checked it, was sure I was finished? I was sure. 🙂


From academia to a job: Free advice from a Phd

Werner Erhard and Associates v. Christopher Co...

Image via Wikipedia

[tweetmeme source=”lovestats” only_single=false]I’ve been asked on a number of occasions how I found a job in market research after completing a Phd in psychology. You’d think the opportunities are endless but, like any career path, there are always obstacles.

Interviewers have told me to my face that they refuse to hire or they don’t like Phds. They’ve even given me strange tests to determine whether I’m human or robot, like “What does this abstract painting on my wall mean to you?” (Honestly, that piece of crap is major ugly.)

So here is my advice. It’s free advice so the confidence intervals are wide. Please do ask questions and I promise I’ll answer all of them.

  1. The fact that you have a Phd means you know research and statistics. Don’t waste your cover letter proving this.
  2. The fact that you have a Phd means employers think you don’t know the real world and that you can’t speak casual english. Prove this wrong in your cover letter. Write in business dialect not in dissertation dialect. This is one case where fancy words do NOT impress.
  3. Forget the stats speak. When they ask you what a t-test is, don’t tell them it’s an analysis of mean scores and confidence intervals of a quantitative variable for a second qualitative variable. Speak english. Say it’s a way to determine if two groups of people differ on a measure like height or weight.
  4. Join a few online survey panels so you can get a lay of the land. What questions get asked? How are they asked? Do you want to shoot yourself during the survey because it’s so horrid? This will give you insight about the business you think you want to get into and…
  5. … something to talk about during interviews. In which you will speak like a human being not a professor.
  6. Go to a used bookstore and buy a Market Research 101 textbook. Learn it.
  7. If your field isn’t psychology, you would do well to take a course in social psychology or personality psychology. It will give you great insight into survey question design.
  8. Learn either SAS or SQL. Not the menu driven kind, the syntax programming type. Even if you don’t end up using it on the job, you will be better able to talk to the statisticians and get what you need in the time you need.
  9. Accept that in the business world, projects take 14 days not 14 months, with sample sizes of 200 not 2000, and conclusions that are final not proposals for 8 more years of research. Now is not the time to try to convince your gracious hosts otherwise. They aren’t stupid.

You are already qualified. You just need the right vocabulary and the right perspective on research for business. Any questions?

Read these too

  • Kama Sutra Statistics
  • What’s Your IQ?
  • It’s your turn to discover the nuances of social media research
  • The Joy of Stats with Hans Rosling #MRX
  • My Theory on Declining Response Rate
  • Be All That You Can Be: Be A Research Mentor

    Baby with book

    Image via Wikipedia

    In Canada, there are very few schools that specialize in teaching the skill of market research at the undergraduate level. In fact, I only know one.

    Now, there are many, many programs that inclde a couple courses in statistics, or research design, or marketing. These at least provide some fundamental knowledge so that when you hear a term later on, you at least recognize that it is a term. If you can’t find an MR program, the next best thing is to do a degree in psychology, sociology, geography, or marketing. I may be biased but I think the best option is a major in psychology with a minor in marketing. You can see though, that even if you create an optimal program, none of these focus on the art and science of MR as its own academic area.

    Even those folks who go on to earn graduate degrees fall into the same bucket. Psychology graduate students do their research on psychology topics and probably never take a marketing course. Their research skills are top notch but an internalized perspective on marketing is lacking. And, marketers do their research in marketing and don’t have the background in social psychology to better understand why people buy the way they buy.

    What it means is that most new market researchers come to the table with serious gaps in knowledge. They must resort to learning on the job. If they’re lucky, the person who trains them is a wonderful mentor with many years of experience. But those folks are few and not always readily available to the junior folk. What is more likely the case is that someone barely senior to them tells them just enough to get the job done because they are still trying to learn the skills themselves. In my case, the only mentor I had was an intro marketing textbook that I picked up at a used book store.

    We are fortunate that our MR societies have ongoing training courses and certification. Unfortunately, these cost money and new graduates just don’t have that kind of cash. Nor do their employers have money to invest in a newbie. Which means a lot of people in the MR industry are not as skilled as they should be.

    Maybe this is partly why our industry is struggling through data quality issues. Not enough people understand the psychology behind survey answering. Not enough people understand the myriad precise techniques of writing survey questions. This lack of MR skills leads to bad surveys which leads to bad survey experiences and results in declining response rates.

    So, here’s my idea. It’s not new. If you work with newbies, be that missing link. Be a mentor. Teach them everything you can. Send them to conferences. Make the time to set up lunch and learns not because you have to, but because its the right thing to do. Invest in your company by financing their CMRP certification. This will lead to a better research product, happier employees, and a stronger company.

    I thank you, and your newbies thank you too!

    Related Posts


  • #MRA_FOC #MRX MRA Articles of Incorporation, no longer 53 years old
  • Art, History, and Culture in Chicago Two Balloons and a Great Song
  • Survey Panel Questions – Enough Already! Word Cloud of my Resume – Wicked Cool!
  • Survey Design Tip #4: Brands are people too
  • Oh oh, I Have Another Favourite Statistic!
  • CAUTION: The Scariest Thing EVER

    Figure 20 from Charles Darwin's The Expression...

    Image via Wikipedia

    There are many ways to terrify people. Put a spider on their shoulder, make them stand close to the edge of a cliff, tell them you’re going to visit the in-laws (ftr, mine are great). Different people are scared of different things. There does, however, seem to be one fear that transcends other fears – the fear of statistics and numbers.

    How did this come to be? Were our math teachers horrible people? I doubt it (though one of mine was and that’s a whole seperate post). Were we threatened with having to do extra math if we didn’t finish our brussel sprouts? Doubt that too.

    Here’s my theory. Remember english class where you wrote a beautiful essay and the teacher gave you an A? That A didn’t mean perfect, it meant great job. However, you never got an A in math. You got an 80%. In other words, you got 80% right, and 20% horribly, horribly wrong. You failed at 20%. You sucked for 20%. Even though you did a great job, you still managed to screw up a lot of answers.

    Math insists on having a right answer. It’s right or its wrong. It’s not a teachers perception of your thoughts and ideas and its not even a measure of how much they hate you. For, even if your math teacher hates you, if your answer matches what’s in the teachers edition, you got the mark and the grade.

    It seems to be that, even though nobody is perfect, we are scared of situations where there is no doubt we are wrong. We seem to forget that everyone is wrong at one point or another, and we all have strengths and weaknesses.
    My advice to you is don’t be fearful. Expect to make mistakes. Expect to forget formulas. No one is perfect and no one gets every math problem right.

    Statistics can actually be interesting if you really to listen to them. TV commercials and other marketing materials use lots of bad statistics and they are a source of great amusement, at least for me. And, you will find that people who are okay around numbers are in high demand in the job market. That’s good enough for me!

    Related Posts


  • My Theory on Declining Response Rate
  • Focus groups: The best and only research method
  • Three TV commercials that tickle my fancy and make me go Yeah baby!
  • Check Out the Statistical Outlier on my LinkedIn Cluster Analysis
  • Why do surveys ask the same question 8 billion times?
  • %d bloggers like this: