Along with a group of market researchers from around the world, I was asked to participate in Voxpopme Perspectives – an initiative wherein insights industry experts share ideas about a variety of topics via video. You can read more about it here or watch the videos here. Viewers can then reach out over Twitter or upload their own video response. I’m more of a writer so you’ll catch me blogging rather than vlogging. 🙂
Episode 6: How does market research maintain trust and authority in modern times where fake news and misinformation are perceived to be rife?
There are a few things we can do.
First, despite how expertise is being discredited more and more these days, let’s be more open and transparent about our credentials. More than simply degrees and experience, let’s talk about our membership in recognized industry associations such as Insights Association, MRIA, MRS, AMSRS, and Esomar, as well as ISO certifications. Let’s do more than simply mention we’re members, and instead start our conversations with that fact. Let’s describe what it means to be a member in good standing in terms of the code of standards and ethics we abide by. Let’s put those logos on the first page of our reports, and even include with them some of the ethics and standards statements that are most relevant to the specific project. Let’s use these as reminders for our clients that we always act in their best interest, and in the best interest of the research project, even if the results don’t work out the way we had hoped.
Second, let’s be more transparent with clients. Let’s tell clients about all of the strengths and weaknesses of our research processes, about the things that changed unexpectedly along the way, even if it means disappointing them. When we can’t achieve the response rate, sample size, or cost per complete that they require, let’s tell them right from the beginning and be clear about why it can’t be done. When the results we generate are completely unexpected and don’t line up with our hypotheses or norms, let’s be open and honest about what might have happened and whether there might be a problem. Let’s worry less about not winning a job, and more about demonstrating our commitment to the integrity of results. The secondary bonus of this transparency is that we can educate less experienced buyers on how research can be positively and negatively impacted by a variety of known and unknown variables so that they will be more informed buyers in the future.
Third, let’s be better public advocates. When we see our research in the media, let’s ensure the results, conclusions, and recommendations are clearly properly represented. And when they aren’t, let’s get in touch with the media to help them understand what the issue is, including telling them why margin of error or making a certain generalization isn’t appropriate. And if they refuse to correct the misinterpretation, let’s make a public statement to right the wrong, perhaps with a note on your website sharing details about how the information should be properly interpreted. And along the way, if we learn that certain media channels regularly misinterpret results, let’s reconsider working with those channels and even the clients that work with those channels. Every one of us has a part to play in helping to ensure our research results are properly portrayed.
Along with a group of market researchers from around the world, I was asked to participate in Voxpopme Perspectives – an initiative wherein insights industry experts share ideas about a variety of topics via video. You can read more about it here or watch the videos here. Viewers can then reach out over Twitter or upload their own video response.
Except the video blogging thing wasn’t working for me. I do my best thinking in writing and I’m pretty sure you don’t want to watch me read a post. So instead, I’ll be sharing my thoughts in written posts. Feel free to write back if you’re so included. Stay tuned!
The entirety of this post is available on the Gender Avenger website.
Why are women underrepresented as speakers?
I’ve pondered this question for years but I never knew if my hypothesis was grounded in fact or in stereotype. Fortunately, or unfortunately as the case may be, the opportunity presented itself and here we are pondering real data from a survey I did of 297 male and 252 female computer or data scientists, and market researchers aged 25 to 49 — people who ought to be on their way to securing spots on the conference circuit.
One of the questions in the survey asked people to imagine speaking at an event and to choose any attributes that would describe themselves as a conference speaker. I was careful to include an equal number of both positive and negative attributes so as to avoid leading people to choose a greater percentage of positive (or negative) items.
Curious how men and women viewed thselves? I know you are. Read the entirety of this post on the Gender Avenger website. If you’re braver enough.
Market research isn’t straight forward. As much as you believe one thing, I’ll completely disagree and believe another. Which brings us to this series of posts. Over the coming weeks, a colleague and I are going to tackle an important research question. I’m going to make one argument and they’re going to argue the exact opposite. Do we each have the same opinion, do we each believe in our own argument, are we arguing against our real opinion? You figure it out. Here goes!
Keza Kyanzaire: Telephone interviewing should die
CATI or Computer Assisted Telephone Interviewing Software has been helping market research companies interrupt people during dinner since the dawn of time (the proliferation of the internet). I admit, there are advantages to this form of interviewing, especially considering that when it was originally conceived, the alternatives included going door to door with a clipboard. However, it’s past time to put telephone interviewing to pasture.
According to the CRTC’s 2016 Communications Monitoring Report, more Canadian households exclusively use mobile phones over landline phones, and the shift away from landline phone usage continues to increase. Telephone interviewers are increasingly beginning to reach respondents on mobile phones. Respondents on mobile are often younger, and age can have an adverse effect on the willingness to participate in research and the quality of the data. As well, a respondent reached on a mobile phone may not be in an environment conducive to completing an interview.
Let’s not even mention the fact that telephone interviewers are often mistaken for telemarketers, which comes with its own set of issues. Market researchers have always had to fight to distance themselves from telemarketers in order to establish trust with participants, and legitimacy in the work we do.
Web-based surveys are an effective alternative to telephone interviews. Not only are they a cost saving alternative, (staffing call centres can be extremely costly and they tend to have high turnover rates) they allow the respondent to decide where and when to participate. Particularly in the case of MROC’s or panels, respondents can choose to participate in surveys that are of interest to them and are more engaged and active. Moreover, web-based surveys allow for visual stimuli and more interactive elements that not only make for a better experience for the respondents, but also allows for a depth in the types of questions that can be asked, and data that can be gathered.
Telephone interviewing has been a great resource for the industry, but with the advantages of web-based methods, why are we still running call centres in 2017. It’s past it’s prime and it is time that they are replaced by more effective and advanced methods.
Keza has an Honors Bachelor of Psychology with a Specialization in Cognition. She also completed Humber College’s Research Analyst Postgraduate Program, where she learned how to conduct both social and market research. Keza put her education into practice as a Research Analyst Intern at Numeris, where she conducted statistical analysis and helped inform business decisions. She is now a Researcher at PATTISON Outdoor. And she’s an experienced speaker having taken the stage at #IIeX in Amsterdam.
Annie Pettit: Telephone interviewing is the best thing since sliced bread
I’m not a gambling person but I’ll bet you know about research panels. Well, you’re in the minority. Most people who are not in the market and social research industry don’t know that panels exist. Which begs the question -who are these people who know about research panels? Are they just people desperately seeking places to earn money? What causes some people to seek out incentive based activities? One thing I know for a fact is these people have internet. They can afford monthly internet charges, maybe even high speed internet charges, as well as the cost of a device to access the internet. Internet might be a right but it still is a luxury for many people. Let’s check our academic privilege at the door. So how do we gather opinions from people who aren’t aware of research panels or couldn’t participate in them even if they wanted to?
One alternative is mall or central location research. Send your best face to face interviewers to the nearest mall and have them randomly interrupt passersby. Well, first of all, not everyone lives near a location that is large enough to warrant sending an interviewer. Sorry residents of Nunavut, you’ll never be asked to participate in mall research – your town isn’t large enough for your opinion to matter. You live in Vancouver, Calgary, Saskatoon, Montreal, or Halifax? Your chances are pretty good of meeting up with a mall interviewer – unless you’re one of those people who takes a ten minute detour through the parking lot to avoid the person with a clipboard. Besides, anyone who’s taken an introduction to research class knows what accidental sampling or convenience sampling is. Goodbye ability to generalize to a larger population if mall research is your game.
Which is why I’m a huge fan of telephone research. As it has been since the invention of polls, telephone research continues to lead the fight against self-selection bias. If you’ve never heard of research panels, if you can’t afford high speed internet, if you don’t live in a major city, if you avoid people at the mall, telephone methods still value and seek out your opinions. We are currently living through a time where people don’t trust the polls. Let’s not revert to methods that make it even easier to distrust polls.
If you’re an early career researcher, data scientist, data visualizer, marketer, or similar, and would like to write a #PointCounterpoint article with me, please send a gmail to my full name anniepettit and let me know what topic interests you. Maybe I’ll pick yours!
This list shows the gender ratio of speakers at marketing research and related conferences during 2017.
These data are not 100% accurate. I am not always able to identify whether a speaker is male or female based on their name. Online programs aren’t always up to date, and printed programs often change at the last minute and don’t reflect who was actually on stage. If you are able to correct my numbers, I would be grateful for the help.
Please contribute: Some conferences remove their information immediately afterwards. If you have a PDF or image of a conference program, email it to me so I can include it in this list. If you have a paper program, mail it me or do the counts and simply send me the final numbers.
- ESOMAR Global Qual, Porto, November: 25 female, 17 male=60% female
- MRS Driving Transformation Through Insight, London, October: 15 female, 12 male= 56% female
- ⭐️ AMSRS, Sydney, September: 3 female keynotes, 3 male keynotes, 1 female invited, 1 male invited, 28 female speakers, 19 male speakers=53% female
- ⭐️ MRS, Financial, London, November: 11 female, 12 male=48% female
- ⭐️ Qual360 APAC, Singapore, October: 16 female, 17 male=48% female
- ⭐️ TMRE, Orlando, October, 79 female, 88 male=47% female
- ⭐️ MR and CI Exchange, St Louis, May: 13 female, 16 male speakers=45% female
- MRIA, Toronto, May: 25 female speakers, 33 male speakers, 6 female panelists, 4 male panelists, 1 female keynote, 4 male keynotes=44% female
- CRC, Chicago, October: 37 female, 55 male=40% female
- Market Research Summit, London, May, 22 female, 29 male=43% female
- ESOMAR Congress, Amsterdam, September: 62 female speakers, 83 male speakers =43% female
- MRS, Customer Summit 2017, November, London: 6 female, 8 male=43 % female
- MRMW Europe, Berlin, November: female, male=43% female
- IIEX, Amsterdam, February: 52 female, 76 male=41% female
- MRS, Methodology in Context, London, November: 40 female, 6 male=40% female
- Customer Experience Strategies Summit, April, Toronto: 12 female, 18 male=40% female
- Sysomos Summit, February, North Carolina: 16 female, 25 male=39% female
- Sysomos Summit, September , NYC: 6 female, 10 male=38% female
- MRIA Net Gain, November, Toronto: 6 female, 10 male=38% female
- ILC Insights Leadership Conference (Insights Association) Chicago, September, 13 female, 24 male=35% female
- IIEX, Atlanta, June: 58 female, 108 male speakers=35% female
- 👎🏻ESOMAR Big Data World, New York, November: 10 female, 24 male=29%female
- 👎🏻Sentiment Analysis Symposium, New York, June, 14 female, 35 male=29% female
- 👎🏻Omnishopper International, Spain, November, 4 female, 13 male =24% female
- 👎🏻CX Talks, Atlanta, October: 7 female, 25 male=22 % female
- 👎🏻Big Data & Analytics for Retail Summit, Chicago, June: 5 female, 19 male=21% female
- 👎🏻 Sysomos Summit, June, London: 3 female, 14 male=18% female
- 👎🏻 Insights50 (Insights Association), Chicago, October: 1 female, 7 male=13% female
- 👎🏻 AMAART Forum, Seattle, June: 4 female, 32 male=11% female
- 👎🏻Sentiment, Emotional & Behavioral Analytics, July, San Francisco: 4 female, 36 male=10% female
- PMRC Speakers not available online
Gender Ratios of Years Past:
- The gender split in #MRX conferences: 2016 edition
- The gender split in #MRX conferences: 2015 edition
- Note: 2014 ratios were done in individual posts
Some of you might remember a long-running and regularly updated post I created last year listing the gender ratios at marketing research conferences around the world. I stopped doing it because an entire year of data is sufficient for an industry that depends on data to see what’s happening. The data showed that women were vastly underrepresented as speakers at conferences. Conference organizers could see that this was an industry issue, not a “just them” issue. The data gave us the perfect opportunity to make great progress in how we source speakers.
I’ll admit it can be difficult to see the problem but Twitter and Facebook make the job of spotting single gender panels much easier. Now, I truly don’t care about single gender, or single race, or all young, all old, all differently abled, or whatever the panel bias is. I DO care when the only type of biased panel I ever see is middle-aged, white, male panels. Has any #MRX conference ever had an all black panel not talking about black issues in #MRX ? Or an all woman panel not talking about women’s issues in #MRX? That’s the problem. That’s the statistically improbable problem.
So that’s why posts like this are so disappointing.
twitter account.) There are hundreds of websites listing #WomenIn_________. There are Facebook groups, google plus groups (yeah, tons of techies there!), Reddit groups, you name the digital channel, they have #WomenIn___________ groups. There’s #WomenAlsoKnow. There are thousands of lists of women experts if you look for them. You don’t even need to ask a woman/black person/differently abled person if they know another woman/black person/differently abled person who is an expert on a topic. You just need to know how to use the google. Or the internet explorer if you work for a company that still operates in the dark ages.There are literally hundreds of Twitter lists labeled as #WomenIn_________, insert industry category. (Some of the relevant lists are on my
But with that tweet, and the emails that came my way to complain, I guess I ought to do the counts again. I hope that while we may have not reached peak equality, e.g., at least 45% of one gender, we at least have shown improvement. I hope that instead of 35% of speakers being women, that at least 40% of speakers are women.
Please do send me PDFs of any market research conference agendas you have saved. I’d appreciate the help. So would your friends and colleagues. My gmail address is anniepettit.
Changing the game: Sports Tech with the Toronto Argonauts and the Blue Jays, #BigDataTO #BigData #AI
Notes from the #BigDataTO conference inToronto
Panel: Mark Silver, @silveratola, Stadium Digital; Michael Copeland, @Mike_G_copeland, Toronto Argonauts; Jonathan Carrigan, @J_carrigan, MLSE; Andrew Miller, @BlueJays, Toronto Blue Jays
- There is a diverse fan base across all Toronto teams, and their preferences and values are diverse in terms of who are they and what drives them to watch and attend games. There are many segments of people not just ‘fans.’
- Fandom takes many shapes and sizes and you always need to grow and rebuild the fan base. You can’t appeal to only avid fans. You must also appear to casual fans. You need to go beyond the narrow focus of superfans.
- The strategy of loyalty programs is that they are an engagement tool to gather data for mining, generate in-game activation, let people win prizes by participating, help partners better understand the fans, and this creates wins across the board – for the team, the partners, and the fans.
- The teams want to learn what people are doing during the game as opposed to guessing. Which benefits do they use their points for, what do they choose at the concession stand, are they watching road games. And this is not just for season ticket holders but people across North America watching games. We need to use the data to learn how to scale beyond ticket holders.
- People want more meaningful and personal relationships with their sports teams. We need to learn what food they want, what environment they want in the venue, what relationships they want outside of the game. And we need to filter out the noise and deliver.
- We’ve all done the analogue research. It’s been done for 100 years and it’s not unique to sport. How do we use technology to do it better now. WHO – we need to stop guessing and start using more efficient research. This massive data we have will tell you many things like WHAT do they want. They might want NEW THINGS that you didn’t offer before, an app, an emoji. The data will also ASSIST your team with player recruitment and roster management. We’ve been doing all this for ages and now we want to do it better, more efficiently, most cost effectively.
- Big data is not free though. Not all stadiums have wifi to do wifi research and it’s expensive to invest in putting wifi in a stadium. We need to spread the cost among multiple agencies.
- This isn’t a technology project. Rather it’s a people project. For instance, a chef can do vastly more with ten ingredients than I can. We need to change the way we engage with fans and leverage partner relationships. Yes, we’re investing in technology but the focus is people. We need to translate tools for each part of the business, reimagine how we engage with fans, and how we make a profit. You can buy a beautiful car but you need to learn to drive to take full advantage of that new car.
Cognitive Analytics: Enabling assisted intelligence in human resources recruiting and hiring by Noel Webb, @CognitiveHR, Karen.ai, #BigDataTO #BigData #AI
Notes from the #BigDataTO conference in Toronto
- He realized that HR teams were spending too much time prescreening resumes before they could even meet with the best candidates
- Recruiters only spend 6 seconds reviewing a resume which means they end up accidentally discarding some of the best ones. Time crunches mean they may only be able to get through 20% of candidates. ML can solve these problems .
- 75% of candidates who apply to jobs do not hear back from the company because there are simply too many candidates and not enough time to do so. NLP and chatbots can solve this problem.
- AI will not steal all jobs but it will automate processes and allow you to engage with potential hires in a more meaningful way.
- Shortlisting is a huge challenge for HR as reducing a huge list of resumes into a screened list takes a lot of detailed attention. Technology such as direct keyword matches aren’t the best option as they eliminate people with relevant skills but not the exact words. For instance, know R is just as good as knowing SAS but a keyword search wouldn’t know that. NLP would work much better.
- Personality insights can also be collected using sentiment analysis to get a functional understanding of the Big 5 Personality traits. [Wow, I can’t imagine how valid it is to do personality assessments with resumes which are often written by third parties and without traditional grammar and style]
- Chatbots can take an applicant through hiring and onboarding processes by answering questions that would normally be asked of an operations officer. [imagine how many stupid questions the chatbot would be asked that new hires are too scared to ask people]