Market research isn’t straight forward. As much as you believe one thing, I’ll completely disagree and believe another. Which brings us to this series of posts. Over the coming weeks, a colleague and I are going to tackle an important research question. I’m going to make one argument and they’re going to argue the exact opposite. Do we each have the same opinion, do we each believe in our own argument, are we arguing against our real opinion? You figure it out. Here goes!
Keza Kyanzaire: Telephone interviewing should die
CATI or Computer Assisted Telephone Interviewing Software has been helping market research companies interrupt people during dinner since the dawn of time (the proliferation of the internet). I admit, there are advantages to this form of interviewing, especially considering that when it was originally conceived, the alternatives included going door to door with a clipboard. However, it’s past time to put telephone interviewing to pasture.
According to the CRTC’s 2016 Communications Monitoring Report, more Canadian households exclusively use mobile phones over landline phones, and the shift away from landline phone usage continues to increase. Telephone interviewers are increasingly beginning to reach respondents on mobile phones. Respondents on mobile are often younger, and age can have an adverse effect on the willingness to participate in research and the quality of the data. As well, a respondent reached on a mobile phone may not be in an environment conducive to completing an interview.
Let’s not even mention the fact that telephone interviewers are often mistaken for telemarketers, which comes with its own set of issues. Market researchers have always had to fight to distance themselves from telemarketers in order to establish trust with participants, and legitimacy in the work we do.
Web-based surveys are an effective alternative to telephone interviews. Not only are they a cost saving alternative, (staffing call centres can be extremely costly and they tend to have high turnover rates) they allow the respondent to decide where and when to participate. Particularly in the case of MROC’s or panels, respondents can choose to participate in surveys that are of interest to them and are more engaged and active. Moreover, web-based surveys allow for visual stimuli and more interactive elements that not only make for a better experience for the respondents, but also allows for a depth in the types of questions that can be asked, and data that can be gathered.
Telephone interviewing has been a great resource for the industry, but with the advantages of web-based methods, why are we still running call centres in 2017. It’s past it’s prime and it is time that they are replaced by more effective and advanced methods.
Keza has an Honors Bachelor of Psychology with a Specialization in Cognition. She also completed Humber College’s Research Analyst Postgraduate Program, where she learned how to conduct both social and market research. Keza put her education into practice as a Research Analyst Intern at Numeris, where she conducted statistical analysis and helped inform business decisions. She is now a Researcher at PATTISON Outdoor. And she’s an experienced speaker having taken the stage at #IIeX in Amsterdam.
Annie Pettit: Telephone interviewing is the best thing since sliced bread
I’m not a gambling person but I’ll bet you know about research panels. Well, you’re in the minority. Most people who are not in the market and social research industry don’t know that panels exist. Which begs the question -who are these people who know about research panels? Are they just people desperately seeking places to earn money? What causes some people to seek out incentive based activities? One thing I know for a fact is these people have internet. They can afford monthly internet charges, maybe even high speed internet charges, as well as the cost of a device to access the internet. Internet might be a right but it still is a luxury for many people. Let’s check our academic privilege at the door. So how do we gather opinions from people who aren’t aware of research panels or couldn’t participate in them even if they wanted to?
One alternative is mall or central location research. Send your best face to face interviewers to the nearest mall and have them randomly interrupt passersby. Well, first of all, not everyone lives near a location that is large enough to warrant sending an interviewer. Sorry residents of Nunavut, you’ll never be asked to participate in mall research – your town isn’t large enough for your opinion to matter. You live in Vancouver, Calgary, Saskatoon, Montreal, or Halifax? Your chances are pretty good of meeting up with a mall interviewer – unless you’re one of those people who takes a ten minute detour through the parking lot to avoid the person with a clipboard. Besides, anyone who’s taken an introduction to research class knows what accidental sampling or convenience sampling is. Goodbye ability to generalize to a larger population if mall research is your game.
Which is why I’m a huge fan of telephone research. As it has been since the invention of polls, telephone research continues to lead the fight against self-selection bias. If you’ve never heard of research panels, if you can’t afford high speed internet, if you don’t live in a major city, if you avoid people at the mall, telephone methods still value and seek out your opinions. We are currently living through a time where people don’t trust the polls. Let’s not revert to methods that make it even easier to distrust polls.
If you’re an early career researcher, data scientist, data visualizer, marketer, or similar, and would like to write a #PointCounterpoint article with me, please send a gmail to my full name anniepettit and let me know what topic interests you. Maybe I’ll pick yours!
We’ve all been on the receiving end of that phrase at one time or another. I wonder, though, if it doesn’t have a hidden meaning, a truer meaning. I wonder if it actually means “I haven’t thought out my premise enough yet and I feel like yammering at someone so I’ll take a verbal meandering stroll with you since you aren’t doing anything important.”
Why should a discussion be easier? Whay can’t you get to the point? We should find it easy as pie to verbalize just about any idea in a short space because we’ve been getting strict training for a few years now. The training is called tweeting and no one will act on a tweet that doesn’t make a point in 100 characters.
Of course, there are legitimate reasons for a verbal stroll – you want to toss ideas back and forth, you want to discuss a number of different issues, you want to flirt with the person. But if that’s not it, I have some simple advice.
Get yourself to a washroom with a full size mirror and have the conversation with yourself. Once you’ve figured out your premise, then you can interrupt my farmville game.
- Talkwheel Hopes To Facilitate Easier Conversations At The Workplace (socialtimes.com)
- You Don’t Say: Persuasive Body Language for Flirting and Dating (psychologytoday.com)
- Dumped via text (introvertdating.wordpress.com)
[tweetmeme source=”lovestats” only_single=false]
Well, once you manage to catch your breath after laughing solid for 4 minutes, let’s really think about all the people involved in this little prank.
1: Interviewer: First of all, this interviewer deserves a raise, a bonus, and a promotion for going through this interview without laughing, getting upset, or antagonizing the survey responder. I’m sure he deals with this sort of thing, whether real or fake, all day long every day. And yet, the utmost professionalism on his part. Kudos for a great job.
2: Responder: How did our industry get to such a state where surveys are written so poorly that people leave a tape recorder at their telephone waiting for researchers to call in order to make fun of them? This is nothing for us to be proud of.
3: Data Analyst: How exactly is the data analyst going to handle data which is clearly horrible quality? Will the analyst think of checking for outliers in each question? Will the analyst review the entire set of responses to recognize that it is an across the board outlier and probably a troublemaker? Will these responses lead to completely invalid analysis and conclusions?
4: Survey Author: Of course, we understand the need to use standardized questions in surveys. But, no matter how convinced you are, the world does not consist of people who know how surveys work. There are absolutely people out there who need to be taken through a survey with far more care than what we
permit when writing surveys. Telephone surveys need to be written so that interviewers can speak naturally and help those people who actually need some help. That’s where good data comes from. I’m really curious if the survey author left a place for the interviewer to indicate that this instance was possibly an outlier.
So, enjoy. But the next time you write a survey, keep this in mind. Are you antagonizing yet another survey responder or are you responsible for creating a more positive market research experience?
So what’s next?
We went through an era of worrying about the data quality of telephone surveys. Make sure the interviewers don’t lead responses with unnecessary “uh huh”s. Make sure the interviewers probe every time and often to avoid lazy responses. Make sure to spread calls out over every day of the week and every time of day, including supper time, nap time, and when you’re in the shower, to ensure maximum generalizability.
We’re now in an era of worrying about online data quality. Now, straightlining and incentive grabbers and random responding and heavy responders are topical issues. We’ve designed a bazillion methods to catch deliberate poor responders, a bizillion other methods to be lenient to occasional and accidental poor responders, and we’re still working on identifying the many people who still slip through the cracks.
What is next? Here’s my guess. With the emerging prospect of Twesearch, data quality measures for ensuring data gathered from the internet is fair and honest and true will be of utmost importance. Methods to ignore posts that automatically appear a bazillion times (see the bazillion trend), methods to tease out posts by spammers, methods to identify ‘marketing’ posts.
Data quality is a never ending issue. You think you’ve got it solved, or at least reasonably handled, and then everything just goes out the window when the next method comes along. Such is life~
I completely disagree with my own title!
My world is online. I started out writing online surveys in 1996 when I bugged the computer helpdesk at my graduate school to set me up with an online database. No one else at the university had ever done such a thing and i confused the heck out of them. I wrote my own html code which allowed me to specify font sizes, font colours, page colours, radio buttons, check boxes and text boxes. ooooooo….. so sophisticated. I’d be embarrassed to tell a scripter now that “I write my own code.”
Online research has never tried to say it uses probability sampling but, other methods of research have. There has been a debate over the last year specifically directed at online panels. Well, not really a debate. Some folks have been outraged that online panels do not use probability sampling and therefore they do not qualify to use statistics. To go even further, they suggest that telephone samples do use probability sampling and so results from that type of research are the most valid.
Let me offer up some ideas…
Telephone research – Do you always answer your phone? Is your phone number unlisted? Do you return phone calls? Do you politely tell telephone interviewers that you are busy when in fact you are nursing a bag of cheetos?
Mail research – Do you just throw out all the junk you get in the mail? Do you fill out surveys AND mail them?
Online research – Are you signed up for an online survey panel? Do you click on the survey banners that appear after you run a search and then finish every survey?
It seems to me that no matter how hard you try to use probability sampling, human beings just cannot cooperate. We’re not worms or mice or molecules. People choose when they wish to pay attention or participate. It’s not online panels. It’s research with human participants.
Probability sampling of people? No such thing.