Tag Archives: ethics

Apr 2 2012

The Audience Question I Fear the Most #MRX

I’ve done my fair share of webinar, workshop, and conference presentations. I’ve talked to groups of ten, a hundred, and three hundred people and there is usually an audience member with a point to prove. They’re skeptic of the topic, eager to prove they’re smarter than everyone else, or just keen to listen to their own voice.

They ask questions designed to identify flaws in methodology or logic, and force me to respond with answers like “You’re right, that is a problem” or “True, we should have done a better job with that,” or “Right, sentiment analysis will never be perfect.” But you know, those kinds of questions don’t scare me. Every research project that was ever conducted was rife with logic and methodology errors and I’m glad that researchers are picking up on them. That’s how the cycle of research works. Find a problem this time, fix the problem the next time.

Then there are questions from people who are keen about the work that has been presented and want to learn more about it. They’ve seen the tables and graphs that I’ve prepared. They’ve listened to the pros and cons of the methodology. They’ve seen the real life examples of misleading and poorly done research. With all of that information, they understand the good and bad of what they’re potentially getting into and they’re intrigued. Intrigued enough to ask that dreaded question.

Now first, you have to recognize that I am a researcher to the core. I deliberately chose to take every single research methods and statistics and psychometrics class in school, even going so far as to get special permission to take classes that were outside of my curriculum. Research methods was and is all I care about.

My problem is this. Every conference submission form and every followup email from conference organizers says you must educate and not promote and I wholeheartedly agree with those rules. I don’t want a sales pitch at a conference. I want to learn something new.

But then that horrible question gets asked. It comes in one of two forms. 1) “What software are you using to do this?” and 2) “How much is your software?” Anyone who’s seen me present can attest to my reaction to those questions. They immediately catch me off guard and I stumble through answers like an idiot. Answering these questions makes me feel like I am promoting and selling. Indeed, I feel that simply speaking in front of a crowd is a sales pitch.

Even worse, as soon as I’m asked to talk brands and money, I fear that people will distrust my research objectivity. I fear that people will disregard my passion for quality and honesty and see not the sense, but the cents. That, my research friends, is truly terrifying. But perhaps I’m over thinking things. Perhaps those questions are just nice compliments, confirmations that I’ve done the job I was supposed to do. Perhaps it really was a sales pitch, a sales pitch done the right way. But I still hate getting that question.

Related articles
Oct 12 2011

Here’s the Problem with Social Media Research Ethics #MRX

Immanuel Kant developed his own version of the...

Image via Wikipedia

Wherever I may wander, wherever I may roam, I always come across a divide in the interpretation of market research ethics. Some people think all market research must be permission based. Other people think anything in the public eye is fair game. The gap therein is wide, rife with arguments, and seemingly unsolveable. But I think I’ve figured out what the problem is.

Many people playing in the market research sandbox have no training in market research. They did NOT take an undergraduate degree in market research. They did NOT take statistics and research methods and research ethics courses in school. They did NOT take philosophy or medical or psychology or sociology courses in ethics. They have NOT taken certification courses offered by market research institutions. In fact, many people playing in the market research sandbox come from computer science, database management, and many other non-social science research spaces. They don’t have the same understanding of people as market researchers do.

Our industry is suffering from a massive influx of workers with insufficient or no training in important market research skills and knowledge. Sure, you can pick up things on the job, from your colleagues, from mentors, from conferences, from webinars, and from many other places. But you can’t pick up everything and you can’t pick up everything correctly. And you certainly can’t pick up research ethics without the careful guidance of someone who has spent decades debating that issue with trained and respected colleagues.

It is only with appropriate training that we will be able to reduce the gap. The gap will always be there but as more of us better understood the human condition and the rules that go along with using humans as data points, there will be more points of agreement. So raise your voice. Raise your voice every time it’s needed.

Related articles
Sep 29 2011

Will Communities Kill the 6 Group Project by Nick Priestley #SoMeMR #li #mrx

15.30 PANEL: Will communities kill the 6 group project?

  • The future for research communities in generating insight
  • How online communities fit into the overall market research mix
  • Evaluating cost efficiencies and quality trade-offs

Nick Priestley, Managing Director, Tuned In

  • Are communities mainstream now? We are approaching mass market in terms of awareness. People still have concerns about blurring boundaries but there are more and more success stories. Barrier is internal stake holders.
  • Is it research? should we call it research? It’s more about collaboration and getting input in at the beginning. Forward thinking clients are open to it. Does it replace traditional focus group?
  • is 6 group project under threat? It will never die. It is still relevant. But there are many benefits to online approach. Start with hypotheses, develop over time. You can do this in two hour. Vast geography, you can’t do focus groups in hard to reach areas.
  • What kind of people work best in communities? 1% are super users, 9% are active, 90% don’t do anything. We want more people to be active. Now, people are more familiar with the idea. We need to show the process more to interest more people.
  • Segment communities so you can identify the creative people and the strict people and use each group to their strengths. Find tasks that are creative as possible to encourage people. People often don’t realize they were capable of expressing themselves in that way.
  • Audience – it’s important to listen to people who know nothing and these people aren’ t in communities.
  • Online doesn’t necessarily mean cheaper. You get more value from community. More people, less geography issues, less time of day issues, more natural setting, technology gives you more creative options.
  • Is it the shiny object effect? Do groups need a fresh spark? There is absolutely room for both. [Ah yes, the dichotomy must always be there, qual vs quant, online vs offline, MY favourite must always win. Can we just all use one big tool box with all the shiny tools in it?]
  • Are communities a safe haven with fewer ethics issues? Are communities fake social media? Communities are no different from people in real life.
  • Audience – Will facebook put an end to parties? Same as will communities put an end to groups. The two must work together. [ha, yeah. no more parties for me.]
  • Communities when used properly will replace parts of panel and parts of focus groups. [Because it is a better tool for various objectives. Use the RIGHT tool from your toolbox.]
  • Audience – Are focus groups more boring than they need to be? Can we incorporate fun of communities back into focus group.
  • Audience – What about finding pre-existing communities, that aren’t created by MR. This area is littered with huge fails because of lack of transparency, researchers failing to identify themselves. Perhaps use those areas for recruitment. [remember patientslikeme]
  • Audience – 6 groups aren’t under threat, the creative brainstorming groups are under threat. Communities let you do it over a longer period of time.
  • Audience – Are communities at risk because anyone can do them, even if you aren’t trained? Well, the client can read the entire transcript and confirm it all.
Related articles
Sep 29 2011

Ethical Framework for SMR, Panel #SoMeMR #MRX #li

10.15 PANEL: Establishing an ethical framework for social media research

  • Tracking developments towards new standards for social media research in Europe and the US
  • Dealing with privacy issues: Assessing attitudes towards privacy in online environments
  • Evaluating ethical guidelines for blog and buzz mining
  • When is engagement with commenters necessary?
  • Evaluating appropriate codes of conduct for qualitative approaches to harvesting data via social media channels

Barry Ryan, Standards & Policy Manager, MRS
Jillian Williams, External Relations Team Leader, Highways Agency
Pete Comley, Chairman, Join the Dots (formerly Virtual Surveys)
Simon McDonald, Business Director, Insites Consulting

  • Barry – Data protection, there is identifiable data that must be protected. Copyright – blogposts, photos, videos, all covered under copyright act. Terms and service – of individual sites must be respected. Internal issue – how MR codes are constructed – participation is based on voluntary informed consent, that is our heritage.  (I’m waiting to hear about the stance on observational research, and qualitative researchers abilities to summarize text.)
  • Simon – Does research industry need rules?  We know it’s a work in progress. We don’t want to be restricted from doing things that other companies are able to do. (e.g., buzz companies don’t have to listen to MR ethics)
  • Pete – Believes in guidelines. Was part of ESOMAR guidelines committee.  Not happy with MRS stance. Bodies should be forward looking and represent us. Tone of consultation document does not do that. Document is like Pope and Catholic church. Applying to the letter. MRS won’t stop us from doing SMR. Seriously risks splitting entire MR industry in the UK. It is that serious. Solutions? We must be more inclusive and representative. Must be provisions for MR to do SMR. Long term, MRS code of conduct is the problem. Informed consent is the problem. We are NOT interviewing people here. This is analyzing public data. Concept of informed consent does not apply. We need to relook at code of conduct.
  • Jillian – As a research client, anonymity is important. Masking isn’t satisfactory.  Client does not want to be on the front page of the Daily Mail. Client will take the flack, not the industry. Clients want to comply with guidelines. Clients pay the bill.
  • Barry – Data Protection Act is the problem. Informed Consent is the first thing in it. There is no distinction between public and private. The MRS Code reflects rules of legislation. MRS made it easy for researchers to comply with data protection act. “This data is accessible” is not sufficient but you can work with the data provider directly and then it’s ok. “Subject to data protection rules” is important. If MRS interpeted law incorrectly, please tell them. [Call to MR company internal legal counsel - does anyone see if there have been misinterpretations of Data Protection Act?]
  • Pete – Data Protection Act is pre-internet. How do we survive as industry until and if there are changes?
  • Barry – Whatever comes next will not be better. “The legislator knows best.”  People want the right to be forgotten (drunken photos should be deleted if the person wants them deleted).
  • Simon – Self-regulation is important. Dialogue with respondents means better qulaity data. Consent is important but best research is also important. Their company had a problem where they friended people for research purposes, with multiple layers of consent, but then of course Facebook lets you see friends of friends, and those friends hadn’t given consent.
  • Jillian – SMR is not necessarily representative.
  • Annie – I asked a question about whether observational research is not research since it’s not informed consent and whether masking is an assault on qualitative researchers who mask for a living.
  • Barry – This is not an assault on qualitative research. There is a separate guidelines for qual research. Maybe the MRS heard nothing back from qualitative researchers and it’s not reflected in the paper. In the online space, everything is data, video, photo. Under the data Proection Act, processing data is engagement. Masking is good for privacy, but it doesn’t rectify the potential unlawfullness of the act of taking the data.
  • Pete – Does everything really need to be masked? “I love McDonalds” maybe not.  Anything Pete says here, he risks it being written down and shared. (Hmmmm….. watch out!) Going out of your way to NOT quote something written in the online public space seems odd. What do you do with gray area of semi-private. Inconsequential “This hotel is lovely” doesn’t need masking but someone’s sexual preference does need masking. It is a minefield.
  • Jillian – Doesn’t like masking at all. We want the insight from the quote as opposed to masked verbatims that aren’t exactly correct and could be misunderstood.
  • Question – why are companies doing this if it’s all illegal under data protection act? [Great question - are we waiting for someone to get sued and go to court before we get an update to the legislation?]
Related articles
Aug 31 2011

If It Ain’t Illegal, It’s All Good #MRX

I’ve been to a number of market research conferences where the following argument has been raised in defense of  methodologies that some people deem questionable:

It’s not illegal so we can do it.

It’s made me stop and re-evaluate how I run my life. That whole “Do unto others” thing is far too complicated and theoretical and the books that promote it are thousands of years old and not relevant to me. Besides, it’s greatly to my advantage to base my choices purely on legalities. So here is how I’m going to better my life.

  1. I’m going to save at least $100 every year by keeping any wallets or phones that I find. (Shout, I could have had a new Android phone last week!)
  2. I’m going to get at least 3 free pairs of gloves and 1 free umbrella every year by not telling departing transit passengers that they’ve dropped something.
  3. Conferences presentations will be a lot funnier because I won’t tell people who are about to present that their zipper is undone.
  4. I will save at least 90 minutes every year by no longer holding doors open for people, trying to re-unite lost children with their moms, and offering to taking photos for visiting travelers.
  5. I will make more money by disregarding personal preferences for privacy, anonymity, and respect.
  6. I will gain more self-respect by doing what’s right for me personally instead of wondering how my perfectly legal actions affect the people around me.

It’s all perfectly legal my dear friends so let’s start with item #1.

Related articles