Qualitative Market Research is Bunk #MRX

A photo of a group conducting psychotherapy.

Image via Wikipedia

I have the greatest of respect for psychologists. My original training was as an experimental psychologist, and as such, I obtained permission to take a graduate level class in ethics alongside students training to become clinical psychologists. I learned the ins and outs of the ethics involved with working with people in the most fragile state of their lives, and trust me it’s very complicated. What I learned there has come with me through my entire career. Any time ethical issues arise in the field of market research, I always look back on what I learned in my training to become a psychologist to guide me down the right path. It’s a good place to start because academic Psychologists have some of the strictest ethical codes out there. People’s lives are on the line.

Psychology ethics codes sound very much like a medical doctor’s code. Do no harm. Respect the individual. Care for people in situations where they may be unable to care for themselves. This is what I seek in market research even when some people say that I am being naive and idealistic.  But I’m OK with that. I’d rather be too careful than risk someone being hurt. I sleep well because of this.

Is it an issue of informed consent?

So when I saw the MRS paper on proposed privacy guidelines, I was very concerned. My interpretation of it suggests that it disregards the legitimacy of naturalistic observation research (i.e., practice of scientifically observing people in their natural environment without interfering with them or drawing attention to yourself). Here is one quote (all the quotes in this post are out of context) from the MRS paper:

“That is why as the real and virtual worlds converge further, MRSB believes that researchers must apply the same ethical standards to research online as they do in real life. That is: Obtaining the informed consent of all persons from or about whom data is collected”

To me, this flies in the face of traditional ethical research methods that have been practiced by psychologists, sociologists, anthropologists, and market researchers for decades. All of these codes permit naturalistic observation and recognize it as a valid and ethical practice. Here are just a few excerpts from various ethics codes of psychology organizations.

Canadian Psychological Association Code of Ethics
“Obtain informed consent for all research activities that involve
obtrusive measures, invasion of privacy, more than minimal risk of
harm, or any attempt to change the behaviour of research

American Psychological Association Online Research Ethics
When conducting research online, researchers need to contend with changes in the technology, the ways the technology is typically used, and the norms surrounding this use, because this context is integral to assessing anonymity, privacy, risk and the like. For example, the concept of minimal risk depends upon a comparison of the risk associated with research participation to risk in everyday life. The concept of privacy depends upon participants’ reasonable expectations about whether others will be allowed access to information about them. As online behavior and norms change, the nature of minimal risk and the very concept of privacy themselves change.

American Psychological Association Code of Ethics
“8.05 Dispensing with Informed Consent for Research
Psychologists may dispense with informed consent only (1) where research would not reasonably be assumed to create distress or harm and involves (a) the study of normal educational practices, curricula, or classroom management methods conducted in educational settings; (b) only anonymous questionnaires, naturalistic observations, or archival research for which disclosure of responses would not place participants at risk of criminal or civil liability or damage their financial standing, employability, or reputation, and confidentiality is protected; or (c) the study of factors related to job or organization effectiveness conducted in organizational settings for which there is no risk to participants’ employability, and confidentiality is protected or (2) where otherwise permitted by law or federal or institutional regulations”

British Psychological Society Code of Ethics
“Unless informed consent has been obtained, restrict research based  upon observations of public behaviour to those situations in which  persons being studied would reasonably expect to be observed by  strangers, with reference to local cultural values and to the privacy of  persons who, even while in a public space, may believe they are unobserved.”

Department of Psychology, University of Sheffield
9. Observational Research
9.1 Studies based upon observation must respect the privacy and psychological well-being of the individuals studied. Unless those observed give their consent to being observed, observational research is only acceptable in situations where those observed would expect to be observed by strangers. Additionally, particular account should be taken of local cultural values and of the possibility of intruding upon the privacy of individuals who, even while in a normally public space, may believe they are unobserved.

Is it an issue of permissions?
This for sure is a no. Ethical monitoring and measuring and research platforms would not exist if it was an issue of permissions. Websites have two very simple ways of ensuring their data remains private. First, they can insert code into their website that essentially says “please don’t gather our data.” This code is readily respected by ethical companies. Second, websites can choose to NOT create APIs that permit the sharing of data with external companies. Since Facebook and Twitter and most other major websites provide APIs to give data to third parties, they are the ones who determine what can be shared within their privacy policy. Ethical companies won’t, and physically can NOT, take data that isn’t permitted.
Is it an issue of masking verbatims?

On a separate note, the MRS paper indicates this about the cloaking of data: “While data masking or cloaking may be offered as a privacy solution, from a methodological point of view it is rather unsatisfactory. This approach could distort results if the original wording was materially altered (and even a single word change can do this).”

This, I believe to be an affront to the skills, expertise, and training of most qualitative researchers who perform this task on a daily basis with the utmost of quality in workmanship. Are we to disregard qualitative researchers? Push them outside of the market research world? Ignore the contributions of people whose skills are qualitative because they are incapable of properly summarizing or describing opinions? I can only hope that this was an unintended side effect. Masking is truly such a minor issue that I immediately begin picturing babies and bathwater. Masking is a widely used process that can be taught and learned. So let’s teach it and learn it.

Whether you agree or disagree with me, whether my opinion is in the extreme minority, I implore you to share your opinions with the MRS. Our industry is being shaped right now and your voice matters. Please speak up.


8 responses

  1. Hi Annie, totally agree with what you’re saying. Academia is a great place to turn for the right way of doing things…I refer to your post over here in case it’s of any interest: http://bit.ly/xRUFpL

  2. […] – Ray Poynter puts forward a new ethical framework which moves beyond the old panel based world – Annie Petitt draws on ethical approaches to naturalistic observation widespread in academia – John Griffiths […]

  3. […] leaves us with the ethical considerations, and Reg Baker does a good job (as did Annie Pettit here and Ray Poynter here) in outlining what I think is a reasonable perspective: I confess that I am not […]

  4. In terms of the ethical guidelines for naturalistic observation that you cited, the key quotes, to me, are these from the British guidelines: “with reference to … the privacy of persons who, even while in a public space, may believe they are unobserved.” I feel that the assumption made by the MR industry that all people who, for example, post on Twitter know that everything they say is now in the public domain, is entirely self serving. We WISH they believed that, we would LIKE to believe they know it is not private but, quite simply, some do not know this and may well be aghast at the fact that peope are out there drawing conclusions about them based on their tweets.

    By the way, as I’m typing this I see my email address in the box below and have no idea how this site knows it – the address is not the one I normally use for public discourse. I undoubtedly “provided” it in some way but do not know/remember how. People don’t always know what we assume “everyone should know”.

    1. I totally agree that not everyone knows the the public internet is public. That’s why I’m a massive believer in masking verbatims. Respect the individual, look out for the ignorant.

  5. Hi Annie, I think the points you raise regarding qualitative research and masking of verbatims are interesting. It does assume that participants ‘know what they say and say what they mean’ – which I very often find not to be the case. As you say, I place more value on the meaning behind the words, and the interpretation of this in a wider context than the exact words themselves. I’d go as far as to say that reports over-reliant on exact verbatims and lacking in interpretation by the researcher do this industry more harm that good! I’ll be interested to see what others have to say on this.

    1. Interesting comment. It goes along with the theory that people actually know what and why and how they think. Which science has proven is false. Trained and experienced researchers are absolutely essential to evaluate data properly, whether masked or unmasked.

    2. That is a good point about the over reliance on exact verbatims, I think it’s the difference between ‘analysis’ and ‘reportage’..

%d bloggers like this: