The Evolution of Research Privacy – It’s Only Just Begun November 6, 2009

Chances are I didn’t give you permission to read this. You probably found this blog because of a google search or a twitter search or some other online search service. You found it through google because I checked a box on my wordpress profile page that said yes, google has permission to index this. Or, you saw a link that I personally posted on twitter and you clicked through it. Either way, you only got here because I let you, I gave you permission.

I fully expect that someone at some point will read this. Maybe even a few people. That’s what blogs are for. But, the average person doesn’t maintain a blog and they probably don’t tweet all day long. The average person probably makes the occasional update to their facebook or myspace page, or the occasional comment on youtube, or writes some scathing remark on an opinion site about some crappy product they just bought.

The more enlightened folks might know that companies actually go online and search for comments naming their products to see what people think of them. I do think, however, that most people probably don’t know that companies use automated systems to seek, collect, and evaluate those comments.

As researchers, it might seem obvious that companies would gather data online. Given the current state of the monitoring industry, if I tweet “Scor Bars suck monkeys” or “I abhor Great Canadian Superstores,” someone from those companies might try to get ahold of me to diffuse a potentially negative situation.

But wait… When I tweeted those opinions, did I actually want someone from the chocolate factory to seek me out and try to convince me that Skor Bars are totally rad? Definitely not. If I really wanted a reply from the company, I would have gone to their website and made my request there. I’m pretty sure I can find their website and I’m pretty sure I know how to fill in a form. For me personally, to have a company contact me because of a tweet I wrote would be an invasion of my personal space. Unless, of course, my tweet was @ them specifically.

Everything on the internet is there because each person agreed to put it there. If you didn’t want someone to read it, you wouldn’t have typed it out on youtube or blogger or flickr. You would also password protect your blog and turn off the search engine and RSS features. I think reading the internet for brand information is fair. Companies should read and learn and make solid business decisions based on data that is readily available. Companies should also know when to draw the line between someone who is actually seeking answers and someone who is just yapping off at the mouth.

I think there are still missing pieces though. People need to be educated that whatever they put online will actually be collected. If it isn’t password protected, it will be collected. If they don’t tell google to not index them, it will be collected. I think a lot of people will be peeved that their data is being collected without payment and then analyzed and sold for profit. It will require a lot of education, and it will be a bumpy ride as people struggle to protect their privacy rights. I think this is our responsibility as researchers and we need to spend the time to do it right. But, our industry will be better for it.

Annie

Mobile Surveys – Kill Me Now November 5, 2009

As I write this blog from the comfort of the Toronto Transit Sytem using my Blackberry, I am reminded of various things. Like, what surgery could I get to add two more thumbs to each hand and how much does carpal tunnel surgery cost? Damn, this is slow and painful! But, while I’m here waiting for the traffic to get going, why don’t I just take a few moments of my time to answer a survey.

As I mentioned in a previous blog about the Esomar conference, during which a few folks extolled the virtues and vices of mobile research, I still don’t see mobile research taking off. It has been around for years and has so far failed to gain any solid footing. Sure, phone technology hasn’t been that great and now we have far better systems. IPhones take gaming on your phone to a whole new level and blackberries let your worklife interfere with your personal life with the utmost efficiency.

But, even though I’m probably more techie than about 51% of the population, I can barely get this BB to work. My preferred twitter platform refused to cooperate during the Esomar conference, I’ve made numerous butt calls, and I can never tell when my data limit is about to be reached. I can’t imagine how many glitches and oddities I would have to overcome just to answer the simplest of short surveys.

Perhaps I would make the effort to answer a point of sale mobile survey if it got me 20% off my purchase. Let’s just forget the 45 and 60 minute surveys here, but even 15 minute surveys that are rarely accepted by clients are out of the question. If 3 POS questions take 3 minutes, I might make the effort. Might. Might not. Probably not. I’ll be honest with myself. I won’t. I’ve already gone bug eyed writing this.

I suspect that mobile surveys will follow the same trend as paper and online surveys. Keeners and early adopters will hop on board first and give researchers great hope about the feasibility and valdity of mobile research. Keeners will be paying such close attention to this cool new thing that their friends will hear about it and want to do it too. Those regular folks will join after all the major glitches have been worked through. And then, we will be back where we started. Responders will become bored with the format. Surveys won’t keep up with the unending quantity and quality of free entertainment available on cell phones. People will want incentives and compensation. People will realize that the pain, physical financial and temporal, just isn’t worth it. And mobile surveys will slowly meet the same fate as paper and online surveys.

You have to give researchers credit though. We are desperately trying to solve a long-term problem. We are trying anything we can to reach more and different people in more and different ways. We’re trying to think outside the box even though we haven’t left the box since someone dreamt up the survey.

Mobile surveys? Just another box.

P.S. Still loving surveys!

Annie

I’m Told I Have No Opinion November 3, 2009

I love Merrell shoes. The sole is like walking on sponges and the designs is really cool. I love Kitchenaid appliances because they look good, they are sturdy, and baking is so much easier. I love Tall Girl clothing because the pants are just the right length and the price is right if you get the sales. I could bore you profusely about even more merits and downfalls of each but unfortunately, I am not permitted to have an opinion of them.

You’ve seen those screener questions. “Do you or any member of your household work in the marketing research industry?” Why is this question so important? Why can’t I answer the survey if I do work in the industry?

Perhaps I am going to steal confidential questions.
Perhaps I am going to try to skew data because I have a competitive client.
Perhaps my opinions will be biased because I understand the purposes of the questions.

Well first, various research codes say I must behave ethically which means no stealing and no biasing.
Second, as I rush through the survey like everyone else who is disgusted with low quality questions and bored with ridiculous questions, I am certainly not paying any more attention to the purposes of the questions than anyone else.
And third, why aren’t my opinions valid? Doesn’t Merrel and Kitchenaid and Tall Girl want to know what one of their most loyal customers thinks? Even worse, since my spouse lives in the same household as someone who works in the marketing reseach industry, why aren’t his opinions valid? He’s never even heard of a likert scale and would probably stab himself in the heart if I tried to explain it to him.

Besides, if I truly am going to lie and cheat and bias answers, why would I ever confess that I work in market research so please don’t show me your survey?

And if you’re curious, I DO answer competitive surveys (just like you do). My rule is answer every question, except the screener, completely honestly or don’t finish the survey. Just how I’d want to be treated.

Annie

CAUTION: The Scariest Thing EVER November 2, 2009

There are many ways to terrify people. Put a spider on their shoulder, make them stand close to the edge of a cliff, tell them you’re going to visit the in-laws (ftr, mine are great). Different people are scared of different things. There does, however, seem to be one fear that transcends other fears – the fear of statistics and numbers.

How did this come to be? Were our math teachers horrible people? I doubt it (though one of mine was and that’s a whole seperate post). Were we threatened with having to do extra math if we didn’t finish our brussel sprouts? Doubt that too.

Here’s my theory. Remember english class where you wrote a beautiful essay and the teacher gave you an A? That A didn’t mean perfect, it meant great job. However, you never got an A in math. You got an 80%. In other words, you got 80% right, and 20% horribly, horribly wrong. You failed at 20%. You sucked for 20%. Even though you did a great job, you still managed to screw up a lot of answers.

Math insists on having a right answer. It’s right or its wrong. It’s not a teachers perception of your thoughts and ideas and its not even a measure of how much they hate you. For, even if your math teacher hates you, if your answer matches what’s in the teachers edition, you got the mark and the grade.

It seems to be that, even though nobody is perfect, we are scared of situations where there is no doubt we are wrong. We seem to forget that everyone is wrong at one point or another, and we all have strengths and weaknesses.
My advice to you is don’t be fearful. Expect to make mistakes. Expect to forget formulas. No one is perfect and no one gets every math problem right.

Statistics can actually be interesting if you really to listen to them. TV commercials and other marketing materials use lots of bad statistics and they are a source of great amusement, at least for me. And, you will find that people who are okay around numbers are in high demand in the job market. That’s good enough for me!

Annie

Esomar Chicago Day 2: Four days later! November 1, 2009

Day 2 brought lots of interesting topics as well. We started off with updates on the ESOMAR 26 document, the ARF quality initiative, and the ISO process. I did not know that the first ESOMAR code was developed in 1948! I did not know that people from 26 countries were involved in the ISO process. It was not a surprise, however, to hear the findings of the ARF quality study .

• data quality is not the serious issue that was previously thought
• panel duplication is really only about 16%
• heavy responders provide good data
• response rates are not the best indicators of quality data.

After the ARF presentation indicating a complete lack of ability to identify a weighting system that would equalize panels, one unlucky speaker had the unfortunate time slot of presenting his work on how the CART system can indeed weight data sources to be equal. Personally, I’m still on the side that panels cannot be weighted to be equal. I’ve participated in so many parallel studies for so many different types of surveys and categories of products  and have yet to seen a perfectly successful case. Sure, you can always weight a few variables into comparability, but you lose out on a bunch more. Maybe CART is the magic solution. Time will tell.

Once again, survey panels took the hit on not being representative of the general population. Really, come on folks, what method of marketing research IS representative of the general population. None. End of story. In the end, the only kind of representativity that matters is making sure your sample suits the purpose. And, if your results never predict the marketplace, get out of the business.

Kim Dedeker was kind enough to share a few thoughts on the state of the industry since she first caused a good storm. Her advice – the industry can only be sustained through quality, we need to continue having and creating leadership through getting involved.

The last major topic of the day was mobile research. I must confess that I am STILL not a believer. I just don’t see how the ability to answer a 20 minute survey on a 2 inch screen as you ride the bus or pay for your new purchase is going to bring the survey industry back from the deadly response rate dive.

Feel free to also read a few other thoughts I had about ESOMAR Chicago in a piece I wrote for research-live.

Below is Kim speaking and a presentation about social media that mentioned the tweet-up at Lux Bar.