Tag Archives: screener

Should you really screen out non-users? #ESOMAR #MRX

esomarLive blogging from #ESOMAR Congress 2014 in Nice, France. Any errors or bad jokes are my own.

Going to the Edges for Inspiration: Why it’s right to talk to ‘extreme’ consumers even if you are a mass market brand by Elaine Ho, Sense Worldwide, UK
Jacky Parsons, Sense Worldwide, UK, Jayne Hickey, PepsiCo, USA, Marlene Cohen, PepsiCo, USA, Nick Graham, PepsiCo, USA, Tom Lilley, Sense Worldwide, UK

  • What inspires you to get up every morning?  [um, SQL coding? is that the right answer?]
  • They like to talk to consumers who have extreme opinions – in the case of Korea, someone with a very relaxed attitude when everyone around her dresses more formally
  • “Inspirational” pair of shoes was grubby and ripped – for this brand, the grubbier the better as it expresses who you are. And, the laces were fully tied, permanently – she cut the shoes in order to slip the shoes on and off quickly as she entered the house. Consequently, they designed shoes with zippers at the heel or an elasticated back.

  • Extreme consumers unlock insights and provide inspiration [is an extreme consumer someone who doesn’t care about fads or what their friends do and does what they individually like and want?]
  • 3 types of extreme consumers, not mutually exclusive [a point no one ever makes about their segmentation]
  • Type 1 – extreme users or non-users – lovers or rejectors, don’t screen these people out [people doing your research will love you!] For example, bare foot funners for a sneaker brand, why did they stop wearing runners, what are the benefits of running barefoot?
  • Type 2 – the expert user – Their role in life gives them an affinity of your brand, they may not use or be aware of your product. Soldiers are experts in armaments and warfare, but also in standing around for a long time, taking long hikes in horrid footwear, blisters all the time. Women wear high heel shoes [i refuse to call 4 inch heels shoes, those are decorations!].
  • Type 3 – leading edge creative consumer – at the forefront of trends, passion and imagination to co-create solutions. These are the people they want for the pepsi now network.
  • Built a community of 50 people around the world. Wanted a community that was an extension of their team. Use these folks at the beginning of the process, not at the end to check if your work was crap or not.
  • Let them advise on music, storyboarding, casting, packaging before you even start building the advert.
  • They bring the community offline to brainstorm and problem solve one on one with the marketing team. They actually pitted the marketing team against the community.  The community had a very different perspective than the marketing team. They recognized a disconnect and it disrupted the agenda for the rest of the day.
  • Took two years to do and they failed multiple times along the way.
  • Extreme consumers live in the future, they are canaries in the coalmine. They represent the aspirationso fthe mainstream.
  • [How do you innovate a drink, pepsi? Honestly, I’d like to know. More sugar, less sugar? Bigger/smaller can? But I do very much appreciate the message.]
  • [You’re over time. I think you owe everyone a mooncake!

The Way of Insight Beyond Technique: Creating an insights culture to inspire transformation by Melissa Dagless, Shionogi Limited, UK, Takashi Takenoshita, Shionogi Limited, UK, Vivek Banerji, Insight Dojo, UK

  • Do = The Way
  • Waza = Technique as art
  • An insights culture creates better business decisions, inspires people and makes them happy
  • Had a new menopause prescription drug that wouldn’t be free for people. Price was a big barrier for the doctors
  • 5 practices to follow in insights projects
  • 1) Receptive mastery – Picasso drew a simple bull after many drafts, it looks easy even though it required a lot of skill. In this example, women are afraid of hormones. The impasse prevented adoption. [LOVE the video which used an effect to turn real video into line drawings thus masking the people]
  • 2) Co-creation – it’s not insights work, it’s starting at the very beginning, at the time of the initial decision, in their case, they role-played doctors
  • 3) Mindfulness – medication is becoming mainstream, we love words like empathy, observation, sensitivity, immersion [shout out to Irrational Agency] – Cognitive, Emotional, Concern.
  • 4) Pick ideas from any industry not just social sciences. Why not reduce pages and pages of charts
  • 5) Strategy – You can debate whether Steve Jobs used research of not but he sure did use strategy
  • Barriers –
    • outside the job description,
    • lack of conviction, lack of skills knowledge or conviction,
    • politics and heirarchy,
    • band wagon effect
  • Need to define vision and values – Essence, values, promise, and brand personality
  • “The WOW book” – ways of working [I’d love to see their book!]

  • One rule – no ice cream in the back room – i.e., why are you fooling around behind the mirror when your focus group people are spilling their guts out? This is not a time to taste the m&ms. Pay attention.

Research That Sparks: Methods to make market research more inspirational by Annelies Verhaeghe, InSites Consulting, Belgium, Natalie Malevsky, Telefónica Digital, UK, Thijs Van de Broek, InSites Consulting, Belgium

  • Fastest rise in communications has been text based, but the future is visual.
  • Impact of communication: 7% is words, 38% is tone of voice, 55% is facial expressions
  • Set up a consumer consulting board
  • Inspirational research does not give answers
  • Privacy was discovered to be an important topic. They did not give answers, they turned this into questions. How do you want to be perceived in terms of privacy?
  • Consumers have a hard time imagining their future – why would you adopt video communications, can you answer that? [easy, instant easy always working access. that would do it]
  • Barriers – if I watch tv virtually with my friend, do i watch the TV or do I watch the game on TV

Insights to Bring Brands A.L.I.V.E.: The challenge of generating and leveraging insights the Pernod Ricard way by Florence Rainsard, Pernod Ricard, France, Kim Gaspar, Pernod Ricard, France, Mark Whiting, Added Value, France, Nathalie De Rochechouart, Pernod Ricard, France

  • Aim, Learn, Insight, Voice, Energize – Insight brings passion brands to life
  • Pernod Ricard – Share our products with your friends, it’s not just bottles and liquids, it’s part of everyone’s life, every party, alcohol is an integral part of parties and life, “Make a friend a day”  [makes me wonder – does anyone market to introverts? — enjoy this chocolate by yourself, don’t share with anyone]
  • [Sigh, this focus on alcohol being the only way to enjoy life is disappointing. Is it not possible to be happy and have fun other ways?]
  • Entify brand – people often treat brands as human beings that care about them
  • ALIVE gives people an insights tool [wow, everyone’s got an acronym for their ‘unique’ process now]
  • Scoping, Consultation, Prototyping, Trial and review, Apply
  • [Apologies for the less than stellar blogging, busy arguing over newbies in #MRX on twitter]

Screener keeners or rejection correction?

[tweetmeme source=”lovestats” only_single=false]How much is too much? How much is too little? There are lots of things in market research that require a healthy balance between doing the right thing and conducting business. Deciding how many screeners to offer to potential survey responders is one of them.

Most survey panels recognize that screening people out of surveys, no matter why, is bad for two reasons. First, and completely justified, it ticks off panelists who feel their time has been wasted and their opinions ignored. Second, it’s a waste for panels that just used up one survey in their data quality rule of “one survey invite per week” and they didn’t even get a complete in return.

For both of these reasons, many panels strive to handle the problem by offering up a number of surveys in a row to panelists. Panelists receive an invite and then proceed through one or more consecutive screeners until they qualify for a survey. (Let’s not consider what this means for probability sampling.)

But what is the right number of screeners? Is it ok to send someone through ten minutes of screeners? Is it ok to give them two or three screeners?

Photo credit: xenia from morguefile.com

I just spoke with someone who said their company takes people through up to five screeners before they say enough is enough. Panelists are even compensated for each screener they complete. I worry that even though they are being compensated, it is annoying to panelists. Screeners are obviously not surveys. Panelists can tell that they’ve been rejected once, twice, three, four, and five times. Imagine being rejected by five screeners every time you try to participate. It’s just one more source of rejection, something none of us need now or ever.

In fact, I even wonder if there is a rejection effect for which I have a two tailed hypothesis. Does increased rejection cause decreased survey scores due to the annoyance or does increased rejection cause increased survey scores due to the satisfaction of finally getting a survey to answer. I’d love an answer to that!

So what does your experience tell you? Are responders keeners for screeners?

Read these too

  • #MRA_FOC #MRX Karaoke by Jim Longo. I dare you.
  • If Frozen Meat Isn’t a Bribe, What About Raw Meat?
  • It’s Here! Twitterversity is Here! #MRXU
  • Proof that sampling social media data is critical, but you already knew that
  • If you DARE – the chart test of death
  • I’m Told I Have No Opinion

    Marketing Research Association

    Image via Wikipedia

    I love Merrell shoes. The sole is like walking on sponges and the designs is really cool. I love Kitchenaid appliances because they look good, they are sturdy, and baking is so much easier. I love Tall Girl clothing because the pants are just the right length and the price is right if you get the sales. I could bore you profusely about even more merits and downfalls of each but unfortunately, I am not permitted to have an opinion of them.

    You’ve seen those screener questions. “Do you or any member of your household work in the marketing research industry?” Why is this question so important? Why can’t I answer the survey if I do work in the industry?

    Perhaps I am going to steal confidential questions.
    Perhaps I am going to try to skew data because I have a competitive client.
    Perhaps my opinions will be biased because I understand the purposes of the questions.

    Well first, various research codes say I must behave ethically which means no stealing and no biasing.
    Second, as I rush through the survey like everyone else who is disgusted with low quality questions and bored with ridiculous questions, I am certainly not paying any more attention to the purposes of the questions than anyone else.
    And third, why aren’t my opinions valid? Doesn’t Merrel and Kitchenaid and Tall Girl want to know what one of their most loyal customers thinks? Even worse, since my spouse lives in the same household as someone who works in the marketing research industry, why aren’t his opinions valid? He’s never even heard of a likert scale and would probably stab himself in the heart if I tried to explain it to him.

    Besides, if I truly am going to lie and cheat and bias answers, why would I ever confess that I work in market research so please don’t show me your survey?

    And if you’re curious, I DO answer competitive surveys (just like you do). My rule is answer every question, except the screener, completely honestly or don’t finish the survey. Just how I’d want to be treated.


    Related Posts

    Enhanced by Zemanta

    Why do surveys ask the same question 8 billion times? #MRX

    1) Bad survey design

    This is the one that is most annoying. Let’s imagine you started answering a survey. You gave your age and gender, possibly even other details like your level of education or your income. Then, you get horrible message, “Sorry but you do not qualify for this survey, but here is another survey for you.” And then you are presented with the age and gender question again. And possibly again. Any possibly again. What is going on here? Why do you have to answer it so many times? Couldn’t they just remember the question from the first time?

    Well, i wish that was how it worked. Unfortunately, the problem comes down to pure business. Every client has their own unique survey. They may have spent weeks or months building, designing, creating a survey that fits their very specific needs. Even the age and gender questions, as common as they are, have been carefully crafted just for them. (Let’s forget about the quality of the crafting for right now.) When the client actually fields the survey, the survey company doesn’t actually own the survey. They can’t “steal” the age and gender data from one survey and apply it to another survey. Even when it appears that the questions are identical. It’s much like copyright rules for music and art. Just because I can see it and hear it, doesn’t mean i can have it.

    There is a solution though. Survey companies should have one or two standard demographic questions that clients must choose from. Clients must also agree that they do not own the data from those questions, but they may have access to them. Given all of the challenges faced by survey clients and companies, this is an easy problem to solve. I hope they do it soon.


    2) Good survey design

    You’ve also seen those long questions that seem to ask the same thing over and over again. “Do you like product A?” “Do you love product A?” Do you hate product A?” “Do you dislike product A?” For most people, it’s just so darn repetitive. If you think about it carefully though, these questions are not the same. Think about how YOU interpret each of the words. You probably have slightly different definitions than everyone else. Just slightly enough that across a series of questions, you would probably answer them differently from someone else who said they also liked the product. And, across ten questions, all of which seem identical, your total score is probably different from their total score, perhaps even different by a point or two. This is why researchers bore you with so many questions.

    Again, though, there is a solution. In many cases, surveys include far more ‘repetitive’ questions than are needed. Usually, 8 to 10 questions are more than enough to create a wide range of scores among people. Thirty questions? Well, that researcher was just lazy and didn’t have to worry about field costs.

    There are some other very legitimate reasons for asking the same question 8 billion times, but you’ve probably got the point now. Now go answer a survey! Take a survey now!

    Related Articles

    Why Do I Always Screen Out of Surveys

    Books about survey research and survey design.

    Image via Wikipedia

    If you are a regular survey participant, you’ve probably experienced this. You start answering a survey, and before you get any real questions, you’re told…
    Most people don’t like to be screened out of surveys. You started answering the survey because you wanted to finish it. So why does this happen? There are both good and bad reasons for this.
    Sometimes when researchers expect that 100 twenty year old women will complete the survey, they find out that 150 twenty year old women want to complete it. And other times, when they expect to get 100 participants, they only get 50 participants. Then they’re in trouble because they have to find an additional 50 people and still meet the client’s timelines.
    Estimating how many people will want to answer a survey is a tough thing. Researchers know in a general sense how many people will complete a survey. They predict it based on the age, gender, and other demographics of people they sent the survey to. They also predict it based on the topic of the survey and the length of the survey. In the end though, it is just an estimate and sometimes, the number is over or underestimated.
    Never forget that researchers value your participation in surveys. So, when you see that screen-out question, view it as the researcher treating you with respect. They don’t want you to spend your time answering a survey when your opinions won’t be used. Why? Because, as soon as that survey is answered by the number of people required, the data is downloaded and quickly shipped off for analysis. It’s a busy world built on ‘give it to me yesterday’ and marketing research is in the same boat as every other business. I certainly wouldn’t want to spend 30 minutes answering a survey only to find out that my results weren’t downloaded and analyzed.
    This isn’t to say that this is a good or right way to do things though. Most survey companies have many, maybe even hundreds, of surveys in the loop. Wouldn’t it be nice if you never screened out? If you were automatically sent to the next survey which was relevant to you, and you didn’t even see the transition? Ah, can’t wait for that technology to get there for everyone! It’ll come, just not soon enough for me.

    Related Articles

    %d bloggers like this: