Tag Archives: do it yourself

It’s a dog eat DIY world at the #AMSRS 2015 National Conference

  What started out as a summary of the conference turned into an entirely different post – DIY surveys. You’ll just have to wait for my summary then!

My understanding is that this was the first time SurveyMonkey spoke at an #AMSRS conference. It resulted in what seemed to be perceived by the audience as a controversial question and it was asked in an antagonistic way – what does SurveyMonkey intend to do about the quality of surveys prepared by nonprofessionals. This is a question with a multi-faceted answer.

First of all, let me begin by reminding everyone that out of all the surveys prepared by professional, fully-trained survey researchers, most of those surveys incorporate at least a couple of bad questions. Positively keyed grids abound, long grids abound, poorly worded and leading questions abound, overly lengthly surveys abound. For all of our concerns about amateurs writing surveys, I sometimes feel as though the pot is calling the kettle black.

But really, this isn’t a SurveyMonkey question at all. This is a DIY question. And it isn’t a controversial question at all. The DIY issue has been raised for a few years at North American conferences. It’s an issue with which every industry must deal. Taxis are dealing with Uber. Hotels are dealing with AirBnB. Electricians, painters, and lawn care services in my neighbourhood are dealing with me. Naturally, my electrical and painting work isn’t up to snuff with the professionals and I’m okay with that. But my lawn care services go above and beyond what the professionals can do. I am better than the so-called experts in this area. Basically, I am the master of my own domain – I decide for myself who will do the jobs I need doing. I won’t tell you who will do the jobs at your home and you won’t tell me who will do my jobs. Let me reassure you, I don’t plan to do any home surgery.

You can look at this from another point of view as well. If the electricians and painters did their job extremely well, extremely conveniently, and at a fair price, I would most certainly hire the pros. And the same goes for survey companies. If we worked within our potential clients’ schedules, with excellent quality, with excellent outcomes, and with excellent prices, potential clients who didn’t have solid research skills wouldn’t bother to do the research themselves. We, survey researchers, have created an environment where potential clients do not see the value in what we do. Perhaps we’ve let them down in the past, perhaps our colleagues have let them down in the past. 

And of course, there’s another aspect to the DIY industry. For every client who does their own research work, no matter how skilled and experienced they are, that’s one less job you will get hired to do. I often wonder how much concern over DIY is simply the fear of lost business. In this sense, I see it as a re-organization of jobs. If research companies lose jobs to companies using DIY, then those DIY company will need to hire more researchers. The jobs are still there, they’re just in different places. 

But to get back to the heart of the question, what should DIY companies do to protect the quality of the work, to protect their industry, when do-it-yourselfers insist on DIY? Well, DIY companies can offer help in many forms. Webinars, blog posts, and white papers are great ways to share knowledge about survey writing and analysis. Question and survey templates make it really easy for newbies to write better surveys. And why not offer personalized survey advice from a professional. There are many things that DIY companies can do and already do.

Better yet, what should non-DIY companies do? A better job, that’s what. Write awesome surveys, not satisfactory surveys. Write awesome reports, not sufficient reports. Give awesome presentations, not acceptable presentations. Be prompt, quick, and flexible, and don’t drag clients from person to person over days and weeks. When potential clients see the value that professional services provide, DIY won’t even come to mind.

And of course, what should research associations do? Advocate for the industry. Show Joe nonresearcher what they miss out on by not hiring a professional. Create guidelines and standards to which DIY companies can aspire and prove themselves. 

It’s a DIY world out there. Get on board or be very, very worried.

DISH Network’s Research Culture Shift by Patti Fries, DISH Network#CRC2014 #MRX

CRC_brochure2013Live blogging from the Corporate Researchers Conference in Chicago. Any errors or bad jokes are my own.

DISH Network’s Research Culture Shift by Patti Fries, DISH Network

  • dish network actually has more revenue than Starbucks. but the brand value of Starbucks is twice as much
  • ask consumers – If dish was a car, what kind of car would it be – grey car with four wheels – perception is everything
  • describe yourself in five words – Charlie Ergen founder of dish was adventurous, competitive, risk taker, tenacious, optimistic – he put 100 million dollars to put a satellite into space, 40% success rate on rocket launches in china in 1995 [please don’t blow up, please don’t blow up, please don’t blow up]
  • First rocket in 1995
  • Consumer research and insights launched in 2012
  • why spend one more penny on research than you had to, they couldn’t keep up with sales
  • now, need to focus efforts on where going long term because its turning into a commodity
  • speakers five words: energetic, passionate, ambitious, trusting, optimistic
  • Research history – it had spartan beginnings, they did do roundtables, talk to people at the DMV, it wasn’t structured, they did DIY surveys, people weren’t sharing their research results, no one knew what else was happening in the company
  • department stores generate NPS of 62. Cable services generate NPS of 14.
  • where would you rather work? a place that gives you opportunity to improve perceptions of a brand.
  • “Our culture works hard to prevent change” ~Seth Godin
  • How do  you bring research in when nobody cared about it?
  • Culture shift: beginning, who, data, education, brand, future
  • Beginning – do what it takes, she did 72 projects by herself in the first 7 months, she coded open-end data herself, did the tabulation, analyses, reports, presented. She had to build awareness and get people to want what she was selling. she wanted a 1000% budget increase. Biggest increase they’ve ever given. Got permission to build a small and mighty team. Develop vision and mission – objective, simple, strategic, inspiring
  • Who – the target was 18+ with a pulse. we target everyone and we target them all the exact same way. Did a household level segmentation with financial information. Lower income doesn’t mean lower interest in TV. Needed to do needs based segmentation too. How do you marry lower income with their services. Worked on predictive models to make a better experience.
  • Data – Less is more. Reports are not based on $$ value per page. would rather two pages of actionable data than 400 pages of numbers we can’t understand. focus on things that will have the greatest impact. need to find a way to move fast. have 14 million subscribers, most have 2 set-top boxes, and they can monitor every click but don’t worry because they are doing zip all with it! zillions of data points not being used. But when you use this data, you need to get it down to one single page of actionable results. 35 suppliers bug her everyday to work with her but she only works with 6.  [how exhausting is that]
  • Education – have you ever been to a focus group? they had to teach people how to do it. they put hundreds of their marketers through focus groups in two days. these people had never heard the voice of the consumer. told consumers they could write a letter to the CEO but then they brought the CEO to the focus groups to talk to them. he talked to them for an hour. there was difficulty getting traction. people weren’t necessarily interested in learning more. chose to empower others by teaching them qualitative tools. had them go in the homes of consumers to see they could do some of their own work. Make it fun, interactive sessions.
  • Brand – make brave recommendations. price focus has made them a commodity. consumers were seeing message madness – 25 completely different commercials. Brand awkwardness – weird commercials that don’t make sense like the cowboy hat commercial. Brand wasn’t building while competitors were growing. Worked to build partners with apple, southwest – that’s better than a grey car. Now you can talk about value. Moved their NPS score up to 38 and the hopper NPS to to 43. Won lots of awards but consumers don’t care about features than won awards.
  • Future – ended up winning the hearts and minds of consumers. awareness is fine but brand advocacy takes your business to the next level. Create a brand feel and make it different than a grey car with four wheels. Build on consumer motivations. It’s more than price and features.  Elevate collaboration. Met with the C-suite to push their thinking, got everyone in the same room so that everyone would be on board. Came up with 972 ideas to change the trajectory of the company and only 4 had anything to do with price.
  • If you’re going to take a side, take the side of the consumer. an exciting category shouldn’t feel horrific to consumers.

What’s Beyond Mobile? New MR: General Mills Goes All In by Ryan Backer, General Mills #CRC2014 #MRX #GreatTalk

CRC_brochure2013Live blogging from the Corporate Researchers Conference in Chicago. Any errors or bad jokes are my own.

What’s Beyond Mobile? New MR: General Mills Goes All In by Ryan Backer, General Mills

  • 2011 saw opportunity for mobile technology; 2013 felt confident they’d figured out mobile; 2014 formed emerging technologies team
  • 2011, they bet big. spent a year trying to get smart. convinced that by 2014 80% of research would be mobile.
  • Believers because 1) they felt they had to, mobile is surpassing desktop, default access point for all things internet, even surveys designed for desktops, particularly for multi-culturals and millennials, attention spans keep shrinking 2) pride themselves on bringing innovation to the marketplace, love idea of getting richer insights using bells and whistles of the phone – location, audio, visual, in the moment qualitative, feedback WHILE it’s happening 3) it felt right, traditional research feels ancient, we can now ask people to take a picture of their snack drawer and then see products they would never imagine sitting right beside your product, have people take pictures in the stores of how their product is always out of stock, it felt more natural in terms of ethnographic research
  • How did it go? GEMO – good enough, move on. there are plenty of warts but let’s move on
  • They heavied up and sold hard – did a lot of hand holding for any research project that seemed to work best with mobile so that every person had a positive experience with it. Had major meetings with large teams to showcase the mobile research so that everyone could see they could do mobile research too
  • They redefined best practice.
    • Ethnography – Not limited to certain geography, more honest feedback, it’s not a stranger interviewing you, you’re in your own home
    • Shopper satisfaction – Do it while you’re still in the store, not the next day after you’ve forgotten everything
    • IHUT – save on shipping costs, send them to store to buy something, ask questions about finding the product, comparing the products
    • Instant A&U – increased speed because everyone has phone with them at all times, can push a survey at any time
  • Invented new capabilities – answered questions they’ve never been able to get answered before, ask questions they’ve never had before
    • Mobile Missions: send people on a task
    • Geo-intercepts: without physically being in a store
    • LaunchWatch: get results on brand new products immediately, real time basket analyses
    • OTC ad-tests – typically in a controled environment but now they can do it “On The Couch,” as ads are experienced for the first time while you’re on your couch
  • Mistakes along the way
    • Lift and Shift ROR – You can’t just put a standard survey on a phone, know when to put audio video so that the research is done well not just somewhere else, know that results will be different because the method and timing and location is different
    • Multimedia paralysis – how do you handle hundreds and thousands of visuals, it’s very time consuming
    • New Method Flops – They never turn work away, but sometimes it just doesn’t work
    • DIY Scissors – They are many DIY solutions, anyone thinks they can do research and it results in “Running with scissors” research
  • Abandoned the mobile team now, but no longer have an in-house team, it’s just regular research now
  • Next is still mobile but also other things
  • We’ve reached the tipping point. Stop talking about mobile and start talking about device agnostic.
  • Close to 80% now, but not by dollars
  • They have a solid rolodex of suppliers and now recommend those suppliers to their teams
  • Know your barriers
  • data stitching is not easy, gamification is on their list, still need true full service – know research AND technology AND storytelling
  • Exponential change
    • Internet of things – connect all device and eeverything in between
    • quantified self – behavioral is now diary but we’re going way beyond that
    • augmented reality and virtual reality – could be THE next big thing
    • 3D printing – especially in food industry
    • artificial intelligence – natural language interpretation, unstructured data interpretation
    • robotics
  • looking for new applications of mobile, new data streams, and new technology/gadgets
    • Geofence consumers to within 4 inches when they are in front of a box of cheerios, how much time in front of those cheerios, build your own granola bar game
    • data streams previously unavailable – fitbit, wearables, consuming calories, how you burn calories
    • fridge with internet, clip on your shirt and takes a pictures every 30 seconds [need one!], coupons and menu ideas direct to your fridge monitor

The when and why of DIY #MRX

334 - Mr. Fixit's

At the CASRO online research conference, one of the panels focused on DIY research and included a couple panel providers and a couple DIY companies. This panel was of particular interest to me because I’ve watched how DIY has taken quite a pounding in the last few years but for the wrong reasons.

I asked the panel about when qualified researchers should use Do It Yourself research and the answers included when you need results better, faster, and cheaper. Ok, a very generic and unhelpful response.

Then, another audience member asked when should DIY research NOT be used. Sadly, the panel could not offer a single idea about when DIY research was not appropriate. Given that one of my mantras is “Every research method has pros and cons,” this was a completely disatisfactory answer. And misleading.

So here is my opinion on when DIY should and should NOT be used.

DO use DIY research when:

  • A qualified researcher has written the survey and designed the methodology
  • The survey is very simple, short, and contains no complicated skip patterns 
  • You need results extremely quickly
  • You have the population of target responders and ‘random’ sampling is not necessarily required
  • You understand statistics well enough to know when sample sizes are too small, when to use a t-test or a chi-sqare, and why MOE is under hot debate
  • You need to test a simple methodological issue prior to launching the full study (e.g., will the distribution of responses be better served with a 5 point or 3 point scale)

Do NOT use DIY research when:

  • You don’t need anyone to proof read your survey because you never make mistakes
  • An expert in survey design has not created the survey
  • An expert in sampling/weighting has not developed and implemented the sampling plan
  • An expert in data analysis will not be analyzing the results
  • (so assuming that a qualified researcher is managing all aspects of the research…)
  • You are running a complicated  tracker – weekly/monthly, complex balancing, multi-country
  • Your survey incorporates multiple and varied skip and logic patterns
  • You require complicated census/target balancing and weighting

It’s a pretty easy answer.

Plumbers plumb.

Engineers engineer.

Researchers research.

Twesearch is the new DIY research tool

Image representing Twitter as depicted in Crun...

Image via CrunchBase

In the research user world, there are those who believe in writing and analyzing their own surveys and there are those who believe that experts should write and analyze surveys for them. I’m probably biased, but I believe that survey writing is such a precise and difficult skill, that you should always have an expert write for you.

Along the same premise, I can see DIY twesearch (yeah, twitter research) becoming a hot topic. This one scares me to the same degree as DIY surveys. I think it’s great that brands can do quick searches of their brand name and find out what consumers are chatting about. But, I worry that the untrained human brain isn’t sufficiently non-biased to be able to monitor and interpret the findings fairly.

Know how you can’t find typos in things you’ve written yourself? Heard of the self-fulfilling prophesy? I know just how easy it is for someone to accidentally miss what they don’t want to see and find everything they do want to see even if it isn’t there. The untrained researcher isn’t always equipped with the skills to prepare a comprehensive research process that will give fair and reliable results.

So by all means, do some DIY twesearch. But when you’re ready turn that chatter into business decisions, make sure you bring in some research pros.

Christmas‘ by kevindooley via Flickr
Image is licenced under a Creative Commons Attribution licence

Related Posts


  • This is why Twitter will die
  • Only crazy people are on twitter
  • Mugging, Sugging and now Rugging: I take a hard stance on privacy
  • I’m sorry but representative samples are 100% unattainable
  • What’s Gonna Kill You? An Infographic That Actually Works #MRX
  • %d bloggers like this: