Tag Archives: Roddy Knowles

How a Mobile-Enabled World is Changing Research Presented by Roddy Knowles, Director of Mobile Research #CRC2014 #MRX

CRC_brochure2013Live blogging from the Corporate Researchers Conference in Chicago. Any errors or bad jokes are my own.

How a Mobile-Enabled World is Changing Research
Presented by Roddy Knowles, Director of Mobile Research

  • “This is the year of mobile” we say this every year
  • what steps do you take for mobile friendly design
  • 1 in 5 survey starts on a mobile device; 2 in 5 panel enrollments is on a mobile device – this is a 100% increase over last year
  • over time, people realize that surveys aren’t always designed for mobile devices
  • meet responders when and how they prefer – at home, work, on the bus
  • mobile reminds us that real paper take our surveys, we lose the human element sometimes
  • Mobile helps with feasibility, data quality, and representativeness
  • Data quality
    • bias towards visible answer choices
    • scale biases
    • count biases – few choices selected on a long list
    • straightlining – mitigated by good design
    • you need to test your specific situation to be aware of potential problems with your survey
  • Data comparability
    • data generally comparable
    • [remember – even if you give the same survey to the same people just one day apart, the data will be different]
    • excluding mobile people from a desktop survey means the data will be less representative, less tech savvy people, fewer early adopters, fewer shopping-centric people, certain tech occupations excluded
  • Best practices
    • avoid wide grid qrid on a smartphone – people still do this!
    • responsive design is not a large font grid on a smartphone
    • keep it short, try for ten minutes
    • use fewer answer options where possible
    • aim for a 5 point scale
    • make sure all scale points are visible without scrolling
    • allow “fat finger” responses on a phone, tiny radio buttons mean you will hit the wrong button
    • avoid need to scroll, pinch, and zoom
    • open ends are shorter but ask the questions well – don’t ask for a novel, ask for a succinct response
    • you can use audio/visual but test it first. if people can’t see the video your data will be poor quality
    • don’t use flash
    • use responsive design – PROPERLY, make sure text size is good
  • they’ve created a scoring system to show four buckets – mobile optimized (you might get a hand-written thank you note if you score this high), mobile friendly, mobile possible, mobile incompatible
  • let’s not torture panelists
  • not every survey is designed to be a mobile survey so don’t do it if it’s not
  • response rates have doubled, dropouts have dropped, fewer reminders, more efficient [impressive!]

Mission Possible: Innovative Solutions to Challenging Project Briefs Presented by Roddy Knowles, Director of Mobile Research #CRC2014 #MRX

CRC_brochure2013Live blogging from the Corporate Researchers Conference in Chicago. Any errors or bad jokes are my own.

Mission Possible: Innovative Solutions to Challenging Project Briefs
Presented by Roddy Knowles, Director of Mobile Research

  • Mission: Invade personal space – What is the relationship between toilet paper and flushing issues – we need you to capture in the moment usage of toilet paper
  • Mobile survey was easy, quick – how many sheets, are you folding/wiping/crumpling, how many times do you wipe?
  • how do you make a survey mobile friendly?
    • break it up into chunks – served 10 questions at a time
    • did a side by side of chunks vs full survey to compare
    • initially difficult to set up, but faster in field, resulted in better member experience, provided better data, challenge made us stronger because we realized we could handle the process
  • IMG_3533[1]Mission: Tag, track and talk
    • measure shopper at 3 points in shopper journey – pre shop planning, in-store, making/consuming product
    • difficult to do without mobile
    • in-store researcher or facility testing was not natural and not preferred
    • first profiled respondents and then recruited the right people, surveyed about planning for shopping and then sent them to the store. they didn’t know what product was going to be assigned to them
    • reimbursed people for the product
    • mobile IHUT validated with phones, added depth with open-ends and media
    • let them see how people use the product differently than expected
  • Mission: understand categories of websites while shopping impacts purchase decisions
    • monitor shoppers full digital footprint
    • metered panelist on mobile phone and computer
    • looked at app usage on smart phone and social media listening
    • let them see what else they are doing at the same time – email, games, while they’re occupied with the product
  • IMG_3534[1]Mission: Unicorn hunting
    • find people who are at the very very beginning of a purchase decision – needle in a haystack
    • follow entire process and see when it does or doesn’t end up in a purchse
    • used metered panel – computer and mobile
    • specifically monitored auto websites, bluebook
    • opt in for GPS tracking, matched up to dealership locations
    • surveyed during and after, and really try to get right at the point of the experience instead of relying on memory
  • Mission: Track Santa Claus
    • study over the last two years, track data around black friday, cyber monday, christmas, new years
    • used metered panel again, which websites, which phone apps, which stores they visit
    • used a mobile diary, had people check in with them while they were shopping, a short survey about categories, basket spend, receipts from shoppers
    • mobile is starting to look more like PC, see bumps around the exact holidays
    • many stores opened extremely early on these holidays but the data showed that people did go earlier and there were spikes in traffic at the early time slots. the people going really early spend more
  • integration is key, this is how you meet objectives in creative way

Holiday Shopping with All Screens 24/7 by Maria Domoslawska, and Roddy Knowles #FOCI14 #MRX

Live blogging from the #FOCI14 conference in Universal City. Any errors or bad jokes are my own.foci14

Holiday Shopping with All Screens 24/7
Maria Domoslawska and Roddy Knowles, RESEARCH NOW

  • multimode starting with online, going to metered PCs, mobile phones, and then mobile diary GPS tracking which adds more engagement and gets us closer to the consumer
  • sample sizes of thousands, conducted around Thanksgiving
  • partnered with Experian for Hitwise data, tracked the top 20 retailers
  • 3% of people used mobile for purchasing but we will see mobile did play a central role
  • saw spikes for black friday and cyber monday on PC traffic
  • even though people aren’t necessarily shopping on their phone, the phone is with them at all times, they use their phones while they shop a lot more than they used to
  • 40% of consumers say they LIKE the huge crowds in the UK, and plan to do this
  • 31% of people made only unplanned purchases, impulse
  • 26% made planned and unplanned purchases
  • purchase planning by store – more unplanned at CJP, Macy’s, Kohl’s – clothing stores
  • in the UK, they plan to buy after Christmas, they give gifts on boxing day because they know they can get a better deal
  • more planned purchases for bigger ticket items like appliances and electronics
  • flowers and small gifts were mostly unplanned
  • ask people to take photos of their receipt – would people actually do it? and are panelists buying what they said they bought – did they forget something?
  • used the mobile diary to determine if people could actually find what they were looking for
  • were able to capture basket spend via diary data and determine average expenditure by store, matched well with responder data so people were able to recall quite accurately
  • 74% say gift giving makes them feel good
  • Key takeaways:
    • multi-country tracking help find subtle changes in your market which can become a big creative idea for your next program
    • mobile is in shopper’s lives and will not go away. it is an essential device for shopping even if they aren’t actually purchasing on their device
    • getting close to the point of experience yields rich and accurate data on shopping behaviours
    • not every datapoint is going to add value but layering survey data with new variables of passive behavioural data can activate your hypothesis

Other Posts

A “How-To” Session on Modularizing a Live Survey for Mobile Optimization by Chris Neal and Roddy Knowles #FOCI14 #MRX

Live blogging from the #FOCI14 conference in Universal City. Any errors or bad jokes are my own.foci14

A “How-To” Session on Modularizing a Live Survey for Mobile Optimization
& Roddy Knowles, RESEARCH NOW

  • conducted a modularized survey for smartphone survey takers, studied hotels for personal travel and tablets for personal use, excluded tablet takers to keep the methodology clean
  • people don’t want to answer a 20 minute survey on a phone but clients have projects that legitimately need 20 minutes of answers
  • data balanced and weighted to census
  • age was the  biggest phone vs computer difference
  • kept survey to 5 minutes, asked no open ended questions, minimize the word count, break grids into individual questions to avoid burden of scrolling and hitting a tiny button with a giant finger
  • avoid using a brand logo even though you really want to. space is at a premium
  • avoid flash on your surveys, avoid images and watermarks, avoid rich media even though it’s way cool – they don’t always work well on every phone
  • data with more variability is easier to impute – continuous works great, scale variables work great, 3 ordinal groups doesn’t work so well, nominal doesn’t work so well at all
  • long answer options lists are more challenging – vertical scrolling on a smartphone is difficult, affects how many options responders choose, ease of fewer clicks often wins out
  • branching is not your friend. if you must branch, have the survey programmers account for the missing data ahead of time, impute all the top level variables and avoid imputing the bottom level branched variables
  • Predictive mean matching works better than simply using a regression model to replace missing data
  • hot decking (or data stitching which combines several people into one)  replaces missing data with that from someone who looks the same, worked really well though answers to “other” or “none of the above” didn’t work as well
  • hot decking works better if you have nominal data
  • good to have a set of data that EVERYONE answers
  • smartphone survey takers aren’t going away, we need to reach people on their own terms, we cannot force people into our terms
  • we have lots of good tools and don’t need to reinvent the wheel. [i.e., write shorter surveys gosh darn it!!!]

Other Posts

How Mobile Stacks up to Online Surveys by Roddy Knowles, Research Now #MRA_National #MRX

… Live blogging from Disney Orlando, any errors are my own…

How Mobile Stacks Up to Traditional Online: A Comparison of Studies

Steve von Bevern, VP, Client Services and Operations, Research Now Mobile (presented by Roddy Knowles)


  • Today’s path to purchase is more complex than ever
  • People use mobile for discover, evaluation, buying, accessing, using, getting support, and much more
  • Mobile users multi-task, 51% listen to music, 52% watch TV, 43% use internet, 28% play video games, 17% read a book 16% read newspaper/magazine
  • What is mobile – nott just phones (dumb phones), it’s tablets that can display surveys
  •  1.4 billion smartphones! 268 million tablets in active use!!
  • Who uses devices? Similar age and gender to census though slightly more younger, slightly less older; same ethnicities as census, similar income as census though slightly
  • Mobile open ends are richer – longer responses and more of these longer responses [do 99% of people provide a survey answer because they are forced to? and the provide a crap answer as a result?]
  • Do we get the same answers both ways? In this case study, when weighted back to census, results were very similar, sport opinions were similar, smoking opinions were similar
  • How do responders feel about it? Older responders feel it isn’t as easy on mobile, on average people don’t find mobile more difficult if the survey is properly designed
  • How fun is the survey?  Mobile is seen as more fun than online, online surveys are no longer new and interesting, maybe this will change over time as people become used to mobile surveys
  • Mobile responders prefer mobile surveys overwhelming. Get people where they want to be
  • Image representing e-Rewards as depicted in Cr...Like online, you must target to ensure mobile and online deliver representative data.
  • Online or mobile are not the solution for every audience. You must choose the method that’s right for the situation.
  • Don’t rely only on one method or you will miss people who prefer other methods
  • Mobile allows rich media uploads, seamless option for people
  • In the moment surveys work well too, get close to the point of experience and a computer isn’t always the right way to do it, don’t rely on imperfect memory, a 5 minute survey while they are at breakfast can work well; we know memory is fallible, will you remember everything from this conference next week?
  • Geolocated surveys are an advantage. Target people entering a store, determine who walked past a store  [I always turn off my geolocation though I know my signals are tracked by my phone provider]
  • Home ethnography – 97% scanned barcodes, 99% uploaded pictures, 82% uploaded audio, 80% of people with pets uploaded videos
  • Sporting event two day diary – 69% completed two diaries, 62% uploaded images, 19% uploaded video, very detailed openends, high level of engagement
  • Holistic Insights – Behaviour data like app downloads, music played plus survey data result in deeper insights
  • Retail surveys – good way to get shopper feedback on in-store displays, respondents can go in and upload photos of instore displays and scan barcodes; geofencing means the survey is only available when the responder is in the store ensuring strong validity
  • Mobile is not online and online is not mobile. You can’t just use them in place of each other.
  • Define – who you want to reach, consider multi-mode to ensure broad representation
  • Dive deep – be creative, enhance open-ends with audio and video, use rich media when it’s fit for purpose, let people write it out or audio record as they wish
  • Design – be pragmatic and make the most of respondent time and screen real estate, streamline and simplify, know which questions work and don’t work, use multiple points of engagement, think like a respondent
  • When you’re doing mobile, don’t ask people to constantly scroll, don’t ask them to rank 15 options
  • Stick to 7 responses so they don’t have to scroll, limit “please specify” to where you really need it, eliminate superfluous words and phrases
  • Don’t use lots of cute and colors and fancy just because you can, practical must come first
  • Grids and mobile don’t play well together [let me rephrase… DO NOT USE GRIDS ON YOUR PHONE. How freakishly tiny are your fingers? Come on!]
  • Test out what you’re asking your responders to do first, take a video while you’re pouring the milk
%d bloggers like this: