Live note taking from the UXPA webinar. Any errors are my own.
- Traditional user experience UX measures – first click accuracy, task accuracy, time to complete tasks, click patterns, conversation rates
- Self report measures include difficulty ratings, satisfaction ratings, think aloud protocols, debriefing interview
- But these use a filter, people can think about what they want to share with us, includes biases, people feel they are being tested and evaluated, people feel bad if they can’t do what they think you want them to do
- Implicit methods can offer a lot
- Eye tracking, EDA (sweat, electrodermal activity), behavioral analysis (eye rolling), verbalization analysis (negative words even though they’re saying something in a positive way “I’m not upset”), pupil dilation
- We can learn HOW people navifate, WHY they focus, and what, how long, and how often they focus on things, the combination of methods is more accurate, do not use eye tracking in isolation
- When to use eye tracking? Do people see things that aid in task completion? Look patterns including location, duration, and path, intended visual hierarchy vs actual look pattern, evaluates user experience
- Eye tracking assesses engagement – desirability, accessibility, trustworthy, useful, valuable, usable
- Engagement – Number of fixations, Processing – fixation durations, findability – time to first fixation, processing order – gaze path, comprehension – repeat fixations, workload excitement – pupil dilation
- Heat map of one person isn’t useful, need to average across many people
- Red usually means more attention, gaze opacity maps are interesting as they present in white and gray so you see more clearly what people are looking at, gaze plots work better for one or two people, side by side gaze plots are better or use a heat map
- Use eye tracking over time to see if changes to design improved findability
- What sample size is needed? No good answer [what about until the data stops changing?]
- Have used it qualitatively, no assumptions like everyone looks at left more in design A, Live broadcast the eye tracking to stake holders, small samples are ok
- Heat maps show a lot of differences when averaging across different groups of 8 people, you can’t make assumptions about these 8 people for another group of people; gets a bit cleaner with 15 people but there’s still variation, increasing it to 30 generates a lot more clarity [ah yes, the academic literature states that 30 is the bare minimum for averages]
- Can compare eye tracking of new users and experienced users, experienced users go directly to what they need
- Use eye tracking to prepare better paper diaries, how do people understand the questionnaire design, is the questionnaire too complicated, people wouldn’t put pen to paper for a long time, realized people wanted to write down the name of the TV show first, not the time and channel of the show
- People do not read dense text, if something is confusing you can’t just add more text to make it easier, use bulleted lists, bold headings, pare text down, people want to accomplish a task and we need to help them do that
- People read pages with questions on them differently than other pages, they skip a lot of instructions but they try to process the questions
- People instantly jump over instructions and go directly to trying to answer the questions, repeat fixations before interacting indicates confusing, moving back and forth and back and forth trying to figure out what to do
- People attend to the username and password no matter what else is on a page, if there is something people must read, it needs to be treated differently, put in a location where you know people are reading it
- Compare attentions to icons and motivational language, do people see the logos, sometimes learn people understand the old-fashioned version more because they are used to it
- Can evaluate age-related differences, older people gravitated to the center, read more of the left side of the screen, didn’t focus on the right side of the screen which would have helped them
- People consume mobile content much more quickly, scan down the right side of the screen and barely look at the left
- Can compare websites e.g. Facebook vs Instagram formatting
- Modern eye tracking – can use overhead tracking, device under a book, eyeglasses, attached to a monitor
- Mobile device stand – non-invasive stand that doesn’t get in the way of the participant, but it’s not a natural way to hold it, poor tracking quality, and where do you put the camera, hand can get in the way of the camera
- Glasses require something on your face, it can be dizzying for live feed watching
- Don’t sit next to the participants because they want to talk to you while they do it, ideal is in a different room but you can simply put a big cardboard barrier in front
- Does not work well – consider attention and gaze
- when attention and gaze are on the screen – answering your email on the screen, eye tracking works well
- If they need to go get information elsewhere, sometimes gaze and attention will be off the screen but at times will both also be on the screen, this will also work fine
- When attention are gaze are in different places – hmmm…. Let me think about that, I need to figure something out, or listening to someone but looking elsewhere – this won’t work well if they are thinking about something else while looking at your product, worse case scenario
It doesn’t matter whether you like it, it’s common practice to share on Facebook and Twitter your support, or lack thereof, for political candidates and issues. It doesn’t seem to make a lot of sense when so many people hold firmly to their beliefs and won’t change their mind anyways. So why bother?
For me, there are two main reasons.
- It reminds people that there are opposing views out there and the entirety of the world does not agree with them. It helps prevent some of the silos.
- It reminds people that others they respect and perhaps look up to for being intelligent and good people disagree with them. This helps them to see that diverging opinions probably have merit even if they hate those ideas.
With the increased use of social media algoeithms, it’s getting easier to go an entire day and not see any opinions that differ from yours. We need to see and appreciate the diversity of people and opinion.
Live blogging from the Net Gain 2015 conference in Toronto, Canada. Any errors or bad jokes are my own.
Brand Building in a Digital, Social and Mobile Age
Joel Rubinson, President and Founder of Rubinson Partners Inc.
- Picture of brand success has to change
- We are no longer in a push word, consumers pull information at their leisure
- We engage in shopping behaviours even when we aren’t really shopping, we are always IN the path to purchase
- Brands must become media
- Starbucks is the best example of a marketer that gets it. 40 million fans on facebook. millions of website visits. millions have downloaded their app. Every interaction generates data they can use, can be used for personalization, to amplify brand communications. They no longer have to pay for every message.
- The rise of math experts in advertising – lift from using math to place advertising is a repeatable success
- Programmatic messaging is key. Think about impressions that are served up one user at a time. marketers goal is serve the most relevant ad at the right price. And this needs to scale.
- Research is missing in action when it comes to math – we lack digital metrics, still rely on survey based tracking, we have a post-mortem mind set, we are failing to change how marketing works
- We must get serious about integrating digital – why isn’t this happening, why are we locked in a survey world
- Our comfort zone is surveys. We know how to construct 20 minute surveys. Our learning zone is the mobile area where we unpack our surveys into smaller pieces.
- The panic zone is digital, we don’t understand it. We must move digital into the comfort zone.
- lets start by just looking at the data, look at page views, look at themes in social media, how big is your brand audience, how many likes on facebook, how many twitter followers, how many newsletter signups. These are unambiguous measures. Look at clicking and sharing and conversions.
- Stop treating social media as a hobby, not specialty projects, not ancillary thing to look at. You must find ways to increase positive word of mouth.
- Do we really need feedback from consumers every single day on attributes they never consider? Can’t social media which is much more organic do this?
- Bring in data that you can’t get from a survey that has action value. Some online panel companies already use a social login called OAuth. Append all the Facebook data to your survey and use it for targeting.
- Data aggregators have lots of profiling information for targeting ads throughout the web which means different people get different ads based on cookies from their browser
- You can also link in frequent shopper data to your survey data.
- You don’t have to guess whether an ad is working. You can run an experiment and serve the ad to one group of people and see the change in group behaviour.
- MR needs to know that brand meaning is done completely different now. People seek out knowledge, digital delivers information in real time. But marketing research hasn’t changed.
- Think digital and do something big. Shift some money into datascience or integration. Conduct in the moment research with smartphones.
If you can’t say it in just a few words, you don’t understand it well enough. It’s a motto I love and a problem I encountered this week when that was my question. Every page I read was a Charles Dickens essay that went into such a multitude of differences that I was more confused afterwards. Just tell me the difference, damn it! Let me now make your life easier.
Facebook page: All the posts that show in the page and in people’s feeds will come from you. Think of this as someone yelling at people from the mountain top.
Facebook group: All the posts that show in the page and in people’s feeds will come from anyone who writes on your page. Think of this as a bunch of friends chatting with each other.
Yeah, sure there are lots of other specific differences but that is the one big difference that will help you figure out which one you actually want.
Welcome to my Facebook page. This morning, I woke up to this photo in my newsfeed. It had 20 comments and 15 likes by the time I saw it. And now, it has 27 likes and 5 more comments. Except for one comment, mine, every single person left a rude, mean, inappropriate comment that they would never say to someone’s face. But hey, it’s fine because everyone else did it and this person will never find out.
Out of no respect for the poster, and only common courtesy to my fellow humans, I have blacked out the names of each person who felt it was perfectly OK to laugh at someone else’s expense. Imagine how furious they would be if they saw personally critical comments of themselves online.
Who is the poster you ask? Well, I only friend three kinds of people on Facebook. My family – it’s not them. Personal friends and past acquaintances – it’s not them. Professional market researchers, marketers, advertisers, and other professional people in my business – yes it’s them.
This particular individual is currently the CEO of a company of more than 500 people (I feel sad for those employees) and the LinkedIn profile claims the poster is client oriented (I feel sad for the clients).
Please, I beg of you. When you see this show up in your newsfeed, comments by people you know and respect, let them know that this kind of behaviour is not acceptable. It won’t stop until someone has the guts to speak up and demand that it stop.
Why are so many people on depression medications? Why do so many people get plastic surgery? Why are so many people anorexic and bulimic? Why do people commit suicide? Why don’t people report being raped? Why are deliberately being mean?
While you’re at it, read this post from a woman who saw it happen to her.
WAPOR opened with a bang as David Fan described the statistical techniques he used to organize the accepted papers into relevant bunches. The key terms included cluster analysis and the traveling salesman approach as a number of presenters were asked to determine which of the other accepted papers were most similar to theirs. One of the methodological issues that had to be dealt with was that some presenters were forced to back out at the last minute such that the carefully designed grouping didn’t end up being perfect. Alas, as with every research project, errors creep in.
And in case you’re curious, no, there was no parade of WAPOR figure heads each welcoming us with a short prepared talk. There were no dance routines, fun videos, or Nice tourism representatives. Yes, a room full of data geeks got a truly geeky talk from the head geek. I’m still chuckling about it. 🙂
Rather than summarize the talks I went to, I’ll mention a few interesting tidbits and a few thoughts that came to mind for me.
- Do you ever consider responder needs, not your own needs? When you’re designing surveys, do you ever really think about what the responder needs as part of the research process? I know you want quality data and you want to design surveys that generate quality data, but do you really think about the fact that responders may want to answer a survey on a phone because they can take it to a private room or a quiet room? Similarly, do you realize that people may not want to answer a phone survey because there are other people in the room or it’s too noisy for them? Stop fussing over whether you do or don’t want people to take a survey on their phone. Give them the tools to give you the best data they can – from a quiet room, a private room, or anywhere.
- People don’t fan pages they don’t like. One of the speakers mentioned that people don’t fan brand pages if they aren’t truly fans of the brand. Well, that’s not completely true. Many people ‘fan’ or ‘like’ a page so that they can leave a complaint or criticism on it. Or, they want to monitor what the brand is doing to see how it compares to their loyal brands. Or, they like the page to learn about discounts and coupons that they can redeem with their own brand. Whether Facebook or Twitter, it doesn’t matter what the social network names the buttons – people will click on the button that suits their purpose.
- Social media data has yet to be validated. Someone also mentioned that social media data is taking a while to become widely used because the data itself hasn’t been validated yet. For instance, if someone tweets that they went to McDonald’s, did they really go to McDonald’s. I found this comment kind of funny coming from someone in the survey world. Hm… if someone says on a survey that they went to McDonald’s, did they really go to McDonald’s? Something to ponder!
- Why are Google, Facebook, and Microsoft so far ahead in research? This comment came up as a tangent and was never answered by the speaker, but I’ll take it on here. Why? Because they aren’t research companies. They don’t have to fuss and fret and worry that their norms and standards will be royally screwed up. They aren’t worried about fitting 412 questions into 5 minutes of survey time. They aren’t trying to figure out how to make their product ‘fun.’ We DO have to worry about these things. Actually, I disagree that we have to worry. If we keep worrying as we have been, then Google and Facebook and Microsoft will wipe our faces with their research. If we don’t get with the times and become our own thought leaders, that’s what’s going to happen. Be aware of your norms and be cautious as you change them. Make the research experience enjoyable as it should be. It’s your business at stake. Stop talking. Start doing. (me included!)
- Are AAPOR guidelines too American? You know, I never really thought of that before. There are a number of organizations in the research world that want to be global. Given that WAPOR is the world version of AAPOR, I must conclude that AAPOR does want to be global. Yes, as was mentioned during today’s talk, most of the AAPOR guidelines are drawn with first world, English countries in mind – everyone has a phone, everyone has a smart phone, everyone has a physical legal home. Do the AAPOR guidelines make it easy or even possible for people in other countries to conduct ‘good’ research? It’s worth a ponder.
- Let’s stop the probability/non-probability debate. Hear hear! I don’t believe there is such thing as a probability sample in the human world (generally speaking). Yet, AAPOR continues to promote the idea. You see, even if you COULD know an entire population and select a random sample, people will still decline to participate, quit participating, answer questions incorrectly, misread questions, lie on questions, etc. The assumption is that probability samples create perfect data and this is just never the case. I would love it if we could just drop the whole probability superiority complex and get on with our work.
- Candy is a legitimate snack. Breaktime featured a fine selection of…. candy? yes, candy. For the second time today, I was happily shocked. Someone later mentioned that fruit was also available but I don’t know what that is and I didn’t see it. So they lied.
And that, my friends, is the Day 1 wrap!
GUN control, he said GUN control! 🙂
If you haven’t heard of the Facebook ‘scandal’ by now then I’m jealous of the holiday you’ve just taken on a beautiful tropical island with no internet access. The gist of the scandal is that the feeds of 689,003 people were curated differently than everyone else to gauge the subsequent effect on emotion.
While most people’s feeds are curated based on which friends you like, share, and comment on more often, the feeds of these people were curated in addition, by considering the positive and negative words they included. In both conditions, Facebook chose which of your friends’ posts you would see though in the Test condition, you might be offered a greater proportion of their positive or negative posts. The conclusion was that you can indeed affect people’s emotions based on what they read. You can read the published study here.
I honestly don’t know where I stand on the ethics of this study right now. Ethics interest me but I’m not an ethicist. So instead, let me think about this from a scientific point of view.
Do you deliberately manipulate emotions in the work you do? As a marketing researcher, your job is ONLY to manipulate emotions. You know very well that this brand of cola or that brand of chips or the other brand of clothing cannot boast better taste, feel, look, or workmanship. All of those features are in the eye, or taste buds, of the beholder. Through careful research, we seek to learn what makes different kinds of people happy about certain products so that marketers can tout the benefits of their products. But, at the same time, we also seek to learn what disappoints and makes people unhappy about the products and services they use such that those weaknesses can be exploited by marketers.
Through a strange twist of fate, a colleague and I recently conducted a tiny study. We found the results quite interesting, and wrote a quick blog post about it. Then the Facebook news broke. As Facebook did on a larger scale, I will confess that I manipulated the emotions of about 300 people.
Previously, I saw on a number of studies that age breaks are inconsistent. Sometimes researchers create an 18 to 34 age break, and other times they create an 18 to 35 age break. In other words, sometimes you’re the youngest person in a group, and sometimes you’re the oldest person in a group. Would you rather be the oldest person in a young group, or the youngest person in an old group? What did we find? Well, people did indeed express greater happiness when they were part of the younger group, even though they were the oldest person in that group. I deliberately and knowingly manipulated happiness. Just like Facebook did. Do you hate me now? Do you think I’m unethical? You can read the post here.
As marketing researchers, every bit of research we do, every interaction we have with people, is intended to manipulate emotions. We collect data that marketers use to criticise our favourite products. We collect data so that politicians can directly criticise other politicians through their negative ad campaigns. Has that bothered you yet? Has that bothered you enough to warrant outcries in social media? Have you campaigned for an immediate ban of television, radio, and viewing products on the shelves at supermarkets knowing that those things are intended to manipulate our emotions?
Since you know that your research is intended to affect emotions, do you inform your research participants about the potential negative consequences of participating in your research? Do you tell them that seeing their age in the older age bracket may make them unhappy, that viewing critical ads may make them unhappy, that being asked to select up to five negative attributes might make them unhappy?
Given that we’ve done it this way for so long, have we become complacent about the ethics of the research we conduct? In this age of big data, is it time to take a fresh look at the ethics of marketing research?
Read more opinions:
Are you jealous?
Who wouldn’t be jealous of like counts and comment counts like these. Sixteen thousand likes for a stupid comment about serving Skittles at red lights. No pictures, no links, no free Skittles. Just a short little comment. But wait, I was one of those people who chuckled at the comment and then clicked on the like button. In fact, Skittles is one of the only 3 or 4 brands that I “like” on Facebook.
It’s this kind of success that every other consumer brand dreams of and tries to figure out. What could they possibly say in a status update that would convince thousands of people play along and click that like button. What exactly is Skittles doing that can be replicated in order to bring hundreds of thousands of consumers into the fold as fans, real fans, engaged, happy, eager fans?
The major problem is this. Brands know they need to be on Facebook which results in status updates that bear no likeness to the brand itself. Sure, it’s great to be patriotic and say Happy Birthday to your country but how does that relate to me, the Duracell fan? And how does Father’s Day really relate to the Generate Motors brand – Dads have cars?
In these two cases, and for many many more brands, the personality and essence of the brands don’t come through the messages. No one has bothered to stop and think about what their Facebook essence is. Birthday messages about celebrities, important dates in history, and recipes for fruit salad might be mildly entertaining but they don’t remind consumers of why they love your brand. They don’t generate likes and comments because people don’t like them and they don’t want to comment on them.
So here are my 6 tips for better Facebook pages.
- Know WHAT your brand’s personality is. If you don’t know, then you’re not ready for Facebook.
- Know WHY you are on Facebook. (and “to sell more widgets” doesn’t count)
- Write messages that embody your brand whether the message is silly or serious.
- Get into the mind of your consumer and ask yourself, “Will anyone truly and honestly care if I write this.”
- Don’t write messages for the sake of “It’s Tuesday. We always write a message on Tuesday.”
- Don’t feel that you must include pictures or links or free offers. People don’t want stuff. They want a feeling.