What We (Don’t) Learn From Eye Tracking by Jennifer Romano Bergstrom, Facebook, @romanocog


Live note taking from the UXPA webinar. Any errors are my own.

  • Traditional user experience UX measures – first click accuracy, task accuracy, time to complete tasks, click patterns, conversation rates
  • Self report measures include difficulty ratings, satisfaction ratings, think aloud protocols, debriefing interview
  • But these use a filter, people can think about what they want to share with us, includes biases, people feel they are being tested and evaluated, people feel bad if they can’t do what they think you want them to do
  • Implicit methods can offer a lot
  • Eye tracking, EDA (sweat, electrodermal activity), behavioral analysis (eye rolling), verbalization analysis (negative words even though they’re saying something in a positive way “I’m not upset”), pupil dilation
  • We can learn HOW people navifate, WHY they focus, and what, how long, and how often they focus on things, the combination of methods is more accurate, do not use eye tracking in isolation
  • When to use eye tracking? Do people see things that aid in task completion? Look patterns including location, duration, and path, intended visual hierarchy vs actual look pattern, evaluates user experience
  • Eye tracking assesses engagement – desirability, accessibility, trustworthy, useful, valuable, usable
  • Engagement – Number of fixations, Processing – fixation durations, findability – time to first fixation, processing order – gaze path, comprehension – repeat fixations, workload excitement – pupil dilation
  • Heat map of one person isn’t useful, need to average across many people
  • Red usually means more attention, gaze opacity maps are interesting as they present in white and gray so you see more clearly what people are looking at, gaze plots work better for one or two people, side by side gaze plots are better or use a heat map
  • Use eye tracking over time to see if changes to design improved findability
  • What sample size is needed? No good answer [what about until the data stops changing?]
  • Have used it qualitatively, no assumptions like everyone looks at left more in design A, Live broadcast the eye tracking to stake holders, small samples are ok
  • Heat maps show a lot of differences when averaging across different groups of 8 people, you can’t make assumptions about these 8 people for another group of people; gets a bit cleaner with 15 people but there’s still variation, increasing it to 30 generates a lot more clarity [ah yes, the academic literature states that 30 is the bare minimum for averages]
  • Can compare eye tracking of new users and experienced users, experienced users go directly to what they need
  • Use eye tracking to prepare better paper diaries, how do people understand the questionnaire design, is the questionnaire too complicated, people wouldn’t put pen to paper for a long time, realized people wanted to write down the name of the TV show first, not the time and channel of the show
  • People do not read dense text, if something is confusing you can’t just add more text to make it easier, use bulleted lists, bold headings, pare text down, people want to accomplish a task and we need to help them do that
  • People read pages with questions on them differently than other pages, they skip a lot of instructions but they try to process the questions
  • People instantly jump over instructions and go directly to trying to answer the questions, repeat fixations before interacting indicates confusing, moving back and forth and back and forth trying to figure out what to do
  • People attend to the username and password no matter what else is on a page, if there is something people must read, it needs to be treated differently, put in a location where you know people are reading it
  • Compare attentions to icons and motivational language, do people see the logos, sometimes learn people understand the old-fashioned version more because they are used to it
  • Can evaluate age-related differences, older people gravitated to the center, read more of the left side of the screen, didn’t focus on the right side of the screen which would have helped them
  • People consume mobile content much more quickly, scan down the right side of the screen and barely look at the left
  • Can compare websites e.g. Facebook vs Instagram formatting
  • Modern eye tracking – can use overhead tracking, device under a book, eyeglasses, attached to a monitor
  • Mobile device stand – non-invasive stand that doesn’t get in the way of the participant, but it’s not a natural way to hold it, poor tracking quality, and where do you put the camera, hand can get in the way of the camera
  • Glasses require something on your face, it can be dizzying for live feed watching
  • Don’t sit next to the participants because they want to talk to you while they do it, ideal is in a different room but you can simply put a big cardboard barrier in front
  • Does not work well – consider attention and gaze
  • when attention and gaze are on the screen – answering your email on the screen, eye tracking works well
  • If they need to go get information elsewhere, sometimes gaze and attention will be off the screen but at times will both also be on the screen, this will also work fine
  • When attention are gaze are in different places – hmmm…. Let me think about that, I need to figure something out, or listening to someone but looking elsewhere – this won’t work well if they are thinking about something else while looking at your product, worse case scenario
Advertisements
%d bloggers like this: