Live blogging from the #CASRO tech conference in Chicago. Any errors or bad jokes are my own.
Track: Innovation in Storytelling & Data Visualization
“MaxDiff Gamification” by J.J. De Simone, Data Analyst, Insights Meta
- people are asked to select best and worst options from 50 or 60 options, in small sets, over and over
- similar to rules of a lot of card games – texas hold’em
- gamification is application of game rules strategies and aesthetics to non-game settings
- incentives and rewards are a kind of gamification
- match solitaire card game with max diff method
- 4 cards with features or attributes – choose the best card and then the worst card, card then flies to the bottom of the screen until there are now 4 best options, then they choose the best of the best. then there are a few more rounds like this. effective data can even be obtained at 2 rounds. what is the ‘right’ number of rounds.
- more exciting and more familiar to people than traditional options
- millennials grew up with video games
- needed to compare quality of the data in both methods, as well as satisfaction metrics
- half way through regular survey was the A/B test of regular vs gamification method
- not everyone sees all the features due to random assignment of features
- took about 18.5 minutes on average, but people in the game group spent about 3 seconds longer [3 seconds? who cares, point is that it didn’t take longer. Ooooor why didn’t it take longer or shorter if people were having fun]
- game group said the length of the survey was the right amount of time, more so than the traditional group, but the effect size was very small
- enjoyment of the survey experience was much higher than the traditional groups, also effects for gender and age – females and younger people enjoyed it more than males and older
- sets of data correlated about .97, no demographic differences [doesn’t mean either set of data is correct though]
- ran a monte carlo simulation – many samples from the existing dataset
- the larger the same size and the more rounds you do, the higher the correlation [yes….]
- 2 or 3 or 4 rounds wasn’t particularly different, maybe 3 rounds with a slightly smaller sample is a good way to go, 1 round is not a good idea
- when enjoyment is high, they may expend more cognition on the data, data may be better quality
- you can generate good data with about 200 people participating
- can change game for individual company needs, format colours and shapes
- good way to break up a long survey
- you can allow people to play the game after the required number of rounds, free data donation when they don’t have to. .4% played an extra round. 1% played 2 or more extra rounds. [now THAT is a measure of engagement]
- Should a panel be representative of the population?
- Humanizing surveys: Why did you screen me out after I told you my age?
- Peanut Labs Ask-Me-Anything with special guest Kristin Luck
- What is a convenience sample?
- Behavioural Economics Can Finally Explain Human Behaviour