Live blogging from the Net Gain 2015 conference in Toronto, Canada. Any errors or bad jokes are my own.
Prediction Science: How Well Can We Predict the Future?
Jon Puleston, VP of Innovation at Lightspeed GMI
- [Jon had the room take a quiz on a random series of questions]
- What has he learned about predictions? Better at predicting certain things, behaviors. Not so good at predicting prices.
- You can isolate people who are good at predicting, perhaps find them and use their super power 🙂
- Prediction isn’t dependent on sample size. One person can be sufficient. It’s more about sample diversity and the intelligence of that sample.
- 16 is a crowd if they are all well-informed. That’s all you need.
- You could ask 1000 people or 5 people about the weather next week, but you really only need to ask the 1 person who saw a weather forecast.
- How do you aggregate crowd wisdom – mean/median/mode
- Crowds mean that errors get distilled out
- A crowd of people predicted the price of the ipad within 1%
- But without any knowledge, the crowd is ignorant. People NOW can’t predict the weight of a cow.
- Prediction is littered with cognitive biases – 68% of people will say that a coin will toss heads because we always hear ‘heads’ first
- People’s preferences for wine depend on whether you ask “Who prefers red wine?” vs “Who prefers white wine?”
- People who check their emails before breakfast are more likely to say people check their email before breakfast
- Emotions get in the way of making valid predictions. We are more positive about our own teams vs other teams when predicting score counts.
- Do you you clean up after a meeting vs Do people clean up after meetings. People say they do but they don’t. [I do. Even when I wasn’t in the meeting.]
- Read: http://www.amazon.com/Expert-Political-Judgment-Good-Know/dp/0691128715
- only 48% of stock market gurus stock market predictions were correct
- People who ‘bet’ 1 unit not as good as 2, but betting 3 is a little better. People who bet against are really good.
- After 15 people predict and other people see that, they start to predict the same way, the answers don’t move.
- But in an independent voting method, larger surveys are better than 15 people
- Best predictive market situations allow sharing of information, let people discuss and debate
- e.g., in guess which mug is most popular, someone will suggest mugs are good gifts, someone will suggest lots of people garden, people decide that the gardening mug will be most popular
- Think of board room meetings where they didn’t discuss things before they vote on a decision. Stray comments are problematic.
- Try dividing up the herd and then recombine the three groups back into one. Helps improve accuracy just like how we run 3 focus groups not 1.
- Let people change their opinions in surveys [We NEVER let people do that!]