Live note taking at the MRIA 2016 annual conference in Montreal. Any errors or bad jokes are my own.
- [Jeremy begins by getting the crowd screaming and handing out books. We love free books 🙂 ]
- Your next breakthrough is closer than you think, but how do you know if you’re making the right choice
- There are too many ideas too close to us and we can’t evaluate them all
- Companies are not structured to adapt, they no longer last 75 years
- How can a man use paper origami to save lives? He now has 23 patents and works at NASA. But he quit his job to be an origami master. He was the one who figured out how to fold air bags in your car, among other life saving folding problems.
- He interviewed his dad and learned so many things he would have never learned otherwise
- His dad was caught stealing at a grocery store and he was sentenced to clean the grocery store for a month, he noticed a lot of good food was getting thrown out, he took that food and sold it, Turned into selling month old magazine, door to door donuts, Started his own night club at age 16 but he ended up in trouble for not having licenses
- Your competitors are lazier than you – you can put in the time, everyone wants to get better but not everyone wants to put in the effort
- He ended up owning a sports team but weird reasons but then he would personally invite people and then sit with them durin the game, to generate a personal connection, he’d encourage them to bring their friends; He increased attendance massively every year doing this; he was the MVP even though he never played in a game
- His did bought every magazine on any topic and then flip through the sections talking about new idea, then he tried to interpret every new idea in a different way or different industry
- The act of becoming inspired has become overwhelming
- Kodak invented the digital camera in 1975 and then had a website in the 90s to share pictures with their friends, but their goal was defeating a competitor and you know the end result
- Blockbuster didn’t see the end coming, they understood preference and stocked movies relevant to the neighbourhood they were in, but they chose not to buy Netflix
- Who invented grammar checkers? And electronic dictionaries, and the laptop word processor, and PDAs. This is all Smith Corona, a typewriter company. [wow. I remember buying a typewriter to go to university!]
- These companies had smart companies who researched good ideas, but hen they reverted back to their original company idea.
- Why was VIctoria’s Secret successful? The owner thought it was a store for MEN to shop for women. It was because the lingerie was fashionable and that’s what women wanted. The store reinvested in that concept and the store grew massively.
- 3 traps of a farmer – become complacent, repetitive, and protective.
- The owner of ZARA is worth 70 billion and he dresses very boring. They bring designs into the store in 14 days not 14 months. THey don’t advertise because it takes to long and costs too much. ANd they make clothes close by geographically so there are no transportation issues. THey can revise the dress in days and put it back in stores. THere is no designing. It is always tested in the stores on the racks. Zara signifies it’s a fashion city.
- Hunter instincts 1 – insatiable – they are never done. Hunter 2 – curious. How much time do you spend hunting outside your industry. Hunter 3 – willing to destroy
- Google had 1000 engineers to generate a new Facebook. WHich we know failed. Many companies were vastly better at specific aspects and were built by 12 people.
- Don’t build a better thing. Build what people WANT.
- No one person invents GPS or the iPhone. You need a little idea you can make big.
- Opposing the mainstream fuels success – Redbull tastes awful, has a smaller can, a bigger price, and scoffs at the legal team
Demand that your conferences be Diversity Approved! (Tweet this post!)
When Canada’s new Prime Minister, Justin Trudeau, was asked why his cabinet was 50% male and 50% female, his answer was simple. Because it’s 2015. Such a simple answer to a long standing problem.
As I look back over 2015, I see that “because it’s 2015” didn’t apply to every market research conference. Some conferences had speaker lists that were 70% male. Some conferences had speaker panels that were 100% male. No conferences had attendee lists nor industry lists that were 100% male let alone 70% male.
There are many reasons that men might be over-represented as speakers, but few that are acceptable.
- Random chance. As a lover of statistics, I accept that random chance will create some all male panels. But since I’ve never seen an all female panel, random chance is not what’s at play here. If you’d rather see the math, Greg Martin calculated the chance of having all male speakers here. It’s not good.
- 70% or more of submissions were from men. That also is an acceptable reason. If women aren’t submitting, then they can’t be selected. So on that note, it’s up to you ladies to make sure you submit at every chance you get. And don’t tell me you’re not good enough to speak. I ranted on that excuse already.
- You haven’t heard of any women working in this area. This excuse is unacceptable. You can’t look for speakers only inside your own comfortable friend list. Get out of your box. Get online. There are tons of women talking about every conceivable industry issue. Find one woman and ask her for recommendations. You can start here: Data science, Marketing research, Statistics, Tech.
- The best proposals happen to be from men. This excuse is also unacceptable. It demonstrates that you believe men are better than women. You need to broaden your perception of what ‘better’ means. Men and women speak in different ways so you need to listen in different ways. It’s good for you. Try it.
- Women decline when we ask them to speak. It’s a real shame particularly if women decline invitations more often than men. But any time a woman declines, ask her for a list of people she recommends. And then consider the women on that list. No women in the list? Then specifically ask her if she knows any women.
- It’s a paid talk and they only sent men. Know what? It’s okay to remind companies that their panel isn’t representative of the industry. You can suggest that they send a broader range of people.
- We didn’t realize this was a problem. Inexcusable. Diversity has been an issue for years. People have been pointing this out to market research conferences for years. The right time to fix things is always now.
When was the last time you prepared a sampling matrix balanced on age, gender, and ethnicity and then were pleased when it was 70% female, 70% age 50+, and 90% white? Never, that’s when. You stayed in field and implemented appropriate sampling techniques until your demographics were representative. This is absolutely no different.
So, to every conference organizer out there, ESOMAR, CASRO, MRA, MRIA, ARF, MRS, AMSRS, ESRA, AAPOR, I challenge you to review and correct your speaker list before announcing it.
- What percentage of submissions are from men versus women? Only when submissions are far from balanced is it acceptable for the acceptance list to be unbalanced.
- Are there any all male panels? Are there any all female panels? (By the way, all female panels talking about female issues do NOT count.)
- Are more than 55% of speakers male? Are more than 55% of speakers female?
- Is the invited speaker list well balanced? There is zero reason for invited speakers to NOT be representative.
- Did you actively ask companies to assist with ensuring that speakers were diverse?
If you can give appropriate answer to those questions, I invite you to publicly advertise your conference as Diversity Approved.
Will you accept this challenge for every conference you run in 2016? Will you:
- Post the gender ratio of submissions
- Post the gender ratio of acceptances
- Proudly advertise that your conference is “Diversity Approved”
Demand that your conferences be Diversity Approved! (Tweet this demand!)
Panel Discussion on Political Polling & Media in Canada: “Election Polling in the West – Has it Changed The Research Industry For the Better?” #MRIA14 #MRX
Live blogging from the #MRIA national conference in Saskatoon, Saskatchewan. Any errors or bad jokes are my own.
Moderator: Steve Mossop, President, Insights West; Panelists: Éric Grenier, polling analyst and the author of ThreeHundredEight.com; Tim Olafson, Co-founder, Stone-Olafson; Scott MacKay, President – Probe Research Inc.; Lang McGilp, Senior Research Executive, Insightrix Research
- [note – there is lots of debate and differences of opinions among the speakers, i have not indicated who said what]
- many voters change their minds at the very last minute, political polling is not broken in canada
- 22% of voters changed their mind in the last few days of the election
- we always ask “how would you vote today.” We need to ask the right questions
- there is still a role for public election polling, parties have the information so the public should have it as well
- campaigns will be dominated by internal polls because they will put out poll results themselves
- no more trust between public and pollsters and we need to rebuild that trust; better polling costs money, money that we don’t have, need more cooperation between pollsters and journalists is reported the right way
- pollster in-fighting looks terrible, media sees the fights and focus on the people who got it wrong
- tired of giving away free polling to build up brand recognition
- pollsters doing a crappy job of setting the context, they focus on a time frame or election, they don’t look back at the last election at society at the bigger picture
- we need to get rid of the ban on polling publication
- industry needs to be less competitive and more open with best practices
- we have civic reasons to do the polling, good for democracy
- there are many people who want us to get polling wrong
- there are too many free polls, angus reid in the west complains the most
- some people think more pollsters is better – 280 pollsters were doing it in the US, Canada probably only needs around 12. comparatively, not as many in Canada
- do engineers or lawyers offer free engineering and free lawyering? Free undervalues our work
- some firms refuse to release any public that is not paid for
- paid for polls are more accurate because you ask more questions
- we trivialize elections with so many polls based on insufficient survey questions; will the media cover costs of a 60 question survey
- polls these days are just horse race measures
- how can we prove that polling works – we’ve called elections accurately for the last 50 years, except when it’s wrong [margin of error people]
- polling used to be much more accurate, record was unblemished. what happened? we started using online panels. some panels aren’t good for this kind of research. telephone method is not dead. it works well. panels won’t work in smaller regions. Do not write off the phone at this point.
- there are region specific panels that were built carefully, based on telephone recruit. These panels are extremely accurate.
- method doesn’t matter. society has changed. it used to be the newspaper in the evening and news on TV at night. Now news is instant all day long.
- not a lot of telephone any more in ontario but any methodology can get it right
- turnout determines accuracy of polls, it’s luck
- voter turnout is declining especially among younger people which means we will need to build likely voter models, this is new for many people
- some region have publicly available voter lists, can be purchased, can determine who has and hasn’t voted
- is it intention, past behaviour that predicts best?
- we don’t ask the right questions, need to probe the undecided better, shouldn’t focus solely on undecided voters and they could be leaning heavily into one camp
- maybe we don’t know what’s going on
- how can we do a better job of predicting elections? voter models which we really haven’t been using [seriously? you aren’t using models? i’m seriously shocked.]
- a publication ban is not a polling ban, we should keep polling until the end so we get a better sense of what’s going on
- perhaps publish your numbers as an exit poll
- people dislike polling because ‘we’re wrecking democracy,’ we’re telling people ahead of time what will happen
- need more transparency, show the numbers, show the questions, show the weighting – this helps to avoid in-fighting
- prediction markets – one happened in BC and followed the polls exactly but it was wrong at the outcome like all of the polls
Using Digital Technologies to Connect with Citizens and Shape Public Policy in the City of Markham by Frank Scarpitti, Mayor, and Adam Froman #MRIA14 #MRX
Live blogging from the #MRIA national conference in Saskatoon, Saskatchewan. Any errors or bad jokes are my own.
Frank Scarpitti, Mayor, City of Markham; Adam Froman, CEO, Delvinia & AskingCanadians
- 60% of their city was born outside of Canada
- known as Canada’s high tech capital, over 900 tech companies call Markham home
- known as a knowledge based economy
- want to attract tech, life science, and finance companies
- one of fastest growing areas in Canada, population will grow by 50% in 20 years
- 25 years ago, the wired telephone was how we communicated at a distance, now mobile phones are used for phone calls, and much more.
- now pushing for internet voting
- in 2003, experienced a 400% increase in internet voting
- offered in 2006 again, further increase in use of the internet, increased turnout to 40% but that compares well to typical numbers of low 30s or lower
- 2007, launched online program “Click with Markham” Feedback from residents on Markham priorities and action plans
- Can get 300 people to an in-person event but got over 7000 online
- “do it yourself Markham” humorous online videos to engage voters and increase awareness of decisions and impacts in their own neighbourhoods, recognize importance of issues and voting. videos of doing your own garbage, street maintenance (See below)
- Go where the people are digital to reach young, tech savvy residence
- Have their own portal for more than 50 services – websites let you see what you can buy, portal connects you directly with the service, introduced mobile app to make access to services portable
- consumer/citizens control the conversation
- how do you deal with transparency and accountability with a government
- built this program on crowdsourcing initiative
- even got data and submissions from 10 to 18 year olds [impressive!]
- had a team go into the schools and engage
- opendata presents a lot of opportunity in the MR world
- 98 municipalities will now offer internet voting, Canada is a leader in this, it’s too late to discuss, we’re already doing this
Keynote: Jeff Hayzlett, Bestselling Author & Global Business Celebrity, The Hayzlett Group #MRIA14 #MRX
Live blogging from the #MRIA national conference in Saskatoon, Saskatchewan. Any errors or bad jokes are my own.
Jeff Hayzlett, Bestselling Author & Global Business Celebrity, The Hayzlett Group
- His shows are viewed 40 times more than any other show, it’s the way they pick their shows
- Used to spend 160 million per year on research
- research should be pushing the edge more than any other group but they are the most conservative group
- Kodak went out of business on the day the first digital photograph was produced
- Kodak was never in the film business, they were in the emotional memories business
- “Kodak moments expire”
- you don’t get in a car and lock the steering and gas in place, you make adjustments all along the way
- We need to be agents of change
- don’t ask for permission, don’t make committees, just do it, see it and fix it
- we need problem solvers, not problem seekers. don’t come to me with a problem, bring me a solution
- drink the koolaid
- need people who operate between the silos, drive change
- fear is a problem, fear only lasts for a couple of seconds, you must overcome those first few seconds and you will accomplish great things
- a good maestro plays a lot of bad notes, that’s the nature of business, you will screw it up the first time
- be brave enough to ask “What are you talking about”
- create as much tension as you possibly can – no pain, no gain
- your job is to go to the edge of the stage, some people will try to pull you back to the middle – HR, legal – but that’s not their job.
- if you aren’t creating tension you’re not growing
- radical transparency – biggest age of this ever, possibly too far right now – – when one group doesn’t do it, the other group helps out
- recall the Domino’s commercial where the say “We Suck”. Wonder if the guy who said that in the first meeting is still there. Someone decided we’re going to care more about what’s in the box than getting the box to you on time. Sales are through the roof now.
- when you see the elephant in the room, speak up
- can you raise the bar, poke someone in the eye
- if you make a mistake, no one is going to die
- why do businesses promise? life should be about promises. set mutual conditions of satisfaction. if you make a promise, you can negotiate but I’d better something in return.
- research is seen as a slowgate
- 118 elevator pitch – this should not be a 48 page hour like PowerPoint pres
- 8 seconds – average attention span of an adult
- 110 seconds – average elevator ride
- you have 8 seconds to give the hook – something that makes you lean in, be provocative
- 110 seconds to explain the value, i don’t care who you did business with, it is enough time
- if Moses can do it in 2 slides with 5 points each you can do it in 110 seconds [HA HA HA that was funny 🙂 ]
- you need to live the brand, look professional
- a brand is a promise to deliver, you must shelter it and nurture it
- if you suck offline, you will suck online, only faster
- put your Kodak moment on the back of your business card, your personal branding
- never be scared when you are in good company
Panel discussion with client-side researchers – “What clients want”
Moderator: David Ian Gray, President & Founder, DIG360 Consulting Ltd.
Panelists: Greg Ambrose, Senior Manager, Consumer Research & Insights, Tim Hortons; Bonnie Baird, Manager of Planning & Research, Tourism Saskatchewan; Susan Williams, Senior Director, Strategic Insights at Cadillac Fairview, Cadillac Fairview Corporation; John Tabone, Principal, Member Value & Research Services Chartered Professional Accountants of Canada
- other than better, cheaper, faster…
- clients shine when they partner with good people
- find a way to get to the executive table, a way to position themselves
- need strong relationships between clients and suppliers, iterative, trusting, collaborative
- suppliers are suppliers within their own organization
- need to learn from suppliers on how to be the consultant within their organization
- don’t need to change suppliers when the same old project is going just fine the same old way; but i want to see new things and if you aren’t showing me, i’ll go find them elsewhere. they’d like to work with the same suppliers but you aren’t offering what the technology companies are offering
- they want to put good research in front of key decisions makers, needs to be timely and a good story, need it to sing a little bit, needs to make them think further and more in depth
- [Pause – time to consider the three Mounties who were killed in Moncton as this is headquarters for the Mounties]
- suppliers want to meet the executive team on day one but that’s not realistic
- every company doesn’t need to know all the new technologies but you need to understand them and know where you fit within them
- need to make sure they’re working with the right people on the right projects, work across projects the supplier gets to understand the business better and the relationship is better, you need months and years to develop relationships
- best suppliers know how to listen, know where you want to go, it’s overwhelming when suppliers list out 20 tools and options but I need something to solve my specific problem
- it’s okay for vendors to say i don’t know if we have that or can offer that
- how can you get the key message across in five minutes
- you may not have to go through the key client contact every time when you have a good relationship
- use the words that your team wants to hear if it will make the technique sound more cool and it actually what they want/need
- probably won’t take a chance on a new vendor for a large job, six figures
- please get the company name right, please know what their products are. it is a job interview so prepare for it.
- RFP process is your first stab at the relationship – on time, typos, answer all the questions, don’t assume we know what you’re talking about, name dropping is not impressive but it helps for context, foot in the door is telling about related projects not the company secrets
- when a client recommends a new supplier to their team, they need examples of relevant work, evidence that you are good at it helps sell you as a vendor
- don’t dangle a wonderful person and then have them work only two hours on a project, it’s misleading, unfair and leaves a bad taste in the mouth, deliver the people you say you are going to deliver
- be familiar with the buzz words needed to sell the job internally, your trademarked words, proprietary tools don’t fool them [Heck yeah! it’s all just regression not the Annie Pettit Regression Tool]
- need to know how involved the senior person is, do include the junior people as that pipeline needs to be filled but don’t let me find out they wrote the whole report and i’ve never met them
- overused buzzwords – Insights – do you have a ‘drawer you can open and pull out an insight’, We Can Do It All, Big Data
- challenge me a little bit even when I think i’m right
- why use a supplier when there are so many DIY tools? because they provide the so what, useful decision advice that you don’t get from DIY tools – but they DO use DIY tools for clear simply projects
- when are DIY tools not appropriate? sometimes after the DIY tool to get the aha
- want a few options of good better best with sample sizes, Q lengths, phases. open to a certain amount of grey even at the pricing stage
- ask good questions at the RFP stage, not stupid questions [i would love to hear some of the stupid questions 🙂 ]
- budgets are available if you can demonstrate impactful decisions
- help them find low risk ways to innovate
- can’t always be as open as they want to be due to legal reasons
Annie Pettit, Chief Research Officer, Peanut Labs
My #MRIA14 presentation in just 5 minutes…
“ICE”ing On the Cake -De-Constructing and Disrupting the Customer Experience by Anne Kossatz and Lesley Haibach #MRIA14 #MRX
“ICE”ing On the Cake -De-Constructing and Disrupting the Customer Experience
Anne Kossatz, Manager Strategic Marketing Research, RBC
Lesley Haibach, Vice President, Ipsos Reid Loyalty
- the customer experience had flat lined, results were very stable especially at the corporate level
- wanted agent level feedback, needed faster presentation of results
- what makes an experience GREAT?
- focus is on the experience – pre-call and post-call also have an influence on telephone customer experience
- you need to focus on far more than just corporate level – apple and disney focus on the individual consumer
- are you thinking about the music playing, what the line will be, what the store smells like
- need to look at where in the interaction someone feels they’ve had a good experience
- what do you THINK the line will be and what would be better – 10 people in line imagined versus 5 or 0 to make you feel better
- 3 stages of study – understand expectations vs reality – key stakeholder interviews conducted
- move to customer observation and interviews with an ethnographic experience, had consumers do an information gathering phone call
- final stage is frontline staff validation sessions – brought in staff from all departments and found the pain points and what could be done to fix those
- customer journey extends beyond the experience itself – IVR to live agent to transfer – people expect ease of navigation, empathy and engagement, warmth of transfer
- also what is motivating them to make the call, is there any follow-up – these also matter
- what are the door openers in the experience – what is the mood, headspace during the precall – transaction you don’t recognize, fear or anger or stress, 1 or 5 rings really matters. if you’re just changing your address while you watch TV at the same time, the mood is totally different.
- people would prefer to stay on the line with an agent rather than stay on line with the agent, when you go to the branch you are technically put on hold but the teller keeps you engaged
- client validation – you have to give your personal details several time even before you get to the right person – need to wait to hear the issue before the person is validated and has to give their personal information
- client experience – it must be client first, need a warm transfer, sometimes problem is resolved but the client doesn’t realize it’s been fixed
- HR realized it needed to hire not necessarily entry level people, but those with the ability to engage and delight customers
- it’s time to rebuild the experience and the measurement of that experience
Epic Win! Successful Gamification in Marketing Research by Briana Brownell and Carl Gutwin #MRIA14 #MRX
Epic Win! Successful Gamification in Marketing Research
Briana Brownell, Manager, Analytics, Insightrix Research Inc.
Carl Gutwin, Professor of Computer Science, University of Saskatchewan
- 1- gamify to increase engagement, encourage people to do things they wouldn’t necessarily do otherwise
- 2 – get something new, get more information from each person, information that wasn’t available via a survey
- 3- reach the unreachable, people who might not take a survey
- spent half the time of the project just coming up with a good idea to turn into a game
- what is the core task – fundamental action you want the user to perform – their age, count the light bulbs in their house, need to match that action with the survey and the game
- add around that some game mechanics – the process of playing the game, and the game itself
- most game mechanics aren’t compatible with surveys – points and scores work. but survey needs a truth, a valid answer not an imagined outcome. you can’t let people choose which questions they want and in what order.
- Game 1: research game show for an ad test. could they identify the sponsor, identify slogans and products, did they understand the purpose of the ad [makes me wonder – if we’re encourage people to pay attention to an but they never actually pay attention to ads, what exactly are we measuring]
- added time pressure, cheers and jeers for correct and incorrect answers, music
- where is the barrier between survey and game?
- found two mistakes – in the full game, more people were getting it right, the feedback mechanism caused people to learn when they were getting right and wrong answers, refielded without the problematic feedback. smaller difference found for a time pressure section which made people give shorter responses, an undesirable outcome.
- it was motivating. people were asked if they wanted to quit or continue. Difference of 12 vs 21 “would you like to continue, traditional vs gamified version.
- no differences by gender, age, tenure, or gaming experience
- Game 2: shopkeeper for a choice based conjoint. Game was little people coming in your shop and you need to help them pick a product. Also tried to do it with a little alien man on another planet – spacemonsters.
- major questions – was the data different, was the game motivating, was the possibility of getting incorrect answers motivating, did the games realism affect the results. There ISN’T a correct answer to a survey question so need to work this out.
- The data was not different at all – when people pick the best value, it doesn’t matter if they pick for themselves, for other people, or for aliens
- The realism did not affect the results. equally motivated in all scenarios.
- The possibility of getting incorrect answers didn’t affect it.
- Was the game motivating? actually, it was demotivating. They did 47 extra games the traditional way but only 37 with the game version. It was too different from reality.
- is it worth it too add gamification? make sure you match the task with the game mechanism first.
- don’t make the game too gamey
- think about auxiliary data – what can i get that I can’t get elsewhere