Live blogged from the 2015 MRIA National Conference in Toronto. Any errors or bad jokes are my own.
Tell Your Stories Here
Luke Stringer, Twitter
- #DaveCalls – a fun trending twitter hashtag with Patrick Stewart where people started holding anything up to their ear with a serious face
- Did a study with people around a sports game
- People are bummed out when you take away their cell phones
- People in the no cell phone room seemed less connected to others, people like to show each other things
- greater emotional intensity in the moment for the same moment and the same demographic
- Commander Hadfield knows how to use Twitter
- 1 billion tweets every 2 days – more words written on twitter in the next two years than were ever printed in every book
- 312000 about breakfast every day, 600 000 selfie tweets every day, 22 00 tweets about going drinking in the next 25 minutes
- Content is predictable, peaks at the same moments every day
- Four buckets of data in Canada [really? this is how the world works, not just canada]
- to keep up to date, with stories around the world
- connect with each other
- discovery, live search
- 43% of Canadians say they use twitter to procrastinate
- Connecting with people – share moments of our lives
- Partnered with NeuroInsight company, examine how different areas of the brain are firing. Looked at personal relevance and emotional relevance and long term memory encoding. This study showed twitter had the highest response in digital medium. The only thing that could beat twitter was personal mail with a stamp on it.
- 63% of Canadians follow celebrities on twitter – music, tv, fashion. Numbers change by demographics. Huge opportunities for brands.
- 55% of Canadians follow brands but half of people say they aren’t a current customer, want to hear about offers and promotions but they also just like the content
- Got 6 times ROI for every dollar spent on movies on twitter
Designing for Efficiency
Kimberly Sanderson, RBC
- Team needs to be simple, central, and consistent
- Avoid recreating solutions to problems, give yourself more time to work on ad hoc tasks
- The efficient days of famous people look nothing like what the “perfect day ought to look like”
- you should get to know your peak energy level, know if you’re a morning or night person
- stop multi-tasking, write a to-do list, limit your distractions including emails, work in 60 minutes spurts, leverage technology
- multi-tasking lowers your IQ by 10 points
- try branding your team at work, even if you aren’t an external product that needs a brand
- stop activities that don’t add value to your team.
- don’t hire the same people on your team. hire people who complement, who don’t have the skills you already have. don’t look for duplication in your hires. use folders. archive old files. avoid keeping duplicates. Separate ongoing and completed work. Create backups. If all else fails, have a search function. Don’t save on your hard drive.
- use a central hub for their reports, templates for everything, every employee has transferable research skills but also specializations
Tested for Life in Canada
Cedric Painvin, Canadian Tire
- handed out a towel to everyone in the audience. it is designed to stay cold for a couple of hours once you get it wet
- want to be known for innovation not being an old company, want to be known for quality, want to be known for life in canada better than any other retailer in canada
- They have a group of people who test products and when these canadians approve of a product, the product gets a sticker on the package
- Products are picked by the marketing group and the merchants. The analytics team looks for testers in their database. Product is shipped to the person. Sometimes they just say use it, other times they are told to use it the way they normally would. Give 2 to 3 weeks to use the product.
- 70 000 people wanted to be a part of their testing team. but all they knew was their name and email address. Started using just an excel sheet but then switched to their own online panel the next year in 2014. Added a gated community in 2015 – like facebook for the most engaged product testers they have – 4000 people here.
- 15 000 testers, have collected 150 videos from testers, films created by professional media, 1000s of profile variables – every question answer is added to the profiling data, 190 products tested, 20 different product categories. Get feedback in 1 to 2 days. Response rate to screeners is around 60%.
- Sent a window scraper to their testers who overwhelming found a glitch. The company fixed the glitch and now version 2.0 is in stores.
Live blogged from the 2015 MRIA National Conference in Toronto. Any errors or bad jokes are my own.
Panel – Digital and Social Media impact on Research
Twitter/Google/Facebook/Rogers – Sponsored by Ipsos
Gayle Lunn, Ipsos UU
- Industrial revolution, mass production/consumption which changed the quality of people’s lives, digital now lets us to do things differently and impacts the way we think
- Luke Stringer from Twitter – startling change joining Twitter – speed of things changing is very fast, challenges the research method. Product is very different today from what it will be in one week or one month. Culture is very collaborative, work spaces are collaborative, completely open concept [booooooooooooooo!] There are tables around so people can meet and talk. Culture of failure is embraced, test and fail fast. They test every minute change to the product.
- Alexandra Cohn from Google – Came from ipsos, overwhelming amount of data and knowledge. Like drinking from a firehouse. Acronyms for everything. What do you need to know and what can you ignore. Speed is overwhelming. Completely different mentally, not asking, the data is behavioural. Fully transparent and collaborative culture, always available to anyone. No one keeps data to themselves.
- When you join twitter, you give your name and email. They don’t know who you are as an individual. Why do you use twitter – they don’t know that. You can apply traditional techniques to understand that. You have to ask questions.
- Not everything can be done in no time. There’s a lot of pressure to do research fast, get answers cheaply.
- Will google surveys replace traditional surveys?
- Collecting too much data is a bad things, especially from consumers.
- Despite all the internal knowledge, they still need to pair internal data collection with elicited data. It’s enrichment of data.
- Google surveys is only ten questions by design. Partnering with the full service companies is where they get their strength because google isn’t researchers. They provide the data, they are the platform. Still need researchers to interpret the data, researchers know how to ask the good questions. Not just anyone can ask a question.
- Twitter is a mobile first company. Rich media is limited, on purpose. There is always a place for longer form surveys, rightly or wrongly.
- Data scientist is a title that has to be earned.
- A lot of collaboration is behind the scenes, not public, and you can’t talk about it. But there is a lot of collaboration among clients. Collaboration is driven from the outside. Clients can’t operate in silos anymore, they need data from a wide range of people.
- Digital is held to a higher standard than other media. People expect you can link data but you can’t always do so. You can’t always use JUST data to complete a model. Big expectations that models will drive income.
- Right now we pay for view but maybe we will eventually pay only if someone follows or only if someone makes a purchase – pay for desired behaviour.
- Like to append their data to market data to see true ROI. Research explains why one campaign worked and another did not. Which specific type of content drove the result.
- Just “knowing” something works is not enough. They want and can get precise numbers.
- What is the value of a Twitter like versus a Facebook like? Need to be able to measure this. Models are wonderful but they don’t matter. Metrics matter.
- TV ads existed for a long time and we had ways of measuring the outcome, same with print and radio. Twitter has to figure it out for themselves instantly, not over decades.
- Industry is evolving but not fast enough. Privacy laws are very challenging. Internal struggles between product development and engineers, all have their own internal objectives.
- Rely on industry to tell them what the cool things are, where the industry is going. Partnered with academics to understand impact of being exposed to twitter versus other media in terms of how brains are reacting. Have and are exploring neuroscience to understand their product.
- Investing in machine learning and artificial intelligence. If you want to learn the direction of the company, look at who is on their board.
- How do i read irony in a google search query? 2/3 of top AI people in the world work for google.
- How can we collaborate more effectively. External partners come with ideas as opposed to we just want to work with you.
- It doesn’t have to be unique data. We just need to properly marry the data.
- It’s okay to fail in this safe space, with appropriate checks and balances
- Sometimes, retweets and shares are the right measure for you. Other times, ROI is the right measure.
- How do you attribute a click when someone saw six ads along the way and then finally clicked on one.
- Collaboration can lead to indecision because no ones in charge. Must hold everyone accountable for specific objectives. Make it directly related to reviews. Course correction is ok. Autonomy is important.
- Collaboration is not leading my consensus. One person must be responsible for the outcome.
Even though this infographic is out of date, having first been published in January 2014, the points it lays out are still relevant today. Jade Furubayashi from Simply Measured describes the Twitter practices of the top brands including how many times they tweet every day and how engagement is affected by the number of followers. But don’t misinterpret that correlation by buying yourself 100,000 followers. Paid followers won’t add to your engagement and they won’t love and adore your brand by sharing, tweeting, and retweeting. Only genuine brand love creates engagement.
- Interesting infographic: How your brain sees a logo (lovestats.wordpress.com)
- Missing Data: Whose problem is it anyways? (web.peanutlabs.com)
- 13 tips for giving the worst presentation ever (lovestats.wordpress.com)
- How women should ask for a raise if they don’t want to follow Microsoft’s CEO advice of Trust Karma (lovestats.wordpress.com)
WAPOR opened with a bang as David Fan described the statistical techniques he used to organize the accepted papers into relevant bunches. The key terms included cluster analysis and the traveling salesman approach as a number of presenters were asked to determine which of the other accepted papers were most similar to theirs. One of the methodological issues that had to be dealt with was that some presenters were forced to back out at the last minute such that the carefully designed grouping didn’t end up being perfect. Alas, as with every research project, errors creep in.
And in case you’re curious, no, there was no parade of WAPOR figure heads each welcoming us with a short prepared talk. There were no dance routines, fun videos, or Nice tourism representatives. Yes, a room full of data geeks got a truly geeky talk from the head geek. I’m still chuckling about it. 🙂
Rather than summarize the talks I went to, I’ll mention a few interesting tidbits and a few thoughts that came to mind for me.
- Do you ever consider responder needs, not your own needs? When you’re designing surveys, do you ever really think about what the responder needs as part of the research process? I know you want quality data and you want to design surveys that generate quality data, but do you really think about the fact that responders may want to answer a survey on a phone because they can take it to a private room or a quiet room? Similarly, do you realize that people may not want to answer a phone survey because there are other people in the room or it’s too noisy for them? Stop fussing over whether you do or don’t want people to take a survey on their phone. Give them the tools to give you the best data they can – from a quiet room, a private room, or anywhere.
- People don’t fan pages they don’t like. One of the speakers mentioned that people don’t fan brand pages if they aren’t truly fans of the brand. Well, that’s not completely true. Many people ‘fan’ or ‘like’ a page so that they can leave a complaint or criticism on it. Or, they want to monitor what the brand is doing to see how it compares to their loyal brands. Or, they like the page to learn about discounts and coupons that they can redeem with their own brand. Whether Facebook or Twitter, it doesn’t matter what the social network names the buttons – people will click on the button that suits their purpose.
- Social media data has yet to be validated. Someone also mentioned that social media data is taking a while to become widely used because the data itself hasn’t been validated yet. For instance, if someone tweets that they went to McDonald’s, did they really go to McDonald’s. I found this comment kind of funny coming from someone in the survey world. Hm… if someone says on a survey that they went to McDonald’s, did they really go to McDonald’s? Something to ponder!
- Why are Google, Facebook, and Microsoft so far ahead in research? This comment came up as a tangent and was never answered by the speaker, but I’ll take it on here. Why? Because they aren’t research companies. They don’t have to fuss and fret and worry that their norms and standards will be royally screwed up. They aren’t worried about fitting 412 questions into 5 minutes of survey time. They aren’t trying to figure out how to make their product ‘fun.’ We DO have to worry about these things. Actually, I disagree that we have to worry. If we keep worrying as we have been, then Google and Facebook and Microsoft will wipe our faces with their research. If we don’t get with the times and become our own thought leaders, that’s what’s going to happen. Be aware of your norms and be cautious as you change them. Make the research experience enjoyable as it should be. It’s your business at stake. Stop talking. Start doing. (me included!)
- Are AAPOR guidelines too American? You know, I never really thought of that before. There are a number of organizations in the research world that want to be global. Given that WAPOR is the world version of AAPOR, I must conclude that AAPOR does want to be global. Yes, as was mentioned during today’s talk, most of the AAPOR guidelines are drawn with first world, English countries in mind – everyone has a phone, everyone has a smart phone, everyone has a physical legal home. Do the AAPOR guidelines make it easy or even possible for people in other countries to conduct ‘good’ research? It’s worth a ponder.
- Let’s stop the probability/non-probability debate. Hear hear! I don’t believe there is such thing as a probability sample in the human world (generally speaking). Yet, AAPOR continues to promote the idea. You see, even if you COULD know an entire population and select a random sample, people will still decline to participate, quit participating, answer questions incorrectly, misread questions, lie on questions, etc. The assumption is that probability samples create perfect data and this is just never the case. I would love it if we could just drop the whole probability superiority complex and get on with our work.
- Candy is a legitimate snack. Breaktime featured a fine selection of…. candy? yes, candy. For the second time today, I was happily shocked. Someone later mentioned that fruit was also available but I don’t know what that is and I didn’t see it. So they lied.
And that, my friends, is the Day 1 wrap!
GUN control, he said GUN control! 🙂
- The Conference Presenter Gender Gap #WAPOR #AAPOR #MRX (lovestats.wordpress.com)
- I poo poo on your significance tests #AAPOR #MRX (lovestats.wordpress.com)
- How many survey contacts is enough? #AAPOR #mrx (lovestats.wordpress.com)
- President of American Association of Buggy-Whip Manufacturers takes a strong stand against internal combustion engine, argues that the so-called “automobile” has “little grounding in theory” and that “results can vary widely based on the particular fuel t (andrewgelman.com)
I’m on Twitter a lot. I tweet a lot, I read it a lot. But I seem to be gravitating more and more to Facebook even though the powers that be tell me Facebook is about to implode. But I’ll tell you why I’m dating Twitter less than I used to.
- It’s getting harder and harder to talk to a person. As people realize the value of branding, more and more Twitter accounts are being named after companies and brands. They tweet endlessly all day long and attempt to engage in conversation. But I really can’t remember the last time I picked up a box of Cheerios and had a fun and interesting conversation with it. I talk to people, not brands. Even though I deliberately follow people over brands, my Twitterstream is an endless list of brands and companies, as if every human has stopped tweeting. Why not sign each tweet with -Annie, or put -Annie in your user bio. Yup, change your user bio every day depending on who’s tweeting.
- More and more people are begging to be followed. Outright asking for people to follow up them. No, they aren’t buying followers and I do appreciate that. But I don’t appreciate tweets along the lines of “Hey LoveStats, I love your tweets. Please follow us.” You see, if you tweeted with me even a couple of times, I would have reviewed your tweets and determined for myself whether your tweets were of interest to me. In other words, if you’ve tweeted with me and I’m not following you, it’s likely because I don’t want to follow you.
- Fewer and fewer tweets are personal. As I already said, I prefer to talk to people. And have lunch and play with their kids and do embarrassing things. It’s fun to read these things because first of all, well, they’re fun. And second of all, it helps me get to know you as a person, you as someone I’d like to talk to. I firmly believe there is a healthy balance between professional and serious, and fun and friendly. As a market research community, we are losing the right balance.
In conclusion, please treat me like a person wearing pink socks and eating chocolate, not a robot that might open a wallet for you.
There are a few basic rules to creating a great Twitterstream. The most obvious ones are
- Follow people with similar interests
- Don’t follow people who only tweet marketing messages
- Don’t follow people who only tweet inspirational messages
- Don’t follow people just because they followed you first
These tips will ensure that your Twitterstream is generally interesting and relevant. But, you’ll still see lots of irrelevant and uninteresting tweets with this method as most people like to create a bit of variety by tweeting off-topic once in while. It does potentially create a more fun reading pane for those just looking for something fun to read.
But, if you’re rushed or really need to focus, an open Twitterstream can sometimes bog you down. One of the best things you can do is create the perfect search string. For instance, here is the search string I use
The essential features are
- Choose your favorite hashtags and insert “OR” between each of them
- Choose terms you NEVER want to see. In my case, my stream is completely wiped of all things bieber and directioners. Just insert “-” before each of the dreaded words
- Choose a language. I only want to see English tweets so I type “lang:en: at the end of my search string
- Run the search and then save it
As you use the search string every day, take note of which hashtags need to be added to the list to include or exclude more tweets. Re-save your search string and then delete the old one. After a few days of adding and removing terms, you’ll end up with the perfect, highly relevant search string. Enjoy!
[To delete an old search string, run the old search string. Then the delete option will appear in the corner.]
Once you know what you really want to do, how do you get there? I’ve been asked a few times how I became a research methodologist and can someone else become a methodologist as well. It’s really quite simple. It’s not easy, but it is simple.
The advice I give is don’t wait for someone to offer you a job as a methodologist or to declare that you are a methodologist. If you know what you want, then it is your job to make it happen. No one else.
- If you are a project manager and you want to be a methodologist. Start tweeting about methodology. Share educational white papers and intriguing blog posts that helped you learn about methodology.
- If you’re in report prep and you want to be a methodologist, start a blog about methodology. Share your thoughts and opinions about good and bad methodology in easy to read, weekly or monthly opinion pieces. Be brand and bold and take a stand.
- If you’re a data analyst and you want to be a methodologist, stay a few hours late after work and re-analyze datasets with methodology in mind. Analyze speeding rates by survey topic, straightlining rates by number of grids, random responding by demographics. Then write a paper and share with your colleagues.
No matter what your current role is, you can find a way to incorporate what really excites you into your day. If you do it well, a good boss will find ways for you to expand into that space. And if your current boss doesn’t, one of your Twitter followers or blog readers will. Enjoy your new job!
- Finding the why in your market research results #MRX (lovestats.wordpress.com)
- A Halloween chocolate lesson in nominal, ordinal, interval, and ratio data #MRX (lovestats.wordpress.com)
- SAS vs SPSS: Pick one and forever hold your peace #MRX (lovestats.wordpress.com)