Day 2 brought lots of interesting topics as well. We started off with updates on the ESOMAR 26 document, the ARF quality initiative, and the ISO process. I did not know that the first ESOMAR code was developed in 1948! I did not know that people from 26 countries were involved in the ISO process. It was not a surprise, however, to hear the findings of the ARF quality study .
- data quality is not the serious issue that was previously thought
- panel duplication is really only about 16%
- heavy responders provide good data
- response rates are not the best indicators of quality data.
After the ARF presentation indicating a complete lack of ability to identify a weighting system that would equalize panels, one unlucky speaker had the unfortunate time slot of presenting his work on how the CART system can indeed weight data sources to be equal. Personally, I’m still on the side that panels cannot be weighted to be equal. I’ve participated in so many parallel studies for so many different types of surveys and categories of products and have yet to seen a perfectly successful case. Sure, you can always weight a few variables into comparability, but you lose out on a bunch more. Maybe CART is the magic solution. Time will tell.
Once again, survey panels took the hit on not being representative of the general population. Really, come on folks, what method of marketing research IS representative of the general population. None. End of story. In the end, the only kind of representativity that matters is making sure your sample suits the purpose. And, if your results never predict the marketplace, get out of the business.
Kim Dedeker was kind enough to share a few thoughts on the state of the industry since she first caused a good storm. Her advice – the industry can only be sustained through quality, we need to continue having and creating leadership through getting involved.
The last major topic of the day was mobile research. I must confess that I am STILL not a believer. I just don’t see how the ability to answer a 20 minute survey on a 2 inch screen as you ride the bus or pay for your new purchase is going to bring the survey industry back from the deadly response rate dive.
Feel free to also read a few other thoughts I had about ESOMAR Chicago in a piece I wrote for research-live.
Below is Kim speaking and a presentation about social media that mentioned the tweet-up at Lux Bar.