Tag Archives: WAPOR

WAPOR Day 3: Margin of Error is too complicated to understand #AAPOR #MRX

All good things must come to an end and so to has WAPOR. But, today was still a full day of sessions so here are a few of my take aways.

  • Anything other than Margin of Error is too hard to understand.  AAPOR, and by association, WAPOR have been having a rough time lately with discussions, rants, arguments or whatever you want to call them regarding margin of error. In today’s business meeting, someone mentioned that using anything other than margin of error is just too difficult to understand. Well you know what, margin of error is difficult to understand yet we’re still all on the same page. The fact that it, or any other measure, is difficult to understand is absolutely no excuse. We aren’t stupid. Journalists are stupid. Pollsters aren’t stupid. Let’s find a measure that works, that makes sense, and start using it. I don’t care how ‘difficult’ it is.
  • Should you debrief after observational research? It’s probably been a hundred years now that various North American associations have agreed that observational research does not require researchers to inform the people they observe. As long as the researchers do not interfere, don’t talk to, don’t manipulate, don’t affect the people around them, don’t sneak around, don’t hide, don’t misconstrue, they are free to listen and observe what people are saying and doing in public spaces. No permission required.  But, I learned today that academics in Germany must debrief people afterwards. I’m having a hard time wrapping my head around this one. Chances are that at some point in the future, most if not all research organizations are going to have extremely similar ethics codes. How will we reconcile this one?
  • Newspapers are the most trustworthy. I found this kind of humorous. Newspapers first, TV second, social media is further down the line. It kind of made me think that the longer it takes to take the news and make it public, the more likely people are to trust it. Hence, since daily newspapers generally take about 24 hours to turn news into the medium, there’s a lot of time to determine where an error was made and refrain from printing it. Television makes somewhat of an effort to broadcast news as quickly as possible but even they take some time. If an item doesn’t make “live, breaking news”, it still has to wait until 6pm or 10pm to be shared widely. Again, lots of time to discover and error and correct it. But this blog post? I could have written it the very second I heard each tidbit (and I normally do) which means I could have misheard or mistyped something without realizing it. Hit “submit” and that false news is out there.

WAPOR Day 2: People don’t lie on government surveys #AAPOR #MRX

Day two of WAPOR has come and is nearly gone, but my brain continues to ponder and debate all that I heard today. I hope you enjoy a few of the ramblings from my macaron infested brain.

  • People don’t lie on government surveys. Wow. That’s news to me! My presentation focused on how people don’t always provide exactly correct answers to surveys for various reasons – the answer isn’t there, they misread something, they deliberately gave a false answer. But, while people may feel more incentive to answer government surveys honestly, those surveys are certainly not immune to errors. Even the most carefully worded and carefully pre-tested survey will be misread and misinterpreted. And, some people will choose to answer incorrectly for a variety of reasons – privacy, anti-government sentiment, etc. There is no such thing as “immune to errors.” Don’t fool yourself.
  • How do you measure non-internet users? Well, this was a fun one! One speaker described setting up a probability panel (i know, i know, those don’t really exist). In order to ensure that internet usage was not a confounding variable, they provided a 3G tablet to every single person on the panel. This would ensure that everyone used the same browser, had the same screen size, had the same internet connection, and more. Of course, as soon as you give a tablet to a non-internet user, they suddenly become….. an internet user. So how do you understand perceptions and opinions from non-internet users. Chicken and egg! Back to paper you go!
  • Stop back translating. I don’t work much in non-English languages so it was interesting to hear this one. The authors are suggesting a few ideas:  questionnaire writers should write definitions of each question, preliminary draft translations should be provided by skilled translators, and finally, those two sets of information should go to the final translator. This is how you avoid “military rule” being translated as “role of the military” or “rules the military has” or “leadership of the military.” Interesting concept, and I’d love to know whether it’s efficient in practice.
  • Great presenter or great researcher: Pick one. I was reminded on many occasions today that, as a group, researchers are not great presenters. We face the screen instead of the audience,  we mumble, we read slides, and we speak too quietly. We focus on sharing equations instead of sharing learnings, we spend two thirds of the time explaining the method instead of sharing our insights. Let’s make it a priority to become better speakers. I know it won’t happen over night but I’ve progressed from being absolutely terrible to reasonably ok in a short matter of just 15 years. You can do it too.

America: You aren’t representative of the world #MRX #WAPOR

At one of yesterday’s talks, someone mentioned they sometimes felt that #AAPOR didn’t realize the rest of the world isn’t American.

A second presentation this morning was a brutal reminder. Here are a few interesting tidbits I heard from Gina Cheung at the University of Michigan.

– they sometimes needed to send two interviewers to every house because men could only talk to men and women could only talk to women
– they didn’t understand why there were such long pauses in the interview times until they realized people had to stop to pray
– though many people had good bandwidth, they still needed to find internet cafés to upload data at reasonable speeds.

Researchers really need to remember that the world does not live in neighbourhoods like theirs, have jobs like theirs, eat like they do, or have access to services like they do. Even in our own country. Think outside your country. Think drastically less privileged.

Food for thought.

WAPOR Day 1: Why are Google, Facebook, and Microsoft so far ahead of us in research? #MRX

WAPOR opened with a bang as David Fan described the statistical techniques he used to organize the accepted papers into relevant bunches. The key terms included cluster analysis and the traveling salesman approach as a number of presenters were asked to determine which of the other accepted papers were most similar to theirs. One of the methodological issues that had to be dealt with was that some presenters were forced to back out at the last minute such that the carefully designed grouping didn’t end up being perfect. Alas, as with every research project, errors creep in.

And in case you’re curious, no, there was no parade of WAPOR figure heads each welcoming us with a short prepared talk. There were no dance routines, fun videos, or Nice tourism representatives. Yes, a room full of data geeks got a truly geeky talk from the head geek.  I’m still chuckling about it. 🙂

Rather than summarize the talks I went to, I’ll mention a few interesting tidbits and a few thoughts that came to mind for me.

  • Do you ever consider responder needs, not your own needs? When you’re designing surveys, do you ever really think about what the responder needs as part of the research process? I know you want quality data and you want to design surveys that generate quality data, but do you really think about the fact that responders may want to answer a survey on a phone because they can take it to a private room or a quiet room?  Similarly, do you realize that people may not want to answer a phone survey because there are other people in the room or it’s too noisy for them? Stop fussing over whether you do or don’t want people to take a survey on their phone. Give them the tools to give you the best data they can – from a quiet room, a private room, or anywhere.
  • People don’t fan pages they don’t like. One of the speakers mentioned that people don’t fan brand pages if they aren’t truly fans of the brand. Well, that’s not completely true. Many people ‘fan’ or ‘like’ a page so that they can leave a complaint or criticism on it. Or, they want to monitor what the brand is doing to see how it compares to their loyal brands. Or, they like the page to learn about discounts and coupons that they can redeem with their own brand. Whether Facebook or Twitter, it doesn’t matter what the social network names the buttons – people will click on the button that suits their purpose.
  • Social media data has yet to be validated. Someone also mentioned that social media data is taking a while to become widely used because the data itself hasn’t been validated yet. For instance, if someone tweets that they went to McDonald’s, did they really go to McDonald’s. I found this comment kind of funny coming from someone in the survey world. Hm… if someone says on a survey that they went to McDonald’s, did they really go to McDonald’s? Something to ponder!
  • Why are Google, Facebook, and Microsoft so far ahead in research? This comment came up as a tangent and was never answered by the speaker, but I’ll take it on here. Why? Because they aren’t research companies. They don’t have to fuss and fret and worry that their norms and standards will be royally screwed up. They aren’t worried about fitting 412 questions into 5 minutes of survey time. They aren’t trying to figure out how to make their product ‘fun.’ We DO have to worry about these things. Actually, I disagree that we have to worry. If we keep worrying as we have been, then Google and Facebook and Microsoft will wipe our faces with their research. If we don’t get with the times and become our own thought leaders, that’s what’s going to happen. Be aware of your norms and be cautious as you change them. Make the research experience enjoyable as it should be. It’s your business at stake. Stop talking. Start doing. (me included!)
  • Are AAPOR guidelines too American? You know, I never really thought of that before. There are a number of organizations in the research world that want to be global. Given that WAPOR is the world version of AAPOR, I must conclude that AAPOR does want to be global. Yes, as was mentioned during today’s talk, most of the AAPOR guidelines are drawn with first world, English countries in mind – everyone has a phone, everyone has a smart phone, everyone has a physical legal home. Do the AAPOR guidelines make it easy or even possible for people in other countries to conduct ‘good’ research? It’s worth a ponder.
  • Let’s stop the probability/non-probability debate. Hear hear! I don’t believe there is such thing as a probability sample in the human world (generally speaking). Yet, AAPOR continues to promote the idea. You see, even if you COULD know an entire population and select a random sample, people will still decline to participate, quit participating, answer questions incorrectly, misread questions, lie on questions, etc. The assumption is that probability samples create perfect data and this is just never the case. I would love it if we could just drop the whole probability superiority complex and get on with our work.
  • Candy is a legitimate snack. Breaktime featured a fine selection of…. candy? yes, candy. For the second time today, I was happily shocked. Someone later mentioned that fruit was also available but I don’t know what that is and I didn’t see it. So they lied.

And that, my friends, is the Day 1 wrap!

GUN control, he said GUN control!  🙂

The Conference Presenter Gender Gap #WAPOR #AAPOR #MRX

I always like to evaluate the speaker gender ratio at conferences. This is my first time at a World Association of Public Opinion Researchers, and so a quick check is definitely in order.

I’ve got the conference guide in front of me and I’ve coded the first names of all the first speakers for each paper. Given the much higher rate of attendance by non-English speakers, my ability to infer gender is greatly reduced. Since there are many names I am unable to code, my numbers may not generalize well. Here goes.

Of the 108 first names I could infer from, 47 were female names and 61 were male names. If you can’t divide 47 by 108, that’s 44%. In my books, that’s a slight bias towards male speakers.

At least one of these is the cause:
1) fewer women than men submit papers
2) women don’t brag enough in their submissions and so they aren’t chosen
3) women’s submissions are less likely to be chosen due to sexism of the judging panel, whether conscious or unconscious. (Everyone is sexist is one way or the other, be honest with yourself.)

If I had to guess, I’d say #1 is 50% of the problem, #2 is 45% of the problem, and #3 is 5% of the problem.






The Bakery Review: A Daily Blog of the Macarons of Nice, France

You’ve come to expect nothing less of me. When I travel, wherever I travel, I seek out the best of sweetness, whatever it may be. And here in Nice, France, I shall do no less. My plan, whether possible, is to review the macarons from a different bakery every single day while I’m here. I don’t know whether I will pass by enough shops on my way to and from the WAPOR and ESOMAR conferences, but I’m willing to go slightly out of my way to make things happen. Instead of a new post everyday (which would drive some of you nuts!), I’ll just keeping adding to the top of this post. Come back every day to see what I eat!

Sept 10, 2014. May I simply start by saying YUM! I stopped by today’s shop a couple days ago but declined to buy any macarons because they were priced a fair bit more than other stores. I got used to paying 1.2EU each but this shop was 1.5 and 2EU each. Yes, both are exorbitant prices for a tiny little cookie when you could buy an entire box of packaged cookies for less. For my last evening in Nice, though, I broke down and shelled out the dough at a shop called Le Notre. For three! And I’m sure you can see why. The first macaron you see is Jasmine. And yes, it has gold flakes on it which stayed nicely attached instead of just blowing off. It tasted a bit like flowers which is to be expected given the flavour, but it wasn’t overwhelming as some floral treats are. It was crispy, creamy, and nicely done inside and out. The pink one is cherry and it was decorated with a thin chocolate coating in bright pink. Inside was purple cherry jam and it didn’t quite taste like cherry. It was gushy and it completely broke apart. But, even though I’m being very critical, it was still delicious. The last kitty macaron was flavoured badou. I asked what that meant and he said badou. So, I can’t tell you what flavour it was. But, it was crispy on the outside and creamy on the inside as expected. The inside was pink and white and kind of marshmallowey. I wish I could tell you the flavour as it was tastey. And awfully cute!

IMG_3472[1]Sept 9, 2014. My apologies for this belated post. Further apologies for the fact that the shop I went to was a repeat shop, Gosset Boulanger Patissier. But I wasn’t disappointed. I could see the label for the salted caramel macaron so I asked for it. It was a up to par in terms of crispy outside and nice instead. The second macaron made me try out my french because it was not labeled with a flavour and I knew I wouldn’t be able to understand a description of it. So, I ended up asking for the orange one with green on it. What a surprise! It turned out to be orange flavoured with a hint of chili to it. I know, I know. The chili macaron I had the other night was horrid but this one managed to get the flavours just right. Not too much chili and not some strange flavour combination. Besides that, they were shiny on the outside and chewy on the inside which was el perfecto! So, I’m pretty sure I’m forgiven since these were yum!

6macaronsSept 8, 2014. After yesterday’s horrid macarons, it seems strange that I yet again write about the macarons I ate at the Acropolis Congress. But, since they were so horrid yesterday, I thought today’s macarons earned them a chance on the blog again. What we were served today were your typical macarons – shiny on top, slightly crispy on the outside, moist on the inside, not too sweet. In order, they were pistachio (very nice flavour, mushy inside), chocolate (nice brownie flavour), coffee (i was hoping for praline and I’m not a coffee fan but I could see that people would like this one), vanilla (very nice flavour, not just a ‘plain’ macaron), lemon (great lemony flavour despite being crunched, but it was closer to lemon pudding inside), raspberry (i’m not a big fan of raspberry nor of jam but given that, this macaron was fine). Actually, this is a picture of the SECOND set of macarons, but these macarons made it back to my hotel room so I could write about them. Anyways, they weren’t SO delicious that I required 12 macarons, but a macaron on a table is a macaron that requires eating. Let the hating begin… “Oh, how do you stay so thin! I wish I could eat like that!”  Well, I don’t eat like that. It’s called conference eating and it stops the second I get on the plane. That’s how. 🙂

horrible awful macaronSept 7, 2014. Stop reading today’s update right now. I mean it. Stop. Well, I warned you. Today’s macaron was awful. Horrible. Can’t imagine anything worse.  So, the source of today’s atrocity is the Acropolis Congress which is hosting this week’s conference. I pretty much ran over to the table as soon as I realized there were macarons on it. Politely, I took just one. I’m always very polite when it comes to sweets. Anyways, with glee, I popped that little green baby into my mouth and nearly… well, let’s say I politely finished it. I just had to open it up to see what the double hockey sticks was in. Peanut butter? With chili? Well, the outside was soft and mushy and it smelled awful. Maybe this was a gourmet flavour that is far behind my unrefined palette. I don’t care. It was so bad I just had to check out the pink one too. Back to the table… Fortunately, the pink one was still a little crispy on the outside but I wisened up on my methodology. I DID NOT EAT THIS ONE. I opened that puppy up and it had some weird brown filling that ALSO SMELLED AWFUL.  In other words, I have no idea what flavour it was and I really don’t care. To give credit, I don’t know how you’re supposed to serve good quality, fresh macarons to 1000 people when a macaron only stays fresh for about ten minutes. So don’t even try.  Hey, is it inappropriate to whip the leftover macaron bits out my hotel window? Just curious. Not that I did. But I did do this.

IMG_3115[1]Sept 6, 2014. I’ve never chaired a conference session before and today was my first time. The speakers were kind as pie and intrigued the audience enough to generate lots of questions. They made my job easy! But, I’ll take that as another reason to celebrate which of course means…. the daily macaron! I think I went to heaven! Today’s macarons are brought to you by the number OMG and the letter I NEED MORE! I believe the name of the shop was Serain Cappe and I will have to find it again. I know the macarons look unassuming. The same smooth outside, filling peeking out in the middle. They were both very fat which makes me think they were made at the store instead of shipped in from elsewhere. The brown one is praline and it caused me to literally “OMG YUM” over my notebook. It was a little sweeter than maybe it should have been but it was otherwise perfect. The meringue stayed crispy, the inside was chewy, the filling didn’t completely soak into the meringue. Serious oh yeah! The vanilla macaron was also smooth, crispy, and quite nice. Not sure why I asked for vanilla since I’ve already had a vanilla macaron and I’d say they’re generally boring, but such is life. The most popular ice cream flavour is vanilla and that just might be true with macarons too.

IMG_3112[1]Sept 5, 2015. Today was presentation day at WAPOR and the audience had lots of great questions for me. Naturally, a celebration of the blessed event was in order.  So here goes. I stopped at a place called Mimosa, which was actually a very nice candy store. I’ve seen a number of candy stores here and they are NOTHING like what we have at home. I’ll have to take a bunch of pictures. Anyways, the macarons obviously weren’t made there but they had a bunch of different, interesting flavours. The unappetizing grey macaron is licorice! It was a little sparkly, crispy on the outside and soft on the inside. It wasn’t the best licorice flavour, it was a little odd, but the sweetness was good. And, the filling didn’t melt into the meringue. The second macaron was lime & cactus. Despite the picture, it’s actually kind of green on the outside and very pink on the inside. It too was crispy on the outside but it was mushy and wet on the inside. The lime was a nice flavour, a good sweet and sour mix with the cactus. I’m already looking forward to tomorrow!

macarons, gossetSept 4, 2014. I didn’t manage to find any macarons between the hotel I’m staying at and the conference hotel but have no fear.  To compensate for hours of sitting on my butt, a little exercise was in order. Aaaaaaaand, that included Gosset Boulanger Patissier. Sadly, they didn’t label the flavours so I simply practiced my stunning french (Deux macarons s’il vous plait) and pointed at two – the two most boring ones I could see. I think the white one was vanilla and the light brown one was caramel. Both macaroons had the traditional smooth shiny outside. They were crispy on the outside and smooshy on the inside. You could actually see the filling inside as opposed to the filling having been completely absorbed by the meringue. They were nicely sweet, just like the three bears. However, and this a big one, I do believe that these macaroons were not made in the store. They were set in plastic trays, there were only about 8 kinds, and they really looked like they just got dropped off at the store every few days. So, while they were good, I prefer to patronize 100% self-sufficient bakeries. Which means I won’t be going back for a second look.

macaron, nice, france, wapor, esomar, L'Art GourmandSept 3, 2014. Today my mission was simply to arrive before the conference started (first plane was late, but I caught the second plane because it too was late, hence I succeeded!). The second priority was to find the conference hotel and some dinner. Which means I had to walk by a bakery. Today I found L’Art Gourmand which is a STUNNING candy shop. They had many kinds of homemade style candy from jelly to nougat to chocolate and, yes, macarons. I chose a chocolate and an almond macaron. Both macarons looked very homemade and lacked the typical shiny smooth appearance you may be used to. But, they were the right size and shape, though not machine perfect as i suspect many are. They were also hefty and had a heavy filling. The chocolate macaron didn’t seem quite sweet enough and tasted more like chocolate powder. The almond macaron had a light taste, very nice. I wouldn’t go back for the macarons, given that there are likely to be many many other choices but if I can find the shop again, I will fork out a wad of dough for a one time ever nougat treat ($5 per 100 grams!).

Related Posts

%d bloggers like this: