Tag Archives: #PAPOR

2016: The year of the outsider #PAPOR #MRX 

live blogged at #PAPOR in San Francisco. Any errors or bad jokes are my own.

The Summer of Our Discontent, Stuart Elway, Elway Research

  • regarding state of washington
  • it’s generally democratic
  • between elections, more people are independents and then they drift to democrat
  • indepents are more social liberals
  • has become more libertarian
  • dont expect a rebellion to start in washington state
  • [sorry, too many details for me to share well]

Californian’s Opinion of Political Outsiders, Mark Baldassare, PPIC

  • california regularly elects outsiders – Reagan, Schwarzenegger
  • flavour is often outsider vs insider, several outsiders have run recently
  • blog post on the topic – http://ppic.org/main/blog_detail.asp?i=1922
  • they favour new ideas over experience
  • 3 things are important – approval ratings of elected officials, people who prefer outsiders give officials lower approval, negative attitudes of the two party system
  • majority think a third party is needed – more likely to be interested in new ideas over experience
  • [sorry, too many details for me to share well]

Trump’s Beguiling Ascent: What 50-State Polling Says About the Surprise GOP Frontrunner, Jon Cohen & Kevin Stay, SurveyMonkey

  • 38% of people said they’d be scared if trump is the GOP nominee
  • 25% would be surprised
  • 24% would be hopeful
  • 21% would be angry
  • 14% would be excited
  • List is very different as expected between democrats and republicans, but not exactly opposite
  • quality polling is scale, heterogeneity , correctable self-selection bias
  • most important quality for candidates is standing up for principles, strong leader, honest and trustworthy – experience is lowest on the list
  • Views on Trump’s muslim statement change by the minute – at the time of this data: 48% approve, 49% disapprove, split as expected by party
  • terrorism is the top issue for republicans; jobs AND terrosiam are top for independants; jobs is top for democrats
  • for republicans – day before paris 9% said terrorism was top, after paris 22%
  • support for Cruz is increasing
  • half of trump voters are absolutely certain they will vote for trump; but only 17% of bush voters are absolutely certain
  • among republicans, cruz is the second choice even among trump voters
  • trump has fewer voters who go to religious services weekly, least of all candidates; carson and cruz are on the high end
  • trump voters look demographically the same but carson has fewer male voters and cruz has fewer female voters
  • trump voters are much less educated, rubio voters are much more educated

Improvements to survey modes #PAPOR #MRX 

What Are They Thinking? How IVR Captures Public Opinion For a Democracy, Mary McDougall, Survox

  • many choices, online is cheapest followed by IVR followed by phone interview
  • many still do not have internet – seniors, non-white, low income, no high school degree
  • phone can help you reach those people, can still do specific targeting
  • good idea to include multiple modes to test for any mode effects
  • technology is no longer a barrier for choosing a data collection strategy
  • ignoring cell phones is poor sampling
  • use labor startegically to allow IVR
  • tested IVR on political polling, 300 completes in 2.5 hours, met the quotas, once a survey was started it was generally completed

The Promising Role of Fax in Surveys of Clinical Establishments: Observations from a Multi-mode Survey of Ambulatory Surgery Centers, Natalie Teixeira, Anne Herleth, and Vasudha Narayanan, Weststat; Kelsey O’Yong, Los Angeles Department of Public Health

  • we often want responses from an organization not a company
  • 500 medical facilities, 60 questions about staffing and infection control practices
  • used multimode – telephone, postal, web, and fax
  • many people requested the survey by fax and many people did convert modes
  • because fax was so successful, reminder calls were combined with fax automatically and saw successful conversions to this method
  • this does not follow the current trend
  • fax is immediate and keeps gatekeepers engaged, maybe it was seen as a novelty
  • [“innovative fax methodology” so funny to hear that phrase. I have never ever ever considered fax as a methodology. And yet, it CAN be effective. 🙂 ]
  • options to use “mass” faxing exist

The Pros and Cons of Persistence During Telephone Recruitment for an Establishment Survey, Paul Weinfurter and Vasudha Narayanan, Westat

  • half of restaurant issues are employees coming to work ill, new law was coming into effect regarding sick pay
  • recruit 300 restaurants to recruit 1 manager, 1 owner, and a couple food preparers
  • telephone recruitment and in person interviews, english, spanish, mandarin, 15 minutes, $20 gift card
  • most of the time they couldn’t get a manager on the phone and they received double the original sample of restaurants to contact
  • it was assumed that restaurants would participate because the sponsor was health inspectors, but it was not mandatory and they couldn’t be told it was mandatory, there were many scams related to this so people just declined, also all of the health inspectors weren’t even aware of the study
  • 73% were unreachable after 3 calls, hard to get a person of authority during open hours
  • increased call attempts to five times, but continued on when they thought recruitment was likely
  • recruited 77 more from people who were called more than 5 times
  • as a result, data were not limited to a quicker to reach sample
  • people called up to ten times remained noncommittal and never were interviewed
  • there wasn’t an ideal number of calls to get maximum recruits and minimum costs
  • but the method wasn’t really objective, the focus was on restaurants that seemed like they might be reachable
  • possibly more representation than if they had stopped all their recruitment at five calls
  • [would love to see results crossed by number of attempts]

Uses of survey and polling data collection: practical and ethical implications #PAPOR #MRX 

Live blogged at #PAPOR in San Francisco. Any errors or bad jokes are my own.

Are California’s Registered Independents Shy Partisans?, David Kordus, Public Policy Institute of California

  • number of independant voters has doubled in the last twenty years
  • automatic voter registration via the DMV will add new voters
  • independents are not one homogenous group
  • on average, they really are the middle between republicans and democrats, not necessarily more moderate

Exploring the Financial Landscape Facing Veterans in Nevada: Financial Literacy, Decision-making, and Payday Loans, Justin S. Gardner & Christopher Stream, UNLV, Runner-Up Student Paper Competition Winner

  • payday lending only started in the 1990s, more of them in military areas
  • largest security clearance issues were financial, capped interest rate of payday loans
  • 375 respondents, lots of disabled veterans who can’t work
  • use as medical loans is very low, many use it to pay off student loans or other debts, paying for housing also major use
  • most learned about it from tv commercials, or friends and family. If family are encouraging them to do this, something needs to change
  • people who don’t feel prepared for emergencies are more likely to use
  • majority had salary under $50 000, likely to need another loan in the future
  • 20% had used payday, it is cyclical, once you’re in the cycle it’s difficult to break out of it
  • half people could walk there from their home, didn’t need a car

What Constitutes Informed Consent? Understanding Respondents’ Need for Transparency, Nicole Buttermore, Randall Thomas, Frances M. Barlas, & Mansour Fahimi, GfK

  • biggest threat is release of name of participant but should participants be told sponsor of the study?
  • problem is nonresponse and survey bias if people know who the sponsor is
  • 6% thought taking a survey could have a negative impact on their life – worried about data breach, who has access to data, company might be hacked, improper use of data, questions might make me feel uncomfortable
  • 95% think surveys have no or minimal risk to my mental health – about 23% have quit a survey because it made them feel uncomfortable
  • about 20% said a survey has made them feel very uncomfortable – ask abour race, income, too much personal information, can’t give the exact answer they want to, feel political surveys are slanted, surveys are boring, don’t know how to answer the question
  • respondents want to know how personal information will be used and how privacy will be protected
  • want to know how long it will take, the topic, and the points for it
  • about twenty percent want to know company doing the research and company paying for the research

Recent Changes to the Telecommunications Consumer Protection Act, Bob Davis, Davis Research

  • this is not legal advice
  • TCPA issue is regarding calls using automated telephone equipment
  • lawyers like to threaten to sue but settle
  • vicarious liability – responsibility of the superior for the acts of their subordinates, i.e., contract work, sponsor of research
  • any phone with a redial button is an autodialer – so only the old phones where you stick your finger in the hole and turn the dial is not an autodialer
  • if you can get permission, then get it
  • regularly scrub your landline system to make sure there are no cell phones in it
  • use a non-predictive dialing system
  • ask that suppliers are TCPA compliant
  • international partners dialing into the US need to follow the rules as well
  • talk with your lawyer ahead of time so you can say you have already talked to a lawyer and they don’t think you are weak

Analysis, design, and sampling methods #PAPOR #MRX 

Live blogged at #PAPOR in San Francisco. Any errors or bad jokes are my own.

Enhancing the use of Qualitative Research to Understand Public Opinion, Paul J. Lavrakas, Independent Consultant; Margaret R. Roller, Roller Research

  • thinks research has become to quantitative because qual is typically not as rigorous but this should and can change
  • public opinion in not a number generated from polls, polls are imperfect and limited
  • aapor has lost sight of this [you’re a brave person to say this! very glad to hear it at a conference]
  • we need more balance, we aren’t a survey research organization, we are a public opinion organization, our conference programs are extremely biased quantitative
  • there should be criteria to judge the trustworthyness of research – was it fit for purpose
  • credible, transferable, dependability, confirmability
  • all qual resaerch should be credible, analyzable, transparent, useful
  • credible – sample repreentation and data collection
  • do qual researchers seriously consider non-response bias?
  • credibility – scope deals with coverage design and nonresponse, data gathering – information obtained, researcher effects, participant effects
  • analyzability – intercoder reliability, transcription quaity
  • transparency – thick descriptions of details in final documents

Comparisons of Fully Balanced, Minimally Balanced, and Unbalanced Rating Scales, Mingnan Liu, Sarah Cho, and Noble Kuriakose, SurveyMonkey

  • there are many ways to ask the same question
  • is it a good time or a bad time? – fully balanced
  • is it a good time or not? – minimally balanced
  • do you or do you not think it is getting better?
  • are things headed in the right direction?
  • [my preference – avoid introducing any balancing in the question, only put it in the answer. For instance: What do you think about buying  a house? Good time, Bad time]
  • results – effect sizes are very small, no differences between the groups
  • in many different questions tested, there was no difference in the formats

Conflicting Thoughts: The Effect of Information on Support for an Increase in the Federal Minimum Wage Level, Joshua Cooper & Alejandra Gimenez, Brigham Young University, First Place Student Paper Competition Winner

  • Used paper surveys for the experiment, 13000 respondents, 25 forms
  • Would you favor or oppose raising the minimum wage.
  • Some were told how many people would increase their income, some were told how many jobs would be lost, some were told both
  • Negative info opposed a wage increase, positive info in favor of wage increase, people who were told both opposed a wage increase
  • independents were more likely to say don’t know
  • negatively strongly outweighs the good across all types of respondents regardless of gnder, income, religion, partyID
  • jobs matter, more than anything

Mobile devices and modular survey design by Paul Johnson #PAPOR #MRX 

Live blogged at the #PAPOR conference in San Francisco. Any errors or bad jokes are my own.

  • now we can sample by individuals, phone numbers, location, transaction
  • can reach by an application, eail, text, IVR but make sure you have permission for the method you use (TCPA)
  • 55+ prefer to dial an 800 number for a survey, young perfer prefer an SMS contact method; important to provide as many methods as possible so people can choose the method they prefer
  • mobile devices give you lots of extra data – purchase history, health information, social network information, passive listening – make sure you have permission to collect the information you need; give something back in terms of sharing results or hiding commercials
  • Over 25% of your sample is already taking surveys on a mobile device, you should check what device people are using, skip questions that wont render well on small screens
  • remove unnecessary graphics, background templates are not helpful
  • keep surveys under 20 minutes [i always advise 10 minutes]
  • use large buttons, minimal scrolling; never scroll left/right
  • avoid using radio buttons, aim for large buttons intead
  • for openends, put a large box to encourage people to us a lot of words
  • mobile open ends have just as much content although there may be fewer words, more acronyms, more profanity
  • be sure to use a back button if you use auto-next
  • if you include flash or images be sure to ask whether people saw the image
  • consider modularizing your surveys, ensure one module has all the important variables, give everyone a random module, let people answer more modules if they wish
  • How to fill in missing data  – data imputation or respondent matching [both are artificial data remember! you don’t have a sense of truth. you’re inferring answers to infer results.   Why are we SOOOOO against missing data?]
  • most people will actually finish all the modules if you ask politely
  • you will find differences between modular and not but the end conclusions are the same [seriously, in what world do two sets of surveys ever give the same result? why should this be different?]
%d bloggers like this: