It’s a dog eat DIY world at the #AMSRS 2015 National Conference


  What started out as a summary of the conference turned into an entirely different post – DIY surveys. You’ll just have to wait for my summary then!

My understanding is that this was the first time SurveyMonkey spoke at an #AMSRS conference. It resulted in what seemed to be perceived by the audience as a controversial question and it was asked in an antagonistic way – what does SurveyMonkey intend to do about the quality of surveys prepared by nonprofessionals. This is a question with a multi-faceted answer.

First of all, let me begin by reminding everyone that out of all the surveys prepared by professional, fully-trained survey researchers, most of those surveys incorporate at least a couple of bad questions. Positively keyed grids abound, long grids abound, poorly worded and leading questions abound, overly lengthly surveys abound. For all of our concerns about amateurs writing surveys, I sometimes feel as though the pot is calling the kettle black.

But really, this isn’t a SurveyMonkey question at all. This is a DIY question. And it isn’t a controversial question at all. The DIY issue has been raised for a few years at North American conferences. It’s an issue with which every industry must deal. Taxis are dealing with Uber. Hotels are dealing with AirBnB. Electricians, painters, and lawn care services in my neighbourhood are dealing with me. Naturally, my electrical and painting work isn’t up to snuff with the professionals and I’m okay with that. But my lawn care services go above and beyond what the professionals can do. I am better than the so-called experts in this area. Basically, I am the master of my own domain – I decide for myself who will do the jobs I need doing. I won’t tell you who will do the jobs at your home and you won’t tell me who will do my jobs. Let me reassure you, I don’t plan to do any home surgery.

You can look at this from another point of view as well. If the electricians and painters did their job extremely well, extremely conveniently, and at a fair price, I would most certainly hire the pros. And the same goes for survey companies. If we worked within our potential clients’ schedules, with excellent quality, with excellent outcomes, and with excellent prices, potential clients who didn’t have solid research skills wouldn’t bother to do the research themselves. We, survey researchers, have created an environment where potential clients do not see the value in what we do. Perhaps we’ve let them down in the past, perhaps our colleagues have let them down in the past. 

And of course, there’s another aspect to the DIY industry. For every client who does their own research work, no matter how skilled and experienced they are, that’s one less job you will get hired to do. I often wonder how much concern over DIY is simply the fear of lost business. In this sense, I see it as a re-organization of jobs. If research companies lose jobs to companies using DIY, then those DIY company will need to hire more researchers. The jobs are still there, they’re just in different places. 

But to get back to the heart of the question, what should DIY companies do to protect the quality of the work, to protect their industry, when do-it-yourselfers insist on DIY? Well, DIY companies can offer help in many forms. Webinars, blog posts, and white papers are great ways to share knowledge about survey writing and analysis. Question and survey templates make it really easy for newbies to write better surveys. And why not offer personalized survey advice from a professional. There are many things that DIY companies can do and already do.

Better yet, what should non-DIY companies do? A better job, that’s what. Write awesome surveys, not satisfactory surveys. Write awesome reports, not sufficient reports. Give awesome presentations, not acceptable presentations. Be prompt, quick, and flexible, and don’t drag clients from person to person over days and weeks. When potential clients see the value that professional services provide, DIY won’t even come to mind.

And of course, what should research associations do? Advocate for the industry. Show Joe nonresearcher what they miss out on by not hiring a professional. Create guidelines and standards to which DIY companies can aspire and prove themselves. 

It’s a DIY world out there. Get on board or be very, very worried.

4 responses

  1. The problem may have also been that the talk was a sale pitch – there were software suppliers at that conference who had paid for exhibition booths (not us), or who had paid to attend, who have never been allowed to get main stage free advertising.

    1. Ah yes, I noticed that as well. I’ll never understand why companies think a sales pitch is the right way to go. Isn’t a great talk a great sales pitch? What it does mean is that when associations ask me for recommendations about speakers, I never recommend those folks.

  2. I disagree that the fear of losing business is what is the major motivator for criticism of DIY. The real criticism is that they allow someone to write a questionnaire with the likely quality of an unskilled amateur. I have seen the drivel used by consulting firms as questionnaires with access to these DIY solutions – and these are MBA’s! I can recall the many years of learning that it took to write a good questionnaire, always supervised by people with decades of industry experience. I used to pride my self on being at the 90% level as a quantitative researcher designing questionnaires, with the perception that there was always 10% more to learn. How could any practitioner expect to perform at quality levels without that training and supervision? The answer is clearly they cannot.

    1. Unfortunately, many people do not appreciate the skill that goes into a great questionnaire. I will personally ponder for ten minutes over what seems to be the most trivial question as i can see the pros and cons of three different ways of posing the question. It’s hard to convince some people that spending several hours to write a ten question survey has any merit. The speed of business often interferes with quality and surveys are one of the things that suffer the most. I’m glad your main concern is quality. I wish that was the driving force for all people.

%d bloggers like this: