How to do a survey in six steps
Question: What's the difference between a questionnaire and a survey?
Answer: A questionnaire is a series of questions and answers on a topic; a survey is the overall process of obtaining useful information using a questionnaire.
Question: OK then, what are the steps in the process?
Answer: Until recently, I was stumped on that one, but after a lot of help from others and some thinking, I have an answer for you...
Answer: A questionnaire is a series of questions and answers on a topic; a survey is the overall process of obtaining useful information using a questionnaire.
Question: OK then, what are the steps in the process?
Answer: Until recently, I was stumped on that one, but after a lot of help from others and some thinking, I have an answer for you...
Summary: the six steps
- Questions: Start with the questions but don't stop there
- Goals: Work out the goals for your survey
- Users: Talk to your users about the topics in your survey
- Build: Create and test your questionnaire
- Deploy: Send it out and watch the responses as they come back
- Analyze: Investigate the data and create your report
1. Questions: Start with the questions but don't stop there
You have to know what questions you want to ask. And the way to get to the right questions is to iterate: generate some questions, work on them, revise, try again.2. Goals: Work out the goals for your survey
One definition of 'the right questions' is: ones that gather useful answers. Think about the answers you will get. What decision or change will you make based on the answers?Then think about who you want to answer those questions. Who are the right users to reach, and how will you get to them?
If you don't plan to make a decision or to change anything: stop now. Save your users' goodwill for another time when you really, really need their answers.
Once you've got your goals sorted out, iterate back to step 1 - revise your questions to match the survey goals.
3. Users: Talk to your users about the topics in your survey
Now you know what answers will be useful, it's time to find out if your users want to give you those answers or not. Meet some of them. (Face-to-face is best, but phone meetings will do in a pinch). This is also a chance to find out whether your plans for reaching your users are practical.Do they understand your questions in the same way that you do? Do they want to talk to you about those topics? Is there another topic, more important to them, that you need to hear about?
If you talking to your users seems like a problem to you, then stop working on your survey and start working on that problem instead.
Once you've talked to your users about the topics in your survey, iterate back to step 1. You'll have lots of ideas about how to improve your questions. Might be a good idea to have another little look at step 2 while you're about it.
4. Build: Create and test your questionnaire
All those great questions ... time to put them together. There are survey tools at every price from free to thousands, with features to match and learning curves to match the features. Or think about maybe an email questionnaire, or a Word document. (OK, maybe not the Word document unless that's really the only option available). Or if you're a programmer, or have some around the place, then try roll-your-own.This is also the time to:
Then test, test, test. Try it out yourself, usability test it, run a pilot (same survey but on a very small sample). Don't forget to test the pre-notice, invitation and reminders too.
If you don't have time to test the questionnaire, you don't have time to do the survey.
(And guess what: those tests will make you want to go back to step 1 and tweak your questions some more.)
5. Deploy: Send it out and watch the responses as they come back
This step is the most nerve-wracking but also the most fun. At last, you hit launch and out go your questionnaires, or the links to them, or whatever deployment you have chosen.Watch those responses carefully, especially the first few. Anything odd happening? Do the answers seem sensible, individually and collectively? If not, pull the plug quickly. Better to grovel a bit than to waste that precious user goodwill on a survey that's not working properly.
6. Analyze: Investigate the data and create your report
And at last, you can settle down to getting to grips with your data. Chances are you'll have to to a bit of clean-up, making some decisions about part-completed questionnaires and other anomalies.It's also a good idea to do one last round of iteration, going back to your goals to make sure that your report refers to them.
Summary: the six steps
- Questions: Start with the questions but don't stop there
- Goals: Work out the goals for your survey
- Users: Talk to your users about the topics in your survey
- Build: Create and test your questionnaire
- Deploy: Send it out and watch the responses as they come back
- Analyze: Investigate the data and create your report
Comments
Great overview of the key steps. The only thing I'd challenge is the placement of Questions before Goals.
I know that people — designers and/or stakeholders — often start with questions in mind. But I don't see how any project can really be effective if it doesn't set out to establish goals before doing anything else. So while the practical reality might kind of:
1. Questions
2. Goals
I think it's best to encourage people to do:
1. Goals
2. Questions
You could even include a note that this order doesn't mean you have to throw out those existing questions. Rather, they just need to be put aside for a bit and/or used as a prompt for discussing goals (e.g. "why do you want to ask that question? what information need will it solve?").
[One other very minor thing: the title "Users" made me think you were going to talk about researching who your users are instead doing research with users, to essentially test your questionnaire.)
Posted by: Jessica Enders | December 5, 2011 1:48 AM
Jessica,
Thanks for your comment. I'd also love to encourage people to start with Goals, but I find that they just don't!
So I've gone for a process that I believe is realistic, and then hope to lure everyone into iterating. The key is: start with the questions *but don't stop there*.
I suppose it's also my background in project management. My experience of projects is that they nearly always start somewhere in the middle, even if it's close to 'the beginning of the middle'; it's very, very rare to come across one where you really start absolutely from nothing. There's nearly always some prior work that you have to take account of.
I'll brood about the name of the 'users' step. It's an unexpected step for many people and has been quite a challenge to describe correctly - and to name. But very, very important.
Posted by: Caroline Jarrett | December 7, 2011 5:28 AM