Recipe for a Good Survey Design
It may seem like it’s easy to cook up a good survey. You write up some questions about the topic of interest, program your survey, and send it out. Done! There’s just one problem. The survey data comes in and the data just doesn’t “taste” good. How can you make your survey better? Here are a few tips:
1. Target the right audience
Before you launch your survey you need to ask yourself who you want to respond and how to best reach this audience. This seems pretty easy right? You’d think so, but not always. Imagine you are trying to design a program to increase technology skills of a group of people who are not tech savvy. You want to know what types of training they need. Collecting data from social media platforms might not be the best approach in this instance. A better approach might be a paper survey, phone survey, or focus group.
Before deciding where and how to launch your survey, ask yourself:
Who do I want to respond to my survey?
Are all the types of respondents (e.g. male/female, age groups, types of participants/consumers, etc.) likely to use the platform I’m using to launch my survey?
Do I need to launch the survey on multiple platforms to reach all of my intended audience?
If I’m offering incentives to complete the survey, will the incentives appeal to all the types of individuals I’m targeting?
2. Keep it short
Have you ever started taking a survey and after a while started filling in answers without reading the question just so you could finish? Surveys that are too long tend to encourage this type of behavior and lower the quality of your data. If you’re using the survey information to make key decisions its better to have less data that is high quality than more data that is low quality.
3. Stay focused
It’s easy to get excited and to try and ask everything you could possibly want to know about the program, product, or group you’re researching. This can be problematic. Not only can it lead to surveys that are way too long, which may cause people to skip through key questions, but constantly changing topics can also be confusing for the survey respondent. Surveys that try to cram multiple topics and research questions may also run the risk of under developing each topic. Key decisions should not be made based unless a topic is explored more fully.
4. Be specific and intentional in your wording
The other day I received a call to participate in a survey over the phone. The first question I was asked was, “How do you feel California is doing?”. My immediate thought was, “In respect to what? economics? politics? the environment?” Now there may be a reason for asking a question this way that I don’t know about, but if an organization wants to know my attitudes about a particular issue they need to lead with that. Try and ask yourself what are the possible ways a person could respond to this question. If you feel like there are too many possible responses that are off topic try and rephrase the question to narrow down the range of possible responses. It also never hurts to do a test run to see how people answer.
5. Create balanced item response options
There are many different types of response scales you can use in a survey. Each option measures a different area – frequency, attitudes, level of agreement, level of participation, and so on. However, there is one thing that all your scales should have in common – balance. For example, a five-point agreement scale has two options expressing agreement (agree or strongly agree), two options expressing disagreement (disagree or strongly disagree), and one option for neither agree or disagree.
6. Use “N/A” or “I don’t know”
Another big mistake can occur when a respondent is forced to select a response option because he or she does not have the knowledge to form an opinion, and “N/A” or “I don’t know” is not an available option. Make sure you add an “N/A” option where appropriate. Otherwise the quality of your data may decrease.
7. Demographic questions are important
It is important to include demographic questions in your survey, and to explore how the respondents map on to your
intended audience. For example, imagine your survey data shows that 95% of the respondents recommend a change in your products color pallet. You go to your boss and recommend the change. After the change is made sales go down dramatically. Looking back at the survey results you realize that a majority of the survey respondents were female and in their early twenties, but the majority of the buyers of your products are male and in their late forties. The two groups have very different ideas of what color the product should be. By including demographic survey questions and identifying response patterns you are better prepared to make informed decisions based on the findings.
