Creating great surveys can influence the success of your business. But most times, surveys are not created to optimize on the results and effects they can produce. How do you get a high response rate and interesting answers to the questions you’re asking?
In this webinar, KISSmetrics UX Research Lead Chuck Liu discusses what he’s learned about effective survey techniques and strategies over the years. He provides insight into tips for designing survey questions to ways to get surveys in front of people and more.
As the 15th employee at KISSmetrics, Chuck has years of experience helping the product, design and sales teams figure out how to get answers to the questions they have. Besides creating surveys and performing user interviews, he also shows wire frames to new customers to figure out a better design. Needless to say, his experience ranges on a wide spectrum.
Before creating a survey, ask yourself: what do you want to learn?
Chuck says it’s important to figure out what actions you want to take after your survey is done. Surveys should not be thrown together because you simply want to do a survey. They should be created because you want to answer a specific question that you currently don’t have the answer to.
5 Tips for Designing Survey Questions
At KISSmetrics, we’re all about lean analytics. But how do surveys actually play apart in lean analytics or lean companies? For one, you pick a metric or key performance indicator (KPI) and you figure out a benchmark where you really want to move your mark on and figure out where to improve. Usually the problem is you may not have enough data to make an improvement.
Lean is all about identifying and quantifying risk.
You get to ask people in your representative demographic what can be done better and address their frustrations and pain points. This gets rid of all the headaches that you have later when you actually build your product or business out. You’re able to validate most of your assumptions beforehand. By doing this, you’re saving yourself a lot of time and resources. Surveys has helped us at KISSmetrics really figure out how to prioritize our feature set and what we build, so we don’t throw away our time.
Don’t just ask questions.
Focus on what decisions you want to make from these surveys. When you want to figure out what to ask, this is where we survey. For example, if you want to find out about how to acquire more customers or get more signups, and you’re not sure about what to pursue based on feature set, this is a great time for you to survey to ask potential or existing customers about their experiences, frustrations or pain points.
1. Keep surveys short.
Under 5 minutes is ideal. The timing is really important, because asking more questions will likely lower the response rate you have in your surveys. Survey Monkey did a study on the impact of the number of questions on ‘completion rate’ and found it leads to lower completion rates. There’s a sharp drop-off when you hit 15 questions. But what is considered good or bad when it comes to completion rate? By industry standards, 40% is awesome, 30% is great, 20% is good and about 15% is expected. At KISSmetrics, we see a 15% completion rate when it comes to our larger surveys, such as reaching out to customers on our blog to figure out what kind of blog content they’re interested in.
In Chuck’s survey experience at KISSmetrics, he’s researched current customers, potential customers and people who went through a trial and ended up going with a competitor. What he found over time is the sweet spot is at the five question limit.
In this graph, you see the graph decreasing sharply after five questions. If you can deal with low completion rates because you have a relatively large audience (hundreds of millions), then it may not be an issue for you. But for KISSmetrics as a startup with a smaller customer base, we cannot survey as much as we’d like to. We will usually try to keep surveys shorter.
How well did this perform? Chuck got a 14% click-through-rate (CTR) from nudge to screener. This relatively low percentage was expected, because a nudge isn’t necessarily a popup where you get the users full attention. He received 202 responses in 7 days and all 202 opted in for further studies. Chuck basically created a pool of 202 people to reach out to for future surveys. If you’re planning on doing a lot of questions in your surveys, watch out for questions you can ask them in a further study. If you’re giving them a shorter study and ask them to opt-in to a further study, you’ll find that because the initial experience was short and pleasing to them, they are more likely to help you out in the future.
Note: Satisfaction surveys see 10% or lower response rate.
2. Start broad then get detailed.
This is more about the flow of your survey. The completion rates decrease if you don’t get the broad questions out of the way first. This is because it’s jarring to get specific questions out of the way first and then ask broad questions.
Some broad questions that you can start with include:
- How satisfied are you?
- What is your primary motivation? This is great if you’re trying to do research on why people are on your site or using your products in general. This is especially useful if you’re a startup and trying to figure out a product-market fit.
- What best describes why you came here today? This is a great way for you to benchmark the primary motivation and what users are doing on your site well.
- What is your experience with using X? Please rate your experience on ease of use on a scale of 1-5.
- How satisfied are you with X/Y/Z? Amazon and Apple usually ask you how satisfied you are with the professionalism of support staff or technical knowledge.
3. Ask what you need to know.
Avoid what’s nice to know. This usually happens when you expose the questions to more people, the survey can suddenly get bigger with things that are nice to know versus what you need to know. You want to ask a bunch of different questions slightly not related to each other. Do the questions really help you make a decision on what you’re asking in the first place? Another way to ask this is: What are you going to DO with this survey data? Before you run the survey, you should know what you want to be able to do when you receive survey results.
4. Avoid hypothetical questions.
It’s really difficult for people to know what they would want if something doesn’t exist. If you ask: “Would you buy X if we offered it?” most of the time, people say yes, because it solves their problems and pain points. This sort of validation is something you should watch out for, because you don’t have much backbone on how it would be used. The better way to ask this is to ask about their current experience: “Please describe your frustrations with this situation and describe some workarounds.”
5. Test survey internally.
This is to spot check and to figure out if questions are phrased correctly or in a non-confusing way. This is especially important for startups or companies that have smaller customer counts. Surveying multiple times gets interpreted as spam. Unless you’ve already segmented your markets, sending multiple blasts generally annoys customers. Because you can’t send too many surveys, you need to make sure the questions are clear and make sense.
To learn about three ways to get surveys in front of people, three strategies for segmenting survey results and KISSmetrics’ favorite tools and workflows, you’ll want to watch the rest of the webinar here.