Survey Best Practices

SCTR Communications
February 20, 2023
Online Survey on Computer by AndreyPopov from Getty Images| Canva Pro
Online Survey on Computer by AndreyPopov | Canva Pro

Have you ever found yourself in need of feedback about behaviors and attitudes? Do you manage a program and want to learn from your stakeholders how well you may be meeting their needs? Do you want to collect data for a research project but find yourself short on resources? Well, then a survey may be for you.

We’ve taken a look at some of the best advice out there (see Learn More below) to bring you this easy-to-use tool as a reference to assist in survey development. While it’s not meant to be comprehensive, the following summarizes some of the key tips shared in the references reviewed. Whether it’s your first time tasked with creating a survey, or you’re a seasoned surveyor, you’ll find some tips to get you started and some gentle reminders to assist your survey in doing the job you’re hoping it will do.

 

Survey Design

Define objectives, detail desired data, look for existing sources for data, write the questions, pilot/pretest, revise as need, conduct survey. Survey development pro tips: Circle back to review the survey’s objectives throughout the planning process. Stay connected with survey stakeholders to ensure questions will generate desired data. 

 

Survey Question Basics

A good question will…

  • be understandable.
  • be interpreted in the same way by all.
  • allow for accurate responses.
  • be something that respondents can and will answer.

 

General Rules for Survey Questions

1. Order the Questions

When ordering your questions, consider …

  • Starting with an easy open-ended question central to the survey’s topic.
  • Grouping similar questions and creating sections for multiple categories of questions that follow a logical order.
  • Positioning sensitive questions in the middle of the survey.
  • Mixing difficult questions with engaging questions to limit the burden on the respondent.
  • Ending with the questions you may need to document demographics, affiliations, and credentials.

2. Plan Your Questions

Research shows… 

  • The longer the survey, the lower the response rate.
  • The longer the survey, the greater the perception of burden.

Keep your questions short and your words simple. Avoid jargon or cultural references that may not resonate with your full audience. Always ensure questions will be easily understood. Eliminate unnecessary or redundant questions.

Know Your Audience - Avoid language that is inappropriate for your full audience – this may include technical language, acronyms, or abbreviations.

3. Avoid Influencing the Response 

  • Don’t lead respondents to your preferred responses
  • Balance out positive vs. negative leads to eliminate bias

black x markWhat issues do you have with the current mentor/mentee event offerings?
(Assumes there are issues)

black x markRank the issues you have with the administrative team.
(Assumes there are issues)

black x markIn which areas has our administrative team been the most helpful?
(Assumes the team has been helpful)

green check markRate the efforts of our administrative team.
(Neutral) Response: Sliding scale from not at all helpful to very helpful

Pro Tip - Send out multiple surveys to cancel out bias, sending half with a positive lead (good – fair – poor) and half with a negative lead (poor – fair – good) in rating scales.

4. Craft Response Options

Avoid “don’t know” or “neutral” for subjective questions. These responses give recipients the option of not providing an opinion when one is needed. They can be used when seeking facts or objective data. 

Subjective:

How helpful are the following resources? Or, The following resources would help me advance my research.

(avoid “don’t know” or “neutral” if you’re seeking an opinion)

(“not applicable” may apply)

Objective:

Do you use the following resources?

5. Use Consistent Language 

In formatting questions, be consistent in… 

  • Tense
  • Point of view
  • Response options

Balanced Response Options - Yes – No; True – False; Good – Fair – Poor; Too much – About right – Too little; Always – Often – Sometimes – Seldom – Never; Extremely – Very - Moderately – Slightly - Not at all; Excellent - Above Average – Average - Below Average - Very Poor

6. Limit Recall Burden

  • Recalling specific details from the past can be challenging.
  • Make requests reasonable and clear rather than open to interpretation.
  • Specifics should be consistent across questions and the timeline used should be relevant to the task.

black x markIn the past month…
If the survey is taken November 24, is “the past month” 10/24-11/24, the month of October (the last full month) or the first 24 days of Nov.?
green check markIn October 2022… or, In the past 30 days…

7. Match Responses to Questions

  • Make sure question roots agree with the response options.
  • Create responses to multiple choice questions and rating scales that are mutually exclusive (unless multiple responses are enabled).
  • Design questions to provide the specifics needed to get an appropriate response.

How many times a month do you meet with… 

Are you looking for frequency (1 <-> 10), ranking (more than <-> less than), or opinion (too many <-> not enough).

In asking for frequency, response options need to have numbers or number ranges and be mutually exclusive.

Don't use never, sometimes, often; Use instead frequency measures like 1-3, 4-6, 7-9, >9 times

8. Avoid Double-Barreled Questions

Don’t ask more than one question simultaneously.

  • How would you rate the quality of our equipment and ease of use?
  • Were you helped by a skilled technician or provided instructions to use the equipment safely?

Design individual questions to collect the specific responses needed.

  • How would you rate the quality of our equipment?
  • How easy was the equipment to use?
  • Were you provided adequate instruction specific to your safe use of the equipment?
  • Was a skilled technician available to assist during your use of the equipment?

9. Protect Data Integrity

Customize response fields (i.e., add a field qualifier) to provide guides for how data should be provided. 

  • last name, first name 
  • MM-DD-YYYY
  • ###-###-####

Individuals may provide the same category of data in a wide variety of formats. When compiling your results, it is useful to have all responses in the same format to avoid the need for reformatting for consistency. Survey responses should not be manipulated after they are provided to avoid creating data errors.

Guide Response Formatting - A phone number may be provided in a variety of formats, (123)456-1890; 123-456-7890; 1234567890,  so, it’s best to provide a guide for the formatting of responses.  Note: For REDCap surveys, this can be done in the Field Note.

 

Tips for Increasing Response Rates

1. Don’t Antagonize

Avoid using inflammatory language in questions. Respondents may quit the survey or provide an emotional response prompted by the way a question is presented. How a question will be perceived may be hard to predict, but try to avoid questions that run the risk of upsetting or antagonizing respondents. In particular, consider how questions may be perceived as threatening.

2. Provide Incentive

Tangible rewards (e.g., money, gift card, swag) may provide incentives but may also create bias. They could attract respondents only interested in the incentive. Consider appealing to your respondents’ desire to help, offer feedback and enable change.

  • Provide an introduction with some explanation for why the survey is being conducted.
  • Establish trust by clarifying how the results will be used.
  • Identify the change (direct and indirect) that may be generated.

3. Reconsider Required Responses

Requiring question responses may lead to survey drop-offs. It may not always be easy to predict which questions may be perceived as difficult or uncomfortable to answer. It is better to receive a higher number of surveys, even with incomplete responses, than the lower number of ”completed” surveys you will receive if the demands of the survey turn respondents away.

 

Download the Infographic

Survey Best Practices (PDF)

 

Learn More

AEM Education & Training | Educator's blueprint: A how-to guide for survey design 

Otolaryngology–Head and Neck Surgery | A Practical Guide to Surveys and Questionnaires

Pew Research Center | Writing Survey Questions

SurveyMonkey.com | How much time are respondents willing to spend on your survey?