Effective Survey Design

If you have already reviewed To Survey or Not to Survey and determined that a survey is the best approach for your project, consider these guiding questions and best practices as you begin designing your questionnaire.

Within the Design step, there are several smaller steps. The first step is determining the specific information you are hoping to gather. If you have already identified research objectives for your study, that’s an excellent place to start! Your overarching objectives for the study should closely align with the data you’re hoping to gather to inform your subsequent actions. As you work to identify the information needed to address your research objectives, keep in mind that survey questionnaires can only collect information about experiences, opinions, personal characteristics, and behaviors. A survey cannot directly measure learning. 

Many survey designers find it helpful to brainstorm a list of all the questionnaire statements, also known as items, that come to mind while considering what data should be collected. This can be a helpful way to get the creative juices flowing! It is important, however, to then review each item closely for alignment with your study objectives. Once you have a handful of items drafted, ask yourself the guiding questions below and take some notes for each item. 

• Is this item necessary to acquire some of the information I need? 

• Can this item provide accurate and relevant data? 

• What is the justification for including this item? 

• Which study objectives does this item align with? 

• Is there a more appropriate data collection method for obtaining this information? 

• Does relevant or related data already exist within the University or elsewhere? 

It’s important to note that this brainstorming process is iterative, and often involves several rounds of drafting and revising questionnaire items. Items that do not align with the research objectives or do not have a strong justification for being included in the questionnaire should be removed from consideration. Others may then be drafted to fill in any identified gaps. Any items better-suited for alternative data collection methods should similarly be eliminated, or reframed to focus on participant experiences, opinions, characteristics, or behaviors. 

Step two of the survey design process involves applying a new lens to your draft items. As you read through each of the items in your first draft, consider who can provide the information you are planning to collect. Do the draft items apply to a wide variety of people, or only to one or more specific groups? Can your survey reach those audiences? Once you have identified your audience, take another pass through the draft items and consider: 

• What interest, if any, will the audience have in the topic? 

• What is the background and educational level of your audience(s)? 

• What other characteristics of this group relate to your study? How do you know? 

Step three of the design phase involves transforming your draft items into strong questions. To do so, we must first consider what type of questions will collect the most meaningful data for the study at hand. The cards below provide a brief overview of some of the most common types of survey questions, as well as key benefits and drawbacks associated with each.  

Understanding the various question types employed by surveys is a fundamental piece of the survey design puzzle, but crafting strong survey questions goes beyond mere familiarity with question formatting. Crafting strong survey questions also entails considering the clarity, specificity, and neutrality of each question written to ensure that respondents can comprehend and accurately respond to the questions asked.

Clarity and Simplicity: Use clear and simple language to ensure that respondents understand.

Clear, simple

How many days per week do you exercise?

Precision: Avoid vague language or ambiguous wordings to prevent misinterpretations by participants.

Precise

How old were you on January 1, 2024?

Avoid “Double-barreled” Questions: Ask about one concept at a time to prevent confusion.

Separated

Q1: How would you rate the effectiveness of Nike athletic shoes?

Q2: How would you rate the affordability of Nike athletic shoes?

Neutrality: Phrase questions in a neutral and unbiased manner. Don’t use language that leads respondents or makes assumptions.

Neutral

Please rank the technology companies listed below in order of preference.

The final step of designing a survey is creating an instrument that will allow respondents to provide thoughtful and accurate answers to the questions you have crafted. In today’s digital age, online survey platforms such as Qualtrics offer convenient and efficient means for collecting survey data.