Creating Survey Questions

Defining a Survey

A survey is a method of collecting and gathering information from a predefined group.

In the context of research, program evaluations, and quality improvement activities, a survey is a method of collecting and gathering information from a predefined group (e.g., clients from a specific program) (Anheier & Scherer, 2015).

Defining a Questionnaire

A questionnaire is a set of questions.

A questionnaire is a set of questions and is commonly used to survey information from a predefined group.

Well-Crafted Questions = Quality Data = Valuable Findings

The information gathered from the predefined group is only valuable if the survey questions were thoughtfully designed and free from common mistakes.

9 Common Mistakes to Avoid When Creating Survey Questions

Multi-Barreled Questions

A multi-barreled question asks about multiple issues in one question, making it hard for people to answer clearly. For example, "How satisfied are you with the waiting time and the communication from your physician during your visit at hospital?" combines two topics, so one answer might not reflect opinions on both. To avoid confusion, split such questions into separate ones.

Multi-Barreled Question Example

Overlapping Response Options

Overlapping response options occur when the choices in a survey question are not mutually exclusive, meaning a person could potentially choose more than one option because the ranges or categories overlap. This creates confusion, as it's unclear which answer best applies to the person.

Overlapping Response Option Example

Unbalanced Response Options

Unbalanced response options occur when the choices in a survey question are not evenly distributed or do not represent a full spectrum of possible answers. This can lead to biased responses, as some options may seem more appealing or relevant than others, for example, in a survey question asking: “How would you rate the quality of care you received at hospital?” with response options a) Excellent, b) Very Good, c) Good, and d) Poor. This is unbalanced because it has more positive options (Excellent, Very Good, Good) than negative ones (Poor). People may feel pushed toward selecting a more favourable option.

Unbalanced Response Option Example

Leading Questions

A leading question subtly suggests a specific answer, influencing the person's response. It often contains wording that bias the person toward a particular choice rather than allowing for an unbiased answer. For example, the question: "How much did you value the excellent care you received from our healthcare team?". This is leading because it assumes the care they received was excellent, which may not be accurate for everyone. It pressures the person to agree with the assumption, potentially skewing the results. To avoid leading questions, it's important to phrase questions neutrally, allowing the person to provide their honest opinion without being influenced by the wording.

Leading Question Example

Acronyms Not Defined At Least Once

Acronyms not defined at least once in survey questions can confuse people. When acronyms are used without clarification, some people may not be familiar with the term or its meaning, potentially leading to inaccurate or incomplete answers. Acronyms should be defined the first time they are referenced in the survey. This ensures that all people understand the terminology and can respond more accurately.

Acronyms Not Defined At Least Once Example

Yes/No Response Options for Complex Questions

Using "Yes/No" options for complex questions can oversimplify the issue and lead to inaccurate or incomplete responses. Complex questions often involve multiple factors, nuances, or gray areas that a simple binary choice cannot capture. For example, if the question asks: "How satisfied are you with the overall quality of care at the hospital?" (Yes/No). The Yes/No response option for this question doesn't account for people who may have partial satisfaction. A better approach would be to offer more nuanced options, such as a) Very Satisfied, b) Satisfied, c) Neutral, d) Dissatisfied and e) Very Dissatisfied.

Yes/No Response Option for Complex Question Example

Excluding “Prefer Not to Answer” Response Options

Excluding a "Prefer not to answer" option in a survey can lead to biased or incomplete data. People who are uncomfortable or unwilling to answer a question may be forced to skip it (which could result in incomplete responses) or select an inaccurate answer to proceed. For example, the question "How long have you been a patient at this hospital?" might make some people uncomfortable. Without a "Prefer not to answer" option, a person who doesn't want to disclose their length of stay may leave the question blank or choose a misleading answer, skewing the data. If a "Prefer not to answer" is not provided, the survey instructions can indicate that people are welcome to skip any question they don't feel comfortable answering. This ensures that people are not forced to provide inaccurate or uncomfortable responses and helps maintain the quality of the data collected.

Excluding “Prefer Not to Answer” Response Option Example

No Survey Purpose Provided

Not providing the purpose of the survey in the instructions or introductory section can lead to confusion, mistrust, and incomplete responses. When people do not understand why they are being asked to provide certain information, they may feel reluctant to participate fully. To avoid this, it's essential to clearly state the purpose of the survey at the beginning, explaining why the data is being collected, how it will be used, and assuring people that their information will be kept anonymous and/or confidential (understand the difference between "anonymous" and "confidential" in the next section). This transparency helps build trust, encourages honest responses, and improves the quality of the data collected.

Survey Purpose Not Included Example

Using Anonymous and Confidential Interchangeably

Using "anonymous" and "confidential" interchangeably in a survey can lead to confusion about how people's data will be handled. While both terms relate to privacy, they have different meanings, and misusing them can cause misinterpretations.

Anonymous means people's identities are not collected or linked to their responses. There is no way to trace answers back to an individual.

Confidential means that people's identities are collected, but the information will be kept secure and only shared with authorized individuals or used in aggregate form. People's identities are known to the researchers but protected.

For example, if a survey states that responses are "anonymous", but then asks for identifying information like email addresses or demographic details, it creates a contradiction. People may hesitate to answer truthfully. It is essential to avoid misinterpretations using the correct term based on how the team will handle responses and clearly explain them to the participants. If responses are genuinely anonymous, ensure no identifying information is collected. If confidential, clarify how the identifying information is being kept secure.

Using Anonymous and Confidential Interchangeably


References

Anheier, H. K., & Scherer, N. (2015). Survey research centers and companies. In J. D. Wright (Ed.), International encyclopedia of the social & behavioral sciences (2nd ed., pp. 747–751). Elsevier. https://doi.org/10.1016/B978-0-08-097086-8.41079-2