Templates Create a poll

How to avoid bias in survey questions

Bias creeps in when a question or its answers push people towards one response. It is rarely deliberate — and usually easy to fix once you know what to look for.

Leading questions

A leading question signals the “right” answer through its wording.

Instead of “How much did you enjoy our excellent workshop?”

Better “How would you rate the workshop?”

“Excellent” and “how much did you enjoy” both assume a positive experience. A neutral rating lets people who were bored say so.

Loaded assumptions

A loaded question takes something for granted that may not be true for everyone answering.

Instead of “Which of our new features do you use most?”

Better First “Have you used any of the features added this month?”, then the feature question for those who have.

People who haven’t used any of them are forced to pick one anyway, and the results overstate how much the features are used. If a question doesn’t apply to everyone, make it optional so people can skip it.

Unbalanced answer scales

A scale should have as many steps on the negative side as on the positive side.

Instead of “Excellent / Very good / Good / Poor”

Better “Very good / Good / Neither good nor poor / Poor / Very poor”

The first scale has three positive answers and one negative one, so the average is pulled upwards before anyone answers. A 0–10 rating question avoids the problem: both ends are labelled (“0 · Worst”, “10 · Best”) and the steps are equal.

The order of the answers

When people skim a long list, options near the top tend to get a little more attention. Two ways to reduce that in VotePoll:

  • Turn on Randomize option order for multiple-choice and ranking questions, so each participant sees the options in a different starting order.
  • For a list with a natural order (dates, sizes, age groups), keep that order — shuffling it would only make answering harder.

The order of questions matters too. Ask general questions before specific ones: “How satisfied are you with your job overall?” gives a different picture when it comes straight after five questions about workload.

Answers people feel they “should” give

On topics like exercise, donations or following company rules, people tend to give the answer that sounds best, especially when they think someone will know it was them.

  • Don’t ask for names unless you need them. VotePoll only asks participants for a name if you turn that on.
  • Leave Show who voted for what off, so answers aren’t linked to names in the results.
  • Say in the poll description who will see the answers and what you will do with them.
  • Ask about behaviour in a concrete time frame (“in the past month”) rather than in general.

How to run an anonymous feedback survey goes into this in more detail.

Who answers

Even a perfectly worded poll only tells you what the people who answered think. If you share an employee survey in one team’s chat, you have that team’s view, not the company’s. Share the link where everyone you want to hear from will see it, and give people enough time before the poll closes.

A quick checklist

  1. Does any question contain “and” joining two topics? Split it.
  2. Does any question contain a word that praises or criticises? Remove it.
  3. Does every question apply to everyone? If not, make it optional.
  4. Are the scales balanced and do the options cover every answer without overlapping?
  5. Would people answer differently if their name were attached? Then don’t attach it.