Some good practices to design multiple-choice questions

Multiple choice questions usually follow a conventional format. There is the question (sometimes called the stem) followed below by a few response selections. In line with that template, here are some recommendations for law firms or their advisors who design such questions. The graphics come from DLA Piper’s European Acquisition Finance Debt Report 2015.

  1. State the question in the form of an inquiry, not a fill-in-the-blank sentence. For example, “What is the primary method your firm uses to reduce unwanted departures of associates?” is a question. Compare it to a statement like “Our primary method to reduce unwanted departures of associates is …”
  2. Try to keep most or all of your questions in the same style, typically a question followed by selections that are each five-to-seven words (quick for readers to absorb) and something like four-to-six selections listed vertically.
  3. Have each question address a single topic. You don’t want your analysis to have to guess at the interaction of several ideas on the resulting data.
  4. Limit the number of selections. Pedagogical researchers say that three to four is the ideal number, but they are also focused on questions with a single correct answer and a handful of plausible incorrect answers.
  5. ┬áTry to keep each of the selections approximately the same length. Don’t mix terse, one-to-three-word choices with sentence-length choices because the variability might influence which ones are picked. Here is an example from the DLA Report where the Likert scale has consistent lengths and a clear question. A Likert scale has a logical sequence of selections, unlike a question with selections that are unrelated to each other.
  6. Choose selections that are reasonable and realistic.
  7. Include clear instructions, especially if respondents are allowed to “select all that apply” or “select no more than three.” You might need to repeat the instructions before each set of questions.
  8. Avoid complicated vocabulary and technical jargon. You don’t want to cause would-be participants to drop out because the question demands too much parsing and interpretation. In further pursuit of clarity, avoid negative constructions. Here is an example of a question that might force respondents to read and re-read it.
  9. Consider whether multiple choice is the best approach. There are times when qualitative comments are ideal. Likewise, true-false and fill-in-the-blank questions may be more suitable.

Leave a Reply

Your email address will not be published. Required fields are marked *