Use scale questions, but think about text labels

Quite often law firms ask respondents to answer a question with a value from a scale. Those values should represent balanced positions on the scale. That is, they should have the same equal conceptual distance from one point to the next. For example, researchers have shown the perceived balance on the strongly disagree-disagree-neutral-agree-strongly agree scale.

Most survey designers set the bottom point as the worst possible situation and the top point as the best possible, then evenly spread the scale points in-between.

The text selected for the spectrum of choices deserves an extended discussion. Sometimes questions on surveys add text only to the polar values of a scale. For example, “Choose from a scale of 1 to 6 where 1 indicates “Yes, definitely” and 6 indicates “No, definitely not.” Alternatively, the question could supply intermediate scale positions with text: 2 indicates “Yes, probably”, 3 indicates “Maybe”, etc.

DLA Piper Compliance 2017 [pg. 6] used a 10-point scale and text at the extremes and in middle position:

It is hard to create text descriptions of positions on a scale that respondents perceive as equally spaced. If you put only numbers, respondents will unconsciously space the choices: but you will not have as clear a way to indicate what was in the mind of the respondents. On the other hand, words are inherently ambiguous and introduce all kinds of variability in interpretation by respondents.

Often the responses to a well-crafted scale question come back reasonably “normal,” as in the oft-seen bell-curve normal distribution. The midpoint gets the most responses and on either side the numbers drop or rise fairly symmetrically. Here is an example from a five-point scale.

 

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.