Number of selections in multiple-choice questions

I wanted to know how many selections are typical in multiple-choice questions asked in law-firm research surveys 1, To start figuring that out, I picked the Davies Ward report from 2011.

Since I do not have the actual online survey distributed by any of the firms, and therefore can’t see which questions were multiple choice, I had to reverse engineer the Davies Ward report to make the best determination possible. To the firm’s huge credit, however, its report helps in that determination because it states the question asked on the survey at the bottom of the plot that displays the results from that question. Studying those plots and the question asked, I identified 16 questions that were probably multiple-choice.

It helps to understand the findings to see them visualized.

The plot below shows on the horizontal axis how many selections were in those 16 questions. The fewest selections were three; the most were nine. Note that this number includes an “Other” selection if that was available to the respondents. The vertical axis and the height of the corresponding column tells how many questions had that number of selections. So, for example, four questions had five selections (the third column from the right).

Over all 16 questions, the average number of selections was 4.875. In general, the questions with six or more selections dealt with more complex situations, such as outside counsel management methods or alternative fee arrangements. The questions with fewer selections generally had categories created by the firm, such as when asking for years of experience in private practice the selections consisted of four ranges of years, e.g., 0-5, 6-10.

For a future post we might pick another report, derive the comparable figures, and combine them with the Davies Ward figures to develop an even more representative distribution of numbers of selections.

Notes:

  1. My set includes reports By Allen & Overy, Berwin Leightner Paisner, Carlton Fields, Davies Ward Phillips, Eversheds, Goulston & Storrs, Haynes and Boone, Hogan Lovells, Littler Mendelson, Norton Rose Fulbright, Proskauer Rose, Ropes & Gray, Seyfarth Shaw, White & Case, and Winston & Strawn

Leave a Reply

Your email address will not be published. Required fields are marked *