Density of questions in survey reports by law firms

For the most part, survey reports present approximately one question asked per page of report. The plot that follows shows data on how many questions were asked per report page. The data comes from a random set of reports that made reasonably clear how many questions its survey asked.

Stated differently, for every question asked, reports devote a bit less than a page to present the plot of findings (or table or list) and discuss the findings. Often pages also have quotations and other material. As you consider the plot, bear in mind that every report has at least a cover page and a back page that does not address a question.

The next plot shows how many pages are in each of the reports covered by the preceding plot and how many questions are explicitly discussed. The sweet spot appears to be about 20 questions in 25 pages, which again works out to roughly one question per page after you subtract the questionless cover, table of contents and introductory pages, and back page.

 

For survey ranking questions, a technique to assure that the scale was applied correctly

If you are collecting data with a survey, you might ask the invitees to rank various selections on a scale.  “Please rank the following five methods of knowledge management on their effectiveness using a scale of 1 (least) to 5 (most)” followed by a list of five methods.  Ranking yields more useful data than “Pick all that you believe are effective” since the latter does not differentiate between methods: each one picked appears equally effective.

But ranking spawns the risk that respondents will confuse which end of the scale is most effective and which least.  They might not read carefully and therefore put the number 1 for their most effective method – after all, being Number 1 is best, right? – and put the number 5 for their least effective method.

One method some surveys adopt to guard against respondents misreading the direction of the scale is to add a question after the ranking question.  The follow-on question asks them to check the most effective method.  Software can quickly confirm that the respondent understood and applied the scale correctly since the 5 on the first question matches the checked method on the second question.