The reports at hand deal each in their own way with the four basic demographic attributes (position of respondent, industry, geography, and revenue). We can visualize the relative levels of disclosure by applying a simple technique.
The technique starts with the category assigned to each law firm for a given demographic attribute. For example, we categorized the firms on how they disclosed the positions of respondents with four shorthand descriptions: “No position information”, “Text no numbers”, “Some numbers”, and “Breakdown and percents”. It’s simple to convert each description to a number, such as in our example with one standing for “No position information” up to four standing for “Breakdown and percents.” The same conversion of text description to an integer counterpart was done to the other three demographics, where each time the higher number indicates a better explanation of the report regarding that demographic attribute.
Adding the four integers creates a total score for each firm. The plot below shows the distribution of those total scores by firm.
The firm that did the best on this method of assessment totaled 15 points out of a maximum possible of 15 (three times four categories plus one times three categories for the demographic attribute that had only three levels). At the other end, one firm earned the lowest score possible on each of the four attributes, and thus a total score of four. [Another plot could break up the bar of each firm into the four segments that correspond to the four demographic attributes.]
Our hope is that someday every law-firm research survey will disclose in its report breakdowns by these fundamental attributes together with the percentage of respondents in each. By then, perhaps another level of demographic disclosure will raise the bar yet again.