From the period 2013 through now, we have found 154 research surveys where a law firm conducted or sponsored the survey and a PDF report was published. That group includes 55 different law firms.
We categorized the firms according to five geographical bases: United States firms, United Kingdom firms, vereins, combinations of U.S. and U.K. firms (“USUK”), and the rest of the world (“RoW” — Australia, Canada, Israel, New Zealand, and South Africa). We thought we would find that the largest firms, either the vereins or the USUK firms, would write the longest reports. Our reasoning was that they could reach more participants and could analyze the more voluminous data more extensively (and perhaps add more marketing pages about themselves).
Quite true! As can be seen in the table below, the average number of pages and the median number of pages for the five geographical groupings of firms each stand at approximately the same number. How many surveys are included in each category is shown in the column entitled “Number”. Nevertheless, the two large classes of firms do indeed produce more pages of reports.
We tested the difference between the average number of pages for the USUK reports and average pages for the US reports. We selected those two groups because they had the largest gap [30.2 versus 22.5].
A statistical test called the t-test looks at two averages and the dispersion of values that make up each average. It tells you how likely it is that the difference of those averages is statistically significant, meaning that if random samples of survey reports were taken repeatedly from law firms in each geography, less than 5% of the time a gap of that amount or more would show up. If that threshold is not met, you can’t say that the differences are due to anything other than chance. If the threshold is met, statistician say that the difference can be relied on, in that it is statistically significant. On our data, the t-test was 1.2 and the p-value is 0.24, much above the threshold of 0.05 for statistical significance. The swing between USUK average pages and US average pages may look material, but on the data available, we can’t conclude that something other than random variation accounts for it.