Many firms conduct surveys in an annual series. To add a nuance, we have encountered a handful of examples of two surveys per year, such as by Haynes & Boone. Most of them do so each year on the same topic but lapse after three or four years.
The plot below shows all series identified to date as a histogram of how many surveys have appeared in the series. So, for example, five series have continued for thee years (the second column from the left). The plot visualizes a total of 187 research surveys by 32 law firms. Of those firms, two have sponsored at least three series (CMS and DLA Piper). Seven other firms claim at least two series.
Stated differently, just under half of all the research surveys found to date (349) are part of a multi-year series. A handful of the series were interrupted for a year but most of them were continuous.
As can be seen from the left-most column, 16 of the series either carried on for only two years, or 2018 may be their second year and we do not know how long they will continue. On the far right looms the Methuselah of all series, the 13 year run by Dykema Gossett on merger and acquisition activity. Overall, law firms have demonstrated admirable consistency and commitment.
Not only are the sheer volume of surveys and breadth of surveying firms remarkable, but the topics studied range widely. Some explore broad concerns, such as worldwide legal risks, while some address narrow niches, such as telemedicine. A coding of survey topics assigned them as those relating primarily to (1) a practice area, such as employment discrimination or tax; (2) current events, such as Brexit or the GDPR; (3) the legal profession, such as outside counsel management or innovation by law firms, and (4) industry, such as manufacturing or technology suggests provides one perspective.
The plot below presents the breakdown from almost 350 surveys uncovered to date. The column chart shows how many surveys fall into each of the four meta-topics.
For example, 150 were classified as focused on an area of law, which we labelled as “Practice” surveys. To convey the relative proportions of each of these four meta-topics, the column widths are scaled by the number of surveys.
Someone could conceivably categorize the surveys by the scope of their research (narrow, broader, average, wide, and large, for example). Coding on that basis, it would soon become clear, introduces too many subjective judgments for the end result to have much meaning. How would one rank “Indianapolis and Social Entrepreneurship: An Examination and Recommendations for the Future” [Taft Stettinius Entrep 2018] as compared to “GDPR Readiness Report” [Technology Law GDPR 2017]? Perhaps a measure in line with total dollars involved in the survey topic or the number of employees involved? Even if the meta-topics are taken to be correctly (and meaningfully) assigned, within a meta-topic the same challenges would exist. The methodological obstacles to reaching consensus on a categorization of scope loom very large.
The plot that follows shows an increasing number of research surveys each year since approximately 2005. We assign to a survey the year its results became public.
Not that all research surveys by law firms have been identified, far from it. In fact, in the years before 2010 our efforts to unearth research surveys have not been conscientious. That said, were we to plumb those earlier years, we would certainly find more as well as in the 2008 to 2012 period. [Even the firms themselves have lost track of the surveys they conducted years ago. People have left, memories have failed, mergers have consummated or groups departed, files have disappeared.]
Nevertheless, 2017 has so far surfaced at least 82 surveys, and 2018 has generated more than half that number in a half year, a pace that would exceed the 2017 total. We should also note that our investigations have been directed at surveys in the English language.
We have located a total of 348 research surveys, of which 189 were announced during the past five and a half years. A total of 83 different law firms played a role in those surveys.
The plot below shows by the height of each column the number of surveys announced each year, with the most (83) during 2017. This year looks set to exceed that figure because through the middle of 2018 more than 50 research surveys have been identified.
Through the middle of the plot the reader can see the number of different law firms that conducted a research survey during each year. For example the 26 surveys found for the year 2013 were the efforts of 20 different law firms. The trend of the number of firms that are surveying is definitely increasing.
The horizontal black lines in each column partition it by law firm. Each colored bank represents a law firm. Most of the partitioned segments are one unit high, since most of the firms only sponsored a single survey during a year. Yet some firms were prolific. For example in 2017 two firms in the top third of the 2017 column accounted for approximately 10 of surveys that year as shown because their segments of five are tall (three firms below them, starting at the 25 mark on the y axis, also churned out a similar number of surveys). The number 47 appears in the middle of those two segments toward the top.
The portion of Akerman RealEstate 2016 [pg. 13] shown below, which omits the remaining six funders, shows year-over-year changes with paired bars. [We note in passing the unusual page-number with its the vertical firm name, divider line and number.]
Carlton Fields CA 2015 [pg. 7] conveys trend data for nine years and a projection for a tenth year. This line chart also shades the four years of decreased spending.
Another visualization of changes over time comes from Morrison Foerster MA 2017 [pg. 7]. This plot covers five years, with data collected twice a year, and presents the data as segmented column charts.
DLA Piper Debt 2015 [pg. 7] uses a side-by-side bar plot to give data from past years’ projections.
The plurality of plots in survey reports have no borders, as is the case with this unusual column plot from Akerman RealEstate 2016 [pg. 5]. The plot is unusual because it’s base rests on the bottom of the page, the x-axis line extends to the far left and right, and because there are no axis titles: those titles are shown next to the appropriate column.
Howes Percival SocialMedia 2018 [pg. 7] unusually outlines plots with dashed lines. Intermixed with that design choice are plots outlined in solid borders like the one below on the lower left.
Osborne Clarke Consumer 2018 [pg. 6] borders its plots with a solid line on all four sides. Some people may feel that this enclosure adds nothing to the presentation, merely more ink on the page.
In a final example, from Goodwin Law Rollover 2018 [pg. 6], horizontal lines bracket the plots at the top and bottom.
The final page of most survey reports typically contributes aesthetically much more than substantively. The page usually provides contact information for the law firm, has some legal boilerplate, and mostly consists of empty space. Quite often in more recent reports the back page offers some social media links as in this half-page from Mills \& Reeve CommonLaw 2017. Twitter and YouTube icons show up. King Wood AustDirs 2015 closes with seven social media icons!
Burness Paul CorpScotland 2016 ends with a plush purple page, as shown in the top half below:
Goodwin Law Rollover 2018 ends in blackness brightened in the bottom portion by the firm’s icon and four lines of dense legal disclaimers.
Osborne Clarke Consumer 2018 [pg. 9] applies on most of its pages a two-column layout, and sometimes puts a plot in each column.
Another design choice has two columns of text but sometimes spreads a wide plot across both columns. Here’s an example from Norton Rose AustAgribus 2018 [pg. 14].
King Wood AustraliaDirs 2015 [pg. 25] spreads an unusual polar chart across the right two columns of a three-column layout. The gray bars on either side are the edge of the page, by the way, not a design element.
Going one column further, Burness Paull CorpScotland 2016 [pg. 15] stretches an unusual pie chart across all three columns.
DLA Piper Debt 2015 [pg. 8] extends a plot into the right margin.
In most survey reports, the law firm does not explain which survey software it used to create its set of online questions and capture the responses of the participants. Such software has been available for years. According to one online site, “The first online survey software and questionnaire tools initially surfaced in the late 1990s.” The author goes to differentiate the capabilities of free software from paid versions.
“Typically, paid versions of online survey software offer added capabilities such as:
Survey logic — paid tools often provide the option to add a follow up question. This is based on the answer you’ve provided to the previous question.
Export data — There are several tools that won’t let you export your survey data, unless you start using the paid version.
Custom logo — Looking to get rid of the survey tool’s logo and make it your own? With most paid versions this is possible.
More question types — Although free survey tools offer plenty of question types, including multiple choice, ratings, drop-downs and radio buttons, paid versions tend to offer even more.”
At various points I have encountered references to the following software:
NoviSurvey (Fulbright Jaworski used this software)
Qualrics In a previous post, Dec. 22, 2017, I mention a survey publicized by Seyfarth Shaw that used Qualtrics to host the survey.
Undoubtedly there are many more offerings that law firms can choose from. In fact, the commentary quoted above claims there are hundreds of offerings, and gives a list of 21.
During the past six or seven years, it appears that all of the largest Canadian law firms have unveiled research surveys.
The plot below shows six of those firms and presents a point above the firm for each survey in each year in which they conducted a survey. For example, Fasken Martineau conducted two in 2017 and two in 2016. The dates and the points are slightly jittered so that none of them overlap.
We found out about these surveys through a search using Google and reviewing the first six pages of hits returned on March 6, 2018. We used the name of the firm and “survey.” It could well be that the firms have conducted other research surveys during the period. We have not searched the websites of each firm.
Blake Cassels, Borden Lardner, Davies Ward, Fasken Martineau, McCarthy Tetrault, and Miller Thomson are known to have sponsored surveys. It may also be that Gowling Lafleur Henderson, carried out one or more surveys before it combined with U.K.-based Wragge Lawrence Graham & Co. to create Gowling WLG in 2015. That merged firm ran two surveys, at least, in 2017.