Potential participants of surveys on a logarithmic scale

Among the hundreds of survey reports that I have located, the number of participants varies enormously. The variance is a function of many factors:

  1. the size and quality of the law firm’s contact list
  2. whether there is a co-contributor, and the quality of its contact list
  3. the mix and amount of efforts to publicize the opportunity to take the survey
  4. the topic of the survey
  5. the length, complexity and design of the survey questionnaire
  6. the period of time that the survey stays open
  7. whether a survey is part of a series
  8. inducements offered for participation
  9. reputation of the law firm.

But some variance in participation numbers relates to the total number of potential participants. All things being equal, a survey targeted at a relatively small number of potential participants will not reach the numbers of a broad-based survey. Stated differently, 100 responses might mean a robust response rate, such as 20% or higher, if only a few hundred people qualify to take a survey, whereas given a huge pool of people who might be appropriate for a survey, the response rate would be an anemic sub-1% response rate.

To start a framework for evaluating potential participant numbers, I looked at 16 survey reports that have between 100 and 110 participants. At least by controlling for the number of actual respondents, I thought I could evaluate factors that influenced the number. But the other factors became too numerous and the data set was too small.

So, since none of the reports stated even the number of email invitations sent out, I estimated my own figures for how many could have been invited. I chose to use a base-10 logarithmic scale to roughly categorize the potential total populations. Thus the smallest category was for narrow-gauged surveys for hundreds of potential participants: the ten squared category (102). The next largest category aimed at roughly 10 times more participants: thousands as ten cubed (103). Even broader surveys would have had a reachable set of possible participants in the tens of thousands, at ten raised to the fourth power (104). At the top end of my very approximate scale are surveys that could conceivably have invited a hundred thousand participants or more (105).

Below is how I categorized the surveys by this estimated log scale and in alphabetical order within increasing bands of potential participants. The quotes come from the report at the page noted. I have shortened them to the core information on which I estimated the scope of the survey’s population.

Even though my categorizations are loose and subjective, the point is that the number of respondents as a percentage of the total possible participants can range from significant percentages down to microscopic percentages. That is to say, \textit{response rates} vary enormously in these — and probably all — law firm research surveys

Narrow

Clifford Chance Debt 2010 [pg. 4] “canvassed the opinion of 100 people involved in distressed debt about their views of the Asia-Pacific distressed debt market.”

CMS GCs 2017 [pg. 26] had “a quantitative survey of 100 senior in-house respondents law departments” that were almost half “drawn from FTSE 350 or FTSEurofirst 300 companies. A further 7% represent Fortune 500 companies.”

DWF Food 2018 [pgs. 3, 8] “surveyed 105 C-suite executives from leading food businesses” that are “in the UK.”

Pepper Hamilton PrivateFunds 2016 [pg. 1] “contacted CFOs and industry professionals across the US” who work in private funds.

Medium

CMS Russia 2009 [pg. 3] explains that its co-coordinator “interview[ed] 100 Russian M&A and corporate decision makers.”

Foley Lardner Telemedicine 2017 [pg. 16] “distributed this survey … and received responses from 107 senior-level executives and health care providers at hospitals, specialty clinics, ancillary services and related organizations.”

Reed Smith LondonWomen 2018 [pg. 22] explains that “A survey was launched via social media which was open to women working in the City of London with a job title equivalent to director, partner, head of department or C-level status.”

Technology Law GDPR 2017 [pg. 2] writes that “In-house legal counsel from 100 different organizations (the majority of which had 1,000+ employees) were invited to participate in a survey.”

Large

Burgess Salmon Infrastructure 2017 [pg. 3] “drew on the opinions of over 100 [infrastructure] industry experts.”

Dykema Gossett Auto 2016 [pg. 3] “distributed its [survey] via e-mail to a group of senior executives and advisers in the automotive industry including CEOs, CFOs and other company officers.”

Freshfields Bruckhaus Crisis 2013 [pg. 3] “commissioned a survey of 102 senior crisis communications professionals from 12 countries across the UK, Europe, Asia and the US.”

Norton Rose ESOP 2014 [pg. 2] “conducted a survey of 104 [Australian] businesses — from startups to established companies.”

Reed Smith Lifesciences 2015 [pg. 4] commissioned a co-coordinator that “surveyed 100 senior executives (CEO, CIO, Director of Strategy) in biotechnology and pharmaceuticals companies” around the world.

Huge

Berwin Leighton Risk 2014 [pg. 2] researched “legal risk” in financial services organizations around the world. “The survey was submitted to participants in electronic format by direct email and was also hosted online at the BLP Legal Risk Consultancy homepage.”

Dykema Gossett MA 2013 [pg. 10] “distributed its [survey] via e-mail to a group of senior executives and advisors, CFOs and other company officers.”

Proskauer Rose Empl 2016 [pgs. 3-4] retained a co-coordinator that “conducted the survey online and by phone with more than 100 respondents who are in-house decision makers on labor and employment matters.”

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.