Balancing survey respondents across industries and geographies

When most law firms conduct a research survey, they primarily hope to get enough respondents so that the results are defensible and generalizable. That is to say, they want enough data from desirable respondents to be able to say that their findings make sense and can be extrapolated beyond their particular group of participants.

Ropes & Gray made a very different decision in its 2017 survey on risk management practices in companies. Working with a research group, the firm deliberately balanced the number of participants by five named industries plus “Other” and across four geographical regions. As can be seen in the table below, adapted from page 5 of the report, out of the 300 total respondents, 100 came from each of America and EMEA, while 70 came from Pacific Asia and 30 from Latin America. Moreover, each industry had exactly 50 participants.

 AmericaEMEAPacific AsiaLatin AmericaTotal
Banking161712550
Private Equity171612550
Asset Management161811550
Life Sciences & Healthcare171711550
Technology171612550
Other171612550
Total1001007030300

The very brief description of the survey’s methodology does not explain why the firm chose those industries, those geographies, or the balanced participant numbers within them. Nor does it delve into how FT Remark, the research firm that assisted Ropes \& Gray, obtained the desired number of respondents.

One reason for the geographic distribution may have been that it proved difficult to obtain equal numbers of respondents for each pair of industry and geography. It may also be that the firm feels that this geographical weighting in some way more accurately represents companies and their risk management approaches around the globe. Many other questions arise regarding the decisions underlying this symmetric data set.

We will close by noting that if a law firm sets its goal to proportionally balance the number of responses by one or more criteria, revenue of the company being another possible parameter, it significantly increases the effort to locate and persuade the requisite number of respondents.

 

Leave a Reply

Your email address will not be published. Required fields are marked *