Most people who follow surveys by law firms assume that the firms collect their data with an online questionnaire. Interviews by telephone, it turns out, play a significant role in survey data collection. In fact, a small number of reports indicate that the law firm, or an organization it commissioned, only conducted interviews to collect data. They did not use an online survey tool.
For example, Allen Overy Innovative 2012 [pg. 5] only used interviews: “Interviews tended to last for about an hour and followed a structured questionnaire.” During a structured interview, the person doing the interview follows a careful script of questions. The script assures that they stick to the same order and wording, and that they collect the same information even if multiple people carry out the interviews. In a way, an online survey questionnaire is a structured interview — but silent.
Ashurst GreekNPL 2017 [pg. 2] also collected its information solely through personal interviews instead of with online survey (50 interviews).
Proskauer Rose Empl 2016 [pg. 3] combined an online questionnaire and phone interviews of 100 people.
A variation on the previous method appears in Allen Overy Models 2014 [pg. 2]. That firm deployed two levels of interviews: “The views of 185 individuals were captured through 20-minute structured telephone conversations. A further 13 individuals participated in a longer in-depth interview.”
Sometimes firms compile their data from an online questionnaire, but then turn to selected interviews to gain depth and color. One example is Herbert Smith CorpDebt 2016 [pg. 2] which followed up with some participants to discuss the survey results. The same two-punch methodology was employed in Reed Smith LondonWomen 2018 [pg. 22], except that the firm went back to several participants who opted in. Pinsent Masons Energy 2017 [pg. 5] explains that “The survey included a combination of qualitative and quantitative questions, and all interviews were conducted over the telephone by appointment.”
CMS GC 2017  pulled off a three-step information gathering, as explained in the snippet below. That firm conducted two surveys plus a series of interviews.
The ratio of interviews to online-survey participants varies widely and cannot always be determined from the survey report. White Case Arbitration 2010 [pg. 3] explains that its data comes from 136 questionnaires and 67 interviews, approximately a two-to-one ratio.
Several firms combine modes of data gathering. They start with a survey emailed to their invitee list or otherwise publicized. At some point later the firm (or the service provider it retained) seeks interviews with a subset of the invitees. (At least we assume that those who were interviewed also completed a survey, but the reports do not confirm that assumption.)
The survey gathers quantitative data while the interviews gather qualitative insights. Interviews cost money, but what firms learn from conversations deepens, clarifies and amplifies the story told by survey data. Interviews also enable the firm to strengthen its connections to participants who care about the topic.
The reports make little of the interview process and provide almost no detail about them in general. They show up as quotes and case studies. DLA Piper Debt 2015 , for example, states that 18 interviews were conducted; commendably it lists the names and organizations of those who were interviewed [pg. 30]. We show the first few in the snippet below.
Reed Smith LondonWomen 2018 [pg. 22] mentions that “Several individuals opted to take part in further discussion through email exchange, in-person meetings and telephone interviews.” As a prelude to those discussions, in the invitation to women to take the survey the firm explained: “We will be inviting those who wish to speak on-the-record to take part in telephone or in-person interviews to impart advice and top tips. If you wish to take part in an interview, please fill in the contact details at the end of the survey.” This background tells us about the opt-in process of the firm, although the report itself does not refer to it.
HoganLovells Cross-Border 2014 [pg. 28] explains that interviews were conducted with 140 “general counsel, senior lawyers, and executives.” As with the other examples here, the report adds no detail about how long the interviews lasted or the questions asked during them.
Clifford Chance Debt 2007 [pg. 3] doesn’t say how many interviews were conducted, only that interviews took place during November 2007. It would have been good for the firm to have said something more about how many people they spoke with and how those people were chosen.
Norton Rose Lit 2017 surveyed invitees, “with a telephone interview campaign following” [pg. 5] and adds later in the report [pg. 38] that there was an “interview campaign following [the online survey] across July, August and early September 2017.”