Data may be neutral, but interpretation of it is never

Legal managers need to appreciate the gap between numbers and interpretation of those numbers.   Stated differently, contrary to the chestnut “the numbers speak for themselves”, a babble of conclusions can be reached from any set of numeric data.

A charming anecdote from the NY Times, Nov. 4, 2016 at B4, captures the multiple voices of numbers due to the subjectivity of inference.  According to the Times, the Bureau of Labor Statistics has an unofficial motto for when they are asked about their employment data.  They don’t indulge in drawing conclusions as to whether the employment glass is half full or half empty:  they respond, “It’s an eight-ounce glass with four ounces of liquid.”

In a different legal context and drawing on the wisdom of TV, “Just the facts, ma’am” – leave the interpretation to us.

Data scientists can presumably measure the glass and the amount of liquid in it, but managers in law firms and departments must come to their own conclusions about fullness.

The larger the law department, the more likely it undertakes data analysis

Instances of data science in U.S. law firms or law departments beyond the most basic are sparse or at least hard to find out about.  Most of the numbers collected by them are summarized and described only, often by Excel or PowerPoint, and there is very little analysis other than trends over time or rankings.

Because the field of legal data science in support of management decisions is nascent, we have little to go on regarding its development.   One survey that explored the topic is the 2016 Chief Legal Officer Survey, conducted by Altman Weil, Inc. in the Fall of 2016.  This year’s survey attracted 331 participants.  The median law department has nine lawyers while the median corporate revenue is $3.5 billion dollars.  Thus, the survey sample was large and consisted mostly of very large companies.

One question on the Altman Weil survey asked “In the last 12 months, have you done any of the following to increase your law department’s efficiency in its delivery of legal services? (Check all that apply.)”  Of the eight choices, page 6 of the Report shows that “Collection and analysis of management metrics” came in fourth, with 39% of the respondents checking it.

Not surprisingly, when you break the respondents into five revenue categories, as shown in the graphic below, the larger the company, the more likely the respondent checked that selection.  The smaller companies on the left had one out of four, approximately, indicating that they worked with management metrics; the larger companies on the right were more like two out of three selected it.  The inference is that bigger departments have more data and more people or IT resources who can dive into it to help their managers make decisions.



When you estimate probabilities, use numbers, not words (and back them up with data)

Clients appreciate lawyers who give them a numeric sense of the likelihood of a legal event happening.  True, everything “might” happen, but what is “likely” to happen and how “likely is it to happen,” is what clients plead for and want to know.  Whenever you can do so, alleviate the uncertainty with a numeric probability.  If you practice this habit, by the way, you will push yourself to search for some data to back up your subjective probability.  If no data can be found, what is the basis for your estimate?

If you say that there is a “pretty good chance” that a lawsuit will be filed, that term could variously mean 40 percent, 60 percent or 75 percent.  As explained in Alternatives to the High Cost of Lit., Vol. 24, April 2006 at 65, other terms such as “likely,” “probably,” and “generally” can mean significantly different odds to different people – notably speakers compared to listeners or writers compared to readers.  The likelihood that both will use close to the same number is very small (there, a deliberate and flagrant instance of the bad practice!).   If we can’t say something like “The likelihood that both will use close to the same number is maybe 20 percent,” aren’t we creating a lot of ambiguity and finger-pointing down the road?

In the end, perhaps a range of numbers is all the best that leaves you, the lawyer, comfortable, with the estimate of probability and yet adds at least a bit more solidity for the client: “We face between a 40 and a 60% chance of being sued.”

Law departments and law firms could create data profiles for key clients

To understand a client better, legal managers could generate what we might call “data profiles.”  The initiative can benefit law departments, but let’s use a law firm example.  The profile would assemble several kinds of figures for a client for each of the past three years.  That trend data could include what kinds of work the client generated based on types of matters (and maybe sub-types) including counts, hours and fees.  It could show the number of partners and associates who recorded time on those matters.  It could tally who at the client called to give assignments and their level.  Perhaps there could be data on the email traffic or the conference calls associated with the client.  The profile could extend to fee write offs and discounts and to margins.

A finance group or practice group that researches, ponders, and assembles such data will be able to create client profiles, akin to dashboards, and probe trends over time.  All kinds of analyses would be invited once the data set has been pulled together.

A “client profile” would be a guideline for the kinds of services the client needs, who its key players are, the direction its business initiatives are going, preferences for firm lawyers, and anything else that would enhance client service and client satisfaction.  It could lead to better cross-selling, better use of associates and paralegals, proposals for fixed fee billing, increased client satisfaction, and more.  With data profiles, the firm could segment client groups into categories, such as “high maintenance” or “risk embracing” or “transactional.”  Once you collect the data, your firm will be able to mine it!

An analysis that quantifies waste in “typical” law departments’ spend on outside counsel

If you are a general counsel, do data findings from a recent analysis comport with your sense of what you spend on law firms and receive back?

Data from a study by the Corporate Executive Board (CEB) appears in a summary nugget of Corp. Counsel, Oct. 2016 at 18.   “[T]he typical legal department spends $20 million per year on outside counsel.”  That figure is robustly more than most of the legal departments in the U.S. spend, since half of them have fewer than five lawyers and outside counsel budgets in the low single-digit millions of dollar.  But set that challenge to the CEB’s statement aside for this post.

Using a plausible blended rate of $350 an hour for outside paralegals, associates and partners (and taking 10% off the $20 million for out-of-pocket disbursements), the CEB is saying that payments for about 50,000 outside counsel (or paralegal) hours were typical.

Shifting from dollars to hours, the summary then adds that “4,600 of those hours were considered to be billing for unnecessary ‘overwork’” – nearly one out of every ten hours invoiced padded the bill but added no value.  At a blended $350 an hour, that amounts to $1,610,000 of overwork.

Worse, CEB found that the wasted time “leads to the 1,900 hours worked by the in-house team labeled as ‘rework’ to get back on   track”!   Consider that a reasonable fully-loaded cost for in-house counsel of companies likely to spend $20 million per year on outside counsel is close to $250 an hour, that rework-hours investment would cost the company something like $475,000.

In short, on this set of data from the CEB, a “typical” legal department spends $18 million on legal fees (having reduced the $20 million figure by 10% for out-of-pocket expenses), but nets only $16 million worth of legal services after overpaying $1,610,000 and fixing the wasted time for another $475,000.

Survey data is more insightful if it controls for key variables (partner compensation and gender)

Senior partners in law firms pour hours into agonizing over how much to pay their partners.  Those decisions take into account many factors and cumulatively, over time, shape the culture of the firm.  Because those partner compensation decisions are so crucial, and analysis of data so integral, firms seek guidance and input from multiple sources, including surveys.  So, when an article today in the New York Times, Oct. 14, 2016 at B3, headlined “Men v. Women in Law: A Pay Divide of 44%” the data analyst in me pored over it.

The headline derived from the finding that a partner’s annual compensation is typically “tied to the amount of business they bring.”   Women brought in an average of $1.7 million of business; men averaged $2.6 million.  Interestingly, where the average male partner brought in 50% more than the average female partner, the pay gap was less, at 44%, so something else went into the determinations.

My three other points are different.  First, it would be very useful if Major, Lindsey & Africa, the preeminent executive search firm in the legal industry who sponsored the survey, controlled for the ages of the partners.   It seems plausible that the average male partner is older than the average female partner, which may account for a chunk of the origination and comp differences – you’ve had more years to accumulate a stable of clients and to become known for a specialty.

Second, it would be insightful if the sponsor compared compensation by gender when it holds practice areas constant.  If Trusts and Estates partners, to pick one practice, differ significantly on compensation by gender (ideally controlling for age differences), that finding would clarify the value of the survey results for decision-makers.

Last, if the comparison were between the average compensation of male partners with origination figures roughly similar to the origination figures of female partners, does the gender gap change?   Take all the male partners who originated $1.6 to $1.8 million and match their comp against all the female partners in the same origination range — that will test discrimination.

Data analysis increases profit but also increases productivity

Law firms want to harness data mostly to increase their profitability.  For example, they might want to spot clients that have lower margins (broadly thought of as fee revenue minus full costs of timekeepers) so that they can cull money-losing clients.  Law departments want to harness data mostly to manage (or reduce) their spending.  As an example of that, they might study total costs of budgeted matters compared to similar non-budgeted matters so that they can decide on the best strategy.  For both, money matters.

But a second goal for data analysis in both firms and departments should be to boost productivity.  If a firm handles matters for a department on a fixed fee, productivity increases profitability.  Moreover, you can think of a legal department as a fixed-fee resource (setting aside the variability of external spend), so even more so than in firms productivity gains increase the company’s profitability.  Achieving more with fewer resources (lawyer/paralegal hours or lower cost hours) may well lead to increased profits or decreased spend, but productivity goals are analytically distinguishable.

This realization came from a book review of James Cortada’s, All the Facts (Oxford 2016) in the Times Literary Supplement, Sept. 30, 2016 at 24.  If improved data and analysis enable lawyer and paralegals to accomplish more quickly what previously took longer, their productivity has increased – regardless of the monetary benefits.

Training for attorneys on data literacy – an example from General Electric

Lawyers in firms and legal departments need training on how to recognize and make use of data that improves their decision-making.  Over a decade ago General Electric’s law department appreciated the value of its lawyers being numerate and knowledgeable about business.  The department “conduct[s] a weeklong advanced business course for lawyers, aimed at 30 of the high-achieing or high-potential individuals and covering such topics as financial analysis, controllership, and GE metrics.”

Ben Heineman, then the General Counsel of GE, described the training in Corp. Counsel, Vol. 13, April 2006 at 89.  Others could adopt a similar tactic and organize training sessions on data literacy.  The topics could include all the categories on this blog!

How data science can help leaders of lawyers make better decisions

Data that leaders of lawyers (managing partners, practice group heads, executive directors, GCs, direct reports to the GC, LDOs, and others) can use to make better decisions are plentiful in law firms and law departments.  Challenges to effective data analysis, however, are also plentiful – and the entire area of collection, data clean up, software tools, data visualization, and interpretation expands, deepens, and changes constantly.

This blog will explain how leaders of lawyers can take advantage of data science and become better managers.