Collect past data going back a few years, but do so carefully

The first time a law firm or law department decides to collect a certain kind of data, legal managers should also decide whether to go back in time for past data. We know the number of EEOC charges handled in 2016, but what about in 2015 and 2014?  Such retrospective data raises a set of concerns and challenges.

The farther back you go, the harder it is to collect accurate data and to feel comfortable that the data has been consistently collected over the period. For example, to figure out how many summer interns stayed more than six weeks becomes harder the more years you go back because full-time-equivalent information might not have been logged.

Maintaining a consistent definition back through time, where conditions possibly were changing, also limits the reach back. For example, it may be problematic to collect costs of e-discovery going back several years because the technology, staff skills, and procedural rules transformed so much  during that time.

As to who should do the retrospective collection, it is a better practice to have one person in charge so that they develop a sense of similar treatment.

For a final step, once the older data has been assembled, it is good to graph it and see whether the visual trend line makes sense to subject matter expert (SME). Another technique is to have a SME at the beginning of the project estimate what they think the numbers will be in the prior years. Yes, those are subjective estimates, but at least they give a basis for testing the numbers collected against someone’s a priori surmise. Obviously, too, the firm or department needs to evaluate whether the value of the data exceeds the cost of reconstructing it.

One final note: whatever the decisions made during collection and whatever the methods, someone needs to carefully keep track of them so that someone else can audit the process or improve it if that appears appropriate.