Orders reveal new method of validating TAR (Technology Assisted Review) search results

Author:Mr Craig Macaulay
Profession:KordaMentha
 
FREE EXCERPT

A ground-breaking U.S. case has added some useful guidance on the 'grey' areas of the best methods to identify relevant documents in discovery and how to validate the accuracy of the results obtained using the Technology Assisted Review ('TAR') discovery process.

A ground-breaking U.S. case has added some useful guidance on the 'grey' areas of the best methods to identify relevant documents in discovery and how to validate the accuracy of the results obtained using the Technology Assisted Review ('TAR') discovery process.

The orders, issued by renowned TAR pioneer Special Master Maura Grossman and Magistrate Judge Jeffrey Gilbert in a large and complex antitrust class action suit in the Northern District of Illinois, described how the parties are to go about:

Increasing transparency by disclosing the source of documents and recording any pre-search deduplication and culling of collected data Conducting keyword searches and machine learning-based TAR searches, and disclosing the details of the methodology to the opposing party, and Validating the results of the search process and demonstrating that an acceptably low number of documents were incorrectly classified as irrelevant to Requests for Production ('RFPs') and excluded from the set of documents provided. The validation protocol section is particularly interesting, as it is relevant to any electronic document collection and search process, whether using machine learning or more traditional methods.

During discovery, the producing party uses either TAR or a manual review process to classify documents as either 'responsive' (relevant) or 'non-responsive' (not relevant) to a RFP. In either case, the results should be validated, by selecting a random sample of documents (the 'Validation Sample') and manually checking whether each one was classified correctly. The way that this sample is selected is an important consideration. In this case, the order stated that the Validation Sample was to be selected as follows:

The orders also stated that, if a manual review process was used instead of TAR, a similar Valuation Sample should be selected, made up of 500 documents classified as 'responsive' and 2,500 documents classified as 'non-responsive'.

The order then discussed the relevance of an appropriate sample size.

Importantly, the order specified that the Validation Sample shall be reviewed and coded by a subject matter expert ('SME') who is knowledgeable about the subject matter of the litigation. This should...

To continue reading

FREE SIGN UP