ChatGPT is the “revolutionary” AI tool everyone is talking about. From composing songs and writing college-level essays to finding and fixing bugs in code, the chatbot is both wowing and worrying the tech industry.

TLS has been inundated with questions from clients asking what we predict the impact ChatGPT will have on the legal tech industry and specifically, eDiscovery. The conversation that tends to follow touches on chilling topics such as AI replacing humans, AI versus human cost-effectiveness, and other concerns that we warm-blooded folk conjure when compared to our cold, data-driven competitors. It goes without saying that the use of AI within eDiscovery is becoming the norm. As our recent article on eDiscovery trends in 2023 suggests, tools like TAR have become critical in increasing efficiencies, reducing costs, and achieving higher degrees of accuracy over human review.

In this spirit, we thought we would put ChatGPT to the test and ask it a series of eDiscovery-related questions. We asked several senior eDiscovery professionals to create the questions based on the following criteria:

•    The question must be specific to eDiscovery.
•    You need to be able to answer the question.
•    You would expect an eDiscovery professional to be able to answer the question.

Our first question is generally broad.

1. How long will it take to produce 20,000 documents?

ChatGPT immediately identifies that there is no source for it to provide a concrete answer but still provides a comprehensive response. The acknowledgment by the bot that the eDiscovery process is a “complex and time-consuming process” and one that should be overseen by an “experienced eDiscovery professional” is encouraging.

Our second question relates to privilege.

2. Why should you not provide a native email if the attachment is privileged in eDiscovery?

 
TLS eDiscovery Director, Preeti Sharma, was impressed by the answer. She did, however, point out that while the response appropriately suggests providing documents in a non-native format, it primarily discusses the email metadata and omits the simple fact that the privileged document itself is contained within the native email as an attachment. What is contained is much more than metadata—it is substantive content.

To really put ChatGPT to the test, our third question touched on multilingual eDiscovery. Any legal professional who has worked on a matter involving multilingual eDiscovery knows just how much of a minefield it can be.

3. In a non-English eDiscovery litigation, when reviewing documents in a foreign language, what is the best way to input search terms?

We shared ChatGPT’s answer with TLS Global Director of Multilingual eDiscovery, Robert Wagner. Rob notes that some of the assertions are partially correct but are equally an “incredibly shallow” take on the topic. He also notes that some of the assertions are radically wrong—and weakly caveated. The main concern is that the type of person using ChatGPT for questions of this intricacy are exactly the those that need the most “hand holding” so they don’t trip over all the important points that were omitted.  

To illustrate this point, ChatGPT mentions using a native speaker for inputting search terms in a foreign language, which is true. What's also true and missing: terms are a technical problem as much as they are a linguistic problem. Search syntax is of equal importance to the linguistic component of terms, and most native speakers lack the required syntax experience. And, those two components say nothing about embedding a translation strategy in the terms that amplifies your litigation strategy.

ChatGPT is clearly a very powerful tool and perhaps even warrants the term revolutionary. In the legal industry alone, firms are already leaning into the adoption of generative AI. At the time of writing, Allen & Overy announced that it had rolled out a chatbot named Harvey. Harvey had been tested by more than 2,000 lawyers across their global offices. Other legal chatbots worth mentioning include Billy Bot and LawDroid’s Copilot.

Our biggest takeaway from this little exercise is that expert knowledge and practical human experience remain key. When dealing with the technicalities of eDiscovery, there is nuance and intricacy that a chatbot might not be trained on or capable of acknowledging. We should be hesitant to rely too heavily on bots like ChatGPT when confronted with technical hurdles and instead defer to a human eDiscovery professional.