Generative AI-Based Document Review Case Study

This case study focuses on the application of Generative AI (GenAI) for document review within the legal field, particularly examining its effectiveness in processing large volumes of documents. The study, conducted by HaystackID in partnership with eDiscovery AI, aims to evaluate the strengths and limitations of GenAI in a controlled environment. The research involved re-reviewing a specific case that had previously undergone conventional review, utilizing a dataset of approximately 30,000 documents selected to be statistically representative of a larger corpus. The methodology included modifying prompts for the AI system to enhance performance and analyzing inter-run variability to understand the impact of prompt changes. Results indicated that the AI system achieved recall rates comparable to traditional human review, with a focus on recall over precision. The findings aim to inform the eDiscovery community about the practical applications and challenges of using GenAI in document review processes.