Tuning Generative AI-Based Systems for Document Review

This case study presents an analysis of the aiR for Review tool developed by Relativity, focusing on its application in the legal industry for document review. The study outlines the challenges faced by legal teams in selecting AI tools that provide accurate and transparent results. It details the extensive testing conducted by HaystackID, which included benchmarking and statistical analysis of aiR for Review's performance across three distinct workflows: Issues Review, Relevance Review, and Relevance + Issues Review. Each mode is evaluated for its effectiveness in classifying documents based on specific legal criteria. The findings indicate that while all modes perform well, they are optimized for different objectives, emphasizing the importance of selecting the appropriate mode based on specific review requirements. The study also highlights the significance of prompt engineering in enhancing AI efficiency and accuracy during the document review process.