Leveraging AI for Standardized COA Administration

This technical report discusses the implementation of an AI-enabled tool designed to enhance the quality of clinical outcome assessments (COAs) in psychedelic clinical trials. The study focuses on improving rater consistency and scoring accuracy, particularly in the evaluation of mental health conditions such as major depressive disorder, treatment-resistant depression, PTSD, and anxiety. The AI model was tested through a real-world study involving independent raters who conducted 125 COA administrations. The findings indicate that the AI model demonstrated strong inter-rater reliability across most items of the Montgomery-Åsberg Depression Rating Scale (MADRS) and performed well in evaluating rater performance using the Rater Applied Performance Scale (RAPS). The report outlines the potential of AI to standardize rater performance, reduce variability, and improve scoring accuracy, thereby enhancing the reliability of trial endpoints. Future steps include validating the model's performance on larger datasets and exploring its applications in rater training and protocol adherence monitoring.