Evaluation of AI Detection Algorithms for Video Systems

This technical report examines the accuracy and reliability of Samsara's AI-powered event-triggered video systems in detecting unsafe driving behaviors. The study's objective was to independently validate the AI features by assessing three key performance metrics: recall, precision of in-cab alerts, and precision of Safety Inbox events. The testing focused on the CM32 camera model and included five unsafe behaviors: mobile phone usage (texting and phone-to-ear), inattentive driving, seat belt non-use, and close following distance. The methodology involved controlled tests under various conditions, including time of day and vehicle type. Results indicated that the AI system achieved 100% accuracy in detecting texting, seat belt non-use, inattentive driving at night, and close following distance. However, detection accuracy for holding the phone to the ear was slightly lower in dark conditions. The study concluded with a 95.5% overall detection rate and confirmed 100% accuracy in Safety Inbox events, demonstrating the system's reliability for fleet management.