Interrogation Method for Evaluating Machine Learning Models

This document is a technical report that introduces a new method for assessing the reliability of machine learning predictions, termed interrogation. The authors compare this method to traditional cross-validation techniques, which can be cumbersome and less effective in certain scenarios. The interrogation approach involves asking a series of probing questions to analyze the logic of prediction functions, breaking them down into various components such as linear and nonlinear interactions. The report details how this method was tested to determine optimal stopping times for training neural networks, demonstrating its ability to identify these points without requiring a validation sample. The interrogation technique is described as model-agnostic, meaning it can be applied to various types of prediction models, including those that are considered black-box models. The document emphasizes the importance of this method in ensuring the reliability of machine learning outputs, akin to how a lawyer interrogates a witness to ascertain truthfulness.