Financial Fraud Alert Review Dataset Description

This document is a technical report that introduces the Financial Fraud Alert Review Dataset (FiFAR), which addresses the limitations of existing public datasets for learning to defer (L2D) algorithms in the context of financial fraud detection. The report outlines the need for such a dataset, emphasizing the challenges in obtaining human predictions for training and evaluation in high-stakes environments like fraud detection. FiFAR includes the predictions of 50 synthetic fraud analysts, designed to simulate diverse human behaviors and biases. The dataset also incorporates realistic definitions of human work capacity constraints, allowing for extensive testing of L2D methods under real-world conditions. The report details the methodology used to generate the dataset, including the simulation of expert behavior and the establishment of capacity constraints. Additionally, it discusses the benchmarking of L2D methods using this dataset across 300 testing scenarios, aiming to facilitate rigorous evaluation and comparison of human-AI collaboration in decision-making systems.