Benchmarking S3-Compatible Object Storage for AI Workloads

This technical report details the benchmarking of S3-compatible object storage designed to support modern AI workloads. The collaboration between Solidigm, MinIO, and Intel focuses on addressing the storage bottleneck associated with AI infrastructure as model sizes and inference workloads increase. The report outlines the performance testing conducted on an 8-node storage cluster, achieving significant throughput metrics for both PUT and GET operations. Key results indicate peak PUT throughput of 120 GiB/s and peak GET throughput of 268 GiB/s, demonstrating the efficiency of the storage architecture. The report also discusses the implications of these findings for AI infrastructure, emphasizing the importance of density, throughput, and operational efficiency. It highlights how the tested architecture aligns with the MinIO ExaPOD reference design, which scales effectively from petabytes to exabytes. The findings suggest that organizations can achieve exascale economics at petascale entry points, making a compelling case for the deployment of high-density storage solutions in AI environments.