Model Performance Evaluation of SubQ 1.1 Preview Models

This technical report presents the findings of an independent benchmark evaluation conducted by Appen Ltd. on the performance of Subquadratic's latest preview models. The evaluation focuses on model performance across a suite of recognized benchmarks, specifically the Needle-in-a-Haystack (NIAH) and LiveCodeBench. The results indicate that Subquadratic's models achieve near-perfect long-context retrieval accuracy, with results showing 100% retrieval accuracy at 1M and 2M token contexts and 98% at 6M. The LiveCodeBench evaluation involved 1,055 competitive programming problems, with the models achieving a pass rate of 78.0% at pass@1 and 89.7% at pass@4. The report outlines the independence of the evaluation process, ensuring that results reflect authentic model performance without external influence. Detailed results and methodologies are available in a confidential full report upon request.