Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models

This technical report presents the development of Nemotron-Cascade, a model designed for general-purpose reasoning using a novel approach called cascaded domain-wise reinforcement learning (Cascade RL). The report outlines the challenges associated with training general-purpose reasoning models, particularly the variability in inference-time response lengths and verification latencies across different domains. It details how Cascade RL orchestrates sequential, domain-wise reinforcement learning to reduce engineering complexity and improve performance across various benchmarks. The report also discusses the effectiveness of reinforcement learning from human feedback (RLHF) as a pre-training step, which enhances the model's reasoning capabilities. The authors provide comprehensive insights into their training framework, data curation processes, and the results achieved, including comparisons with existing models. The report concludes with a transparent sharing of training recipes and methodologies employed in the development of the Nemotron-Cascade model.