Abstract
Reinforcement Learning with Verifiable Rewards (RLVR) has recentlydemonstrated notable success in enhancing the reasoning performance of largelanguage models (LLMs), particularly on mathematics and programming tasks.Similar to how traditional RL helps agents explore and learn new strategies,RLVR is believed to enable LLMs to continuously self-improve, thus acquiringnovel reasoning abilities beyond those of the corresponding base models. Inthis study we critically examine the current state of RLVR by systematicallyprobing the reasoning capability boundaries of RLVR-trained LLMs across variousmodel families, RL algorithms, and math, coding, and visual reasoningbenchmarks, using pass@k at large k values as the evaluation metric.Surprisingly, we find that the current training setup does not elicitfundamentally new reasoning patterns. While RLVR-trained models outperformtheir base models at small k (e.g., k = 1), the base models achieve a higherpass@k score when k is large. Coverage and perplexity analyses show that theobserved reasoning abilities originate from and are bounded by the base model.Treating the base model as an upper bound, our quantitative analysis shows thatsix popular RLVR algorithms perform similarly and remain far from optimal inleveraging the potential of the base model. By contrast, we find thatdistillation can introduce new reasoning patterns from the teacher andgenuinely expand the model's reasoning capabilities. Overall, our findingssuggest that current RLVR methods have not yet realized the potential of RL toelicit truly novel reasoning abilities in LLMs. This highlights the need forimproved RL paradigms, such as continual scaling and multi-turnagent-environment interaction, to unlock this potential.