cs.LGNov 2, 2026

What Drives the Zero-to-One Jump? Diagnosing and Steering Hard-Prompt Learning in RLVR

Authors: Rui Zhu, Yumou Liu, Zhenzhe Zheng, Fan Wu

Organizations: Shanghai Jiao Tong University, Shanghai, China

Abstract

Reinforcement Learning with Verifiable Rewards (RLVR) has substantially improved mathematical reasoning, yet progress on hard prompts often stalls: a prompt can remain unsolved for many training steps before abruptly transitioning to solved--a discrete "zero-to-one" jump whose underlying dynamics and steerability remain poorly understood. We investigate two questions: what drives this transition, and can it be steered? Through event-based analysis measuring the time-to-first-positive (TTFP), we find that negative rollouts preceding successful transitions consistently retain higher early-stage generation entropy, distinguishing near-solvable failures from dead ends. This signal motivates NAPO (Negative-Augmented Policy Optimization), a method integrated with Group Relative Policy Optimization (GRPO) that ranks negatives in unsolved groups via an early-branching driver and assigns graded advantages, injecting informative gradients precisely where standard group-relative updates vanish. Under fixed rollout budgets, NAPO accelerates early solution discovery and improves Pass@256 by 10.0 points on AIME24 and 6.6 points on AIME2025, demonstrating that zero-to-one transitions are diagnosable and steerable through driver-guided negative learning. Our code is available at https://github.com/jameszhu-pixel-time/Z2O.

Figures & tables

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. ThinkPrior: Zero-Rollout Difficulty Priors for Cold-Start Prompt Selection in RLVR

    Sep 8, 2026Tommy Sha, Skylar Zhai, Siqi ZhaoRL for Language Model ReasoningGroup Relative Policy Optimization

  2. Unlearnable, or Unmeasured? On the Reliability of Difficulty Labels in RLVR

    Sep 30, 2026Chandak Chakma, Syed Nazmus Sakib, Nafiul Haque +1Reinforcement LearningRL for Language Model Reasoning

  3. Locked at the Entrance, Open Inside: Where RLVR Narrows the Solution Space

    Aug 29, 2026Qiancheng Zhou, Ruizhe LiReasoning DiversityReinforcement Learning with Verifiable Rewards