Learning More by Training Less: Uncertainty-Guided Progressive Contraction for Efficient Reasoning Post-Training

Large language models demonstrate increasingly strong reasoning capabilities through effective post-training. Yet, prevailing post-training methods optimize over massive numbers of tokens, implicitly assuming that effective learning must be token-intensive. We revisit this assumption in the on-policy distillation (OPD) setting, which naturally admits dense teacher supervision at every generated token. Using the Qwen3 family, we discover a counter-intuitive phenomenon: reasoning can be effectively incentivized by an extremely small fraction of generated tokens--as few as one or two tokens per reasoning trajectory, corresponding to only 0.05% of all tokens. Surprisingly, this sparse supervision in most cases matches or surpasses full-token training in improving reasoning ability, despite excluding the vast majority of generated tokens from the training objective. This phenomenon is consistently observed across nine teacher--student configurations spanning different model scales on mathematical reasoning tasks, and is further validated on coding reasoning, Llama models and Proximal Policy Optimization (PPO)-based reinforcement learning with verifiable reward (RLVR). Interestingly, such extremely sparse supervision may be closer to the natural learning process: rather than correcting every step word by word, one reflects on a few critical reasoning steps, updates prior understanding, and continues the trial-and-error, avoiding micro-level corrections while remaining remarkably effective. Overall, our results challenge the assumption that effective post-training must be token-intensive and point to a new direction for understanding and designing more efficient post-training algorithms.

Long chain-of-thought~(CoT) reasoning improves the problem-solving capability of large reasoning models~(LRMs), but often incurs substantial inference cost and overthinking. Existing efficiency methods primarily rely on prompting, post-hoc compression, or length-regularized optimization; these approaches can be sensitive to model scale, depend on external estimators, or under-allocate reasoning to difficult instances. We propose Draft-Thinking, a lightweight adaptation framework that learns a model-adapted draft CoT: a compact, self-contained working derivation that directly supports the final answer. We identify a Cross-Scale Draft Mismatch: concise traces generated by a strong teacher can be too sparse for a smaller target model. To address this mismatch, Progressive Draft Adaptation first transfers a concise reasoning-form prior through supervised fine-tuning (SFT) and then applies two-stage reinforcement learning (RL) with progressively harder data and relaxed response budgets to calibrate the reasoning granularity to the target model. The method requires neither post-hoc trajectory compression nor an explicit length-penalty reward. At inference time, draft, adaptive, and step-by-step prompts provide efficiency-oriented, balanced, and detailed reasoning modes, respectively. The step-by-step mode largely preserves the original model's accuracy while using fewer tokens. With only 847 training problems and 120 RL optimization steps, Draft-Thinking substantially reduces reasoning cost across six mathematical and scientific reasoning benchmarks. On MATH500, its draft mode reduces the average response length from 5,668 to 986 tokens, an 82.6% reduction, while achieving 90.6% accuracy compared with 93.0% for the original long CoT model.

Reasoning-oriented language models achieve strong performance by generating long chain-of-thought traces at inference time. However, this capability comes with substantial and often excessive computational cost, which can materialize in redundant or inefficient reasoning. We study this setting and introduce Truncated-Reasoning Self-Distillation (TRSD), a lightweight post-training procedure that encourages models to produce correct predictions from partial reasoning traces. In TRSD, a frozen teacher model first generates a full reasoning trace and evaluates the corresponding answer distribution conditioned on the prompt and the complete reasoning to construct a synthetic training target. A student model with the same architecture is then trained to match the teacher's answer distribution while being conditioned only on a truncated prefix of its reasoning trace. Across multiple reasoning benchmarks and token budgets, we demonstrate that TRSD improves robustness to truncated inference, with far reduced accuracy tradeoffs when applied to a diverse set of reasoning models. Moreover, although never explicitly regularized for shorter generation during training, we also find that TRSD-trained models inherently output shorter reasoning traces without truncation, significantly reducing inference-time costs even without artificial interventions.