cs.LGNov 7, 2026

Phase-Conditioned Reward Shaping for Token-Efficient LLM Reasoning

Authors: Zhe Chen, Xiaolin Zhou, Xiyang Hu

Organizations: University of Southern California · Arizona State University

Abstract

Extended reasoning can help language models solve complex problems, but long responses often contain both useful computations and redundant checks. Conventional length penalties encourage shorter responses without distinguishing the functions of intermediate steps. We propose Phase-Conditioned Reward Shaping (PCRS), a framework that combines phase identification with training-time reward allocation. We investigate two classification routes: PCRS-H with a hidden-state classifier and PCRS-T with a frozen text classifier. Our main text classifier implementation assigns overlapping reasoning, execution, and revision labels to native chunks, then uses these labels and exact repetition to adjust existing token penalties. We compare the completed configurations on six mathematical and scientific reasoning benchmarks and examine four reward-weight settings with the text classifier. Compared with a baseline using token-level length penalties, the default text classifier configuration reduces mean response length by 3.13%; sampled Pass@1 is approximately 62.07% for both methods, while oracle Pass@16 rises from 86.08% to 87.67%. These results show how phase-informed penalty allocation can reduce output volume and improve multi-attempt coverage without inference-time classification.

Figures & tables

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. To Think or Not to Think: Allocating Reasoning Where It Helps

    Aug 30, 2026Zhengdong He, Yunfan Zhou, Jianguo Yao +2Efficient Language Model ReasoningRL for Language Model Reasoning

  2. The Art of Efficient Reasoning: Data, Reward, and Optimization

    Feb 24, 2026Taiqiang Wu, Zenan Xu, Bo Zhou +1Efficient Language Model ReasoningRL for Language Model Reasoning

  3. SHAPE: Stage-aware Hierarchical Advantage via Potential Estimation for LLM Reasoning

    Apr 8, 2026Zhengyang Ai, Zikang Shan, Xiaodong Ai +3LLM ReasoningEfficient Language Model Reasoning