Process Reward Model

Follow this topic Ask about this topic

Momentum

6 papers in the last four weeks, up 20% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 69

All topics

Learning Process Rewards via Reasoning State Propagation
Sep 30, 2026Kai Gan, Zi-Hao Zhou, Bo Ye +3Process Reward Model Process-Level Supervision
School of Computer Science and Engineering, Southeast University, Nanjing 210096, China · Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, China · Zhongguancun Academy +1
Inducing Process Supervision from Outcome-Only Reinforcement Learning
Sep 29, 2026Shengda Fan, Xin Cong, Zhong Zhang +2Process Reward Model Process-Level Supervision
Renmin University of China · Tsinghua University · University of Electronic Science and Technology of China +1
Why Deterministic PRM Guidance Underperforms in Discrete Diffusion Reasoning
Sep 28, 2026Yan Zhan, Shaobo Liu, Zhijun GaoDiffusion Language Models Process Reward Model
Peking University · Shenzhen University
Dr.Credit: Rubric-Grounded Process Credit Assignment for Deep Research Agents
Sep 28, 2026Yingjian Zhu, Zhenyi Wang, Jiaxin Guo +6Credit Assignment Process Reward Model
School of Artificial Intelligence, University of Chinese Academy of Sciences · State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences · Alibaba Token Hub, Alibaba Group +1
Counterfactual Rollout Replay: Forkable Environments as Free Process Rewards for Software Engineering Agents
Sep 27, 2026Yuanhao Li, Hongbo Wang, Xuhong Chen +2Counterfactual Learning Software Engineering
State Key Laboratory of Networking and Switching Technology Beijing University of Posts and Telecommunications, Beijing 100876, China · University of Luxembourg, Luxembourg
Rewarding Reasoning, Not Answers: Fixing and Bounding Test-Time Reinforcement Learning on Medical QA
Sep 15, 2026Kailong Fan, Anqi Pu, Yichen Wu +7Process Reward Model Answer
1Harvard Medical School/MGH · 3Harvard University · 2Zhejiang University
Legibility is Not Interpretability: Comparing Judged and Actual Importance in Chain-Of-Thought Reasoning
Sep 3, 2026Kevin Du, Alexander Hoyle, Laura Ruis +1Reasoning Chain Reasoning Traces
ETH Zürich · Work done during an internship at Cohere. · MIT +1
Dense Process Supervision for Search Agents via Fact Utility Estimation
Sep 1, 2026Rongzhi Zhu, Xiangyu Liu, Yi Liu +7Process Reward Model Credit Assignment
State Key Laboratory for Novel Software Technology, Nanjing University, China · Ant Group, China · National Institute of Healthcare Data Science, Nanjing University, China
Mitigating Over-Optimization in PRM-Guided Search in Mathematical Reasoning by Optimizing the Guide
Aug 30, 2026Taejong Joo, Diego KlabjanProcess Reward Model Mathematical Reasoning
Department of Industrial Engineering & Management Sciences Northwestern University
SPEAR: Distilling Domain-Adaptive Reasoning Skeletons via Sequential Symbolic Alignment in Reinforcement Learning
Aug 27, 2026Zhuochun Li, Yuelyu Ji, Yiming Zeng +1Process Reward Model Knowledge Distillation
University of Pittsburgh, Pittsburgh, USA · University of Connecticut, Storrs, USA
SCOUT: Unlocking Enhanced Spatial Reasoning via Structured Chain-of-Thought and Multi-Objective Process Reward
Aug 12, 2026Zile Zhou, Huining Yuan, Weichen Zhang +23D Spatial Reasoning Spatial Reasoning
Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
StructReward: Efficient Structured Process Rewards for Self-Correcting Multimodal Reasoning
Aug 8, 2026Yifan Li, Ruxin Sun, Tongzhou ZhaoProcess Reward Model Multimodal Reasoning
Quality-Diversity Stress Tests for Process Reward Models:What Archive Coverage Can and Cannot Certify
Aug 8, 2026Ibne Farabi Shihab, Fariya AfrinProcess Reward Model Stress Testing
Department of Computer Science, Iowa State University · Department of Computer Science, Kalinga Institute of Industrial Technology
The Horizon Gap: Planning, Memory, Execution, Training, and Evaluation for Long-Horizon LLM Agents
Aug 7, 2026Mingguang Chen, Licheng Wang, Bo QuLong-Horizon Task Planning Large Language Model Agents
1DeepGrounding · 2AlphaAvatar
Beyond On-Policy Exploration: Integrating External Policy Rollouts for Reinforcement Learning in Diffusion Language Models
Aug 3, 2026Wonseok Lee, Jimyeong Kim, Jungmin Ko +1Large Language Model Reinforcement Learning Autoregressive Rollout
Interdisciplinary Program in Artificial Intelligence, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826, South Korea · Artificial Intelligence Institute, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826, South Korea · Department of Intelligence and Information, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826, South Korea
BiCAA: Bidirectional Credit Assignment for Search-Augmented Agent
Aug 2, 2026Yibin Huang, Bin Xu, Hailong Cao +1Search Agents Search-Augmented Reasoning
Faculty of Computing, Harbin Institute of Technology
LOTAPO: Leave-One-Turn Attribution for Self-Generated Process Rewards in Multi-Turn Search Reasoning
Jul 15, 2026Qiang Zhu, Jiajun Wu, Longyi WangProcess Reward Model Process-Level Supervision
Zhejiang University
LLMs as a Jury: Cross-Model Consensus Can Outperform Process Reward Models for LLM Reasoning
Jul 11, 2026Ning LiuLlm-As-A-Judge LLM Reasoning Strategies
Independent Researcher
KV-PRM: Efficient Process Reward Modeling via KV-Cache Transfer for Multi-Agent Test-Time Scaling
Jul 10, 2026Peng Kuang, Haibo Jin, Xiaoyu Han +5Process Reward Model Test-Time Scaling
University of Illinois Urbana-Champaign · Imperial College London · City University of Hong Kong
Improving LLM-Generated Process Model Quality Through Reinforcement Learning: The Role of Reward Function Design
Jul 7, 2026Alexander Rombach, Chantale Lauer, Nijat MehdiyevProcess Reward Model Large Language Model Reinforcement Learning
German Research Center for Artificial Intelligence (DFKI), Campus D3 2, Saarbrücken, 66123, Saarland, Germany · Saarland University, Campus D3 2, Saarbrücken, 66123, Saarland, Germany
FaithMed: Training LLMs For Faithful Evidence-Based Medical Reasoning
Jul 1, 2026Zhiyun Zhang, Liwen Sun, Xiang Qian +1Clinical Reasoning Training LLM Reasoning Strategies
Carnegie Mellon University · Stanford University School of Medicine · Xlue
Process Advantage Signal Shaping: A Paradigm-Agnostic Middleware for Process-Supervised RL in LLM Reasoners
Jun 28, 2026Chao Wang, Hongtao Tian, Tao Yang +3Process Reward Model LLM Reasoning Strategies
Tsinghua University · Work done while interning at Tencent. · WeChat, Tencent
The Weakest Link Tells It All: Outcome-Supervised Process Reward Modeling via Learnable Credit Assignment
Jun 26, 2026Tianyu Jia, Yue Fang, Hongxin Ding +6Process Reward Model Credit Assignment
National Engineering Research Center of Software Engineering, Peking University, Beijing, China · School of Computer Science, Peking University, Beijing, China · Key Laboratory of High Confidence Software Technologies, Ministry of Education, Beijing, China +3
Process-Verified Reinforcement Learning for Theorem Proving via Lean
Jun 18, 2026Minsu Kim, Se-Young YunReinforcement Learning With Verifiable Reward Theorem Proving
KAIST AI
VeriBound: PAC-Bayesian Generalization Bounds for Process Reward Models Trained with Formal Verification Tools
Jun 17, 2026Amirul Rahman, Mohammed Sabih AlsharariReinforcement Learning With Verifiable Reward Process Reward Model
University of Malaya
The Hidden Bias of Process Reward Models:PRISM for Rewarding the Right Reasoning
Jun 8, 2026Aakriti Agrawal, Souradip Chakraborty, Armin Saghafian +6Process Reward Model Contrastive Learning
University of Maryland, College Park · Capital One · University of Central Florida
Improving Multimodal Reasoning via Worst Dimension Optimization
Jun 5, 2026Haocheng Lv, Huaping Zhang, Qiuchi Li +2Multimodal Reasoning Process Reward Model
Beijing Institute of Technology
From Correctness to Utility: Gain-Based Prefix Evaluation for LLM Reasoning
Jun 5, 2026Yuhang Zhou, Yixin Cao, Guangnan YeLLM Reasoning Strategies Process Reward Model
Fudan University · Shanghai Innovation Institute
StainFlow: Entity-Stain Tracking and Evidence Linking for Process Rewards in GUI Agents
Jun 5, 2026Haojie Hao, Longkun Hao, Yihang Lou +8Process Reward Model Graphical User Interface Agents
Beihang University · Peking University · Renmin University of China +4
SCI-PRM: A Tool Aware Process Reward Model for Scientific Reasoning Verification
Jun 3, 2026Xiangyu Zhao, Henry Hengyuan Zhao, Yiheng Wang +7Process Reward Model Scientific Discovery
The Hong Kong Polytechnic University · National University of Singapore · Shanghai AI Lab +3
ARBOR: Online Process Rewards via a Reusable Rubric Buffer for Search Agents
Jun 2, 2026Zheng Liu, Longxiang Zhang, Xintong Wang +8Process Reward Model Search Agents
Tsinghua University · Alibaba Group · Peking University +1
Predicting Inference-Time Scaling Gains from Labeled Validation-Set Output Statistics
Jun 2, 2026Luyang Zhang, Jingyan LiBest-Of-N Inference-Time
Carnegie Mellon University · Johns Hopkins University
Off-the-Shelf LLMs as Process Scorers: Training-Free Alternative to PRMs for Mathematical Reasoning
Jun 1, 2026Atoosa Chegini, Soheil FeiziLLM Reasoning Strategies Mathematical Reasoning
Department of Computer Science, University of Maryland
Latent Reward Steering: An Adaptive Inference-Time Framework that Implicitly Promotes Cognitive Behaviors in Reasoning LLMs
May 30, 2026Jiakang Li, Guanyu Zhu, Can Jin +8LLM Reasoning Strategies Process Reward Model
Rutgers University · South China Agricultural University · Columbia University +4
The Paradox of Outcome Optimization: A Causal Information-Theoretic Bound on Reasoning Shortcuts in LLMs
May 30, 2026Zihan Chen, Yiming Zhang, Wenxiang Geng +2LLM Reasoning Strategies Process Reward Model
1HFIPS, Chinese Academy of Sciences · University of Science and Technology of China
EST-PRM: Stress-Testing Process Reward Models Before They Become Load-Bearing
May 30, 2026Ibne Farabi Shihab, Fariya Afrin, Sanjeda Akter +1Process Reward Model Correctness
Department of Computer Science, Iowa State University · Department of Computer Science, Kalinga Institute of Industrial Technology · Department of Civil, Construction & Environmental Engineering, Iowa State University
Rubric-Guided Process Reward for Stepwise Model Routing
May 28, 2026Shenghao Ye, Yu Guo, Zhengheng Li +2Process Reward Model Large Language Model Routing
University of Science and Technology of China · Southeast University
PRO-CUA: Process-Reward Optimization for Computer Use Agents
May 27, 2026Yifei He, Rui Yang, Hao Bai +2Process Reward Model On-Policy
University of Illinois Urbana-Champaign
Retrieval, Reward, and Training Protocols: What Matters in Training Search Agents?
May 27, 2026Yibo Zhao, Zichen Ding, Jiayi Wu +2Search Agents Process Reward Model
School of Data Science and Engineering, East China Normal University · Shanghai AI Laboratory
Search-E1: Self-Distillation Drives Self-Evolution in Search-Augmented Reasoning
May 21, 2026Zihan Liang, Yufei Ma, Ben Chen +4Search-Augmented Reasoning Unsupervised On-Policy Self-Distillation
PAIR: Prefix-Aware Internal Reward Model for Multi-Turn Agent Optimization
May 18, 2026Wonjoong Kim, Yeonjun In, Sangwu Park +2Process Reward Model Inference Cost
KAIST · Yonsei University
Process Rewards with Learned Reliability
May 15, 2026Jinyuan Li, Langlin Huang, Chengsong Huang +5Process Reward Model
Washington University in St. Louis · Singapore University of Technology and Design
Unsupervised Process Reward Models
May 11, 2026Artyom Gadetsky, Maxim Kodryan, Siba Smarak Panigrahi +2Process Reward Model LLM Reasoning Strategies
Swiss Federal Institute of Technology (EPFL)
Reinforcement Learning Measurement Model
May 10, 2026Wenqian Xu, Feng JiProcess Reward Model Offline Reinforcement Learning
Department of Applied Psychology and Human Development, University of Toronto, ON, Canada
EvoMAS: Learning Execution-Time Workflows for Multi-Agent Systems
May 9, 2026Chengdong Xu, Kaiqiang Ke, Ziheng Liu +4Multi-Agent Workflows Multi-Agent Coordination
Sun Yat-Sen University · ShanghaiTech University · Zhejiang University +1
Distributional Process Reward Models: Calibrated Prediction of Future Rewards via Conditional Optimal Transport
May 7, 2026Rachel Ma, Dylan Hadfield-Menell, Kristjan GreenewaldProcess Reward Model Quantile Regression
MIT CSAIL · MIT-IBM Watson AI Lab · IBM Research
Every Step Counts: Step-Level Credit Assignment for Tool-Integrated Text-to-SQL
May 6, 2026Yaxun Dai, Baolin Sun, Junying Wang +6Text-To-Sql Credit Assignment
Institute of Computer Science and Technology, Soochow University, China · Ant Digital Technologies, Ant Group · School of Management, University of Science and Technology of China
SCPRM: A Schema-aware Cumulative Process Reward Model for Knowledge Graph Question Answering
May 4, 2026Jiujiu Chen, Yazheng Liu, Sihong Xie +1Process Reward Model Knowledge Graphs
HKUST (GZ)
Verifiable Counterfactual Supervision for Process Reward Models
May 4, 2026Yinghui Chi, Yuanhong WangProcess Reward Model Counterfactual Learning
Jilin University
Decoding-Time Debiasing via Process Reward Models: From Controlled Fill-in to Open-Ended Generation
May 4, 2026Muneeb Ur Raheem KhanLarge Language Model Bias Debiasing
Lahore University of Management Sciences (LUMS)
LSR-Ben: A Logical and Scientific Reasoning Benchmark for Evaluating Process Reward Models
May 2, 2026Zhouhao Sun, Xuan Zhang, Xiao Ding +10Reasoning Benchmark Process Reward Model
Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology, China · Beijing Academy of Artificial Intelligence, Beijing, China
Improving Vision-language Models with Perception-centric Process Reward Models
Apr 27, 2026Yingqian Min, Kun Zhou, Yifan Li +6Recent Vision-Language Models Process Reward Model
Gaoling School of Artificial Intelligence, Renmin University of China. · Bytedance. · University of California, San Diego. +1
Rewarding the Scientific Process: Process-Level Reward Modeling for Agentic Data Analysis
Apr 27, 2026Zhisong Qiu, Shuofei Qiao, Kewei Xu +4Process Reward Model Data Science Agents
Zhejiang University Ant Group Zhejiang University - Ant Group Joint Laboratory of Knowledge Graph Hangzhou, China · Zhejiang University Hangzhou, China · Ant Group Hangzhou, China +2
Process Supervision of Confidence Margin for Calibrated LLM Reasoning
Apr 25, 2026Liaoyaqi Wang, Chunsheng Zuo, William Jurayj +2Confidence Estimation Large Language Model Reinforcement Learning
Johns Hopkins University
ProcessThinker: Enhancing Multi-modal Large Language Models Reasoning via Rollout-based Process Reward
Apr 23, 2026Jingpei Wu, Xiao Han, Weixiang Shen +3Multimodal Reasoning Process Reward Model
LMU Munich · Harvard University · University of Cambridge +2
Process Reward Models Meet Planning: Generating Precise and Scalable Datasets for Step-Level Rewards
Apr 20, 2026Raffaele Pisano, Roberto NavigliProcess Reward Model Table Reasoning Datasets
Babelscape · Sapienza University of Rome
Internalizing Outcome Supervision into Process Supervision: A New Paradigm for Reinforcement Learning for Reasoning
Apr 19, 2026Fei Ding, Yongkang Zhang, Runhao Liu +4Process Reward Model Process-Level Supervision
Alibaba Group · Tsinghua University
MedPRMBench: A Fine-grained Benchmark for Process Reward Models in Medical Reasoning
Apr 19, 2026Lingyan Wu, Xiang Zheng, Weiqi Zhai +5Clinical Reasoning Training Reasoning Benchmark
Alibaba Group China
RAWR: Reward Assignment Without Rollouts in Verifiable Domains
Mar 18, 2026Corentin Royer, Anna Hedström, Debarun Bhattacharjya +3Process Reward Model Reinforcement Learning With Verifiable Reward
International Business Machines · Department of Computer Science, ETH Zurich, 8092 Zurich, Switzerland · ETH AI Center +2
Truncated Step-Level Sampling with Process Rewards for Retrieval-Augmented Reasoning
Feb 26, 2026Chris Samarinas, Haw-Shiuan Chang, Hamed ZamaniProcess Reward Model Credit Assignment
Center for Intelligent Information Retrieval University of Massachusetts Amherst Amherst, MA, United States

Sep 30, 2026, cs.AIJ/K move · Enter open · S save

Learning Process Rewards via Reasoning State Propagation

Kai Gan, Zi-Hao Zhou, Bo Ye, Jian Zhao+2

School of Computer Science and Engineering, Southeast University, Nanjing 210096, China · Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, China · Zhongguancun Academy+1

Process Reward Model Process-Level Supervision Propagation Break

Open paper AskPDF arXiv

Process reward models (PRMs) have demonstrated notable effectiveness in test-time scaling and reinforcement learning by providing fine-grained signals for evaluating intermediate reasoning states, but their training relies heavily on costly process annotations. A natural way to alleviate this dependence is to complement limited process supervision with scalable outcome supervision. However, existing PRMs often model reasoning prefixes independently, providing no explicit mechanism for effectively using final outcome to guide the learning of intermediate reasoning states. We introduce Reasoning State Propagation (RSP), which represents each reasoning prefix with a binary validity state and models transitions between successive states across the reasoning trajectory. Specifically, RSP predicts a break probability that a valid state becomes invalid and a repair probability that an invalid state returns to valid. By propagating these transitions, RSP connects intermediate states to the final state, allowing process annotations to supervise intermediate states while outcome labels supervise the final state and can provide learning signals to preceding steps. Across reasoning search, response selection, and reinforcement learning, RSP consistently outperforms representative PRM baselines, with average improvements over Qwen2.5-Math-PRM of 5.6% in beam search and 2.1% in reinforcement learning.

Momentum

Related topics

Latest papers 69