Process Reward Models

Also known as PRM

Process reward models estimate whether intermediate steps in a reasoning chain or agent trajectory are valid, useful, and moving toward a goal, rather than judging only the final answer. They are trained from human step labels, formal or execution-based checks, rubrics, and signals inferred from successful or failed outcomes; some methods avoid a separate model and derive rewards from rollouts or internal probabilities. These rewards support reinforcement learning and test-time methods such as reranking, beam search, and tree search across math, coding, retrieval, multimodal, and agent tasks. A central challenge is that noisy or miscalibrated step scores can compound errors or be exploited, and it remains difficult to make process evaluation reliable across domains and task settings.

Latest papers

141 papers22 papers in the last four weeks, up 214% on the four weeks before. 0.1% of all new papers.

Clearing the FoG: Training KG Agents to Retrieve the Evidence Behind Their Answers
Oct 27, 2026Zongyuan Sun, Shuai Zhao, Yingjun Du +3Knowledge Graph Question Answering Evidence-Grounded QA
Beijing University of Posts and Telecommunications, Beijing, China · University of Amsterdam, Amsterdam, Netherlands · University of Helsinki, Helsinki, Finland +1
AudioRSI: Recursively Self-Improving Audio Reasoners
Oct 27, 2026Jiajun Fan, Roger Ren, Jingyuan Li +6Audio Reasoning Audio-Language Models
UIUC
When Fine-Grained Credit Assignment Helps and Hurts: Multimodal Agentic Reinforcement Learning through Video Editing
Oct 26, 2026Lei Zhao, Zongheng Cao, Xinyu HuReinforcement Learning Fine-Grained Credit Assignment
University of Pennsylvania · Philo Labs
Implicit Beliefs as Scalable Process Rewards for Strategic Persuasion
Oct 24, 2026Minqian Liu, Yuexi Shen, Heajun An +5Persuasion in Language Models Process Reward Models
Virginia Tech · UC Santa Barbara · STIR Lab +1
AdaPRA: Process Reward Agents with Adaptive Rubrics for Data Science
Oct 23, 2026Atharva Naik, Vivek Chari, Pol Llado +5Data Analysis Agents Process Reward Models
Carnegie Mellon University · Microsoft
Predictive Long-Horizon Progress Modeling with Action-Aware Quantification
Oct 23, 2026Tianchi Xu, Yalin Zheng, Yangxin Zhu +4Robot Task Progress Estimation Long-Horizon Robotic Manipulation
RIMBOT · Peking University · Tsinghua University
A Strong MoE Is All You Need for Process Supervision
Oct 21, 2026Guanzhi Deng, Zengyan Liu, Haibo Wang +5Process Supervision Process Reward Models
City University of Hong Kong · Carnegie Mellon University · University of Hong Kong
Policy-Adaptive Credit Assignment for Spatial Reasoning with Verifiable Rewards
Oct 19, 2026Hailiang Yang, Shengchao ChenProcess Reward Models Step-Level Credit Assignment in RL
School of Computer Science and Engineering, University of New South Wales · Australian AI Institute, University of Technology Sydney
Process Reward Modeling with Reliability- and Conservatism-Aware Bayesian Belief
Oct 16, 2026Zhongjian Qiao, Zhixiang Zhou, Jiafei Lyu +4MLLM Reasoning Process Reward Models
CityUHK · Shanghai Innovation Institute · Tencent +1
Agentic In-Context Learning: What Algorithms Can Transformers Learn in Closed-Loop Interaction?
Oct 16, 2026Shiguang Wu, Yaqing Wang, Quanming YaoIn-Context Learning In-Context RL
Tsinghua University · Beijing Institute of Mathematical Sciences and Applications
KnowGPO: Knowledge-Grouped Policy Optimization for Search Agents
Oct 15, 2026Siqi Chen, Zhengxi Lu, Yuchen Yan +10RL for LLM Agents Reinforcement Learning
Zhejiang University · Tencent
Lean4AgenticRL: Advancing Agentic RL with Verifiable Multi-level Process Rewards
Oct 15, 2026Ruida Wang, Gerald Friedland, Tong Zhang +1Agentic RL RL for Tool Use
Amazon Web Services · University of Illinois Urbana-Champaign
All-in-One Memory: Process-Rewarded Reinforcement Learning for Memory Agents
Oct 15, 2026Bingbing Wang, Jingjie Lin, Weiming Qiao +2Persistent Memory for Language Models Memory-Augmented Language Model Agents
Harbin Institute of Technology, Shenzhen · The Hong Kong Polytechnic University · Fudan University +1
Unlocking Token Rewards via Training-Free Reward Attribution
Oct 13, 2026Sitong Wu, Haoru Tan, Bin Xia +6Token-Level Credit Assignment RL for Language Model Reasoning
Chinese University of Hong Kong · University of Hong Kong · Hong Kong University of Science and Technology
Adaptive-Group Process Rewards for GUI Agents
Oct 13, 2026Zirui Shang, Yang Liu, Xin Shu +3RL for GUI Agents Process Reward Models
Beijing Key Laboratory of Intelligent Information Technology, School of Computer Science & Technology, Beijing Institute of Technology · State Key Laboratory of General Artificial Intelligence, BIGAI · Wuhan University +1
LeanPRM: Lean-Grounded Process Supervision for Natural Language Proof Verification
Oct 13, 2026Rodney Lafuente-Mercado, Salman Rahman, Jingyan Shen +3Process Supervision Process Reward Models
University of Texas at Austin · University of California, Los Angeles · New York University
RewardWeaver: Long-Horizon Interactive Learning for Language Agents via Self-Evolving Reward Adaptation
Oct 7, 2026Hengbo Xiao, Boyao Zhang, Purui Liu +4RL for LLM Agents Rubric-Based RL
BoT-GRPO: Efficient Process-Reward RL for Reasoning via Bag-of-Token Aggregation
Oct 7, 2026Yingxiang Yang, Weihang Xiao, Zhunxuan Wang +2RL for Language Model Reasoning Group Relative Policy Optimization
Amazon AGI · Virginia Tech
World Potential Model: Pretrained World Knowledge as Progress Potentials
Oct 7, 2026Jun Zhao, Jixin Tang, Yang Shu +6Long-Horizon LLM Agents RL for LLM Agents
National University of Singapore · Fudan University · Tsinghua University +1
EnGRICH: Enhancing Generative Reward Modeling with Critiques from Humans
Oct 4, 2026Xuancheng Li, Beining Wang, Haitao Li +7Reward Modeling Process Reward Models
Department of Computer Science and Technology, Tsinghua University, Beijing, China · Tencent, Beijing, China
Learning Process Rewards via Reasoning State Propagation
Sep 30, 2026Kai Gan, Zi-Hao Zhou, Bo Ye +3Process Reward Models RL for Language Model Reasoning
School of Computer Science and Engineering, Southeast University, Nanjing 210096, China · Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, China · Zhongguancun Academy +1
ProgressCompass: Embodied Progress Reward Models Are Lost Without the Right Context
Sep 29, 2026Jianshu Zhang, Keliang Wu, Chengxuan Qian +6Robot Task Progress Estimation Process Reward Models
Northwestern · UCSB · UIUC +1
Inducing Process Supervision from Outcome-Only Reinforcement Learning
Sep 29, 2026Shengda Fan, Xin Cong, Zhong Zhang +2Reinforcement Learning Process Reward Models
Renmin University of China · Tsinghua University · University of Electronic Science and Technology of China +1
Why Deterministic PRM Guidance Underperforms in Discrete Diffusion Reasoning
Sep 28, 2026Yan Zhan, Shaobo Liu, Zhijun GaoProcess Reward Models Diffusion Language Model Decoding
Peking University · Shenzhen University
ARS: Agentic Reward System for Robot Learning
Sep 28, 2026Sheng Hu, Weiyi Lu, Lingbing Zeng +5RL for Robotics Process Reward Models
AIRC, Midea Group
From Trajectories to Grounded Preferences: Process Preference Synthesis via Interaction Element Graphs for Web PRMs
Sep 26, 2026Yangzhe Peng, Xiaoyang Wang, Yiyang Zhao +2Process Reward Models Synthetic Data Generation
Huazhong University of Science and Technology · Shanghai Artificial Intelligence Laboratory · Zhejiang University +1
LLaDA-PRM: A Bidirectional Step-Level Reasoning Evaluator
Sep 19, 2026Yiming Feng, Naihao Deng, Yulong Chen +1Reasoning Evaluation Language Model Reasoning Evaluation
University of Michigan · University of Aberdeen · University of Cambridge
Rewarding Reasoning, Not Answers: Fixing and Bounding Test-Time Reinforcement Learning on Medical QA
Sep 15, 2026Kailong Fan, Anqi Pu, Yichen Wu +7Process Reward Models RL for Language Model Reasoning
1Harvard Medical School/MGH · 3Harvard University · 2Zhejiang University
SocialRL: Refining LLMs' Social Intelligence through Multi-turn Reinforcement Learning and Reward Design
Sep 9, 2026Jianing Wang, Xintao Wang, Aili Chen +7RL for Language Models Reinforcement Learning
Fudan University · Hello Group
ConsensusBench: Benchmark of Consensus Nodes for LLM Reasoning via Outcome Reward Densifying
Sep 4, 2026Shi-Qi Yan, Chao-Hong Tan, Qian Chen +3RL for Language Model Reasoning LLM Reasoning
Alibaba Token Hub, Alibaba Group

Oct 27, 2026J/K move · Enter open · S save

Clearing the FoG: Training KG Agents to Retrieve the Evidence Behind Their Answers

Zongyuan Sun, Shuai Zhao, Yingjun Du, Ke Shu+2

Beijing University of Posts and Telecommunications, Beijing, China · University of Amsterdam, Amsterdam, Netherlands · University of Helsinki, Helsinki, Finland+1

Knowledge Graph Question Answering Evidence-Grounded QA LLM Agent Training Process Reward Models

Open paper AskPDF arXiv

Large language models (LLMs) can answer many factual questions correctly, but a correct answer is not necessarily a supported one. For an LLM agent, a knowledge graph (KG) plays two roles: as a source of answers, it supplies facts that the model lacks; as a source of evidence, it supplies the supporting triples that make an answer traceable and auditable. However, existing KG agents treat the KG only as a source of answers: their training data carries answers but not evidence, and their training rewards check the answer but not the retrieved evidence, so agents often answer correctly without retrieving the supporting triples. We propose FoG, a framework that trains KG agents to retrieve the evidence behind their answers. FoG consists of evidence-grounded synthesis, a KG interaction agent, and an evidence-coverage process reward (ECPR): the synthesis recursively hides explicit entities behind verifiable sub-questions, so that every question carries its supporting triples by construction; the agent records the triples returned by the KG; and ECPR credits each supporting triple to the turn that first retrieves it. Experiments on three KGQA benchmarks demonstrate that FoG enables 3B--4B open-source agents to outperform substantially larger baselines in answer accuracy and evidence coverage, with zero-shot cross-KG transfer.