Process Reward Models

Also known as PRM

Process reward models estimate whether intermediate steps in a reasoning chain or agent trajectory are valid, useful, and moving toward a goal, rather than judging only the final answer. They are trained from human step labels, formal or execution-based checks, rubrics, and signals inferred from successful or failed outcomes; some methods avoid a separate model and derive rewards from rollouts or internal probabilities. These rewards support reinforcement learning and test-time methods such as reranking, beam search, and tree search across math, coding, retrieval, multimodal, and agent tasks. A central challenge is that noisy or miscalibrated step scores can compound errors or be exploited, and it remains difficult to make process evaluation reliable across domains and task settings.

Latest papers

141 papers22 papers in the last four weeks, up 214% on the four weeks before. 0.1% of all new papers.

MAPLE: Elevating Medical Reasoning from Statistical Consensus to Process-Led Alignment
Mar 9, 2026Kailong Fan, Anqi Pu, Yichen Wu +7Medical LLMs RL for Language Models
Zhejiang University · Harvard Medical School · Harvard University
DC-W2S: Dual-Consensus Weak-to-Strong Training for Reliable Process Reward Modeling in Biological Reasoning
Mar 9, 2026Chi-Min Chan, Ehsan Hajiramezanali, Xiner Li +6Process Reward Models Weak-to-Strong Generalization
The Hong Kong University of Science and Technology, Hong Kong · Genentech, Inc., South San Francisco, CA, USA.
Post-Training with Policy Gradients: Optimality and the Base Model Barrier
Mar 7, 2026Alireza Mousavi-Hosseini, Murat A. ErdogduLanguage Model Post-Training Policy Gradient
University of Toronto, Vector Institute. · University of Toronto, Vector Institute, and Numurho.
Verify to Amplify: Improving Reasoning via Learned Chain-of-Thought Verification
Mar 3, 2026Maria-Florina Balcan, Avrim Blum, Kiriaki Fragkia +2LLM Evaluation LLM Answer Verification
Carnegie Mellon University · Toyota Technological Institute at Chicago · Northwestern University
PRISM: Pushing the Frontier of Deep Think via Process Reward Model-Guided Inference
Mar 3, 2026Rituraj Sharma, Weiyuan Chen, Noah Provenzano +1LLM Reasoning Inference-Time Guidance
Virginia Tech
Truncated Step-Level Sampling with Process Rewards for Retrieval-Augmented Reasoning
Feb 26, 2026Chris Samarinas, Haw-Shiuan Chang, Hamed ZamaniReinforcement Learning Process Reward Models
Center for Intelligent Information Retrieval University of Massachusetts Amherst Amherst, MA, United States
ContextRL: Enhancing MLLM's Knowledge Discovery Efficiency with Context-Augmented RL
Feb 26, 2026Xingyu Lu, Jinpeng Wang, YiFan Zhang +12RL for VLMs Reinforcement Learning with Verifiable Rewards
Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Harbin Institute of Technology, Shenzhen, Shenzhen, China · Chinese Academy of Sciences, Beijing, China +2
Reward Under Attack: Analyzing the Robustness and Hackability of Process Reward Models
Feb 20, 2026Rishabh Tiwari, Aditya Tomar, Udbhav Bamba +5Process Reward Models Reward Hacking
UC Berkeley · Transmute AI · ICSI +1
WS-GRPO: Weakly-Supervised Group-Relative Policy Optimization for Rollout-Efficient Reasoning
Feb 19, 2026Gagan Mundada, Zihan Huang, Rohan Surana +8Efficient Language Model Reasoning RL for Language Model Reasoning
University of California, San Diego · University of Toronto · 3Adobe Research +2
Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation
Feb 10, 2026Pei-Chi Pan, Yingbin Liang, Sen LinReward Modeling Reward Hacking
Department of Computer Science University of Houston · The Ohio State University · University of Houston
Process Rewards for Outcome-Guided Reasoning Steps in LLM Reinforcement Learning
Feb 8, 2026Mohammad Rezaei, Jens Lehmann, Sahar VahdatiLLM Mathematical Reasoning Reinforcement Learning
Faculty of Computer Science, Technical University of Dresden, Dresden, Germany · Amazon Science · Technical University of Dresden, Dresden, Germany +1
rePIRL: Learn PRM with Inverse RL for LLM Reasoning
Feb 8, 2026Xian Wu, Kaijie Zhu, Ying Zhang +2Reinforcement Learning Process Reward Models
Meta AI · Department of Computer Science, University of California, Santa Barbara · Independent Researcher +1
MAS-ProVe: Understanding the Process Verification of Multi-Agent Systems
Feb 3, 2026Vishal Venkataramani, Haizhou Shi, Zixuan Ke +6Multi-Agent LLM Systems LLM-as-a-Judge
1Rutgers University · 2Salesforce AI Research
SCRIBE: Structured Mid-Level Supervision for Tool-Using Language Models
Jan 7, 2026Yuxuan Jiang, Francis FerraroProcess Reward Models RL for Language Model Reasoning
University of Maryland, Baltimore County
ScalePRM: Training Process Reward Models by Scaling Verification Compute Without Ground Truth
Dec 2, 2025Salman Rahman, Sruthi Gorantla, Arpit Gupta +3Process Reward Models Reinforcement Learning with Verifiable Rewards
2UCLA · Work done while as an intern at Amazon AGI. · 1Amazon AGI
Who Gets the Reward & Who Gets the Blame? Evaluation-Aligned Training Signals for Multi-LLM Agents
Nov 11, 2025Chih-Hsuan, Yang, Tanwi Mallick +5Multi-Agent LLM Systems Cooperative Game Theory
Bella · Argonne National Laboratory · & University of Chicago
MASPRM: Multi-Agent System Process Reward Model
Oct 28, 2025Milad Yazdani, Mahdi Mostajabdaveh, Zirui Zhou +1Multi-Agent LLM Systems Inference-Time Search
Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC V6T1Z4, Canada · Huawei Technologies Canada, Burnaby, BC V5C6S7, Canada
Rethinking Reward Models for Multi-Domain Test-Time Scaling
Oct 1, 2025Dong Bok Lee, Seanie Lee, Sangwoo Park +12Reward Modeling LLM Evaluation
KAIST · TH Nürnberg · Microsoft Research Asia
GUI-PRA: Process Reward Agent for GUI Tasks
Sep 27, 2025Tao Xiong, Xavier Hu, Yurun Chen +6Process Reward Models GUI Agents
Zhejiang University · MiLM Plus, Xiaomi Inc.
VRPRM: Process Reward Modeling via Visual Reasoning
Aug 5, 2025Xinquan Chen, Chongying Yue, Bangwei Liu +3Process Reward Models Multimodal Reward Modeling
Shanghai AI Laboratory · Zhejiang University · East China Normal University
The Bidirectional Process Reward Model
Aug 3, 2025Lingyin Zhang, Jun Gao, Xiaoxue Ren +1Process Reward Models Reasoning Verification
School of Computer Science and Technology, Soochow University · Biomedical Basic Research Center of Jiangsu, Soochow University, Suzhou, Jiangsu 215123, China · School of Software Technology, Zhejiang University +1

Mar 9, 2026J/K move · Enter open · S save

MAPLE: Elevating Medical Reasoning from Statistical Consensus to Process-Led Alignment

Kailong Fan, Anqi Pu, Yichen Wu, Wanhua Li+6

Zhejiang University · Harvard Medical School · Harvard University

Medical LLMs RL for Language Models Process Reward Models

Open paper AskPDF arXiv

Recent advances in medical large language models have explored Test-Time Reinforcement Learning (TTRL) to enhance reasoning. However, standard TTRL often relies on majority voting (MV) as a heuristic supervision signal, which can be unreliable in complex medical scenarios where the most frequent reasoning path is not necessarily the clinically correct one. In this work, we propose a novel and unified training paradigm that integrates medical process reward models with TTRL to bridge the gap between test-time scaling (TTS) and parametric model optimization. Specifically, we advance the TTRL framework by replacing the conventional MV with a fine-grained, expert-aligned supervision paradigm using Med-RPM. This integration ensures that reinforcement learning is guided by medical correctness rather than mere consensus, effectively distilling search-based intelligence into the model's parametric memory. Extensive evaluations on four different benchmarks have demonstrated that our developed method consistently and significantly outperforms current TTRL and standalone PRM selection. Our findings establish that transitioning from stochastic heuristics to structured, step-wise rewards is essential for developing reliable and scalable medical AI systems