Process Reward Models

Also known as PRM

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, up 40% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 91

All topics

Unsupervised Process Reward Models
May 11, 2026Artyom Gadetsky, Maxim Kodryan, Siba Smarak Panigrahi +2Reward Modeling Process Reward Models
Swiss Federal Institute of Technology (EPFL)
FormalRewardBench: A Benchmark for Formal Theorem Proving Reward Models
May 11, 2026Zeynel A. Uluşan, Burak S. Akbudak, Can S. Erer +1Reward Modeling Automated Theorem Proving
Koç University, Department of Computer Science and Engineering · Codeway Studios · Bo˘gaziçi University, Department of Computer Engineering +1
BoostAPR: Boosting Automated Program Repair via Execution-Grounded Reinforcement Learning with Dual Reward Models
May 9, 2026Yuanhao Li, Hongbo Wang, Xiaotang Shang +3Automated Program Repair Process Reward Models
State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China · University of Luxembourg
Distributional Process Reward Models: Calibrated Prediction of Future Rewards via Conditional Optimal Transport
May 7, 2026Rachel Ma, Dylan Hadfield-Menell, Kristjan GreenewaldProcess Reward Models Inference-Time Scaling
MIT CSAIL · MIT-IBM Watson AI Lab · IBM Research
Correct Is Not Enough: Training Reasoning Planners with Executor-Grounded Rewards
May 5, 2026Tianyang Han, Hengyu Shi, Junjie Hu +3RL for Code Generation Process Reward Models
1D4 Lab · 2Independent Researcher
SCPRM: A Schema-aware Cumulative Process Reward Model for Knowledge Graph Question Answering
May 4, 2026Jiujiu Chen, Yazheng Liu, Sihong Xie +1LLM Reasoning with Graphs Process Reward Models
HKUST (GZ)
Verifiable Counterfactual Supervision for Process Reward Models
May 4, 2026Yinghui Chi, Yuanhong WangProcess Reward Models Process Supervision
Jilin University
Decoding-Time Debiasing via Process Reward Models: From Controlled Fill-in to Open-Ended Generation
May 4, 2026Muneeb Ur Raheem KhanDebiased ML Language Model Decoding
Lahore University of Management Sciences (LUMS)
LSR-Ben: A Logical and Scientific Reasoning Benchmark for Evaluating Process Reward Models
May 2, 2026Zhouhao Sun, Xuan Zhang, Xiao Ding +10Logical Reasoning Process Reward Models
Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology, China · Beijing Academy of Artificial Intelligence, Beijing, China
Granularity-Regulated Adaptive Computational Efficiency for Optimal Verification in Test-Time Scaling
Apr 28, 2026Ardit Krasniqi, Luan Vejsiu, Elira DervishiInference-Time Optimization Process Reward Models
European University of Tirana
Improving Vision-language Models with Perception-centric Process Reward Models
Apr 27, 2026Yingqian Min, Kun Zhou, Yifan Li +6Process Reward Models Object Hallucination in VLMs
Gaoling School of Artificial Intelligence, Renmin University of China. · Bytedance. · University of California, San Diego. +1
DPRM: A Plug-in Doob h transform-induced Token-Ordering Module for Diffusion Language Models
Apr 27, 2026Dake Bu, Wei Huang, Andi Han +4Doob H-Transform Process Reward Models
City University of Hong Kong · CFAR and IHPC, A*STAR · RIKEN AIP +4
Rewarding the Scientific Process: Process-Level Reward Modeling for Agentic Data Analysis
Apr 27, 2026Zhisong Qiu, Shuofei Qiao, Kewei Xu +4Process Reward Models RL for LLM Agents
Zhejiang University Ant Group Zhejiang University - Ant Group Joint Laboratory of Knowledge Graph Hangzhou, China · Zhejiang University Hangzhou, China · Ant Group Hangzhou, China +2
Reward Models Are Secretly Value Functions: Temporally Coherent Reward Modeling
Apr 24, 2026Alex NikulkovReward Modeling Value Function Estimation
AI at Meta
ProcessThinker: Enhancing Multi-modal Large Language Models Reasoning via Rollout-based Process Reward
Apr 23, 2026Jingpei Wu, Xiao Han, Weixiang Shen +3Process Reward Models Multimodal QA
LMU Munich · Harvard University · University of Cambridge +2
ProMMSearchAgent: A Generalizable Multimodal Search Agent Trained with Process-Oriented Rewards
Apr 22, 2026Wentao Yan, Shengqin Wang, Huichi Zhou +4Multimodal IR Process Reward Models
East China Normal University · Huawei Noah’s Ark Lab · University College London +2
Process Reward Models Meet Planning: Generating Precise and Scalable Datasets for Step-Level Rewards
Apr 20, 2026Raffaele Pisano, Roberto NavigliProcess Reward Models Synthetic Data Generation
Babelscape · Sapienza University of Rome
MedPRMBench: A Fine-grained Benchmark for Process Reward Models in Medical Reasoning
Apr 19, 2026Lingyan Wu, Xiang Zheng, Weiqi Zhai +5Healthcare Process Reward Models
Alibaba Group China
RAWR: Reward Assignment Without Rollouts in Verifiable Domains
Mar 18, 2026Corentin Royer, Anna Hedström, Debarun Bhattacharjya +3Process Reward Models Reinforcement Learning with Verifiable Rewards
International Business Machines · Department of Computer Science, ETH Zurich, 8092 Zurich, Switzerland · ETH AI Center +2
Verify to Amplify: Improving Reasoning via Learned Chain-of-Thought Verification
Mar 3, 2026Maria-Florina Balcan, Avrim Blum, Kiriaki Fragkia +2LLM Evaluation LLM Answer Verification
Carnegie Mellon University · Toyota Technological Institute at Chicago · Northwestern University
Truncated Step-Level Sampling with Process Rewards for Retrieval-Augmented Reasoning
Feb 26, 2026Chris Samarinas, Haw-Shiuan Chang, Hamed ZamaniReinforcement Learning Process Reward Models
Center for Intelligent Information Retrieval University of Massachusetts Amherst Amherst, MA, United States
Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation
Feb 10, 2026Pei-Chi Pan, Yingbin Liang, Sen LinReward Modeling Reward Hacking
Department of Computer Science University of Houston · The Ohio State University · University of Houston
rePIRL: Learn PRM with Inverse RL for LLM Reasoning
Feb 8, 2026Xian Wu, Kaijie Zhu, Ying Zhang +2Reinforcement Learning Process Reward Models
Meta AI · Department of Computer Science, University of California, Santa Barbara · Independent Researcher +1
MAS-ProVe: Understanding the Process Verification of Multi-Agent Systems
Feb 3, 2026Vishal Venkataramani, Haizhou Shi, Zixuan Ke +6Multi-Agent LLM Systems LLM-as-a-Judge
1Rutgers University · 2Salesforce AI Research
SCRIBE: Structured Mid-Level Supervision for Tool-Using Language Models
Jan 7, 2026Yuxuan Jiang, Francis FerraroProcess Reward Models RL for Language Model Reasoning
University of Maryland, Baltimore County
ScalePRM: Training Process Reward Models by Scaling Verification Compute Without Ground Truth
Dec 2, 2025Salman Rahman, Sruthi Gorantla, Arpit Gupta +3Process Reward Models Reinforcement Learning with Verifiable Rewards
2UCLA · Work done while as an intern at Amazon AGI. · 1Amazon AGI
Who Gets the Reward & Who Gets the Blame? Evaluation-Aligned Training Signals for Multi-LLM Agents
Nov 11, 2025Chih-Hsuan, Yang, Tanwi Mallick +5Multi-Agent LLM Systems Cooperative Game Theory
Bella · Argonne National Laboratory · & University of Chicago
MASPRM: Multi-Agent System Process Reward Model
Oct 28, 2025Milad Yazdani, Mahdi Mostajabdaveh, Zirui Zhou +1Multi-Agent LLM Systems Inference-Time Search
Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC V6T1Z4, Canada · Huawei Technologies Canada, Burnaby, BC V5C6S7, Canada
Rethinking Reward Models for Multi-Domain Test-Time Scaling
Oct 1, 2025Dong Bok Lee, Seanie Lee, Sangwoo Park +12Reward Modeling LLM Evaluation
KAIST · TH Nürnberg · Microsoft Research Asia
GUI-PRA: Process Reward Agent for GUI Tasks
Sep 27, 2025Tao Xiong, Xavier Hu, Yurun Chen +6Process Reward Models GUI Agents
Zhejiang University · MiLM Plus, Xiaomi Inc.

May 11, 2026, cs.LGJ/K move · Enter open · S save

Unsupervised Process Reward Models

Artyom Gadetsky, Maxim Kodryan, Siba Smarak Panigrahi, Hang Guo+1

Swiss Federal Institute of Technology (EPFL)

Reward Modeling Process Reward Models Reasoning Verification

Open paper AskPDF arXiv

Process Reward Models (PRMs) are a powerful mechanism for steering large language model reasoning by providing fine-grained, step-level supervision. However, this effectiveness comes at a significant cost: PRMs require expert annotations for every reasoning step, making them costly and difficult to scale. Here, we propose a method for training unsupervised PRMs (uPRM) that requires no human supervision, neither at the level of step-by-step annotations nor through ground-truth verification of final answers. The key idea behind our approach is to define a scoring function, derived from LLM next-token probabilities, that jointly assesses candidate positions of first erroneous steps across a batch of reasoning trajectories. We demonstrate the effectiveness of uPRM across diverse scenarios: (i) uPRM achieves up to 15% absolute accuracy improvements over the LLM-as-a-Judge in identifying first erroneous steps on the ProcessBench dataset; (ii) as a verifier for test-time scaling, uPRM performs comparably to supervised PRMs and outperforms the majority voting baseline by up to 6.9%, and (iii) when used as a reward signal in reinforcement learning, uPRM enables more robust policy optimization throughout training compared to a supervised PRM trained using ground-truth labels. Overall, our results open a path toward scalable reward modeling for complex reasoning tasks.

Momentum

Related topics

Latest papers 91