Process Reward Model

Follow this topic Ask about this topic

Momentum

6 papers in the last four weeks, up 20% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 69

All topics

rePIRL: Learn PRM with Inverse RL for LLM Reasoning
Feb 8, 2026Xian Wu, Kaijie Zhu, Ying Zhang +2Process Reward Model Large Language Model Reinforcement Learning
Meta AI · Department of Computer Science, University of California, Santa Barbara · Independent Researcher +1
SCRIBE: Structured Mid-Level Supervision for Tool-Using Language Models
Jan 7, 2026Yuxuan Jiang, Francis FerraroProcess Reward Model Process-Level Supervision
University of Maryland, Baltimore County
ScalePRM: Training Process Reward Models by Scaling Verification Compute Without Ground Truth
Dec 2, 2025Salman Rahman, Sruthi Gorantla, Arpit Gupta +3Process Reward Model Mathematical Reasoning Benchmarks
2UCLA · Work done while as an intern at Amazon AGI. · 1Amazon AGI
Who Gets the Reward & Who Gets the Blame? Evaluation-Aligned Training Signals for Multi-LLM Agents
Nov 11, 2025Chih-Hsuan, Yang, Tanwi Mallick +5Multi-Llm Agents Process Reward Model
Bella · Argonne National Laboratory · & University of Chicago
MASPRM: Multi-Agent System Process Reward Model
Oct 28, 2025Milad Yazdani, Mahdi Mostajabdaveh, Zirui Zhou +1Process Reward Model Beam Search
Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC V6T1Z4, Canada · Huawei Technologies Canada, Burnaby, BC V5C6S7, Canada
Rethinking Reward Models for Multi-Domain Test-Time Scaling
Oct 1, 2025Dong Bok Lee, Seanie Lee, Sangwoo Park +12Process Reward Model LLM Reasoning Strategies
KAIST · TH Nürnberg · Microsoft Research Asia
GUI-PRA: Process Reward Agent for GUI Tasks
Sep 27, 2025Tao Xiong, Xavier Hu, Yurun Chen +6Process Reward Model
Zhejiang University · MiLM Plus, Xiaomi Inc.
TagPR: Tag-Guided Process Supervision for Personalization Reasoning in Large Language Models
Sep 27, 2025Song Jin, Juntian Zhang, Ruyu Lyu +7Personalization Process Reward Model
Gaoling School of Artificial Intelligence, Renmin University of China · Meituan · Wuhan University
The Bidirectional Process Reward Model
Aug 3, 2025Lingyin Zhang, Jun Gao, Xiaoxue Ren +1Process Reward Model LLM Reasoning Strategies
School of Computer Science and Technology, Soochow University · Biomedical Basic Research Center of Jiangsu, Soochow University, Suzhou, Jiangsu 215123, China · School of Software Technology, Zhejiang University +1

Feb 8, 2026, cs.LGJ/K move · Enter open · S save

rePIRL: Learn PRM with Inverse RL for LLM Reasoning

Xian Wu, Kaijie Zhu, Ying Zhang, Lun Wang+1

Meta AI · Department of Computer Science, University of California, Santa Barbara · Independent Researcher+1

Process Reward Model Large Language Model Reinforcement Learning LLM Reasoning Strategies Offline Reinforcement Learning

Open paper AskPDF arXiv

Process rewards have been widely used in deep reinforcement learning to improve training efficiency, reduce variance, and prevent reward hacking. In LLM reasoning, existing works also explore various solutions for learning effective process reward models (PRM) with or without the help of an expert policy. However, existing methods either rely on strong assumptions about the expert policies (e.g., requiring their reward functions) or suffer intrinsic limitations (e.g., entropy collapse), resulting in weak PRMs or limited generalizability. In this paper, we introduce rePIRL, an inverse RL-inspired framework that learns effective PRMs with minimal assumptions about expert policies. Specifically, we design a dual learning process that updates the policy and the PRM interchangeably. Our learning algorithm has customized techniques to address the challenges of scaling traditional inverse RL to LLMs. We theoretically show that our proposed learning framework can unify both online and offline PRM learning methods, justifying that rePIRL can learn PRMs with minimal assumptions. Empirical evaluations on standardized math and coding reasoning datasets demonstrate the effectiveness of rePIRL over existing methods. We further show the application of our trained PRM in test-time training, test-time scaling, and providing an early signal for training hard problems. Finally, we validate our training recipe and key design choices via a detailed ablation study.

Momentum

Related topics

Latest papers 69