Process Supervision

Follow this topic Ask about this topic

Process supervision studies how to assess and improve the steps an AI system takes, since a correct answer can result from flawed reasoning or a lucky guess. Approaches include human or model critiques, process reward models, and verifiers based on code execution, formal logic, or other checkable evidence; these signals can guide fine-tuning and reinforcement learning. The methods are applied to tasks such as mathematics, coding, tool use, medical reasoning, and multimodal question answering. A central challenge is producing step-level signals that are accurate and affordable while ensuring that visible reasoning is faithful to the process that actually drives the answer.

Latest papers

74 papers11 papers in the last four weeks, down 15% on the four weeks before. 0.1% of all new papers.

GRPO-VPS: Enhancing Group Relative Policy Optimization with Verifiable Process Supervision for Effective Reasoning
Apr 22, 2026Jingyi Wang, Lei Zhu, Tengjin Weng +8RL for Language Model Reasoning Group Relative Policy Optimization
Tsinghua University · Huawei Noah’s Ark Lab · Shenzhen University
Prioritizing the Best: Incentivizing Reliable Multimodal Reasoning by Rewarding Beyond Answer Correctness
Apr 20, 2026Mengzhao Jia, Zhihan Zhang, Meng JiangMultimodal Reasoning Multimodal Reward Modeling
University of Notre Dame
Internalizing Outcome Supervision into Process Supervision: A New Paradigm for Reinforcement Learning for Reasoning
Apr 19, 2026Fei Ding, Yongkang Zhang, Runhao Liu +4RL for Language Model Reasoning Credit Assignment in RL
Alibaba Group · Tsinghua University
Improving Medical VQA through Trajectory-Aware Process Supervision
Apr 10, 2026Halil Ibrahim Gulluk, Olivier GevaertMedical VQA Medical VLMs
Electrical Engineering, Stanford University · Stanford Center For Biomedical Informatics Research
SHAPE: Stage-aware Hierarchical Advantage via Potential Estimation for LLM Reasoning
Apr 8, 2026Zhengyang Ai, Zikang Shan, Xiaodong Ai +3LLM Reasoning Efficient Language Model Reasoning
Huawei Taylor Lab · Center for Data Science, Peking University · Shanghai University of Finance and Economics
Correct Answers from Sound Reasoning: Verifiable Process Supervision for Language Models
Apr 3, 2026Kyuyoung Kim, Kevin Wang, Yunfei Xie +7Process Supervision RL for Language Model Reasoning
KAIST AI · University of Texas, Austin · Rice University +3
Reliable Self-Improvement Training by Verifying Reasoning, Not Just Answers
Mar 23, 2026Xinyu ZhangLanguage Model Self-Improvement Self-Training for Language Models
Anyscale
RAWR: Reward Assignment Without Rollouts in Verifiable Domains
Mar 18, 2026Corentin Royer, Anna Hedström, Debarun Bhattacharjya +3Process Reward Models Reinforcement Learning with Verifiable Rewards
International Business Machines · ETH AI Center · Lirio +1
AgentProcessBench: Diagnosing Step-Level Process Quality in Tool-Using Agents
Mar 15, 2026Shengda Fan, Xuyan Ye, Yupeng Huo +9LLM Agent Evaluation Process-Based Agent Evaluation
Renmin University of China Beijing, China · Beijing Jiaotong University Beijing, China · Shanghai Jiao Tong University Shanghai, China +1
Social-R1: Towards Human-like Social Reasoning in LLMs
Mar 10, 2026Jincenzi Wu, Yuxuan Lei, Jianxun Lian +5Social Reasoning RL for Language Model Reasoning
The Chinese University of Hong Kong · Microsoft Research Asia · Princeton University
PaLMR: Towards Faithful Visual Reasoning via Multimodal Process Alignment
Feb 28, 2026Yantao Li, Qiang Hui, Chenyang Yan +8Multimodal Reasoning Faithfulness in VLMs
National Key Laboratory for Novel Software Technology, Nanjing University · Data Science & Artificial Intelligence Research Institute, China Unicom · Unicom Data Intelligence, China Unicom
Data-Driven Supervision of a Thermal-Hydraulic Process Towards a Physics-Based Digital Twin
Feb 25, 2026Osimone Imhogiemhe, Yoann Jus, Hubert Lejeune +1Fault Detection Process Supervision
LS2N, LS2N - équipe SIMS, Nantes Univ - ECN · Nantes Univ., Centrale Nantes LS2N, CNRS UMR 6004 F-44000 Nantes, France · Cetim +1
Pixels Don't Lie (But Your Detector Might): Bootstrapping MLLM-as-a-Judge for Trustworthy Deepfake Detection and Reasoning Supervision
Feb 23, 2026Kartik Kuckreja, Parul Gupta, Muhammad Haris Khan +1Deepfake Detection Explainable Deepfake Detection
MBZUAI · Monash University *
TagPR: Tag-Guided Process Supervision for Personalization Reasoning in Large Language Models
Sep 27, 2025Song Jin, Juntian Zhang, Ruyu Lyu +7LLM Alignment LLM Personalization
Gaoling School of Artificial Intelligence, Renmin University of China · Meituan · Wuhan University

Apr 22, 2026J/K move · Enter open · S save

GRPO-VPS: Enhancing Group Relative Policy Optimization with Verifiable Process Supervision for Effective Reasoning

Jingyi Wang, Lei Zhu, Tengjin Weng, Song-Li Wu+7

Tsinghua University · Huawei Noah’s Ark Lab · Shenzhen University

RL for Language Model Reasoning Group Relative Policy Optimization Reinforcement Learning with Verifiable Rewards Process Supervision

Open paper AskPDF arXiv

Reinforcement Learning with Verifiable Rewards (RLVR) has advanced the reasoning capabilities of Large Language Models (LLMs) by leveraging direct outcome verification instead of learned reward models. Building on this paradigm, Group Relative Policy Optimization (GRPO) eliminates the need for critic models but suffers from indiscriminate credit assignment for intermediate steps, which limits its ability to identify effective reasoning strategies and incurs overthinking. In this work, we introduce a model-free and verifiable process supervision via probing the model's belief in the correct answer throughout its reasoning trajectory. By segmenting the generation into discrete steps and tracking the conditional probability of the correct answer appended at each segment boundary, we efficiently compute interpretable segment-wise progress measurements to refine GRPO's trajectory-level feedback. This approach enables more targeted and sample-efficient policy updates, while avoiding the need for intermediate supervision derived from costly Monte Carlo rollouts or auxiliary models. Experiments on mathematical and general-domain benchmarks show consistent gains over GRPO across diverse models: up to 2.6-point accuracy improvements and 13.7% reasoning-length reductions on math tasks, and up to 2.4 points and 4% on general-domain tasks, demonstrating strong generalization.