Process Supervision

Follow this topic Ask about this topic

Process supervision studies how to assess and improve the steps an AI system takes, since a correct answer can result from flawed reasoning or a lucky guess. Approaches include human or model critiques, process reward models, and verifiers based on code execution, formal logic, or other checkable evidence; these signals can guide fine-tuning and reinforcement learning. The methods are applied to tasks such as mathematics, coding, tool use, medical reasoning, and multimodal question answering. A central challenge is producing step-level signals that are accurate and affordable while ensuring that visible reasoning is faithful to the process that actually drives the answer.

Latest papers

74 papers11 papers in the last four weeks, down 15% on the four weeks before. 0.1% of all new papers.

AdaPRA: Process Reward Agents with Adaptive Rubrics for Data Science
Oct 23, 2026Atharva Naik, Vivek Chari, Pol Llado +5Data Analysis Agents Process Reward Models
Carnegie Mellon University · Microsoft
A Strong MoE Is All You Need for Process Supervision
Oct 21, 2026Guanzhi Deng, Zengyan Liu, Haibo Wang +5Process Supervision Process Reward Models
City University of Hong Kong · Carnegie Mellon University · University of Hong Kong
PAR: Progress-Aligned Recurrence for Length Generalization in State Tracking
Oct 20, 2026Pan Wang, Yihao Hu, Guangyuan Dong +8Length Generalization Recurrent Language Models
Ant International · National University of Singapore
C3: Rethinking Process Supervision via Confidence-Guided Credit Correction in Multi-Turn Reinforcement Learning
Oct 14, 2026Ai Jian, Haoran Pan, Eryu Guo +5Multi-Turn RL Reinforcement Learning
Beijing University of Posts and Telecommunications · Peking University · Institute of Automation, Chinese Academy of Sciences +2
LeanPRM: Lean-Grounded Process Supervision for Natural Language Proof Verification
Oct 13, 2026Rodney Lafuente-Mercado, Salman Rahman, Jingyan Shen +3Process Supervision Process Reward Models
University of Texas at Austin · University of California, Los Angeles · New York University
Proactive Agentic Orchestration
Oct 9, 2026Xueyang Feng, Chaozhuo Li, Xiaohe Bo +4LLM Agent Orchestration LLM Agent Workflow Optimization
Gaoling School of Artificial Intelligence, Renmin University of China · Beijing Academy of Artificial Intelligence · The Chinese University of Hong Kong Shenzhen
Learning to Act with Task Progress: Distilling Small Agents from Compact Teacher Supervision
Oct 7, 2026Wenxi GanLearning from Demonstration Agent Skill Learning
City University of Hong Kong Shenzhen Loop Area Institute
Collaborative Reasoning Distillation via Cross-Feedback and Coherent Curation
Oct 7, 2026Taehoon Kim, Seunggeun Cho, Dongsu HanReasoning Distillation LLM Reasoning
KAIST
HaPRL: Human-Anchored Process Reinforcement Learning for Visual Search Agent
Sep 29, 2026Zhangquan Chen, Yaoxin Niu, Xiang An +5Human-in-the-Loop Annotation Process Supervision
Tsinghua University · Peng Cheng Laboratory · LMMs-Lab +2
Rewarding Novel Deductions: Solver-guided Process Supervision for Logical Reasoning
Sep 28, 2026Muhammad Asif Ali, Wenqing Wang, Huan Wang +1RL for Language Model Reasoning Process Supervision
FORTE Lab, Faculty of Science, Information Technology University, Lahore, Pakistan · College of Informatics, Huazhong Agricultural University, Wuhan, China · Qatar Computing Research Institute, Hamad Bin Khalifa University, Doha, Qatar
Reinforcement Learning from Intermediate Renders for Image-to-Code Generation
Sep 28, 2026Omri Kaduri, Kate Feingold, Phillip Isola +1RL for Code Generation Image-to-Code Generation
Weizmann Institute of Science · MIT
Combining Hierarchical Cognitive Process with Process Supervision for Interpretable Scene Safety Understanding
Sep 22, 2026Zhiyun Jiang, Hanyong Wang, Binbin Liang +4LLM Interpretability Process Supervision
FLARE: A Full-Lifecycle Dense Supervision Paradigm for Long-Horizon Coding Agents via Generative Reward Model
Sep 20, 2026Jingxuan Xu, Gang Wu, Yanan Wu +13RL for Code Generation Reward Modeling
Independent Researcher · Nanjing University · Peking University +1
ProcessLight: Process Supervision for Large Language Model Based Traffic Signal Control
Sep 19, 2026Huaitao Zhao, Tianlong Zhou, Weijie Wang +2Language Model-Based Control Traffic Signal Control
School of Computer Science and Technology, Tongji University, Shanghai, China
The Imitation Game: When LLMs Learn to Reason Like Programs via Code-Centric Reasoning Data Synthesis
Sep 13, 2026Jinyang Zhang, Weibin Liao, Keqin Bao +10Synthetic Data Generation for Language Models RL for Language Model Reasoning
School of Computer Science, Peking University · Qwen Team, Alibaba Group · EPIC Lab, SJTU +1
Debate-to-Skill: Capability-Bound Process Supervision for Industrial Query-to-Agent Annotation
Sep 10, 2026Shiyu Zhang, Leisheng Cheng, Huifu LiLLM-Assisted Annotation Process Supervision
Baidu, Inc.
Structural Process Supervision for Latent Chain-of-Thought Reasoning
Sep 9, 2026Yiqi Li, Xu Chen, Chen Ju +5Representation Learning Process Supervision
Shanghai Jiao Tong University · Taobao & Tmall Group
IPM-FM: A Foundation Model with Consensus Feature Selection for Industrial Process Monitoring
Sep 8, 2026Liang Cao, Weide Liu, Yan Qin +3Predictive Process Monitoring Process Supervision
Department of Chemical and Biological Engineering University of British Columbia Vancouver, BC, Canada · School of Computing and Artificial Intelligence Jiangxi University of Finance and Economics Nanchang, China · School of Automation Chongqing University Chongqing, China +2
SiLR: Structure-Preserving Admission and Process Reward for LLM Tool Agents
Sep 4, 2026Chenyu Zhou, Qiliang Jiang, Shuning Wu +1Constrained RL Runtime Enforcement for AI Agents
School of Engineering, Institute of Science Tokyo, Japan · College of Control Science and Engineering, Zhejiang University, China · Department of Electrical and Computer Engineering, National University of Singapore, Singapore
Cliff: Learning Process Rewards from the First Mistake
Sep 2, 2026Peixuan Han, Runhui Wang, Ketan Ramaneti +3Reinforcement Learning with Verifiable Rewards LLM Reasoning
1Amazon Web Services · University of Illinois Urbana-Champaign
Monitoring Web Agents Without Internal Signals: Observable Trajectories and Key-Step Supervision
Sep 2, 2026Sitong Pan, Yipeng Shen, Yilin Lu +3AI Agent Monitoring Process Supervision
University of Minnesota · 2Purdue University · 3The Pennsylvania State University
Dense Process Supervision for Search Agents via Fact Utility Estimation
Sep 1, 2026Rongzhi Zhu, Xiangyu Liu, Yi Liu +7Reinforcement Learning Credit Assignment in RL
State Key Laboratory for Novel Software Technology, Nanjing University, China · Ant Group, China · National Institute of Healthcare Data Science, Nanjing University, China
PRO-Step: Step-level Process Reward Optimization for Retrieval-Augmented Generation
Aug 31, 2026MinKeon Kim, Namjun Lee, Jaekwang KimRetrieval-Augmented Generation Retrieval-Augmented Reasoning
Department of Applied Artificial Intelligence, Convergence Program for Social Innovation, Sungkyunkwan University, Seoul, South Korea
From Final Artifacts to Trajectories: Retrospective Process Supervision for Evidence-Grounded Long-Form Generation
Aug 31, 2026Junjie Huang, Jiarui Qin, Di Yin +4LLM Agent Trajectory Synthesis Process Supervision
Shanghai Jiao Tong University · Tencent Youtu Lab
Function-Level Execution Feedback for Code Preference Optimization
Aug 23, 2026Idris Nechnech, Sehwan Kim, Jimin Seo +4Code Generation Preference Optimization
Department of Electrical and Computer Engineering, Seoul National University · Department of Computer Science and Engineering, Konkuk University
CoEvo: Oracle-Grounded Self-Evolution of a Single Model for Multi-Step Causal Reasoning
Aug 18, 2026Jian Zhang, Bingyi Wang, Yizhi LiuCausal Reasoning in Language Models Causal Reasoning
Zhejiang University
CEDAR-GRPO: Process-Aware Reinforcement Learning for General Abductive Reasoning in LLMs
Aug 14, 2026Moein Salimi, Danial Parnian, Shaygan Adim +6Abductive Reasoning in Language Models Abductive Reasoning
Sharif University of Technology · University of Tehran
Advantage-Guided Gate: Reshaping Open-Ended Reasoning for Vision-Based Spatial Intelligence
Aug 8, 2026Ling Lin, Yang Bai, Congcong Zhu +6Visual Spatial Reasoning Inference-Time Search
School of Artificial Intelligence and Data Science, University of Science and Technology of China, Hefei 230000, China · Suzhou Institute for Advanced Research, USTC, Suzhou 215123, China · Institute of Advanced Intelligence and Computing (IAIC), Agency for Science, Technology and Research (A*STAR), Singapore +4
CausalOPD: First-Wrong-Step Supervision for Distilling Causal Chain Reasoning
Aug 4, 2026Jian Zhang, Bingyi Wang, Yizhi LiuCurriculum RL Causal Reasoning in Language Models
Zhejiang University
PAMT: Process-Aligned Reinforcement Learning for Multi-Domain Machine Translation
Aug 4, 2026Yongshi Ye, Biao Fu, Chongxuan Huang +2RL for Language Model Reasoning Process Supervision
Institute of Artificial Intelligence, Xiamen University · Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan (Xiamen University), Ministry of Culture and Tourism · School of Informatics, Xiamen University

Oct 23, 2026J/K move · Enter open · S save

AdaPRA: Process Reward Agents with Adaptive Rubrics for Data Science

Atharva Naik, Vivek Chari, Pol Llado, Daniel Fried+4

Carnegie Mellon University · Microsoft

Data Analysis Agents Process Reward Models Process Supervision

Open paper AskPDF arXiv

Data science agents can automate end-to-end machine learning workflows, yet they remain far from competitive on human leaderboards. Agents make process errors such as insufficient exploration, unsound validation, coding bugs, and poor use of a fixed budget (re-running the same exploratory data analysis and baseline models in every attempt). We propose AdaPRA to address these issues: a persistent coding agent harness that accumulates knowledge across attempts on the same task, with rubric-guided process supervision that identifies and corrects errors. Our process reward agent audits the agent's trajectory and workspace by executing checks, and adapts its rubrics to each task. We find that rubrics seeded from easier tasks generalize to harder tasks, with rubric adaptation further improving performance. With five attempts per task, AdaPRA improves an 80B Qwen model's leaderboard percentile by 24.5 points on DSPredict-Easy and 14.8 points on DSPredict-Hard while reducing process errors. We also show that supervision with rubrics mined from one base model's runs improves another (DeepSeek-V3.2), and that AdaPRA beats DataPRM, a competitive PRM for data science. We also share our code for reproducibility.