External Reward Models

Follow this topic Ask about this topic

Recent momentum

-80%

3 papers in the last 28 days · 0.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-14

1 new paper

A weekly snapshot of new work published in External Reward Models.

Period ending 2026-09-07

5 new papers

A weekly snapshot of new work published in External Reward Models.

96 papers

Latest in External Reward Models

Open your feed →

Proof-Carrying Cognition: Closing the Verification Gap with Reality-Settled Reward
Sep 9, 2026Eshwar Reddy M, Sourav KarmakarVerifiable Rewards Probabilistic Verification
Dense Process Supervision for Search Agents via Fact Utility Estimation
Sep 1, 2026Rongzhi Zhu, Xiangyu Liu, Yi Liu +7Process Reward Modeling External Reward Models
Mitigating Over-Optimization in PRM-Guided Search in Mathematical Reasoning by Optimizing the Guide
Aug 30, 2026Taejong Joo, Diego KlabjanProcess Reward Modeling Mathematical Reasoning
Se-DPO: Self-Evolving Token Credit for Direct Preference Optimization
Aug 10, 2026Wenxiao Zhao, Shu Wang, Ying Nian WuTopic-Disjoint Direct Preference Optimization Data Token-Level Uncertainty
TrustRoboReward: Preference-Ordered Isotonic Score Editing for Multi-Paradigm Robot Reward Models
Aug 9, 2026Yidong Wang, Yan Zhan, Ziteng Feng +16External Reward Models Computer Vision
Quality-Diversity Stress Tests for Process Reward Models:What Archive Coverage Can and Cannot Certify
Aug 8, 2026Ibne Farabi Shihab, Fariya AfrinProcess Reward Modeling External Reward Models
Why Study Emergent Behavior When You Can Regulate It? Aligning Multi-Agent Systems with Reward Prediction
Aug 7, 2026Assaf Caftory, Almog Zemach, Moshe Butman +1Multi-Agent Reinforcement Learning Simulation-Based Reinforcement Learning
RRC: Unlocking Generative Reward Models in LLM Reinforcement Learning via Ranking-Based Reward Construction
Aug 6, 2026Chenglong Wang, Ziming Zhu, Yifu Huo +9External Reward Models Goal-Conditioned Reinforcement Learning
PAMT: Process-Aligned Reinforcement Learning for Multi-Domain Machine Translation
Aug 4, 2026Yongshi Ye, Biao Fu, Chongxuan Huang +2Machine Translation Process Reward Modeling
Rubrics as Privileged Information for Open-Ended Generation
Aug 3, 2026Deepika Bablani, Ajay Gupta, Wanming ChenRubric-Based Reinforcement Learning Rubrics
Rewriting or Reweighting? A Geometric Account in Language Models
Aug 3, 2026Juntong Wang, Shengkun Yang, Xiyuan Wang +1Model Activations External Reward Models
DocPO: Advancing Document Policy Optimization via Tailored Step-Aware Rewards
Aug 1, 2026Yunhao Wang, Binghong Wu, Zhenyu Huang +4External Reward Models Relative Policy Optimization
Beyond Routing Weights: Faithful Response-Level Interpretation of Mixture-of-Experts Reward Models via Contribution Contrast
Jul 31, 2026Yifan Wang, Jinyi Mu, Mayank Jobanputra +4Cold Mixture-Of-Expert Models External Reward Models
OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models
Jul 30, 2026Qiushi Sun, Kanzhi Cheng, Yian Wang +20External Reward Models Large Language Model Judges
SERPO: Self-Evolving Rubric Policy Optimization for Open-Ended Test-Time Reinforcement Learning
Jul 29, 2026Jianze Wang, Kunwang Zheng, Ying Liu +5External Reward Models Open-Ended Responses
Optimal Reward Shaping: Autonomous Car Parking Case Study
Jul 26, 2026Emre Özkaya, Nicolas R. GaugerAutonomous Parking Deep Q-Networks
Oxygen-TryOn: Fashion-Native Foundation Model for Any-item Virtual Try-On
Jul 23, 2026Yong Liu, Xiaolong Fu, Zihang Xu +10Virtual Try-On Garments
Progress Reward Modeling for Robotic Learning: A Comprehensive Survey
Jul 22, 2026Jianshu Zhang, Keliang Wu, Haoran Lu +8Scalable Robot Learning External Reward Models
LenGuard-GPC: Length Guarding with Guided-Prompt Consistency for Spatial Reasoning Reinforce Learning
Jul 19, 2026Xingjian Tao, Yiwei Wang, Yujun Cai +1Spatial Reasoning External Reward Models
Lyapunov Exponent as Physics-Informed Dense Reward: RL Discovery of Stabilization Beyond the Kapitza Pendulum
Jul 15, 2026Slava AndrejevLyapunov Function External Reward Models
Long-term User Engagement Optimization through Model-agnostic Downstream Rewards Learning
Jul 15, 2026Dingsu Wang, Filip Ryzner, Kelly He +12Recommender Systems External Reward Models
Read It Back: Pretrained MLLMs Are Zero-Shot Reward Models for Text-to-Image Generation
Jul 13, 2026Runhui Huang, Qihui Zhang, Zhe Liu +3Text-To-Image External Reward Models
Reason, Reward, Refine: Step-Level Errors Corrections with Structured Feedback for Physics Reasoning in Small Language Models
Jul 6, 2026Raj Jaiswal, Dhruv Jain, Rishabh Dhawan +4Physics Problem External Reward Models
TimeThink: Reasoning with Time for Video LLMs
Jul 6, 2026Handong Li, Longteng Guo, Zikang Liu +8External Reward Models
Unbiased Alignment for Large Language Models with Noisy Preferences
Jul 3, 2026Jialiang Wang, Xianming Liu, Xiong Zhou +2Topic-Disjoint Direct Preference Optimization Data Reinforcement Learning From Human Feedback
PAPA: Online Personalized Active Preference Alignment
Jul 1, 2026Anindya Sarkar, Nasik Muhammad Nafi, Isaac Lyngaas +2Preference Alignment Parameter-Efficient Fine-Tuning Methods
VLM-AR3L: Vision-Language Models for Absolute and Relative Rewards in Reinforcement Learning
Jul 1, 2026Kuan-Chen Chen, Winston Chen, Wei-Fang Sun +1Reward Functions External Reward Models
TempAct: Advancing Temporal Plausibility in Autoregressive Video Generation via Planner-Executor RL
Jun 26, 2026Jing Wang, Xiangxin Zhou, Jiajun Liang +5Autoregressive Video Generation Autoregressive Diffusion
Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards
Jun 25, 2026Ritesh Thawkar, Shravan Venkatraman, Omkar Thawakar +5Multimodal Generation Large Multimodal Models
CFPO: Counterfactual Policy Optimization for Multimodal Reasoning
Jun 22, 2026Zhangyuan Yu, Wanran Sun, Guangjing Yang +2Multimodal Reasoning External Reward Models
RARM: Confidence-Gated Progress Reward Modeling for RL in Manipulation
Jun 20, 2026Pengzhi Yang, Xinyu Wang, Pengyu Jing +7External Reward Models
ARCO: Adaptive Rubric with Co-Evolution for Multi-Step LLM-Based Agents
Jun 19, 2026Zihang Tian, Jingsen Zhang, Rui Li +3Rubric-Based Reinforcement Learning Rubric Framework
Be Your Own Teacher: Steering Protein Language Models via Unsupervised Reward Optimization
Jun 17, 2026Lanqing Li, Shentong Mo, Yang Yu +1Protein Design External Reward Models
Steerable Cultural Preference Optimization of Reward Models
Jun 17, 2026Minsik Oh, Advit Deepak, Sophie Wu +2Large Language Model Alignment Robust Preference Optimization
TuneJury: An Open Metric for Improving Music Generation Preference Alignment
Jun 15, 2026Yonghyun Kim, Junwon Lee, Haiwen Xia +5Generative Music Models Preference Alignment
The Quality-Utility Paradox: Why High-Reward Data Impairs Small Model Mathematical Reasoning
Jun 15, 2026Haolong Qian, Xianliang Yang, Yinuo ma +6Large Reasoning Models External Reward Models
Closing the Reflection Gap: A Free Calibration Bonus for Agentic RL
Jun 12, 2026Yinglun ZhuAgentic Reinforcement Learning Reflections
PersonaDrive: Human-Style Retrieval-Augmented VLA Agents for Closed-Loop Driving Simulation
Jun 10, 2026Mahmoud Srewa, Praneetsai Iddamsetty, Mohammad Abdullah Al Faruque +1Drives Naturalistic Driving Data
Z-Reward: Beyond Scalar Rewards by Internalizing Reasoning into Score Distributions
Jun 8, 2026Xin Jin, Huanqia Cai, Zhen Li +9External Reward Models Reward Signal
A Unifying Lens on Reward Uncertainty in RLHF
Jun 8, 2026Ely Hahami, Yoel Zimmermann, Ray Zhou +1Reinforcement Learning From Human Feedback External Reward Models
DriveReward: A Comprehensive Dataset and Generative Vision-Language Reward Model for Autonomous Driving
Jun 7, 2026Qimao Chen, Fang Li, Yuechen Luo +11Naturalistic Driving Data Autonomous Driving
CATPO: Critique-Augmented Tree Policy Optimization
Jun 6, 2026Ayush Singh, Umang Goyal, Ankur DahiyaVerifiable Rewards External Reward Models
BiasGRPO: Stabilizing Bias Mitigation in High-Variance Reward Landscapes via Group-Relative Policy Optimization
Jun 3, 2026Saket Reddy, Ke Yang, ChengXiang ZhaiLarge Language Model Alignment Reinforcement Learning From Human Feedback
Skill-RM: Unifying Heterogeneous Evaluation Criteria via Agent Skill
Jun 2, 2026Tao Chen, Gangwei Jiang, Pengyu Cheng +10External Reward Models
HARVE: Hacking-Aware Reward-Head Vector Editing for Robust Reward Models
Jun 2, 2026Shuang Liu, Yuxuan Bo, Qiuyang Zhao +4External Reward Models Vectors
EST-PRM: Stress-Testing Process Reward Models Before They Become Load-Bearing
May 30, 2026Ibne Farabi Shihab, Fariya Afrin, Sanjeda Akter +1Process Reward Modeling External Reward Models
Consolidating Rewarded Perturbations for LLM Post-Training
May 29, 2026Zheyu Zhang, Shuo Yang, Gjergji KasneciLarge Language Model Post-Training Reinforcement Learning Post-Training
Federated Variational Preference Alignment with Gumbel-Softmax Prior for Personalized User Preferences
May 29, 2026Jabin Koo, Hoyoung Kim, Minwoo Jang +1Preference Alignment Pairwise Preferences
Pairwise Reference Alignment as a Model-Level Ordinal Observable
May 29, 2026Mujing LiPreference Pairs Feature Alignment
Configurable Reward Model for Balanced Safety Alignment
May 28, 2026Zhengping Jiang, Mehran Khodabandeh, Akash Bharadwaj +5Safety Alignment Large Language Model Alignment
Beyond Trajectory Rewards: Step-level Credit Assignment for Agentic Search via Graph Modeling
May 28, 2026Yuchen Liu, Yingjie Feng, Lixiong Qin +5Trajectory-Level Credit Assignment External Reward Models
RUBRIC-ARROW: Alternating Pointwise Rubric Reward Modeling for LLM Post-training in Non-verifiable Domains
May 27, 2026Haoxiang Jiang, Zihan Dong, Tianci Liu +5Rubric-Based Reinforcement Learning Rubrics
AdaDPO: Self-Adaptive Direct Preference Optimization with Balanced Gradient Updates
May 27, 2026Shaolong Chen, Madalina Ciobanu, Qingqing Mao +1Topic-Disjoint Direct Preference Optimization Data Prior Prompt Optimization Techniques
InterSketch: An Interleaved Reasoning Model with Self-correcting Visual Sketch and Stepwise Reward
May 26, 2026Zhiwei Ning, Wenwen Tong, Xiangli Kong +12Visual Reasoning Sketches
Trait-Aware Policy Optimization for Autoregressive Multi-Trait Essay Scoring
May 25, 2026Zhengyang Wang, Sanwoo Lee, Jiaxin Wang +3Automated Essay Scoring External Reward Models
Token-weighted Direct Preference Optimization with Attention
May 21, 2026Chengyu Huang, Zhuohang Li, Sheng-Yen Chou +1Topic-Disjoint Direct Preference Optimization Data Large Language Model Post-Training
Distribution-Aware Reward: Reinforcement Learning over Predictive Distributions for LLM Regression
May 20, 2026Jungsoo Park, Hyungjoo Chae, Ethan Mendes +4External Reward Models Regression
Introspective X Training: Feedback Conditioning Improves Scaling Across all LLM Training Stages
May 19, 2026Brandon Cui, Ximing Lu, Jaehun Jung +7Large Language Model Pipelines External Reward Models
SafeDiffusion-R1: Online Reward Steering for Safe Diffusion Post-Training
May 18, 2026Komal Kumar, Ankan Deria, Abhishek Basu +3Text-To-Image Diffusion Models Generative Quality
Edit-Compass & EditReward-Compass: A Unified Benchmark for Image Editing and Reward Modeling
May 13, 2026Xuehai Bai, Yang Shi, Yi-Fan Zhang +7Image Editing Evaluation Benchmarks

Sep 9, 2026 · cs.AIJ/K move · Enter open · S save

Proof-Carrying Cognition: Closing the Verification Gap with Reality-Settled Reward

Eshwar Reddy M, Sourav Karmakar

Frontier gains in language-model reasoning come from reinforcement learning on reasoning traces and are concentrated in domains with a cheap, sound verifier. We argue the field's binding constraint is the verification gap: no scalable, incorruptible reward for reasoning outside formal domains. We make four contributions. (1) Theory: in a joint-Gaussian model of best-of-N selection, verifier-gold correlation rho is the exact exchange rate between test-time compute and capability, and an unsound verifier pays a polynomial penalty N^(1/rho^2); a margin-free copula form predicts realized soundness of real LLM judges to 4% median error. (2) Demonstration: in program-synthesis testbeds with executable ground truth, including a pre-registered scaled replication, unsound verifiers lose Soundness-under-Pressure as optimization grows (0.94 to 0.32 at N=4096) while a sound verifier improves monotonically; reality-anchored settlement beats a frozen verifier under i.i.d. and adversarial pressure, driving the hacking gap from ~0.27 to ~0; soundness scales log-linearly with settled labels, with on-policy settlement ~10x more label-efficient than random labeling. With real LLM judges and unit-test execution as gold, a weak judge loses soundness under best-of-N (p<0.001), a stronger judge is more robust, and selection alone manufactures +0.53 hacking gaps from honest samples. Under real GRPO training, a frozen reward model traces the full overoptimization curve (executed reward collapses 90%) while the same model refit on a 10% settlement stream preserves 6x the executed reward. (3) Paradigm: proof-carrying cognition, where reasoning steps are typed probabilistic claims priced by a self-built world model trained only on held-out reality and settled by proper scoring rules. (4) Benchmark: we specify Soundness-under-Pressure as the headline metric for a reality-settled reasoning benchmark.

Verifiable Rewards Probabilistic Verification Reasoning Benchmark Verification External Reward Models Cognitive Science Gaussian Copula

Open paper Ask about it PDF