Reward Modeling

Follow this topic Ask about this topic

Reward modeling turns comparisons, ratings, demonstrations, rubrics, or verifiable outcomes into signals for judging responses and actions. Methods range from pairwise preference models and scalar outcome or process critics to generative rubric judges, tool-based verifiers, and uncertainty-aware or personalized models. These signals guide preference optimization, best-of-N selection, test-time search, and training in language, image and video generation, robotics, and agents. A central challenge is that high preference accuracy may not capture diverse or shifting human intent, while optimization can exploit scoring biases, so evaluation must consider calibration and downstream behavior as well as ranking accuracy.

Latest papers

325 papers49 papers in the last four weeks, up 69% on the four weeks before. 0.3% of all new papers.

Beyond Verifiable Rewards: Rubric-Based GRM for Reinforced Fine-Tuning SWE Agents
Mar 13, 2026Jiawei Huang, Qingping Yang, Renjie Zheng +1Software Engineering Agents Reward Design for RL
ETH Zurich · ByteDance Seed
Trust Your Critic: Robust Reward Modeling and Reinforcement Learning for Faithful Image Editing and Generation
Mar 12, 2026Xiangyu Zhao, Peiyuan Zhang, Junming Lin +7Reward Modeling RL for Image Editing
Shanghai Jiao Tong University · Wuhan University · BUPT +3
Video-Based Reward Modeling for Computer-Use Agents
Mar 10, 2026Linxin Song, Jieyu Zhang, Huanxin Sheng +6Computer-Use Agent Evaluation Reward Modeling
University of Southern California · University of Washington · MBZUAI +1
Reward Prediction with Factorized World States
Mar 10, 2026Yijun Shen, Delong Chen, Xianming Hu +4Reward Modeling State Representation Learning
East China Normal University · HKUST
RubiCap: Rubric-Guided Reinforcement Learning for Dense Image Captioning
Mar 10, 2026Tzu-Heng Huang, Sirajul Salekin, Javier Movellan +2Image Captioning RL for VLMs
Work done during an internship at Apple · University of Wisconsin—Madison · Apple
CRED: Counterfactual Reasoning and Environment Design for Active Preference Learning
Mar 9, 2026Yi-Shiuan Tung, Gyanig Kumar, Wei Jiang +2Active Preference Learning Preference Learning
Department of Computer Science, University of Colorado Boulder
CDRRM: Contrast-Driven Rubric Generation for Reliable and Interpretable Reward Modeling
Mar 9, 2026Dengcan Liu, Fengkai Yang, Xiaohan Wang +7Preference Modeling Reward Modeling
University of Science and Technology of China · Peking University · Meituan +1
TDM-R1: Reinforcing Few-Step Diffusion Models with Non-Differentiable Reward
Mar 8, 2026Yihong Luo, Tianyang Hu, Weijian Luo +1RL for Diffusion Models Reinforcement Learning
Hong Kong University of Science and Technology · The Chinese University of Hong Kong, Shenzhen · hi-Lab, Xiaohongshu Inc +1
When Rubrics Fail: Error Enumeration as Reward in Reference-Free RL Post-Training for Virtual Try-On
Mar 5, 2026Wisdom Ikezogwo, Mehmet Saygin Seyfioglu, Ranjay Krishna +1Reward Modeling Reference-Free Evaluation
University of Washington, Seattle, USA · Amazon, Seattle, USA · Allen Institute for AI
VRM: Teaching Reward Models to Understand Authentic Human Preferences
Mar 5, 2026Biao Liu, Ning Xu, Junming Yang +2Reward Modeling LLM Alignment
Southeast University, Nanjing, China
Causally Robust Reward Learning from Reason-Augmented Preference Feedback
Mar 5, 2026Minjune Hwang, Yigit Korkmaz, Daniel Seita +1Reward Modeling Preference-Based RL
Thomas Lord Department of Computer Science, University of Southern California
A Rubric-Supervised Critic from Sparse Real-World Outcomes
Mar 4, 2026Xingyao Wang, Valerie Chen, Heng Ji +1Reward Modeling AI Agent Evaluation
OpenHands · UIUC · CMU
Expected Reward Prediction, with Applications to Model Routing
Mar 3, 2026Kenan Hasanaliyev, Silas Alberti, Jenny Hamer +5LLM Routing Reward Modeling
Stanford University · Inception Labs · Cognition Labs +2
Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisons
Mar 2, 2026Anthony Liang, Yigit Korkmaz, Jiahui Zhang +14Reward Modeling
Univ. of Southern California · UT Dallas · MIT +4
Fast-Slow Thinking RM: Efficient Integration of Scalar and Generative Reward Models
Mar 2, 2026Jiayun Wu, Peng Zhang, Yuanyuan Lu +3Reward Modeling Efficient Language Model Inference
Fudan University · Meituan
Beyond Length Scaling: Synergizing Breadth and Depth for Generative Reward Models
Mar 2, 2026Qiyuan Zhang, Yufei Wang, Tianhe Wu +5Reward Modeling Reward Model Evaluation
City University of Hong Kong · Tencent Hunyuan · MBZUAI
RLAR: An Agentic Reward System for Multi-task Reinforcement Learning on Large Language Models
Feb 28, 2026Andrew Zhuoer Feng, Cunxiang Wang, Bosi Wen +4RL for Language Models Reward Design for RL
Tsinghua Univeristy · Peking University
Pragma-VL: Towards a Pragmatic Arbitration of Safety and Helpfulness in MLLMs
Feb 28, 2026Ming Wen, Kun Yang, Xin Chen +4Multimodal Large Language Models LLM Safety Alignment
Fudan University · Shanghai Innovation Institute · Ant Group +3
Enhancing Spatial Understanding in Image Generation via Reward Modeling
Feb 27, 2026Zhenyu Tang, Chaoran Feng, Yufan Deng +5T2I Generation Reward Modeling
Peking University · ByteDance Seed
RewardUQ: A Unified Framework for Uncertainty-Aware Reward Models
Feb 27, 2026Daniel Yang, Samuel Stante, Florian Redhardt +5Reward Modeling Reward Model Evaluation
ETH Zurich · ETH AI Center
DesignSense: A Human Preference Dataset and Reward Modeling Framework for Graphic Layout Generation
Feb 26, 2026Varun Gopal, Rishabh Jain, Aradhya Mathur +6Human Preference Modeling Reward Modeling
MDSR, Adobe
TextPecker: Rewarding Structural Anomaly Quantification for Enhancing Visual Text Rendering
Feb 24, 2026Hanshen Zhu, Yuliang Liu, Xuecheng Wu +7Text Rendering in T2I Generation RL for Image Generation
Huazhong University of Science and Technology · ByteDance
CAMEL: Confidence-Gated Reflection for Reward Modeling
Feb 24, 2026Zirui Zhu, Hailun Xu, Yang Luo +4Reward Modeling LLM Alignment
National University of Singapore · TikTok
Janus-Q: End-to-End Event-Driven Trading via Hierarchical-Gated Reward Modeling
Feb 23, 2026Xiang Li, Zikai Wei, Yiyan Qi +6Algorithmic Trading Reward Modeling
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · International Digital Economy Academy, Shenzhen, China
MARS: Margin and Semantic-Aware Data Augmentation for Reward Modeling
Feb 19, 2026Payel Bhattacharjee, Osvaldo Simeone, Ravi TandonReward Modeling Synthetic Data Augmentation
University of Arizona Tucson, AZ, USA · Northeastern University London London, UK
Discovering Implicit Large Language Model Alignment Objectives
Feb 17, 2026Edward Chen, Sanmi Koyejo, Carlos GuestrinLLM Alignment LLM Interpretability
Stanford University.
Automatically Finding Reward Model Biases
Feb 16, 2026Atticus Wang, Iván Arcuschin, Arthur ConmyReward Modeling LLM Auditing
MATS · Massachusetts Institute of Technology · Independent
Interactionless Inverse Reinforcement Learning: A Data-Centric Framework for Durable Alignment
Feb 16, 2026Elias Malomgré, Pieter SimoensAI Alignment Reinforcement Learning
IDLab, Ghent University - imec, Ghent, Belgium
GRRM: Group Relative Reward Modeling for Machine Translation
Feb 15, 2026Sen Yang, Shanbo Cheng, Lu Xu +2Reward Modeling RL for Language Models
National Key Laboratory for Novel Software Technology, Nanjing University · Singapore University of Technology and Design
Small Reward Models via Backward Inference
Feb 14, 2026Yike Wang, Faeze Brahman, Shangbin Feng +3Reward Modeling RL for Language Models
University of Washington · Allen Institute for Artificial Intelligence

Mar 13, 2026J/K move · Enter open · S save

Beyond Verifiable Rewards: Rubric-Based GRM for Reinforced Fine-Tuning SWE Agents

Jiawei Huang, Qingping Yang, Renjie Zheng, Jiaze Chen

ETH Zurich · ByteDance Seed

Software Engineering Agents Reward Design for RL Reward Modeling RL Fine-Tuning Rubric-Based RL

Open paper AskPDF arXiv

Despite recent progress in Large Language Model (LLM) Agents for Software Engineering (SWE) tasks, end-to-end fine-tuning typically relies on verifiable terminal rewards such as whether all unit tests pass. While these binary signals reflect whether the final solution is correct, they provide little guidance for shaping intermediate behaviors during multi-step interactions, thereby limiting improvements in the overall quality of the resolution process. To address this, we introduce a rubric-based Generative Reward Model (GRM) that provides richer learning signals. The GRM is equipped with human-designed rubrics that indicate criteria for encouraging or discouraging specific behavioral patterns, and we leverage this feedback for high-quality training data collection via trajectory filtration. When used for Reinforced Fine-Tuning (RFT) on SWE Tasks, our approach outperforms terminal-score-only rejection sampling: it more effectively suppresses undesirable patterns while promoting beneficial ones, as confirmed by case analyses, and it ultimately improves final test accuracy.