Reward Modeling

Follow this topic Ask about this topic

Momentum

24 papers in the last four weeks, up 118% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 197

All topics

Claim-Level Rubric Rewards for Video Caption Reinforcement Learning
Jul 6, 2026Mingqi Gao, Hongyuan Dong, Yifei Chen +6Reward Modeling Video Captioning
1Tsinghua Shenzhen International Graduate School, Tsinghua University · 3LLM Department, Tencent · University of Chinese Academy of Sciences
Attention Limited Reward Learning
Jul 6, 2026Wenqian XingPairwise Preference Learning Reward Modeling
Management Science and Engineering Department, Stanford University
Reward Lightning: Fast Video Generation via Homologous Preference Distillation
Jul 4, 2026Jiaxiang Cheng, Bing Ma, Xuhua Ren +4Diffusion Model Alignment Reward Modeling
Tencent Hunyuan, China
Rank-Then-Act: Reward-Free Control from Frame-Order Progress
Jul 2, 2026Yuriy Maksyuta, George Bredis, Ruslan Rakhimov +1RL Control Reward Modeling
T-Tech
Many Voices, One Reward: Multi-Role Rubric Generation for LLM Judging and Reward Modeling
Jul 2, 2026Dazhi Fu, Jiuding Yang, Yiwen Guo +1Reward Modeling LLM-as-a-Judge
School of Data Science, The Chinese University of Hong Kong, Shenzhen, China · LIGHTSPEED · Independent Researcher
Arena-T2I Hard: Benchmarking and Improving Faithfulness with Dependency-Aware Checklist
Jun 30, 2026Yuanhao Ban, Tong Xie, Sohyun An +6Reward Modeling Compositional T2I Generation
1Arena Intelligence Inc · 2UCLA
Test-Time Verification for Text-to-SQL via Outcome Reward Models
Jun 29, 2026Mattia Tritto, Giuseppe Farano, Dario Di Palma +4Reward Modeling LLM Answer Verification
Polytechnic University of Bari, Bari, Italy · IBM T.J. Watson Research Center, Yorktown Heights, NY, USA
Pessimism's Paradox: Conservative Offline Training Amplifies Reward Hacking During Online Adaptation in Reasoning Models
Jun 29, 2026Subramanyam Sahoo, Aman Chadha, Vinija Jain +1Reward Modeling Reward Hacking
1Horizon Research · 2Apple · 3Meta +1
Your Data Manifold is Secretly a Reward Model: Shell-LCC for Text-to-Video Generation
Jun 29, 2026Shihao Zhang, Yunzhi Li, Yuguang Yan +4Reward Modeling Diffusion Model Guidance
Huawei Central Research Institute · Guangdong University of Technology
To Reason or to Fabricate: Reasoning Without Shortcuts via Hint-Anchored Pairwise Aggregation
Jun 28, 2026Jiuheng Lin, Chen Zhang, Yansong FengCoT Faithfulness Reward Modeling
Wangxuan Institute of Computer Technology, Peking University
HPRO: Hierarchical Progressive Reward Optimization via Preference Extraction for Emotional Text-to-Speech
Jun 26, 2026Sihang Nie, Xiaofen Xing, Rui Xing +5Reward Modeling Speech Prosody
South China University of Technology, China · Huya Inc., China · Tongyi Fun Team, Alibaba Group, China +1
Qwen-Image-2.0-RL Technical Report
Jun 25, 2026Yixian Xu, Kaiyuan Gao, Yuxiang Chen +25Reward Modeling RL for Diffusion Models
Equal contribution. · Corresponding author.
PEBS: Per-rater Empirical-Bayes Shrinkage for RLHF Reward-Model Calibration
Jun 25, 2026Arnav RajReward Modeling Post-Hoc Calibration
Department of Computer Science and Engineering, Indian Institute of Technology Delhi, New Delhi, India.
SpatialFlow-GRPO: Where Spatial Credit Drives Image Editing
Jun 25, 2026Yankai Yang, Yancheng Long, Wei Chen +7Reward Modeling Group Relative Policy Optimization
Harbin Institute of Technology, Shenzhen · Kuaishou Technology
The Verification Horizon: No Silver Bullet for Coding Agent Rewards
Jun 24, 2026Binghai Wang, Chenlong Zhang, Dayiheng Liu +10Reward Modeling Reward Hacking
Qwen Team
MAPL: Multi-Objective Preference Learning for Robot Locomotion
Jun 24, 2026Xiyue Chen, Muhan Lin, Shuyang Shi +1Multi-Objective Reinforcement Learning Reward Modeling
Department of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47906, USA · Department of Computer Science, Purdue University, West Lafayette, IN 47906, USA · independent researcher
Themis: An explainable AI-enabled framework for Reinforcement Learning with Human Feedback
Jun 23, 2026Andreas Chouliaras, Luke Connolly, Dimitris ChatzpoulosReward Modeling Human-in-the-Loop AI
School of Computer Science, University College Dublin, Belfield, Dublin 4, Ireland
Discretizing Reward Models
Jun 19, 2026Vijay Viswanathan, Shiqi Wang, Devamanyu Hazarika +4Reward Modeling Reinforcement Learning
1Carnegie Mellon University · 2Meta Superintelligence Labs
PrivacyAlign: Contextual Privacy Alignment for LLM Agents
Jun 19, 2026Manveer Singh Tamber, Abhay Puri, Marc-Etienne Brunet +3Reward Modeling Privacy Leakage in Language Models
University of Waterloo · †Work done during internship at ServiceNow AI Research · 2ServiceNow AI Research +2
Your Mouse and Eyes Secretly Leak Your Preference: LLM Alignment using Implicit Feedback from Users
Jun 18, 2026Haw-Shiuan Chang, Jeffrey Gomez, Mehul Patwari +2Reward Modeling LLM Alignment
University of Massachusetts, Amherst, USA · York University, Canada
Reward as An Agent for Embodied World Models
Jun 18, 2026Pu Li, Zhigang Lin, Qiang Wu +3Reward Modeling Reward Hacking
Kairos Team
Uncertainty-Aware Reward Modeling for Stable RLHF
Jun 18, 2026Licheng Pan, Haocheng Yang, Haoxuan Li +7Reward Modeling Reward Hacking
Zhejiang University · Xiaohongshu Inc · National University of Singapore +1
Learning User Simulators with Turing Rewards
Jun 17, 2026Yingshan Susan Wang, Cedegao E. Zhang, Linlu Qiu +5Reward Modeling RL for Language Models
Massachusetts Institute of Technology · Stanford University · MIT-IBM Watson AI Lab
Online Reward-Punishment Learning from Fixed-Channel Perceptual Event Streams without Environment Rewards
Jun 17, 2026Zirong LiReward Modeling Reinforcement Learning
Tiangong University
Steerable Cultural Preference Optimization of Reward Models
Jun 17, 2026Minsik Oh, Advit Deepak, Sophie Wu +2Reward Modeling Cultural Bias in Language Models
1Stanford University · University of Amsterdam
TuneJury: An Open Metric for Improving Music Generation Preference Alignment
Jun 15, 2026Yonghyun Kim, Junwon Lee, Haiwen Xia +5Text-to-Music Generation Music Generation Evaluation
Georgia Tech · KAIST · Peking University +3
Graph Structured Combinatorial Semi-Bandit with Nonlinear Reward Associations through Separable Signals
Jun 12, 2026Christoph Bauschmann, Setareh MaghsudiReward Modeling Multi-Armed Bandits
Department of Electrical Engineering and Information Technology Ruhr University Bochum Bochum, Germany
Reward Modeling for Multi-Agent Orchestration
Jun 11, 2026King Yeung Tsang, Zihao Zhao, Vishal Venkataramani +5Reward Modeling Multi-Agent Orchestration
Rutgers University · Salesforce AI Research
Understanding helpfulness and harmless tension in reward models
Jun 11, 2026Eshaan Tanwar, Pepa AtanasovaReward Modeling LLM Alignment
University of Copenhagen
Plan-and-Verify Video Reward Reasoning with Spatio-Temporal Scene Graph Grounding
Jun 10, 2026Hyomin Kim, Junghye Kim, Joanie Hayoun Chung +4Reward Modeling Text-to-Video Generation
Department of Artificial Intelligence, Korea University · Department of Statistics, Korea University
SARM2: Multi-Task Stage Aware Reward Modeling for Self Improving Robotic Manipulation
Jun 9, 2026Qianzhong Chen, Hau Zheng, Justin Yu +8RL for Robotics Reward Modeling
Stanford University · UC Berkeley · Shanghai Jiao Tong University
Reasoning Arena: Trace Tournaments When Verifiable Rewards Fall Short
Jun 8, 2026Han Zhou, Adam X. Yang, Laurence Aitchison +2Pairwise Preference Learning Reward Modeling
University of Cambridge · 2Mistral AI · *Work done as an AI Scientist Intern at Mistral AI.
Z-Reward: Beyond Scalar Rewards by Internalizing Reasoning into Score Distributions
Jun 8, 2026Xin Jin, Huanqia Cai, Zhen Li +9Reward Modeling Reasoning Distillation
Z-Image Team, Alibaba Group · VCIP, CS, Nankai University
A Unifying Lens on Reward Uncertainty in RLHF
Jun 8, 2026Ely Hahami, Yoel Zimmermann, Ray Zhou +1Reward Modeling KL-Regularized RL
Harvard College, Harvard University, Cambridge, MA, USA · John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA, USA
DynaCF: Mitigating Shortcut Learning in Reward Models via Dynamic Counterfactual Sensitivity
Jun 8, 2026Fengyuan Liu, Yongliang Miao, Zirui He +3Pairwise Preference Learning Reward Modeling
The Chinese University of Hong Kong, Shenzhen · New Jersey Institute of Technology · Institute of Computing Technology, CAS
Support Vector Rubrics: Closing the Gap Between Self-Generated and Human Rubrics
Jun 6, 2026Mengyuan Sun, Yu Li, Zhuohao Yu +2Reward Modeling Rubric-Based Evaluation
National Engineering Research Center for Software Engineering, Peking University · University of Science and Technology of China
PAFO: Pareto Fairness Optimization for Personalized Reward Modeling
Jun 6, 2026Xiaoyan Zhao, Haoting Ni, Yang Zhang +3Reward Modeling LLM Alignment
National University of Singapore · University of Science and Technology of China · Peking University
Beyond Rubrics: Exploration-Guided Evaluation Skills for Reward Modeling
Jun 5, 2026Xing Yue, Linjuan Wu, Daoxin Zhang +2Reward Modeling LLM Evaluation
Zhejiang University · Xiaohongshu Inc.
BiasGRPO: Stabilizing Bias Mitigation in High-Variance Reward Landscapes via Group-Relative Policy Optimization
Jun 3, 2026Saket Reddy, Ke Yang, ChengXiang ZhaiReward Modeling RL for Language Models
University of Illinois - Urbana-Champaign
Sparse Mixture-of-Experts Reward Models Learn Interpretable and Specialized Experts for Personalized Preference Modeling
Jun 2, 2026Yifan Wang, Jinyi Mu, Mayank Jobanputra +5Pairwise Preference Learning Reward Modeling
Saarland University · Independent Researcher · Bielefeld University +2
Skill-RM: Unifying Heterogeneous Evaluation Criteria via Agent Skill
Jun 2, 2026Tao Chen, Gangwei Jiang, Pengyu Cheng +10Reward Modeling Reward Model Evaluation
1Qwen Large Model Application Team, Alibaba · 2Sun Yat-sen University · 3The Chinese University of Hong Kong +3
Mitigating False Credit Propagation: Probabilistic Graphical Reward Aggregation for Rubric-Based Reinforcement Learning
Jun 2, 2026Can Lv, Mingju Chen, Heng Chang +1Reward Modeling RL for Language Models
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University · Tsinghua University
ARBOR: Online Process Rewards via a Reusable Rubric Buffer for Search Agents
Jun 2, 2026Zheng Liu, Longxiang Zhang, Xintong Wang +8Reward Modeling Process Reward Models
Tsinghua University · Alibaba Group · Peking University +1
Follow-Your-Preference++: Rethinking Preference Alignment for Image Inpainting
Jun 2, 2026Junkun Yuan, Yutao Shen, Toru Aonishi +2Pairwise Preference Learning Reward Modeling
Zhejiang University · The University of Tokyo · Tsinghua University
HARVE: Hacking-Aware Reward-Head Vector Editing for Robust Reward Models
Jun 2, 2026Shuang Liu, Yuxuan Bo, Qiuyang Zhao +4Reward Modeling Reward Hacking
Carnegie Mellon University · University of Virginia · Harvard University +4
Mitigating Perceptual Judgment Bias in Multimodal LLM-as-a-Judge via Perceptual Perturbation and Reward Modeling
Jun 1, 2026Seojeong Park, Jiho Choi, Junyong Kang +3Reward Modeling LLM-as-a-Judge
Graduate School of Artificial Intelligence, KAIST, Republic of Korea · KRAFTON
Expected Value Alignment for Generative Reward Modeling in Formal Mathematics Verification
May 31, 2026Shihao Ji, Haotao Tan, Zihui Song +1Reward Modeling Automated Theorem Proving
Position: Good Embodied Reward Models Need Bad Behavior Data
May 31, 2026Ran Tian, Yilin Wu, Andrea BajcsyReward Modeling Preference Alignment
UC Berkeley · NVIDIA Research · Carnegie Mellon University
From Empathy to Personalized Empathy: Adapting Empathetic Strategies to Individual Users
May 30, 2026Wuqiang Zheng, Chengbing Wang, Yilin Yang +6Reward Modeling LLM Personalization
University of Science and Technology of China · 2Huawei Technologies · 3China Academy of Cyber
Latent Reward Steering: An Adaptive Inference-Time Framework that Implicitly Promotes Cognitive Behaviors in Reasoning LLMs
May 30, 2026Jiakang Li, Guanyu Zhu, Can Jin +8Reward Modeling Language Model Steering
Rutgers University · South China Agricultural University · Columbia University +4
CARE-RL: Capability-Aware Reinforcement Learning for Mitigating Cross-Domain Conflicts
May 30, 2026Rui Zhang, Xinle Wu, Yao LuReward Modeling Reinforcement Learning
National University of Singapore Singapore
The Representation-Rationalizability Tradeoff in Reward Learning
May 29, 2026Jing Dong, Yaoliang Yu, Pascal PourpartPairwise Preference Learning Reward Modeling
Vector Institute · University of Waterloo & Vector Institute
The Flip Side of RLHF: On-Policy Feedback for Reward Model Self-Supervised Improvement
May 29, 2026Xiaobo Wang, Tong Wu, Min Tang +3Reward Modeling LLM Alignment
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China · State Key Laboratory of General Artificial Intelligence, BIGAI · University of Science and Technology of China +1
Reward Learning from Best-of- $N$ Preference Data: Targets, Tradeoffs, and Design Principles
May 28, 2026Rattana Pukdee, Maria-Florina Balcan, Pradeep RavikumarPairwise Preference Learning Reward Modeling
Machine Learning Department Carnegie Mellon University
Configurable Reward Model for Balanced Safety Alignment
May 28, 2026Zhengping Jiang, Mehran Khodabandeh, Akash Bharadwaj +5Reward Modeling Language Model Safety Evaluation
Johns Hopkins University · Meta · Meta Superintelligence Labs
Beyond Trajectory Rewards: Step-level Credit Assignment for Agentic Search via Graph Modeling
May 28, 2026Yuchen Liu, Yingjie Feng, Lixiong Qin +5Reward Modeling Agentic RL
Beijing University of Posts and Telecommunications · Li Auto Inc.
Prompt-Level Reward Specifications for Open-Ended Post-Training
May 28, 2026Zijun Weng, Xiaohui Hu, Shuangyong Song +3Open-Ended Generation Reward Modeling
1Fudan University · 2Xingchen AGI Lab, China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd.
RUBRIC-ARROW: Alternating Pointwise Rubric Reward Modeling for LLM Post-training in Non-verifiable Domains
May 27, 2026Haoxiang Jiang, Zihan Dong, Tianci Liu +5Reward Modeling RL for Language Models
University at Albany · 2Rutgers University · 3Purdue University +3
Refining Multidimensional Video Reward Models via Disentangled Influence Functions
May 27, 2026Muyao Wang, Zeke Xie, Hideki NakayamaReward Modeling Training Data Selection
The University of Tokyo · HKUST (Guangzhou)
StoryLens: Preference-Aligned Story Rewriting via Context-Aware Narrative Enrichment
May 27, 2026Hanwen Cui, Yuting Mei, Yuhang Fu +2Reward Modeling Personalized Text Generation
Beijing University of Posts and Telecommunications · AIM3 Lab, Renmin University of China · Nanyang Technological University

Jul 6, 2026, cs.CVJ/K move · Enter open · S save

Claim-Level Rubric Rewards for Video Caption Reinforcement Learning

Mingqi Gao, Hongyuan Dong, Yifei Chen, Zhisheng Zhong+5

1Tsinghua Shenzhen International Graduate School, Tsinghua University · 3LLM Department, Tencent · University of Chinese Academy of Sciences

Reward Modeling Video Captioning Rubric-Based RL Reinforcement Learning with Verifiable Rewards

Open paper AskPDF arXiv

In this paper, we introduce Claim-Level Rubric Rewards (CuRe), a structured reward framework designed to address the reward-design bottleneck in reinforcement learning for dense video captioning. Existing reward designs generally fall into two categories: holistic response-level judgment across heterogeneous criteria, or alignment-based evaluation against reference captions. However, both paradigms suffer from fundamental limitations. Holistic rewards struggle to ensure factual accuracy and are prone to stylistic reward hacking, while reference-based rewards overly rely on rigid textual alignment, failing to preserve the completeness and diversity inherent to open-ended generation tasks. To address these challenges, CuRe reformulates reward modeling as fine-grained claim-level verification. Specifically, CuRe decomposes captions into category-aware atomic claims through a structured rubric, converting holistic evaluation into simpler and more reliable claim-level verification.

Momentum

Related topics

Latest papers 197