Reward Modeling

Follow this topic Ask about this topic

Reward modeling turns comparisons, ratings, demonstrations, rubrics, or verifiable outcomes into signals for judging responses and actions. Methods range from pairwise preference models and scalar outcome or process critics to generative rubric judges, tool-based verifiers, and uncertainty-aware or personalized models. These signals guide preference optimization, best-of-N selection, test-time search, and training in language, image and video generation, robotics, and agents. A central challenge is that high preference accuracy may not capture diverse or shifting human intent, while optimization can exploit scoring biases, so evaluation must consider calibration and downstream behavior as well as ranking accuracy.

Latest papers

325 papers49 papers in the last four weeks, up 69% on the four weeks before. 0.3% of all new papers.

AndroidFlux: Evaluating and Reward Modeling Failure Recovery Capabilities in Mobile Use Multimodal Agents
Oct 25, 2026Gyubeum Lim, Yemo Koo, Ghunwoo Jang +1Multimodal Agent Evaluation Reward Model Evaluation
Georgia Institute of Technology · Konkuk University
CropReward: A Human-Preference Benchmark and Reward Model for Aesthetic Image Cropping
Oct 25, 2026Xiangfei Sheng, Xiaofeng Pan, Leida Li +4Reward Modeling Human Preference Modeling
School of Artificial Intelligence, Xidian University, Xi’an 710071, China · Huawei Technologies Co., Ltd., Shenzhen 518129, China · Laboratory of Experimental Psychology, Department of Brain and Cognition, KU Leuven, 3000 Leuven, Belgium +1
FLORA: An Open Model Suite for Scaling Robotics Data Engines
Oct 22, 2026Sicong Leng, Yunze Man, Zhiqi Li +14Robot Foundation Models Reward Modeling
Memory-Aware Reward Modeling: Learning Human Preferences for LLM Memory Use
Oct 22, 2026Yijiang River Dong, Zexue He, Cheng Yu +9Memory-Augmented Language Models Reward Modeling
University of Cambridge · Stanford University · Technical University of Munich +2
Scaling Laws for Generative Reward Models
Oct 22, 2026Angel Raychev, Nathan Lile, Rafael RafailovReward Modeling RL for Language Models
Stanford University · Thinking Machines
Differentially Private Sparse Reward Estimation with Preference Feedback
Oct 21, 2026Meng Ding, Mingxi Lei, Jie Zhang +2Reward Modeling RL from Human Feedback
Department of Computer Science, University of Massachusetts Boston, Boston, MA, USA · Department of Computer Science and Engineering, University at Buffalo, SUNY, Buffalo, NY, USA · ETH Zürich, Zürich, Switzerland +2
AdvFidelity: Unifying Preference Alignment and Few-Step Image Generation within Reinforcement Learning
Oct 21, 2026Sicong Kang, Fangao Zeng, Yuhan Li +6RL for Image Generation Diffusion-Based Image Generation
Taobao & Tmall group of Alibaba · Shanghai Jiao Tong University
PerSPL: Personalized Semi-supervised Preference Learning
Oct 21, 2026Xiaoan Xu, Tingzhou Wei, Juncheng Dong +3Personalized Reward Modeling User Preference Modeling
Duke University · George Washington University
CollabSciReM: Aligning Language Models for Scientific Conversations via Science-Aware Reward Decomposition
Oct 20, 2026Sasi Kiran Gaddipati, Gollam Rabby, Sören AuerLLM Alignment Reward Modeling
TIB—Leibniz Information Centre for Science and Technology, Hannover, Germany · L3S Research Center, Hannover, Germany
Judging the Justification: Decision Certificate Supervision for Generative Reward Models
Oct 20, 2026Guiquan Sun, Bing Bai, Martin Renqiang Min +1Reward Modeling Faithfulness of Language Model Explanations
School of Computing, University of Connecticut · NEC Labs America
CheckDistill: Adaptive Checklist Distillation for Efficient Distributional Reward Modeling in Image Editing
Oct 19, 2026Yaqi Cai, Zuan Gao, Mengting Chen +2Reward Modeling Multimodal Reward Modeling
Taobao & Tmall group of Alibaba
Steered Not Stirred: Effective Reward Tilting using Surrogate Reward Mixing
Oct 18, 2026Alec Sargood, Olivia Gallup, Georg Maierhofer +4Inference-Time Guidance Reward Modeling
InstaDeep · University College London · University of Oxford +3
Reason-guided Human Reward Modeling for Preference-based Reinforcement Learning
Oct 16, 2026Anshul Nayak, Shahil Shaik, Jonathan Smereka +1Preference-Based RL Reward Modeling
Clemson University, Clemson, SC 29631 USA · U.S. Army DEVCOM Ground Vehicle Systems Center, Warren, MI 48092 USA
Language Models Learn to Improve Language Models
Oct 16, 2026Minhua Lin, Zijun Wang, Zewen Liu +7Language Model Self-Improvement Large Language Model-Guided Optimization
The Pennsylvania State University · UC Santa Cruz · Emory University +2
Reward Engine: Reward-Centric Self-Evolution for Open-Ended Tool Agents
Oct 16, 2026Litong Feng, Ji Qi, Yiran Cheng +3Tool-Using Agents Self-Improving Agents
Huawei openJiuwen Team · Department of Statistics and Data Science, The Chinese University of Hong Kong (CUHK)
Compression Precedes, Content Decides: Emergent Misalignment in Reward Models
Oct 16, 2026Dhruv Puri, Nikhil PrakashReward Modeling Emergent Misalignment
Northeastern University
AVC-Reward: Event-Centric Audio-Video Consistency Evaluation
Oct 15, 2026Yuruo Wu, Kaihang Pan, Kai Shen +6Video Generation Evaluation Audio-Video Generation
Zhejiang University · HiThink Research · Fudan University +1
Blame Errant Words, Not the Canvas: Defect-Aware Spatial Credit Assignment for Visual Text Generation
Oct 14, 2026Sujie Hu, Huaisong Zhang, Pujing Yang +7Text Rendering in T2I Generation Reward Modeling
The Hong Kong University of Science and Technology · Weixin, Tencent · Tsinghua University +3
RM-OPD: On-Policy Distillation for Generative Reward Models
Oct 14, 2026Qinglin Su, Licheng Pan, Zhen Fang +5Reward Modeling RL for Generative Models
Zhejiang University · Xiaohongshu Inc. · University of Science and Technology of China
Rent-Seeking Language Models: Reward Models Set the Prices, Frequent Audits Curb the Rent
Oct 14, 2026Zixiao Wang, Xingbo Du, Duzhen Zhang +1Reward Over-Optimization RL for Language Models
Mohamed bin Zayed University of Artificial Intelligence
Rubrics from the Task, Verdicts from Evidence: Pointwise Rubric Rewards with Tool-Augmented Judges
Oct 13, 2026Tianci Liu, Haoyu WangReward Modeling Rubric-Based Evaluation
University of Tennessee · University at Albany
Diagnose-to-Generate: Eliminating Structural Distortions in Flow Models via Patch-Level Grounded Alignment
Oct 13, 2026Yifan Li, Lishuai Gao, Tong Zhang +8Image Generation Flow-Based Generative Modeling
University of Science and Technology of China, Hefei, AnHui Province, China · Meituan Inc., Beijing, China
IsoVRBench: Video Reward Modeling Without Human Feedback
Oct 12, 2026Shuyue Stella Li, Jacqueline He, Jiahui Chen +5Reward Modeling Video Generation Evaluation
University of Washington · 2Meta Superintelligence Labs
SyncReward: Human-Aligned Synchronization Reward for Audio-Visual Generation
Oct 12, 2026Kaisi Guan, Ying Ba, Wenjing Wang +4Audio-Visual Synchronization Audio-Video Generation
Gaoling School of Artificial Intelligence, Renmin University of China · WeChat, Tencent Inc.
Getting Space Right: Agentic Rewards for Image Generation
Oct 10, 2026Xiaofeng Tan, Jiaxin Lin, Haobo Yuan +5T2I Generation RL for Image Generation
Southeast University · Tencent HY · UC Merced +1
Rubric-CEPR: Self-Evolving Image Editing via Reward-Verified Self-Distillation
Oct 8, 2026Ritesh Thawkar, Shubham Patle, Shravan Venkatraman +1Image Editing Text-Guided Image Editing
Mohamed bin Zayed University of Artificial Intelligence · Aalto University
Judging in Latent Space: Efficient Generative Reward Modeling via Semantics-Preserving Compression
Oct 7, 2026Mingqing Yuan, Xiaobo Liang, Junwei Yang +5Reward Modeling LLM-as-a-Judge
Soochow University · University of Cambridge · Chalmers University of Technology +3
Visual Jev Rewards: Reference-Bound Verification for Multi-Subject Image Generation
Oct 7, 2026Baoteng Li, Wenzhuo Wu, Kongming Liang +1Multi-Subject Image Generation Reward Modeling
School of Artificial Intelligence, Beijing University of Posts and Telecommunications1 · Beijing Key Laboratory of Multimodal Data Intelligent Perception and Governance2
Personalize at Test Time: Learning User Preferences for Image Generation
Oct 6, 2026Jiamu Bai, Jiaming Hu, Yanhong Wu +1User Preference Modeling Diffusion Model Alignment
Penn State University · Meta · Boston University +1
Reinforcement Learning for Hierarchical Reasoning Rewards: Minimax-Optimal Rates with Transformers
Oct 6, 2026Naoki Nishikawa, Taiji SuzukiReward Modeling Transformer-Based RL
The University of Tokyo, RIKEN AIP

Oct 25, 2026J/K move · Enter open · S save

AndroidFlux: Evaluating and Reward Modeling Failure Recovery Capabilities in Mobile Use Multimodal Agents

Gyubeum Lim, Yemo Koo, Ghunwoo Jang, Zsolt Kira

Georgia Institute of Technology · Konkuk University

Multimodal Agent Evaluation Reward Model Evaluation Reward Modeling

Open paper AskPDF arXiv

Mobile-use multimodal agents are typically evaluated by task success from a clean initial state, which does not isolate their ability to recover from intermediate errors. Because mobile environments change dynamically, a general mobile-use agent must complete tasks robustly even from unexpected states and in unseen environments. This requires knowing when and where an agent makes errors, and whether it can recover from errors and subsequent states. To evaluate this capability, we introduce AndroidFlux, a failure-recovery benchmark built on AndroidWorld. AndroidFlux restores recorded trajectories to controlled checkpoints before, at, and after the first recoverable error, so that different agents continue from the same device state and history. On these checkpoints, a complementary reward-modeling benchmark evaluates recovery-action preferences using downstream rollout outcomes. We further release an offline reinforcement learning (RL) dataset that improves both error recovery and overall task success. Our evaluations show that recovery performance only partially correlates with clean-start success, and that explicit reasoning helps only models trained with GUI-domain specific RL.