Multimodal Reward Modeling

Overview

It studies how models can compare or score multimodal outputs and behaviors—such as image edits, generated video or speech, and GUI or robot trajectories—against preferences, instructions, or task progress. Common approaches train vision-language or audio-language judges from pairwise or listwise preferences, or make judgments more interpretable with task-specific rubrics and grounded rationales; other methods derive dense progress signals from video, pretrained representations, or self-evaluation. These rewards guide reinforcement learning and preference optimization, and are tested on generation and editing benchmarks, video-understanding tasks, and embodied simulations or robots. A central challenge is ensuring scores remain calibrated across prompts and domains without encouraging reward hacking or overlooking specific failure modes.

Latest papers

CardsList

99 papers6 papers in the last four weeks, down 33% on the four weeks before. 0.0% of all new papers.

  1. Grana: Compositional Reward Modeling for Long-Horizon Robot Manipulation

    Nov 18, 2026Zi-ang Cao, Kaiyuan Zheng, Yinzhen Xu +14Reward ModelingRobot Task Progress Estimation

  2. Leveraging Psychophysical Attentional Distribution for Gaze-Augmented Reward Modeling

    Nov 10, 2026Chaeho Lee, Junyup Kim, Wansoo Kim +2Reward ModelingPreference Learning for LLMs

  3. RM-VLM: From Natural-Language Task Descriptions to Vision-Grounded Reward Machines

    Nov 5, 2026Roko Parac, Christos Ziakas, Leo Ardon +1Reward MachinesReinforcement Learning

  4. PhySA: Decoupling Aesthetic and Physical Rewards for Humanoid Locomotion

    Nov 5, 2026Yunwan Fu, Sicheng He, Jiageng Mao +1Humanoid Robot LocomotionRobot Locomotion

  5. EmoReward: Self-Evolving Reward Calibration for Emotional Video Generation RL

    Nov 4, 2026Mingzhe Zheng, Yue Wu, Jing Tan +19RL for Video GenerationVideo Generation

  6. ACWM-RM: A Reference-Free Reward Model from Complementary Mismatch Preferences

    Oct 23, 2026Shibo Hong, Siyu Ren, FengJiao Chen +3Multimodal Reward ModelingRL for Video Generation

  7. Harnessing VLM in Embodied RL: From Reward Regressor to Success Detector via Exploration-Augmented Key-Observation Gating

    Oct 11, 2026Xin Liu, Yutong Jiang, Jian Su +4Reward Design for RLReinforcement Learning

  8. Disentangling Task-Relevant Latent Dynamics for Reward Modeling

    Oct 9, 2026Dayou Li, Jionghao Wang, Peihao Wang +9Dense Reward RLMultimodal Reward Modeling

  9. Unsupervised Multimodal Reward Modeling

    Oct 8, 2026Yueying Li, Jingxuan Fan, Kianté Brantley +2Multimodal Reward ModelingVLM Evaluation

  10. CriticHack: Evaluating Visual Rewards Under Robot Policy Optimization

    Oct 1, 2026Jiaxuan Luo, Xingguo Xu, Shanshan Wang +2RL for RoboticsReward Hacking

  11. Think Before You Score: Thinking Reward Model for Visual Generation

    Sep 29, 2026Xuehai Bai, Zhenchen Tang, Yang Shi +9Reward ModelingMultimodal Reward Modeling

  12. SkillRubric: Co-Evolving Actor Guidance and Evaluator Rubrics for Multimodal Agents

    Sep 28, 2026Bingqing Jiang, Guoxi Zhang, Jasper Wang +7Multimodal Reward ModelingRubric-Based RL

  13. ARS: Agentic Reward System for Robot Learning

    Sep 28, 2026Sheng Hu, Weiyi Lu, Lingbing Zeng +5RL for RoboticsProcess Reward Models

  14. Diffusion Reward Models

    Sep 27, 2026Xiangyang Wang, Bingxiang He, Zeyuan Liu +13Reward ModelingMultimodal Reward Modeling

  15. RewardVerse: Rubric-Guided Policy Optimization for Video Reward Modeling

    Sep 19, 2026Zhenchen Tang, Yang Li, Songlin Yang +6Reward ModelingMultimodal Reward Modeling

  16. TryOnReward: Learning Foveated Consistency for Reinforcement Fine-Tuning of Virtual Try-On

    Sep 6, 2026Xueheng Li, Yong Liu, Xiaolong Fu +5Virtual Try-OnMultimodal Reward Modeling

  17. Same Trajectory, Contradictory Rewards (ROBORMBENCH): Paraphrase Fragility in Vision Language Reward Models

    Sep 4, 2026Wonje Jeung, Sangyeon Yoon, Hyesoo Hong +6VLM RobustnessMultimodal Reward Modeling

  18. WorldReward: Reward Modeling for Camera-Conditioned World Models

    Sep 3, 2026Yibin Wang, Zehan Wang, Junshu Tang +13Reward ModelingCamera-Conditioned Video Generation

  19. RubricRM: Generative Reward Modeling via Dynamic Rubrics for Image Generation and Editing

    Aug 27, 2026Zijian Kan, Wei Wang, Long Luo +6Reward ModelingMultimodal Reward Modeling

  20. Beyond Pairwise Feedback: Listwise Vision-Language Supervision for Preference-Based Reward Learning

    Aug 26, 2026Srivalli Katkuri, Maxwell Kawada, Juan WachsPreference-Based RLPreference Learning

  21. VIG: Visual Information Gain as a Reward Signal for Multimodal Chain-of-Thought Compression

    Aug 22, 2026Wen Luo, Xiaohan Yi, Xiaotao Huang +1Multimodal CoT ReasoningMultimodal Token Compression

  22. Robo-Dopamine 2.0: History-Conditioned and OOD-Aware Process Reward Modeling for Robotic Manipulation

    Aug 16, 2026Yijie Xu, Haopeng Jin, Run Zhou +8Process Reward ModelsMultimodal Reward Modeling

  23. RynnValue: Scaling Robotic Value Foundation Models with Temporal Distance

    Aug 10, 2026Dongchi Huang, Hongyin Zhang, Bohan Hou +11Reward ModelingRobot Foundation Models

  24. TrustRoboReward: Preference-Ordered Isotonic Score Editing for Multi-Paradigm Robot Reward Models

    Aug 9, 2026Yidong Wang, Yan Zhan, Ziteng Feng +16Reward ModelingPairwise Preference Evaluation

  25. Evaluation-Verification Reward for Consistent Multi-Reference Image Editing

    Jul 31, 2026Yingmao Miao, Pengfei Zhang, Xiaochen Lv +5Image Editing EvaluationMultimodal Reward Modeling

  26. OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models

    Jul 30, 2026Qiushi Sun, Kanzhi Cheng, Yian Wang +20VLM EvaluationComputer-Use Agent Benchmarks