Multimodal Reward Modeling

It studies how models can compare or score multimodal outputs and behaviors—such as image edits, generated video or speech, and GUI or robot trajectories—against preferences, instructions, or task progress. Common approaches train vision-language or audio-language judges from pairwise or listwise preferences, or make judgments more interpretable with task-specific rubrics and grounded rationales; other methods derive dense progress signals from video, pretrained representations, or self-evaluation. These rewards guide reinforcement learning and preference optimization, and are tested on generation and editing benchmarks, video-understanding tasks, and embodied simulations or robots. A central challenge is ensuring scores remain calibrated across prompts and domains without encouraging reward hacking or overlooking specific failure modes.