RL for Image Generation

RL: Reinforcement Learning

This field uses reinforcement learning to post-train image generators, including diffusion and flow-matching models, autoregressive systems, and unified multimodal models. Common approaches optimize policies with methods such as GRPO, using rewards from human preferences, vision-language models, prompt-specific rubrics, verifiable visual criteria, or the model’s own understanding signals. Researchers also study finer-grained feedback, including token-, region-, and denoising-step rewards, as well as reasoning, reflection, and iterative editing; progress is assessed on tasks such as GenEval and T2I-CompBench. A central challenge is finding rewards and optimization rules that improve faithfulness and image quality without reward hacking, loss of diversity, unstable training, or excessive compute.

Latest papers

93 papers22 papers in the last four weeks, up 340% on the four weeks before. 0.1% of all new papers.

RubricRL: Simple Generalizable Rewards for Text-to-Image Generation
Nov 25, 2025Xuelu Feng, Yunsheng Li, Ziyu Wan +4Reward Modeling T2I Generation
University at Buffalo · Microsoft AI · Nikola Tesla STEM High School
SRUM: Fine-Grained Self-Rewarding for Unified Multimodal Models
Oct 14, 2025Weiyang Jin, Yuwei Niu, Jiaqi Liao +4Unified Multimodal Models T2I Generation
HKU MMLab · The University of Hong Kong · Shenzhen Loop Area Institute +2
Advantage Weighted Matching: Aligning RL with Pretraining in Diffusion Models
Sep 29, 2025Shuchen Xue, Chongjian Ge, Shilong Zhang +2Score Matching Diffusion Model Alignment
UCAS · Adobe Research · HKU +1

Nov 25, 2025J/K move · Enter open · S save

RubricRL: Simple Generalizable Rewards for Text-to-Image Generation

Xuelu Feng, Yunsheng Li, Ziyu Wan, Zixuan Gao+3

University at Buffalo · Microsoft AI · Nikola Tesla STEM High School

Reward Modeling T2I Generation Multimodal Reward Modeling Rubric-Based RL RL for Image Generation

Open paper AskPDF arXiv

Reinforcement learning (RL) has recently emerged as a promising approach for aligning text-to-image generative models with human preferences. A key challenge, however, lies in designing effective and interpretable rewards. Existing methods often rely on either composite metrics (e.g., CLIP, OCR, and realism scores) with fixed weights or a single scalar reward distilled from human preference models, which can limit interpretability and flexibility. We propose RubricRL, a simple and general framework for rubric-based reward design that offers greater interpretability, composability, and user control. Instead of using a black-box scalar signal, RubricRL dynamically constructs a structured rubric for each prompt--a decomposable checklist of fine-grained visual criteria such as object correctness, attribute accuracy, OCR fidelity, and realism--tailored to the input text. Each criterion is independently evaluated by a multimodal judge (e.g., o4-mini), and a prompt-adaptive weighting mechanism emphasizes the most relevant dimensions. This design not only produces interpretable and modular supervision signals for policy optimization (e.g., GRPO or PPO), but also enables users to directly adjust which aspects to reward or penalize. Experiments with an autoregressive text-to-image model demonstrate that RubricRL improves prompt faithfulness, visual detail, and generalizability, while offering a flexible and extensible foundation for interpretable RL alignment across text-to-image architectures.

Latest papers

RubricRL: Simple Generalizable Rewards for Text-to-Image Generation

SRUM: Fine-Grained Self-Rewarding for Unified Multimodal Models

Advantage Weighted Matching: Aligning RL with Pretraining in Diffusion Models