RL for Generative Models

RL: Reinforcement Learning

Ask about this topic

Overview

This field studies how reward and preference feedback can improve models that generate text, images, video, molecules, proteins, and robot actions. Common approaches include policy-gradient methods such as PPO and GRPO, reward-weighted fine-tuning, preference optimization, and reinforcement learning for diffusion and flow-matching models. Rewards may come from verifiable rules, learned evaluators, human comparisons, physical simulators, or downstream task performance. A central challenge is making feedback informative and reliable without reward hacking, unstable training, excessive compute, or loss of output diversity.

Latest papers

CardsList

154 papers24 papers in the last four weeks, up 100% on the four weeks before. 0.1% of all new papers.

  1. AnyDoc: Enhancing Document Generation via Large-Scale HTML/CSS Data Synthesis and Height-Aware Reinforcement Optimization

    Mar 26, 2026Jiawei Lin, Wanrong Zhu, Vlad I Morariu +1Text GenerationSynthetic Data Generation for Language Models

    Xi’an Jiaotong University · Adobe Research

  2. MSRL: Scaling Generative Multimodal Reward Modeling via Multi-Stage Reinforcement Learning

    Mar 26, 2026Chenglong Wang, Yifu Huo, Yang Gan +8Multimodal Reward ModelingRL for Generative Models

    Northeastern University, Shenyang, China · ByteDance

  3. RefReward-SR: LR-Conditioned Reward Modeling for Preference-Aligned Super-Resolution

    Mar 25, 2026Yushuai Song, Weize Quan, Weining Wang +8Image Super-ResolutionHuman Preference Modeling

    Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · OPPO AI Center, OPPO Inc.

  4. UniGRPO: Unified Policy Optimization for Reasoning-Driven Visual Generation

    Mar 24, 2026Jie Liu, Zilyu Ye, Linxiao Yuan +8Image GenerationRL for Generative Models

    The Chinese University of Hong Kong · ByteDance Seed

  5. CellFluxRL: Biologically-Constrained Virtual Cell Modeling via Reinforcement Learning

    Mar 23, 2026Dongxia Wu, Shiye Su, Yuhui Zhang +4Constrained Generative ModelingRL for Image Generation

    Stanford University, Stanford, CA

  6. Incentivizing Generative Zero-Shot Learning via Outcome-Reward Reinforcement Learning with Visual Cues

    Mar 22, 2026Wenjin Hou, Xiaoxiao Sun, Hehe FanZero-Shot LearningZero-Shot Image Classification

    CCAI, Zhejiang University · Stanford University · State Key Laboratory of CAD&CG, Zhejiang University

  7. Correlation-Weighted Multi-Reward Optimization for Compositional Generation

    Mar 19, 2026Jungmyung Wi, Hyunsoo Kim, Donghyun KimCompositional T2I GenerationRL for Generative Models

    Korea University · Korea Institute of Science and Technology

  8. AR-CoPO: Align Autoregressive Video Generation with Contrastive Policy Optimization

    Mar 18, 2026Dailan He, Guanlin Feng, Xingtong Ge +5RL for Video GenerationStreaming Video Generation

    CUHK MMLab, China · Vivix Group Limited, China · HKUST, China +1

  9. Reliable Reasoning in SVG-LLMs via Multi-Task Multi-Reward Reinforcement Learning

    Mar 17, 2026Haomin Wang, Qi Wei, Qianli Ma +4Text-to-SVG GenerationVLM Reasoning

    Shanghai Jiao Tong University · Shanghai AI Laboratory · Nanjing University +1

  10. Collaborative Temporal Feature Generation via Critic-Free Reinforcement Learning for Cross-User Sensor-Based Activity Recognition

    Mar 17, 2026Xiaozhou Ye, Feng Jiang, Zihan Wang +3Human Activity RecognitionDomain Generalization

    School of Artificial Intelligence (School of Future Technology), Nanjing University of Information Science and Technology, Nanjing, China · School of Cyberspace Security, Nanjing University of Information Science and Technology, Nanjing, China · Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, China +1

  11. GoldenStart: Q-Guided Priors and Entropy Control for Distilling Flow Policies

    Mar 15, 2026He Zhang, Ying Sun, Hui XiongPolicy DistillationContinuous Control

    Thrust of Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou) · AI2 Robotics · Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong SAR

  12. ReTabSyn: Realistic Tabular Data Synthesis via Reinforcement Learning

    Mar 11, 2026Xiaofeng Lin, Seungbae Kim, Zhuoya Li +3Synthetic Tabular Data GenerationTabular Data Synthesis

    University of California, Los Angeles, USA · University of South Florida, USA · Cisco, USA +1

  13. Graph-GRPO: Training Graph Flow Models with Reinforcement Learning

    Mar 11, 2026Baoheng Zhu, Deyu Bo, Delvin Ce Zhang +1Graph GenerationFlow-Based Generative Modeling

    Beijing University of Posts and Telecommunications · National University of Singapore · University of Sheffield +1

  14. Towards Unified Multimodal Interleaved Generation via Group Relative Policy Optimization

    Mar 10, 2026Ming Nie, Chunwei Wang, Jianhua Han +2Multimodal GenerationUnified Multimodal Models

    School of Data Science, Fudan University · Noah’s Ark Lab, Huawei · Yinwang Intelligent Technology Co., Ltd.

  15. AdaGen: Learning Adaptive Policy for Image Synthesis

    Mar 7, 2026Zanlin Ni, Yulin Wang, Yeguo Hua +5Diffusion Timestep SchedulingRL for Image Generation

    Department of Automation, BNRist, Tsinghua University, Beijing, China. · Taobao & Tmall Group of Alibaba.

  16. When Rubrics Fail: Error Enumeration as Reward in Reference-Free RL Post-Training for Virtual Try-On

    Mar 5, 2026Wisdom Ikezogwo, Mehmet Saygin Seyfioglu, Ranjay Krishna +1Reward ModelingReference-Free Evaluation

    University of Washington, Seattle, USA · Amazon, Seattle, USA · Allen Institute for AI

  17. Deep Interest Mining for Intent-Enriched Semantic IDs in Multimodal Generative Recommendation

    Mar 3, 2026Yangchen Zeng, Jinze WangRecommender SystemsSemantic ID Recommendation

  18. Real-Time Generative Policy via Langevin-Guided Flow Matching for Autonomous Driving

    Mar 3, 2026Tianze Zhu, Yinuo Wang, Wenjun Zou +6Reinforcement LearningRL for Autonomous Driving

    School of Vehicle and Mobility, Tsinghua University, Beijing, China · College of Artificial Intelligence, Tsinghua University, Beijing, China

  19. Improving Text-to-Image Generation with Intrinsic Self-Confidence Rewards

    Mar 1, 2026Seungwook Kim, Minsu ChoT2I Diffusion ModelsRL for Image Generation

    POSTECH · GenGenAI · RLWRLD

  20. Mesh-Pro: Asynchronous Advantage-guided Ranking Preference Optimization for Artist-style Quadrilateral Mesh Generation

    Feb 28, 2026Zhen Zhou, Jian Liu, Biwen Lei +103D Mesh GenerationRL for Generative Models

    Institute of Automation, Chinese Academy of Sciences · Tencent Hunyuan · Hong Kong University of Science and Technology

  21. Rooted Absorbed Prefix Trajectory Balance with Submodular Replay for GFlowNet Training

    Feb 28, 2026Xi Wang, Wenbo Lu, Shengjie WangGenerative Flow NetworksRL for Generative Models

    Courant Institute School of Mathematics, Computing, and Data Science, New York University

  22. SibylSense: Adaptive Rubric Learning via Memory Tuning and Adversarial Probing

    Feb 24, 2026Yifei Xu, Guilherme Potje, Shivam Shandilya +9Rubric-Based RLReward Design for RL

    Microsoft · University of California, Los Angeles

  23. TrajGPT-R: Generating Urban Mobility Trajectory with Reinforcement Learning-Enhanced Generative Pre-trained Transformer

    Feb 24, 2026Jiawei Wang, Chuang Yang, Jiawei Yong +6Trajectory GenerationRL for Generative Models

    The University of Tokyo, Tokyo 113-0034, Japan · Toyota Motor Corporation, Tokyo 112-0004, Japan

  24. Advances in GRPO for Generation Models: A Survey

    Feb 21, 2026Zexiang Liu, Xianglong He, Yangguang LiRL for Generative ModelsGroup Relative Policy Optimization

    SJTU · THU · CUHK

  25. DP-RFT: Learning to Generate Synthetic Text via Differentially Private Reinforcement Fine-Tuning

    Feb 20, 2026Fangyuan Xu, Sihao Chen, Zinan Lin +13Differentially Private Synthetic Data GenerationSynthetic Data Generation for Language Models

    New York University · Microsoft · University of Southern California

  26. MoRL: Reinforced Reasoning for Unified Motion Understanding and Generation

    Feb 16, 2026Hongpeng Wang, Zeyu Zhang, Wenhao Li +1Human Motion UnderstandingHuman Motion Generation

    The University of Sydney · Peking University · Nanyang Technological University

  27. Partial GFlowNet: Accelerating Convergence in Large State Spaces via Strategic Partitioning

    Feb 12, 2026Xuan Yu, Xu Wang, Rui Zhu +2Reinforcement LearningGenerative Flow Networks

    University of Science and Technology of China (USTC), Hefei, China · Suzhou Institute for Advanced Research, USTC, Suzhou, China · State Key Laboratory of Precision and Intelligent Chemistry, USTC, Hefei, China

  28. Alleviating Sparse Rewards by Modeling Step-Wise and Long-Term Sampling Effects in Flow-Based GRPO

    Feb 6, 2026Yunze Tong, Mushui Liu, Canyu Zhao +7Flow MatchingTemporal Credit Assignment

    1Zhejiang University, Hangzhou, China · 2Taobao & Tmall Group of Alibaba, Hangzhou, China · 3Tsinghua University, Beijing, China

  29. Scientific feedback reveals productive regimes beyond training conventions

    Dec 12, 2025Yoon Pyo Lee, Samrendra Roy, Souvik Chakraborty +1AI for ScienceRL for Generative Models

    Department of Nuclear Engineering, Hanyang University, 222, Wangsimni-ro, Seongdong-gu, Seoul, 04763, Republic of Korea.

  30. MIND-V: Hierarchical World Model for Long-Horizon Robotic Manipulation with RL-based Physical Alignment

    Dec 7, 2025Ruicheng Zhang, Mingyang Zhang, Jun Zhou +6Physical Consistency in Video GenerationLong-Horizon Robotic Manipulation

    Tsinghua University · Sun Yat-sen University · X Square Robot +1