RL for Flow Matching Models

RL: Reinforcement Learning

Follow this topic Ask about this topic

Latest papers

15 papers4 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

QF3: Fast Flow RL with Filtered Q-Gradients
Oct 6, 2026Chung Min Kim, Brent Yi, David McAllister +7Robotic RL RL for Legged Locomotion
UC Berkeley · Amazon FAR
MEND: RL For Flow Models via Proximal Velocity Matching
Oct 5, 2026Shreshth Saini, Neil Birkbeck, Yilin Wang +2Flow Matching RL for Image Generation
The University of Texas at Austin · Google · University of Colorado Boulder
QAMM: Adjoint MeanFlow Matching for Few-Step Offline Reinforcement Learning
Sep 28, 2026Yuehu Gong, Shutong Ding, Mokai Pan +4Flow Matching MeanFlow
School of Data Science, Fudan University · ShanghaiTech University · Shanghai Innovation Institute
Timestep Weighting: A Hidden Key to Effective ELBO-Based Flow-Matching RL
Sep 26, 2026Qinwei Ma, Jingzhe Shi, Simin Fan +2RL for Image Generation Adaptive Loss Weighting
College of AI, Tsinghua University · Department of Electrical and Computer Engineering, Princeton University · EPFL +1
LC-GRPO: Bridging Train-Inference Gap for Flow-Based GRPO with Langevin Correction
Aug 6, 2026Yingqing Guo, Hui Yuan, Zijian He +2RL for Video Generation RL for Image Generation
VINE: Taming Generative Control Policies for Reinforcement Learning
Jul 11, 2026Rushuai Yang, Zhuo Han, Houlin Li +10RL for Robotics Offline RL
1AgiBot · 3Peking University · 2The Hong Kong University of Science and Technology
ReFPO: Reflow Regularization for Flow Matching Policy Gradients
Jun 19, 2026Ge Wang, Yibo Peng, Fan Feng +10Policy Gradient Reinforcement Learning
Ising AI · CUHK-Shenzhen · FNii-Shenzhen
Reversal Q-Learning
Jun 16, 2026Aditya Oberai, Seohong Park, Sergey LevineOff-Policy RL Flow-Based Policy Learning
University of California, Berkeley.
Flow-DPPO: Divergence Proximal Policy Optimization for Flow Matching Models
Jun 9, 2026Bowen Ping, Xiangxin Zhou, Penghui Qi +3RL for Video Generation RL for Image Generation
Xi’an Jiaotong University · Tencent Hunyuan · National University of Singapore
Reinforcement Learning for Flow-Matching Policies with Density Transport
Jun 7, 2026Boshu Lei, Kostas Daniilidis, Antonio LoquercioFlow Matching Robotic RL
University of Pennsylvania · Philadelphia, PA 19103
AdvantageFlow: Regularized Advantage-Weighted RL in Flow Models
May 25, 2026Branislav Kveton, Anup Rao, Subhojyoti Mukherjee +2Rectified Flow RL for Image Generation
Adobe Research
Reinforce Adjoint Matching: Scaling RL Post-Training of Diffusion and Flow-Matching Models
May 11, 2026Andreas Bergmeister, Stefanie Jegelka, Nikolas Nüsken +2Diffusion Model Alignment Flow Matching
1TU Munich, MCML · 2MIT CSAIL · 3King’s College London +2
ScoRe-Flow: Complete Distributional Control via Score-Based Reinforcement Learning for Flow Matching
Apr 13, 2026Xiaotian Qiu, Lukai Chen, Jinhao Li +3RL for Flow Matching Models Flow-Based Policy Learning
Zhejiang University, Hangzhou, China · Shanghai Innovation Institute · Shanghai Jiao Tong University, Shanghai, China +1
Discrete Flow Matching Policy Optimization
Apr 7, 2026Maojiang Su, Po-Chung Hsieh, Weimin Wu +4TV Regularization RL for Flow Matching Models
Center for Foundation Models and Generative AI, Northwestern University, Evanston, IL 60208, USA · Department of Electrical Engineering, National Taiwan University, Taipei 10617, Taiwan · Department of Physics, National Taiwan University, Taipei 10617, Taiwan +1
Alleviating Sparse Rewards by Modeling Step-Wise and Long-Term Sampling Effects in Flow-Based GRPO
Feb 6, 2026Yunze Tong, Mushui Liu, Canyu Zhao +7Flow Matching Temporal Credit Assignment
1Zhejiang University, Hangzhou, China · 2Taobao & Tmall Group of Alibaba, Hangzhou, China · 3Tsinghua University, Beijing, China

Oct 6, 2026J/K move · Enter open · S save

QF3: Fast Flow RL with Filtered Q-Gradients

Chung Min Kim, Brent Yi, David McAllister, Hongsuk Choi+6

UC Berkeley · Amazon FAR

Robotic RL RL for Legged Locomotion Off-Policy RL Flow-Based Policy Learning RL for Flow Matching Models

Open paper AskPDF arXiv

Flow policies have become a standard policy class for learning robot behaviors from demonstrations, but reinforcement learning is still critical for improving pre-trained flow policies or learning them from scratch through interaction. We introduce QF3 (Fast Flow RL with Filtered Q-Gradients), an online off-policy RL algorithm that trains a flow policy with flow matching plus the critic's action gradient, backpropagated through a one-step prediction of the flow's output. To keep updates where the critic and this prediction are reliable, QF3 applies the critic gradient only to action dimensions that stay near the replay action. To our knowledge, QF3 is the first off-policy flow RL method to train humanoid locomotion policies from scratch and transfer them zero-shot to hardware. Paired with a high-throughput off-policy training recipe, it trains humanoid locomotion and motion-tracking policies with a 10x wall-clock speedup over FPO++, a recent on-policy flow RL method. We further apply QF3 to fine-tune pretrained flow-based manipulation policies on both ABC-Sim and Robomimic tasks. These results suggest that QF3 can both learn robot policies from scratch and refine those acquired from demonstrations. Website: https://qf3-rl.github.io/