cs.LGNov 6, 2026

SIFR: Stable Signed Flow Reinforcement

Authors: Runlong Liao, Xingqian Xu, Lizhang Chen, Baiyu Su, Yufan Huang, Zhonghao Wang, Akash Gokul, Ming-Yu Liu, 

Organizations: NVIDIA · The University of Texas at Austin

Abstract

Reward-based fine-tuning of flow models requires effectively promoting high-reward and suppressing low-reward samples, while maintaining training stability over many rounds of iterative updates. Iterative reward fine-tuning provides a simple starting point, but the positive reward weights give no explicit unfitting signal, and repeated reweighting concentrates the distribution and causes collapse. We propose Stable Signed Flow Reinforcement (SIFR), which replaces the reward weights by signed advantages to supply the missing unfitting signal, and constrains the velocity drift from a reference model to stabilize the updates, with a Lagrange multiplier that adapts the regularization strength to a prescribed drift budget. We further show that existing flow RL methods fall within this framework: they implicitly minimize the signed loss, differing in their regularization choices. We validate SIFR on SD3.5 Medium (2.5B) and FLUX.2 Klein (9B) and scale it to Cosmos 3 Super Text2Image (64B): across all 12 backbone--reward pairs with preference and rule-based rewards, it matches or exceeds existing methods, reaches their peak rewards much faster, and keeps improving over long training where they plateau or collapse.

Explore similar work

CardsList
  1. AdvantageFlow: Regularized Advantage-Weighted RL in Flow Models

    May 25, 2026Branislav Kveton, Anup Rao, Subhojyoti Mukherjee +2Rectified FlowRL for Image Generation

  2. MEND: RL For Flow Models via Proximal Velocity Matching

    Oct 5, 2026Shreshth Saini, Neil Birkbeck, Yilin Wang +2Flow MatchingRL for Image Generation

  3. Distribution-Based Regularization for Reward Fine-Tuning Diffusion and Flow Models

    Oct 17, 2026Romeo Passaro, Jeongjae Lee, Jong Chul Ye +4Generative ModelingDiffusion Model Alignment