Reinforcement learning (RL) has emerged as a paradigm for fine-tuning large-scale generative models, such as diffusion and flow models, to align with complex human preferences and user-specified tasks. A fundamental limitation remains
the curse of diversity collapse, where the objective formulation and optimization landscape inherently collapse the policy to a Dirac delta distribution. To address this challenge, we propose
DRIFT (
Dive
Rsity-
Incentivized Reinforcement
Fine-
Tuning for Versatile Image Generation), an innovative framework that systematically incentivizes output diversity throughout the on-policy fine-tuning process, reconciling strong task alignment with high generation diversity to enhance versatility essential for applications that demand diverse candidate generations. We approach the problem across three representative perspectives: i)
sampling a reward-concentrated subset that filters out reward outliers to prevent premature collapse; ii)
prompting with stochastic variations to expand the conditioning space, and iii)
optimization of the intra-group diversity with a potential-based reward shaping mechanism. Experimental results show that DRIFT exhibits clear Pareto dominance in task alignment and generation diversity, achieving 3.88%
∼81.42% higher diversity at matched alignment and 14.38%
∼56.45% higher alignment at matched diversity. Our code is available at \textcolor{magenta}{https://github.com/NJU-RL/DRIFT}.