From Sparse to Dense: Multi-View GRPO for Flow Models via Augmented Condition Space

Jiazi Bu; Pengyang Ling; Yujie Zhou; Yibin Wang; Yuhang Zang; Tianyi Wei; Xiaohang Zhan; Jiaqi Wang

cs.CVMar 13, 2026

From Sparse to Dense: Multi-View GRPO for Flow Models via Augmented Condition Space

Authors: Jiazi Bu, Pengyang Ling, Yujie Zhou, Yibin Wang, Yuhang Zang, Tianyi Wei, Xiaohang Zhan, Jiaqi Wang, +3 more

Organizations: Shanghai Jiao Tong University · S-Lab, Nanyang Technological University · Shanghai AI Laboratory · University of Science and Technology of China · Fudan University · Shanghai Innovation Institute · Adobe Research · Stanford University · The Chinese University of Hong Kong · CPII under InnoHK

Abstract

Group Relative Policy Optimization (GRPO) has emerged as a powerful framework for preference alignment in text-to-image (T2I) flow models. However, we observe that the standard paradigm where evaluating a group of generated samples against a single condition suffers from insufficient exploration of inter-sample relationships, constraining both alignment efficacy and performance ceilings. To address this sparse single-view evaluation scheme, we propose Multi-View GRPO (MV-GRPO), a novel approach that enhances relationship exploration by augmenting the condition space to create a dense multi-view reward mapping. Specifically, for a group of samples generated from one prompt, MV-GRPO leverages a flexible Condition Enhancer to generate semantically adjacent yet diverse captions. These captions enable multi-view advantage re-estimation, capturing diverse semantic attributes and providing richer optimization signals. By deriving the probability distribution of the original samples conditioned on these new captions, we can incorporate them into the training process without costly sample regeneration. Extensive experiments demonstrate that MV-GRPO achieves superior alignment performance over state-of-the-art methods.

Figures & tables

Figure 1: Gallery of MV-GRPO. Our MV-GRPO substantially elevates the generation quality of flow models (Flux.1-dev in this figure), particularly in terms of fine-grained details and photorealism. Prompts are listed in the supplementary material.

Figure 2: Reward Evaluation in GRPO Training . (a) Standard flow-based GRPO methods evaluate generated samples under the single original condition, resulting in sparse reward mapping and insufficient inter-sample relationship exploration. (b) Our MV-GRPO leverages an augmented set of conditions to facilitate a dense multi-view mapping, fostering a comprehensive exploration of relationship among samples.

Figure 3: Reward Ranking Varies with Conditions . Reward rankings of SDE samples across multiple semantically similar yet different conditions exhibit large variations, indicating that relying on a single condition for advantage estimation is inadequate.

Figure 4: Overview of MV-GRPO . MV-GRPO leverages a flexible Condition Enhancer module (a pretrained VLM or LLM) to generate diverse augmented conditions for dense multi-view reward signals, facilitating comprehensive advantage estimation.

Figure 5

Figure 5: Distribution of Probability Drift at Different SDE Steps. Most condition pairs exhibit a drift near zero, demonstrating that the SDE transition probability is effectively preserved when substituting the original with augmented conditions.

Figure 6: Reward Curves during Training. Our MV-GRPO outperforms baselines in both convergence speed and performance ceiling under various training settings.

Reward Model	Method	HPS-v3	UR-v2-A	UR-v2-C	UR-v2-S	HPS-v2	CLIP	IR	UR-v1
/	Flux.1-dev [ 14 ]	0.133	3.250	3.634	3.223	0.305	0.389	1.042	3.604
HPS-v3	Flow-GRPO [ 19 ]	0.147	3.235	3.670	3.236	0.326	0.380	1.149	3.548
	DanceGRPO [ 43 ]	0.145	3.238	3.663	3.224	0.322	0.381	1.165	3.585
	TempFlow-GRPO [ 11 ]	0.150	3.240	3.683	3.299	0.332	0.376	1.184	3.559
	DiffusionNFT [ 46 ]	0.149	3.244	3.676	3.259	0.330	0.364	1.161	3.551
	MV-GRPO (LLM)	0.153	3.232	3.692	3.313	0.336	0.373	1.176	3.557

Table 1: Quantitative comparison of different methods. The best results are in bold , while the second-best result is underlined . UR-v2-A, UR-v2-C, and UR-v2-S denote the Alignment, Coherence, and Style dimensions of UnifiedReward-v2, respectively.

Figure 7: Qualitative Comparisons with Baselines on HPS-v3.

Figure 8: Qualitative Comparisons with Baselines on UnifiedReward-v2.

Table 11

Figure 9: Augmented Conditions. MV-GRPO generates diverse augmented conditions by leveraging variations among SDE samples and multi-view descriptive prompts.

Components	Values & Choices	HPS-v3	UR-v2-A	UR-v2-C	UR-v2-S	HPS-v2	CLIP	IR	UR-v1
(a) Condition Number $K$	w/o $\mathcal{V}_{K}$	0.149	3.238	3.668	3.269	0.332	0.378	1.152	3.553
	$K=3$	0.152	3.235	3.684	3.293	0.335	0.377	1.170	3.567
	$K=6$	0.154	3.233	3.696	3.326	0.338	0.372	1.178	3.573
	$\mathbf{K=12}$	0.155	3.229	3.701	3.320	0.340	0.370	1.193	3.569
(b) Condition Diversity	w/o SDE query	0.153	3.233	3.692	3.304	0.336	0.374	1.183	3.560
	w/o $\mathcal{P}_{\text{VLM}}$	0.152	3.237	3.687	3.295	0.334	0.376	1.174	3.564

Table 4: Ablation experiments on MV-GRPO components and hyperparameters.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Value	Parameter	Value
Random seed	42	Learning rate	$2\times 10^{-6}$
Train batch size	1	Weight decay	$1\times 10^{-4}$
Warmup steps	0	Mixed precision	bfloat16
Dataloader workers	4	Max grad norm	1.0
Eta	0.7	Sampler seed	1223627
Group size	12	Scheduler shift	3

Appendix

Table 5: Hyperparameter settings in our experiments.

Figure 10: VLM Prompt Template . It integrates a descriptive instruction from

\mathcal{P}_{\text{VLM}}

, along with an image and its original text prompt, to construct a complete VLM prompt for querying the VLM Condition Enhancer to obtain augmented conditions.

ID	Instructions
1	Focus on the main subjects and key attributes.
2	Focus on actions, motion, or interactions.
3	Focus on scene layout and spatial relationships.
4	Focus on lighting conditions and shadows.
5	Focus on color palette and contrast.
6	Focus on camera angle, lens feel, and depth of field.

Appendix

Table 6: List of instructions in the set

\mathcal{P}_{\text{VLM}}

used to query the VLM. Each instruction guides the model to focus on a specific visual dimension when generating conditions.

Figure 11: LLM Prompt Template . This template incorporates a specific operation from

\mathcal{P}_{\text{LLM}}

and a memory buffer of prior outputs, guiding the LLM Condition Enhancer to refine the original text prompt into diverse augmented conditions.

ID	Instructions
1	ADD: Expand the prompt by enriching existing elements with descriptive details (<= 20 tokens), strictly avoiding the introduction of new subjects.
2	DELETE: Remove some non-essential details (<= 20 tokens) to produce a concise caption, but the output must remain a valid description of image.
3	PARAPHRASE: Swap words/phrases for close synonyms (adjectives -> near-synonyms; nouns -> another common term for the same thing).

Appendix

Table 7: List of operations in the set

\mathcal{P}_{\text{LLM}}

used to query the LLM. Each operation directs the model to modify the input prompt while maintaining semantic consistency.

Figure 12: Illustration of Equivalent SDE Noise. (a) Standard flow SDE generates samples based on the transition mean anchored to a single condition

\mathbf{c}

. (b) When substituting

\mathbf{c}

with an augmented condition

\mathbf{c}^{\prime}

, the transition mean shifts accordingly. To reach the original samples, an equivalent noise term

\boldsymbol{\epsilon}^{\text{SDE}^{\prime}}

is implicitly required. The

\boldsymbol{\mu}_{\theta}

and

\boldsymbol{\mu}^{\prime}_{\theta}

in the figure are shorthand for

\boldsymbol{\mu}_{\theta}(\boldsymbol{x}_{t},\mathbf{c})

and

\boldsymbol{\mu}_{\theta}(\boldsymbol{x}_{t},\mathbf{c}^{\prime})

, respectively.

Figure 13: Additional Comparison Results on HPS-v3. (1/2)

Figure 14: Additional Comparison Results on HPS-v3. (2/2)

Figure 15: Additional Comparison Results on UnifiedReward-v2. (1/2)

Figure 16: Additional Comparison Results on UnifiedReward-v2. (2/2)

Figure 17: Additional Visual Samples of MV-GRPO. (1/4)

Figure 18: Additional Visual Samples of MV-GRPO. (2/4)

Figure 19: Additional Visual Samples of MV-GRPO. (3/4)

Figure 20: Additional Visual Samples of MV-GRPO. (4/4)

Figure 21: Results using same prompts and different seeds. (HPS-v3)

Figure 22: Results using same prompts and different seeds. (UR-v2)

Figure	Text Prompt
Figure. 1	A sleek silver supercar is parked on a wet city street.
	A massive and brightly colored spacecraft in a deserted landscape, depicted in retro 1960s sci-fi art.
	Close-up view of ancient Greek ruins set against a colourful, starry night sky creating a mystical atmosphere.
	An ornate, lantern-lit courtyard with carved balconies and tiled floors.
	A beautiful woman in a white gown wears gold jewels, warm outdoor light.
	Close-up on a flickering neon installation in a dark futuristic nightclub. The text "Multi-View GRPO" is rendered in intertwining blue and magenta neon tubes, casting colored light onto chrome pipes and exposed wires on a brick wall. Cyberpunk aesthetic, shallow depth of field.

Appendix

Table 8: T2I prompts used in this paper (1/2). Prompts for each figure are listed sequentially, following the order from left to right and top to bottom.

Figure	Text Prompt
Figure. 18	American cowboy with a scruffy appearance in a retrofuturistic style, inspired by the animations of Studio Ghibli.
	Undead army on riding beasts with symbols and music.
	A Monet portrait.
	a man walking alone down the street in a velvet jacket.
	a shiny metallic renaissance steampunk robot in the style of Jan van Eyck.
	A key visual of a young female swat officer with a neon futuristic gas mask in a cyberpunk setting.

Appendix

Table 9: T2I prompts used in this paper (2/2). Prompts for each figure are listed sequentially, following the order from left to right and top to bottom.

Explore similar work

AdaGRPO: A Capability-Aware Adaptive Enhancement for Flow-based GRPO
Jun 5, 2026Jiazi Bu, Pengyang Ling, Yujie Zhou +8T2I Generation Group Relative Policy Optimization
1Shanghai Jiao Tong University · 2S-Lab, Nanyang Technological University · 3Shanghai AI Laboratory +7
Alleviating Sparse Rewards by Modeling Step-Wise and Long-Term Sampling Effects in Flow-Based GRPO
Feb 6, 2026Yunze Tong, Mushui Liu, Canyu Zhao +7Flow Matching Temporal Credit Assignment
1Zhejiang University, Hangzhou, China · 2Taobao & Tmall Group of Alibaba, Hangzhou, China · 3Tsinghua University, Beijing, China
Advances in GRPO for Generation Models: A Survey
Feb 21, 2026Zexiang Liu, Xianglong He, Yangguang LiRL for Generative Models Group Relative Policy Optimization
SJTU · THU · CUHK

Jun 5, 2026J/K move · Enter open · S save

AdaGRPO: A Capability-Aware Adaptive Enhancement for Flow-based GRPO

Jiazi Bu, Pengyang Ling, Yujie Zhou, Yibin Wang+7

1Shanghai Jiao Tong University · 2S-Lab, Nanyang Technological University · 3Shanghai AI Laboratory+7

T2I Generation Group Relative Policy Optimization RL for Image Generation Curriculum Learning RL for Generative Models

Open paper AskPDF arXiv

Group Relative Policy Optimization (GRPO) has demonstrated remarkable success in aligning text-to-image (T2I) flow models with human preferences. However, we have identified that the learning loop of current flow-based GRPO is fundamentally decoupled from the learner's current capability, suffering from critical blind spots at both prompt selection and advantage estimation: (i) Existing methods sample prompts randomly, overlooking the substantial impact of data selection on reinforcement learning (RL) efficacy--a factor proven crucial in GRPO for large language models; (ii) They evaluate sample quality solely relying on intra-group statistics, lacking a global perspective to accurately measure true policy improvement. To address these issues, we propose Adaptive GRPO (AdaGRPO), a novel capability-aware RL algorithm tailored for flow models. Specifically, AdaGRPO consists of two principal components: (i) Online Curriculum Filtering Strategy: Dynamically tracks the model's proficiency and adaptively selects prompts that best match its current learning boundary; (ii) Cross-Level Advantage Fusion: Synergistically integrates fine-grained intra-group advantages with macro-level global advantages, providing a comprehensive and unbiased policy evaluation. As a lightweight, plug-and-play module, AdaGRPO can be seamlessly integrated with existing frameworks such as Flow-GRPO, DanceGRPO, and Flow-CPS. Extensive experiments demonstrate that AdaGRPO consistently drives performance gains while significantly stabilizes GRPO training for flow models.

Abstract

Explore similar work

AdaGRPO: A Capability-Aware Adaptive Enhancement for Flow-based GRPO

Alleviating Sparse Rewards by Modeling Step-Wise and Long-Term Sampling Effects in Flow-Based GRPO

Advances in GRPO for Generation Models: A Survey