Modern Text-to-Image (T2I) diffusion models have achieved remarkable semantic alignment, yet they often suffer from a significant lack of variety, converging on a narrow set of visual solutions for any given prompt. This typicality bias presents a challenge for creative applications that require a wide range of generative outcomes. We identify a fundamental trade-off in current approaches to diversity: modifying model inputs requires costly optimization to incorporate feedback from the generative path. In contrast, acting on spatially-committed intermediate latents tends to disrupt the forming visual structure, leading to artifacts. In this work, we propose to apply repulsion in the Contextual Space as a novel framework for achieving rich diversity in Diffusion Transformers. By intervening in the multimodal attention channels, we apply on-the-fly repulsion during the transformer's forward pass, injecting the intervention between blocks where text conditioning is enriched with emergent image structure. This allows for redirecting the guidance trajectory after it is structurally informed but before the composition is fixed. Our results demonstrate that repulsion in the Contextual Space produces significantly richer diversity without sacrificing visual fidelity or semantic adherence. Furthermore, our method is uniquely efficient, imposing a small computational overhead while remaining effective even in modern "Turbo" and distilled models where traditional trajectory-based interventions typically fail.
Figures & tables
Figure 1. Example results of our Contextual Space repulsion framework using Flux-dev. The base model (top) typically converges on a narrow set of visual solutions. By applying semantic intervention within the internal multi-modal attention channels, our approach (bottom) produces a diverse set of images with minimal computational overhead.
Figure 2. Conceptual comparison of diversity strategies in dual-stream DiT architectures. Here p(i) denotes the prompt embedding for sample i , zt(i) denotes the latent at timestep t for sample i , and the red double-arrow icon indicates the point of diversity manipulation. (a) Upstream : Interventions on noise or prompt embeddings lack structural feedback from the emerging image. (b) Downstream : Repulsion in image latents acts on a fixed visual mode and can push samples off the data manifold, causing artifacts. (c) Ours : By applying on-the-fly repulsion within the Contextual Space (text-attention channels), we steer the model’s generative intent. This allows for a semantically driven intervention synchronized with the emergent visual structure.
Figure 3. Comparison of interpolation and extrapolation between the internal representations of two images. Intermediate frames are generated by denoising the source image while linearly blending its internal features with those of the target; extrapolation extends this vector beyond the endpoints. While Latent Space interpolation leads to structural blurring and artifacts due to spatial misalignment, the Contextual Space maintains high visual fidelity. This demonstrates that the Contextual Space enables smooth semantic transitions by decoupling generative intent from fixed spatial structures.
Figure 4. Qualitative results. For each prompt, we compare the base model results (top) to our results (bottom).
Figure 5. Integration with image editing models. We demonstrate that our method can be successfully integrated into Flux-Kontext to generate high-quality diverse results.
Figure 6. Quantitative evaluation. Pareto frontiers comparing our method against baseline methods using Flux-dev. We evaluate the trade-off between semantic diversity (Vendi Score) and three performance axes: (Left) Human Preference [ImageReward ↑ ], (Middle) Prompt Alignment [VQAScore ↑ ], and (Right) Distributional Fidelity [KID ↓ ]. Our method (red) achieves a superior frontier across all metrics.
Method
SD3.5-Large
SD3.5-Turbo
Flux-dev
Base Model
13.83s
4.18s
10.34s
Ours (Contextual)
18.12s
5.52s
12.80s
SGI
8 Candidates
66.79s
13.15s
47.47s
16 Candidates
76.79s
23.73s
56.32s
32 Candidates
101.44s
46.15s
75.39s
64 Candidates
145.14s
91.30s
113.99s
Table 1. Runtime comparison for generating a group of four images. Our method provides a significant speedup over optimization-based diversity methods like SGI while maintaining a low overhead relative to the base model.
Figure 7. Overall user preference comparison. Distribution of user choices comparing our method with five competing approaches. Bars indicate the percentage of cases in which users preferred our results (green), preferred competing methods (red), or rated both equally (gray).
Figure 8. Ablation of the repulsion scale η . We visualize the impact of the repulsion scale on our results. At η=0 (top row), the base model exhibits low diversity, producing similar architectural styles and environments across multiple seeds. As η increases, our Contextual Space repulsion introduces progressively larger variations, while maintaining high image quality and prompt alignment.
Figure 9. Ablation of Repulsion Space. Pareto frontiers comparing repulsion applied to text attention tokens (Contextual Space, f^T ) versus image attention tokens ( f^I ) within the Flux-dev architecture. We evaluate the trade-off between semantic diversity (Vendi Score) and three performance axes: (Left) Human Preference [ImageReward ↑ ], (Middle) Prompt Alignment [VQAScore ↑ ], and (Right) Distributional Fidelity [KID ↓ ]. Our method (red) achieves a superior frontier across all metrics.
Figure 10. Qualitative Ablation of Repulsion Space. For each prompt, we compare repulsion applied in the image attention space (Image) versus our Contextual Space (Contextual). While image-space repulsion is limited by spatial rigidity, our method achieves more varied compositions.
Figure 11. Qualitative results. For each prompt, we compare the base model results (top) to our results (bottom). All batches were generated using the same random seed initialization. Additional results are provided in Appendix B .
Figure 12. Qualitative comparison of our Contextual Repulsion approach against baseline methods. Each quadrant displays four generated samples per method for a given prompt.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 13. Quantitative evaluation on SD3.5-Large.
Figure 14. Quantitative evaluation on SD3.5-Turbo.
Figure 15. Qualitative results on SD3.5-Large.
Figure 16. Qualitative results on SD3.5-Turbo.
Figure 17. Additional qualitative results on Flux-dev. All batches were generated using the same random seed initialization.
Figure 18. Additional qualitative results on Flux-dev. All batches were generated using the same random seed initialization.
Figure 19. Additional qualitative results on Flux-dev. All batches were generated using the same random seed initialization.
Method
Vendi ( ↑ )
IR ( ↑ )
VQA ( ↑ )
KID ×10−4 ( ↓ )
Base Model
1.780
1.075
0.883
Ours
η=5⋅106
1.810
1.102
0.884
0.066
η=1⋅107
1.831
1.092
0.883
0.103
η=1⋅108
1.869
1.075
0.883
0.157
η=5⋅108
1.898
1.070
0.880
0.172
CADS
s=10−20
1.908
0.377
0.719
0.558
Appendix
Table 2. Detailed metrics for the Flux-dev Pareto frontiers in Figure 6 .
Method
Vendi ( ↑ )
IR ( ↑ )
VQA ( ↑ )
KID ×10−4 ( ↓ )
Base Model
1.819
1.051
0.905
Ours
η=5⋅102
1.851
1.018
0.904
0.619
η=5⋅104
1.878
1.012
0.904
0.627
η=5⋅105
1.941
0.988
0.900
0.625
η=5⋅106
1.980
0.940
0.890
0.445
CADS
s=10−12
2.004
0.131
0.717
0.941
Appendix
Table 3. Detailed metrics for the SD3.5-Large Pareto frontiers in Figure 13 .
Method
Vendi ( ↑ )
IR ( ↑ )
VQA ( ↑ )
KID ×10−4 ( ↓ )
Base Model
1.724
0.978
0.891
Ours
η=1⋅105
1.819
0.914
0.887
1.796
η=5⋅105
1.879
0.899
0.884
1.786
η=1⋅106
1.914
0.864
0.876
1.897
η=1⋅107
2.079
0.562
0.822
1.914
CADS
s=0.1
1.808
0.551
0.772
0.158
Appendix
Table 4. Detailed metrics for the SD3.5-Turbo Pareto frontiers in Figure 14 .
Metric
Choice
Base Model
CADS
SGI
PG
SPARKE
Average
Diversity
Ours
71.6
52.2
56.7
80.0
34.4
61.1
Comp.
12.9
30.0
11.1
14.4
53.1
22.0
Tie
15.5
17.8
32.2
5.6
12.5
16.9
Quality
Ours
49.1
67.8
15.6
82.2
85.9
58.0
Comp.
6.9
11.1
31.1
12.2
3.1
13.1
Tie
44.0
21.1
53.3
5.6
10.9
28.9
Appendix
Table 5. User study results comparing our method against five competing approaches across four evaluation metrics. Values show the percentage of times users preferred our method (Ours), the competitor (Comp.), or rated both equally (Tie). Results are aggregated from 450 pairwise comparisons per metric.
Batch size
Vendi
Vendi (avg. pair)
ImageReward
4
1.819
1.393
0.914
8
2.295
1.401
0.923
16
2.768
1.404
0.928
Appendix
Table 6. Scalability across batch sizes. Quantitative results on SD3.5-Turbo for varying batch sizes. We report the average Vendi score per pair to normalize for batch size constraints.
Model
Timestep interval
Vendi
ImageReward
SD3.5-Turbo
[0,1/4]
1.764
0.829
[1/4,2/4]
1.776
0.811
[2/4,3/4]
1.809
0.745
[3/4,1]
1.988
0.660
[0,1]
2.064
0.501
SD3.5-Large
[0,1/7]
1.849
0.942
Appendix
Table 7. Effect of the timestep interval on diversity and human preference. We evaluate different intervention windows during the diffusion trajectory for SD3.5-Large and SD3.5-Turbo.
SD3.5-Turbo
SD3.5-Large
Block group
Vendi
ImageReward
Vendi
ImageReward
First third
1.878
0.774
1.887
0.895
Middle third
1.947
0.844
1.947
0.902
Last third
1.765
0.913
1.835
0.985
All blocks
1.764
0.829
1.960
0.852
Appendix
Table 8. Performance across different transformer block groups. Results are reported for interventions applied to the first, middle, or last third of the blocks for SD3.5-Large and SD3.5-Turbo.
Text-to-image generation creates images from natural-language prompts using generative models, with research focused on fidelity, control, efficiency, and safety.
Recent text-to-image models built on large-scale Transformer backbones and flow-based objectives deliver strong text-image alignment and high visual quality, yet often produce overly similar samples under a fixed prompt. Existing diversity-enhancement methods alleviate this issue, but typically require expensive sampling or auxiliary optimization, incurring non-trivial overhead. To investigate the root cause of this homogeneity, we examine intermediate Transformer features and observe that the zero-frequency spatial average (DC) component rapidly converges across seeds early in generation, causing early trajectory lock-in that limits downstream variation. Building on this observation, we propose DC Attenuation for diVersity Enhancement (DAVE), a training-free representation-level intervention that selectively attenuates this component in the early regime. DAVE preserves the sampling pipeline with negligible overhead, improving prompt-consistent diversity while maintaining competitive image quality.
Dahee Kwon, Haeun Lee, Jaesik Choi
Korea Advanced Institute of Science and Technology (KAIST) · INEEJI
Image diversity remains a fundamental challenge for text-to-image diffusion models. Low-diversity generation often leads to repetitive outputs, increasing sampling redundancy and hindering both creative exploration and downstream applications. A key factor is the tendency of diffusion models to collapse toward strong modes in the learned distribution. Existing attempts to improve diversity, such as steering-based guidance, often introduce distortions that degrade image quality. To address this issue, we propose Token-Prompt Embedding Space Optimization (TPSO), a training-free and model-agnostic module. TPSO introduces learnable parameters to explore underrepresented regions of the token embedding space, reducing the tendency to repeatedly sample from strong modes of the distribution. Meanwhile, a prompt-level semantic constraint regulates distribution shifts, preventing quality degradation while preserving semantic fidelity. Extensive experiments on MS-COCO across three representative diffusion backbones demonstrate that TPSO substantially improves diversity, boosting performance from 1.10 to 4.18, while maintaining image quality with only a modest inference-time overhead of 3.6% to 8.9%. Code is available at: https://github.com/Open-Debin/TPSO.
Debin Meng, Chen Jin, Zheng Gao +3
Queen Mary University of London · Centre for AI, AstraZeneca, UK · University of Bedfordshire +1
Multimodal Diffusion Transformers (MM-DiTs) jointly process visual and textual representations throughout generation. These models repeatedly update the text tokens through multimodal attention, forming dynamic contextual tokens whose function is not well understood. In this work, we introduce a framework for reading this contextual space through natural-language interrogation. We train a lightweight bottleneck network that maps intermediate contextual tokens into the input space of a frozen Large Language Model (LLM), allowing the LLM to answer questions about the emerging image directly from these hidden representations. Our reader reveals that contextual tokens encode a rich, global representation of the emerging scene: generation-specific semantics, including attributes left underspecified by the prompt, are accessible surprisingly early in denoising, while increasingly fine-grained details become readable over time. Remarkably, this information remains decodable even when the MM-DiT receives an empty prompt, showing that contextual tokens accumulate substantial image-specific information from the evolving visual representation itself. We further find that generations with more readable contextual representations tend to receive higher human-preference scores. Building on these observations, we introduce Contextual Alignment, a training technique that explicitly reinforces the visual-semantic information encoded in the contextual tokens, improving generation quality and distributional coverage. Together, our results establish contextual tokens as both an interpretable view into the internal dynamics of MM-DiTs and an effective target for improving generative models.