Despite the success of foundation models in image synthesis, achieving fine-grained spatial control remains a significant challenge, typically requiring auxiliary structural adapters or complex inference-time steering. While modern vision-language (VLM) text encoders possess the latent capacity to represent spatial coordinates, generative models often fail to leverage this information. We demonstrate that the signal-to-noise ratio (SNR) distribution is a critical determinant of spatial grounding in flow-matching models. Using a novel synthetic benchmark of geometric primitives, we isolate structural mechanics from textural complexity inherent in natural imagery and introduce a rigorous color-aware layout metric to quantify spatial fidelity. We show that training on a centered SNR distribution results in poor layout adherence (25.9%). Instead, we identify high-noise regimes as a "critical learning phase" for spatial grounding. Shifting the training focus toward these stages increases layout adherence to 80.8%, effectively reaching the 83.6% reconstruction ceiling of the underlying variational autoencoder. We find that the necessity for this shift scales with resolution and aspect ratio variance, and that the conservative noise schedules employed by models like FLUX.1 are insufficient for precise layout adherence. Finally, we show that these findings translate to real-world imagery via the COCO dataset (1.0 vs. 57.3 Average Precision), where state-of-the-art layout adherence is unlocked through distribution tuning alone, without architectural modifications.
Figures & tables
Figure 1: From Synthetic Experiments to Real-World Transfer. Layout adherence is first studied in a controlled synthetic setting, using color-filled regions as exact ground truth (left). This allows an isolated validation before extending the approach to real images (right).
Figure 2: Example ⟨text prompt,image⟩ pair from our synthetic dataset: (a) the input text prompt used to condition the scene; (b) a visualization of the occlusion, where rectangles are rendered with semi-transparency to reveal depth ordering; and (c) the final rendered image.
Figure 3: Layout Adherence Metric Sensitivity. We illustrate mIoUτ across different failure modes. The metric effectively penalizes spatial misalignment (c), missing elements (d), and incorrect occlusion ordering (e). Note that spatial misalignment can result in a lower score than an omission, reflecting the metric’s focus on precise spatial grounding.
Figure 4: Shifted logit-normal distribution pln for location parameters μ . Increasing μ shifts density toward higher noise levels (larger values of t ), demonstrating how the distribution of training or sampling steps can be adjusted to prioritize specific denoising stages.
Figure 5: Layout adherence evaluation. (Left) mIoU10 evolution over 100k steps for n=16 samples. (Right) Sensitivity to color match threshold τ across 1,000 text-image pairs. Higher μ values (except μ=3 ) drive convergence toward the empirical VAE upper bound (red).
Figure 6: Qualitative comparison of generated images across varying μ . We compare the ground truth (GT) and VAE baseline against our model for μ∈[0,3.0] . Notably, spatial inaccuracies are easily perceivable for lower values ( μ∈[0,0.5] ). As μ increases, the generated images show noticeably higher fidelity and better spatial alignment with the GT.
Table 7
mIoU2:25
\cmidrule 5-7 Model
Params
Text Encoder
μ
Model ↑
VAE ↑
Δ(Gap)↓
\midrule FLUX.2-4B-base
4B
Qwen3-VL-4B
0.0
25.9
83.6
57.7
1.0
80.8
83.6
2.8
\midrule FLUX.2-9B-base
9B
Qwen3-VL-8B
0.0
32.6
83.6
51.0
1.0
81.5
83.6
2.1
\midrule FLUX.1-dev
12B
CLIP + T5
1.0
7.5
88.6
81.1
Table 3: Architectural Generalization. Strong layout adherence can be achieved across scales, architectures, and text-conditioning strategies through SNR shifting alone. μ=1.0 is used as a common starting point across all models, compared against a centered baseline ( μ=0.0 ). FLUX.1-dev instead requires μ=3.0 to reach comparable spatial grounding. ”Gap” ( Δ ) denotes the difference between VAE reconstruction and generated performance.
Layout Adherence
Image Quality
\cmidrule (r)2-4 \cmidrule (l)5-7 Method
AP↑
AP50↑
AR↑
CLIP↑
FID↓
IS↑
pln(t;μ=3.0,⋅)
57.3
81.8
65.2
28.01
32.38
30.32
pln(t;μ=2.5,⋅)
52.8
78.9
60.5
27.92
32.44
29.62
pln(t;μ=2.0,⋅)
32.0
61.6
40.9
27.85
32.45
29.38
pln(t;μ=1.5,⋅)
14.9
36.3
23.1
27.65
32.00
29.82
pln(t;μ=1.0,⋅)
1.4
4.8
5.4
27.04
33.11
29.27
Table 4: Effect of shift μ on layout adherence and image quality of real-world images. To validate generalization beyond our synthetic benchmark, FLUX.2-9B-base is fine-tuned for 5 epochs on MS-COCO 2017 [ Lin et al.(2014)Lin, Maire, Belongie, Hays, Perona, Ramanan, Dollár, and Zitnick ] at 5122 resolution and aspect ratios up to 3:1 ( rmax=3 ).
Layout Adherence
Image Quality
\cmidrule (r)2-4 \cmidrule (l)5-7 Method
AP↑
AP50↑
AR↑
CLIP↑
FID↓
IS↑
GLIGEN [ Li et al.(2023)Li, Liu, Wu, Mu, Yang, Gao, Li, and Lee ]
42.1
66.8
37.2
30.25
30.68
26.76
MIGC [ Zhou et al.(2024)Zhou, Li, Ma, Zhang, and Yang ]
39.1
66.0
36.8
28.60
31.43
26.14
InstanceDiff [ Wang et al.(2024)Wang, Darrell, Rambhatla, Girdhar, and Misra ]
51.9
71.9
43.7
30.04
29.94
27.23
Ranni [ Feng et al.(2024b)Feng, Gong, Chen, Shen, Liu, and Zhou ]
14.7
22.3
17.0
30.02
31.94
24.44
CreatiLayout [ Zhang et al.(2025)Zhang, Hong, Wang, Shao, Wu, Wu, and Jiang ]
47.8
69.8
42.1
33.32
29.27
30.26
Table 5: Quantitative comparison against specialized layout-to-image models. Evaluated on MS-COCO 2017, our fine-tuned FLUX.2-9B-base achieves state-of-the-art layout adherence and competitive image quality, without requiring architectural modifications.
Figure 7: Qualitative results on COCO val. Bounding boxes indicate ground-truth layout conditions. Our model faithfully generates objects at specified locations across diverse scenes, compositions, and densities (rows 1–3). Row 4 shows representative failure cases.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Loss of pixel-level color precision at μ=3.0 (Example 1/3). Aggregated per-color binary mask intersections across thresholds τ∈{2,5,10,15,20,25} (rows) and GT/VAE/generated images at varying μ (columns). Intersection masks at μ=3.0 fade sharply for low τ ( ≤5 ), indicating loss of color precision rather than spatial misplacement.
Figure 9: Loss of pixel-level color precision at μ=3.0 for τ≤15 in a sparse scene (2/3).
Figure 10: Loss of pixel-level color precision at μ=3.0 for τ≤15 in a dense scene (3/3).
Figure 11: Scale and density-dependent layout adherence. The generative model (blue) saturates the VAE’s capacity (gray). While localization is harder for smaller objects, performance is stable across scene densities, with near-constant median mIoU2:25 for n∈[3,10] .
Layout Adherence
Image Quality
\cmidrule (r)2-4 \cmidrule (l)5-7 Epoch
AP↑
AP50↑
AR↑
CLIP↑
FID↓
IS↑
0
0.3
0.9
1.6
22.90
78.99
18.29
1
17.5
40.1
26.0
27.83
33.78
29.46
2
36.4
66.1
45.1
27.91
32.89
29.57
3
45.5
73.7
53.7
28.06
31.55
29.19
4
51.1
78.3
59.1
28.06
31.88
29.69
Appendix
Table 6: Epoch-ablation of layout adherence and image quality on real-world images. FLUX.2-9B-base is fine-tuned on MS-COCO 2017 [ Lin et al.(2014)Lin, Maire, Belongie, Hays, Perona, Ramanan, Dollár, and Zitnick ] at μ=3.0 and evaluated after each of 10 epochs. Results at E=5 slightly differ from Tab. 4 and Tab. 5 due to training variance.
Figure 12: Compositional failure example (1/2). The two elephants’ heads occasionally fuse into a single shared head (e.g., top-right image), a compositional artifact present in the pre-trained FLUX.2-9B-base model prior to any fine-tuning.
Figure 13: Compositional failure example (2/2). Across samples, the seagull is occasionally rendered with two beaks pointing in different directions (e.g., top-left image), or with multiple heads (e.g., the two bottom-right images), a compositional artifact already present in the pre-trained FLUX.2-9B-base model prior to any fine-tuning.
Flow-based generative modeling learns probability transports from simple noise to data for fast sampling, density estimation, and controllable generation.
Current flow matching models learn to transport the source i.i.d. Gaussian noise into the target distribution of natural images, yet this source distribution carries no notion of spatial structure. Images however are fundamentally local since nearby pixels are strongly correlated. By sampling the noise independently, we hypothesize that models are implicitly encouraged to exploit less noisy neighbors as context during training, partially bypassing the need to properly learn the true local structure of images. The source distribution, in other words, works against the inductive bias of the image domain. To ameliorate this design discrepancy, we propose StructFlow which encodes spatial locality directly into the source by having the pixels within a small region share a common noise component. This structured source produces transport paths that are geometrically aligned with image regions - enabling properties that generic flow matching struggles to provide: fine-grained local editing that naturally respects boundaries, robust structure preservation, and smooth semantic interpolation between images. We show that these benefits also extend to large pre-trained models, demonstrating that StructFlow can even be incorporated through a lightweight post-training phase. Comprehensive experiments on multiple datasets, in unconditional, class and text-conditioned regimes, using different diffusion transformer architectures confirm that StructFlow not only offers competitive image generation quality, but also significantly improves localized controllable re-synthesis.
Pixel-space generative models bypass lossy latent compression, yet necessitate joint learning of global structure and fine-grained details in a high-dimensional space. Standard flow matching interpolates noise toward a fixed clean-image endpoint, leaving the spectral evolution to be learned implicitly. In this paper, we introduce Energy-Guided Flow Matching(EG-FM) that explicitly models a coarse-to-fine generative trajectory by moving endpoint. Specifically, EG-FM replaces the fixed endpoint with a heat-kernel-filtered endpoint that evolves smoothly from low-frequency image to clean image. The fraction of high-frequency signal in moving endpoint is released by an image-specific energy-guided scheduling, leading to the re-targeting of velocity in flow matching. Our framework requires no adaptation of the backbone and training data, bringing negligible cost on the training and inference stages. In our experiment, EG-FM consistently achieves lower FID on the ImageNet class-conditional image generation task at 256×256 with fewer epochs, reaching an FID of 1.55 at 200 epochs and 1.45 at 600 epochs. We continue training the generation task on the setting of 512×512 resolution, yielding a FID of 1.58 after only 40 high-resolution adaptation epochs. Furthermore, we transfer EG-FM on text-to-image generation and achieve 0.85 on GenEval score and 83.9 on DPG-Bench. Code is available at https://github.com/ysng123/EG-FM.
Haoyang Tong, Yu He, Fang Li +6
MAIS & NLPR, CASIA · JD.com · Xi'an Jiaotong University +1
In this work, we analyze the optimization dynamics of generative fine-tuning. We observe that under the Flow Matching framework, the standard MSE objective can be formulated as a Quadratic Form governed by a dynamically evolving Neural Tangent Kernel (NTK). This geometric perspective reveals a latent Data Interaction Matrix, where diagonal terms represent independent sample learning and off-diagonal terms encode residual correlation between heterogeneous features. Although standard training implicitly optimizes these cross-term interferences, it does so without explicit control; moreover, the prevailing data-homogeneity assumption may constrain the model's effective capacity. Motivated by this insight, we propose Semantic Granularity Alignment (SGA), using Text-to-Image synthesis as a testbed. SGA engineers targeted interventions in the vector residual field to mitigate gradient conflicts. Evaluations across DiT and U-Net architectures confirm that SGA advances the efficiency-quality trade-off by accelerating convergence and improving structural integrity.
Zhinan Xiong, Shunqi Yuan
Conservatoire National des Arts et Métiers, Paris, France · Sun Yat-sen University, Guangzhou, China