Unlocking Spatial Grounding in Flow-Matching Models via SNR Tuning

Martin Thißen; Elke Hergenröther

cs.CVOct 21, 2026

Unlocking Spatial Grounding in Flow-Matching Models via SNR Tuning

Authors: Martin Thißen, Elke Hergenröther

Organizations: Department of Computer Science, Darmstadt University of Applied Sciences, Darmstadt, Germany

Abstract

Despite the success of foundation models in image synthesis, achieving fine-grained spatial control remains a significant challenge, typically requiring auxiliary structural adapters or complex inference-time steering. While modern vision-language (VLM) text encoders possess the latent capacity to represent spatial coordinates, generative models often fail to leverage this information. We demonstrate that the signal-to-noise ratio (SNR) distribution is a critical determinant of spatial grounding in flow-matching models. Using a novel synthetic benchmark of geometric primitives, we isolate structural mechanics from textural complexity inherent in natural imagery and introduce a rigorous color-aware layout metric to quantify spatial fidelity. We show that training on a centered SNR distribution results in poor layout adherence (25.9%). Instead, we identify high-noise regimes as a "critical learning phase" for spatial grounding. Shifting the training focus toward these stages increases layout adherence to 80.8%, effectively reaching the 83.6% reconstruction ceiling of the underlying variational autoencoder. We find that the necessity for this shift scales with resolution and aspect ratio variance, and that the conservative noise schedules employed by models like FLUX.1 are insufficient for precise layout adherence. Finally, we show that these findings translate to real-world imagery via the COCO dataset (1.0 vs. 57.3 Average Precision), where state-of-the-art layout adherence is unlocked through distribution tuning alone, without architectural modifications.

Figures & tables

Panel of Figure 1 — Figure 1: From Synthetic Experiments to Real-World Transfer. Layout adherence is first studied in a controlled synthetic setting, using color-filled regions as exact ground truth (left). This allows an isolated validation before extending the approach to real images (right).

Panel of Figure 2 — Figure 2: Example $\langle\textit{text prompt},\textit{image}\rangle$ pair from our synthetic dataset: (a) the input text prompt used to condition the scene; (b) a visualization of the occlusion, where rectangles are rendered with semi-transparency to reveal depth ordering; and (c) the final rendered image.

Panel of Figure 3 — Figure 3: Layout Adherence Metric Sensitivity. We illustrate $\mathrm{mIoU}_{\tau}$ across different failure modes. The metric effectively penalizes spatial misalignment (c), missing elements (d), and incorrect occlusion ordering (e). Note that spatial misalignment can result in a lower score than an omission, reflecting the metric’s focus on precise spatial grounding.

Figure 4: Shifted logit-normal distribution $\mathbf{p_{ln}}$ for location parameters $\mathbf{\mu}$ . Increasing $\mu$ shifts density toward higher noise levels (larger values of $t$ ), demonstrating how the distribution of training or sampling steps can be adjusted to prioritize specific denoising stages.

Figure 5: Layout adherence evaluation. (Left) $\mathrm{mIoU}_{10}$ evolution over 100k steps for $n=16$ samples. (Right) Sensitivity to color match threshold $\tau$ across 1,000 text-image pairs. Higher $\mu$ values (except $\mu=3$ ) drive convergence toward the empirical VAE upper bound (red).

Figure 6: Qualitative comparison of generated images across varying $\mu$ . We compare the ground truth (GT) and VAE baseline against our model for $\mu\in[0,3.0]$ . Notably, spatial inaccuracies are easily perceivable for lower values ( $\mu\in[0,0.5]$ ). As $\mu$ increases, the generated images show noticeably higher fidelity and better spatial alignment with the GT.

Table 7

\cmidrule 5-7 Model	Params	Text Encoder	$\mu$	Model $\uparrow$	VAE $\uparrow$	$\Delta(Gap)\downarrow$
				$\text{mIoU}_{2:25}$
\midrule FLUX.2-4B-base	4B	Qwen3-VL-4B	$0.0$	25.9	83.6	57.7
\midrule FLUX.2-4B-base	4B	Qwen3-VL-4B	$1.0$	80.8	83.6	2.8
\midrule FLUX.2-9B-base	9B	Qwen3-VL-8B	$0.0$	32.6	83.6	51.0
\midrule FLUX.2-9B-base	9B	Qwen3-VL-8B	$1.0$	81.5	83.6	2.1
\midrule FLUX.1-dev	12B	CLIP + T5	$1.0$	7.5	88.6	81.1

Table 3: Architectural Generalization. Strong layout adherence can be achieved across scales, architectures, and text-conditioning strategies through SNR shifting alone.

\mu=1.0

is used as a common starting point across all models, compared against a centered baseline (

\mu=0.0

). FLUX.1-dev instead requires

\mu=3.0

to reach comparable spatial grounding. ”Gap” (

\Delta

) denotes the difference between VAE reconstruction and generated performance.

	Layout Adherence			Image Quality
\cmidrule (r)2-4 \cmidrule (l)5-7 Method	$\mathrm{AP}\uparrow$	$\mathrm{AP}^{50}\uparrow$	$\mathrm{AR}\uparrow$	$\mathrm{CLIP}\uparrow$	$\mathrm{FID}\downarrow$	$\mathrm{IS}\uparrow$
$p_{ln}(t;\mu=3.0,\cdot)$	57.3	81.8	65.2	28.01	32.38	30.32
$p_{ln}(t;\mu=2.5,\cdot)$	52.8	78.9	60.5	27.92	32.44	29.62
$p_{ln}(t;\mu=2.0,\cdot)$	32.0	61.6	40.9	27.85	32.45	29.38
$p_{ln}(t;\mu=1.5,\cdot)$	14.9	36.3	23.1	27.65	32.00	29.82
$p_{ln}(t;\mu=1.0,\cdot)$	1.4	4.8	5.4	27.04	33.11	29.27

Table 4: Effect of shift

\mu

on layout adherence and image quality of real-world images. To validate generalization beyond our synthetic benchmark, FLUX.2-9B-base is fine-tuned for 5 epochs on MS-COCO 2017 [ Lin et al.(2014)Lin, Maire, Belongie, Hays, Perona, Ramanan, Dollár, and Zitnick ] at

512^{2}

resolution and aspect ratios up to 3:1 (

r_{\mathrm{max}}{=}3

	Layout Adherence			Image Quality
\cmidrule (r)2-4 \cmidrule (l)5-7 Method	$\mathrm{AP}\uparrow$	$\mathrm{AP}^{50}\uparrow$	$\mathrm{AR}\uparrow$	$\mathrm{CLIP}\uparrow$	$\mathrm{FID}\downarrow$	$\mathrm{IS}\uparrow$
GLIGEN [ Li et al.(2023)Li, Liu, Wu, Mu, Yang, Gao, Li, and Lee ]	42.1	66.8	37.2	30.25	30.68	26.76
MIGC [ Zhou et al.(2024)Zhou, Li, Ma, Zhang, and Yang ]	39.1	66.0	36.8	28.60	31.43	26.14
InstanceDiff [ Wang et al.(2024)Wang, Darrell, Rambhatla, Girdhar, and Misra ]	51.9	71.9	43.7	30.04	29.94	27.23
Ranni [ Feng et al.(2024b)Feng, Gong, Chen, Shen, Liu, and Zhou ]	14.7	22.3	17.0	30.02	31.94	24.44
CreatiLayout [ Zhang et al.(2025)Zhang, Hong, Wang, Shao, Wu, Wu, and Jiang ]	47.8	69.8	42.1	33.32	29.27	30.26

Table 5: Quantitative comparison against specialized layout-to-image models. Evaluated on MS-COCO 2017, our fine-tuned FLUX.2-9B-base achieves state-of-the-art layout adherence and competitive image quality, without requiring architectural modifications.

Figure 7: Qualitative results on COCO val. Bounding boxes indicate ground-truth layout conditions. Our model faithfully generates objects at specified locations across diverse scenes, compositions, and densities (rows 1–3). Row 4 shows representative failure cases.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Loss of pixel-level color precision at $\mu=3.0$ (Example 1/3). Aggregated per-color binary mask intersections across thresholds $\tau\in\{2,5,10,15,20,25\}$ (rows) and GT/VAE/generated images at varying $\mu$ (columns). Intersection masks at $\mu=3.0$ fade sharply for low $\tau$ ( $\leq 5$ ), indicating loss of color precision rather than spatial misplacement.

Figure 9: Loss of pixel-level color precision at $\mu=3.0$ for $\tau\leq 15$ in a sparse scene (2/3).

Figure 10: Loss of pixel-level color precision at $\mu=3.0$ for $\tau\leq 15$ in a dense scene (3/3).

Figure 11: Scale and density-dependent layout adherence. The generative model (blue) saturates the VAE’s capacity (gray). While localization is harder for smaller objects, performance is stable across scene densities, with near-constant median

\mathrm{mIoU}_{2:25}

for

n\in[3,10]

	Layout Adherence			Image Quality
\cmidrule (r)2-4 \cmidrule (l)5-7 Epoch	$\mathrm{AP}\uparrow$	$\mathrm{AP}^{50}\uparrow$	$\mathrm{AR}\uparrow$	$\mathrm{CLIP}\uparrow$	$\mathrm{FID}\downarrow$	$\mathrm{IS}\uparrow$
$0$	0.3	0.9	1.6	22.90	78.99	18.29
$1$	17.5	40.1	26.0	27.83	33.78	29.46
$2$	36.4	66.1	45.1	27.91	32.89	29.57
$3$	45.5	73.7	53.7	28.06	31.55	29.19
$4$	51.1	78.3	59.1	28.06	31.88	29.69

Appendix

Table 6: Epoch-ablation of layout adherence and image quality on real-world images. FLUX.2-9B-base is fine-tuned on MS-COCO 2017 [ Lin et al.(2014)Lin, Maire, Belongie, Hays, Perona, Ramanan, Dollár, and Zitnick ] at

\mu=3.0

and evaluated after each of 10 epochs. Results at

E=5

slightly differ from Tab. 4 and Tab. 5 due to training variance.

Figure 12: Compositional failure example (1/2). The two elephants’ heads occasionally fuse into a single shared head (e.g., top-right image), a compositional artifact present in the pre-trained FLUX.2-9B-base model prior to any fine-tuning.

Figure 13: Compositional failure example (2/2). Across samples, the seagull is occasionally rendered with two beaks pointing in different directions (e.g., top-left image), or with multiple heads (e.g., the two bottom-right images), a compositional artifact already present in the pre-trained FLUX.2-9B-base model prior to any fine-tuning.

Flow-Based Generative Modeling

Flow-based generative modeling learns probability transports from simple noise to data for fast sampling, density estimation, and controllable generation.

See all 552 papers

Spatially-Grounded Flow Matching: Structured Source Distributions for Image Generation
Aug 16, 2026Arman Zarei, Mahdi M. KalayehGenerative Modeling Flow-Based Generative Modeling
University of Maryland · Netflix
Energy-Guided Flow Matching
Aug 6, 2026Haoyang Tong, Yu He, Fang Li +6Flow Matching Conditional Image Generation
MAIS & NLPR, CASIA · JD.com · Xi'an Jiaotong University +1
The Quadratic Geometry of Flow Matching: Semantic Granularity Alignment for Text-to-Image Synthesis
Mar 11, 2026Zhinan Xiong, Shunqi YuanFlow Matching T2I Generation
Conservatoire National des Arts et Métiers, Paris, France · Sun Yat-sen University, Guangzhou, China

Aug 16, 2026J/K move · Enter open · S save

Spatially-Grounded Flow Matching: Structured Source Distributions for Image Generation

Arman Zarei, Mahdi M. Kalayeh

University of Maryland · Netflix

Generative Modeling Flow-Based Generative Modeling Flow Matching Controllable Image Generation Image Editing

Open paper AskPDF arXiv

Current flow matching models learn to transport the source i.i.d. Gaussian noise into the target distribution of natural images, yet this source distribution carries no notion of spatial structure. Images however are fundamentally local since nearby pixels are strongly correlated. By sampling the noise independently, we hypothesize that models are implicitly encouraged to exploit less noisy neighbors as context during training, partially bypassing the need to properly learn the true local structure of images. The source distribution, in other words, works against the inductive bias of the image domain. To ameliorate this design discrepancy, we propose StructFlow which encodes spatial locality directly into the source by having the pixels within a small region share a common noise component. This structured source produces transport paths that are geometrically aligned with image regions - enabling properties that generic flow matching struggles to provide: fine-grained local editing that naturally respects boundaries, robust structure preservation, and smooth semantic interpolation between images. We show that these benefits also extend to large pre-trained models, demonstrating that StructFlow can even be incorporated through a lightweight post-training phase. Comprehensive experiments on multiple datasets, in unconditional, class and text-conditioned regimes, using different diffusion transformer architectures confirm that StructFlow not only offers competitive image generation quality, but also significantly improves localized controllable re-synthesis.

Abstract

Flow-Based Generative Modeling

Explore similar work

Spatially-Grounded Flow Matching: Structured Source Distributions for Image Generation

Energy-Guided Flow Matching

The Quadratic Geometry of Flow Matching: Semantic Granularity Alignment for Text-to-Image Synthesis