FlashControl: One-Step Controllable Generator via Distillation-Friendly Single-Stream Teachers

Ngan Nguyen; Dung Nguyen; Quan Dao; Dimitris N. Metaxas; Minh Hoai; Cuong Pham; Anh Tran

cs.CVOct 15, 2026

FlashControl: One-Step Controllable Generator via Distillation-Friendly Single-Stream Teachers

Authors: Ngan Nguyen, Dung Nguyen, Quan Dao, Dimitris N. Metaxas, Minh Hoai, Cuong Pham, Anh Tran

Organizations: Qualcomm AI Research · Rutgers University · Adelaide University · Posts and Telecommunications Institute of Technology

Abstract

Diffusion models have become a leading approach in visual generative modeling, particularly effective in text-to-image and controllable generation. Their main drawback is slow inference as it requires many sampling steps. While recent distillation methods enable one-step text-to-image generation, extending these techniques to controllable generation is still challenging. Existing controllable models, such as ControlNet, rely on dual-branch design that make distillation difficult. In this work, we adopt a simpler strategy. Instead of adding extra conditional branches, we finetune a selected subset of layers in the original text-to-image diffusion model to directly enable controllable generation. This results in DeltaControl, a single-flow, multi-step diffusion model capable of supporting spatial-condition generation while matching the performance of dual-branch approaches. Building on this teacher model, we further apply step-distillation to obtain FlashControl, a one-step model capable of spatial controllable generation with a single forward pass.

Figures & tables

Figure 1: Our controllable image generator, FlashControl , turns spatial conditions into high-quality images in just a single step.

Figure 2: Comparison of ControlNet (a) and our proposal DeltaControl (b). The architecture of AutoAdapter is illustrated in (c).

Model	Backbone	NFEs	Canny			Depth			Seg			Pose			Hough			Inpainting
			FID $\downarrow$	Rec $\uparrow$	Prec $\uparrow$	FID $\downarrow$	Rec $\uparrow$	Prec $\uparrow$	FID $\downarrow$	Rec $\uparrow$	Prec $\uparrow$	FID $\downarrow$	Rec $\uparrow$	Prec $\uparrow$	FID $\downarrow$	Rec $\uparrow$	Prec $\uparrow$	FID $\downarrow$	Rec $\uparrow$	Prec $\uparrow$
ControlNet [ 51 ]	SD1.5	50	19.86	0.72	0.65	30.04	0.53	0.42	29.18	0.53	0.52	28.45	0.49	0.56	30.75	0.53	0.49	-	-	-
UniControlNet [ 52 ]	SD1.5	50	17.79	0.63	0.65	21.2	0.57	0.54	23.08	0.47	0.49	27.85	0.36	0.49	31.36	0.46	0.44	-	-	-
Uni-Control [ 31 ]	SD1.5	50	22.90	0.72	0.59	21.3	0.60	0.50	25.50	0.51	0.46	27.4	0.43	0.43	-	-	-	19.85	0.79	0.75
T2I adapter [ 27 ]	SD1.5	50	17.59	0.65	0.56	20.73	0.55	0.54	-	-	-	25.62	0.42	0.48	-	-	-	-	-	-
AnyControl [ 43 ]	SD1.5	50	18.55	0.59	0.62	19.32	0.55	0.57	-	-	-	24.99	0.38	0.56	-	-	-	-	-	-

Table 1: Quantitative comparison across six control tasks regarding to FID, Rec, Prec on COCO-5k. "-": the model does not support that condition.

Figure 3: Qualitative comparison across control modalities including: Canny, Depth, Inpainting, Hough, Pose, and Segmentation. Crossed cells mark unsupported controls.

Table 5

Teacher Architecture	Backbone	Canny $\downarrow$	Depth $\downarrow$	Seg $\downarrow$	Pose $\downarrow$	Hough $\downarrow$	Inpainting $\downarrow$
ControlNet	SD2.1	47.15	49.12	52.62	58.03	53.01	31.74
T2I-Adapter	SD2.1	24.72	29.62	28.51	37.80	33.72	24.15
RepControlNet	SD2.1	21.41	25.29	26.40	34.33	31.01	21.23
DeltaControl (Ours)	SD2.1	19.53	24.27	24.93	32.62	29.13	19.07
OminiControl	FLUX.2-Klein-4B	17.38	23.01	25.42	32.51	30.55	17.40
DeltaControl (Ours)	FLUX.2-Klein-4B	11.27	16.95	24.63	30.91	29.45	15.94

Table 5: Effect of teacher architecture under the same one-step distillation pipeline. All rows use the same distillation recipe; only the teacher architecture changes. Metric is FID on COCO-5k.

Models	Backbone	Canny (RMSE $\downarrow$ )	Depth (RMSE $\downarrow$ )	Seg (mPA $\uparrow$ )
ControlNet	SD1.5	87.30	36.49	0.12
CTRLORA	SD1.5	82.52	33.59	0.16
CtrlAdapter	SD1.5	84.05	38.43	0.13
DeltaControl	SD2.1	76.25	34.75	0.16
FlashControl	SD2.1	76.64	34.12	0.18
RepControlNet	PixArt- $\alpha$	87.56	32.77	0.19

Table 6: Comparison of model performance on COCO-5K using extended alignment metrics. “-” denotes that condition is unavailable.

Figure 6: Training losses during one-step distillation. We compare Single stream , SS (Single Stream) + AutoAdapter , and SS + AA (AutoAdapter) + LSP using both student loss and LoRA-teacher loss across training steps. AutoAdapter makes optimization smoother and lowers the losses compared with the plain single-stream baseline, while adding LSP further suppresses the loss spike around 3.5k–4k steps, indicating more stable distillation.

Figure 7: Qualitative training at different distillation iterations. AutoAdapter improves visual convergence over the plain single-stream baseline, and adding LSP further stabilizes late-stage training and avoids collapse. SS is Single-Stream and AA is AutoAdapter.

Figure 8: Evolution of the control-hidden statistic mismatch

\Delta_{\mathrm{stat}}

during training. All variants start from a similar level, but the full AutoAdapter reduces the mismatch fastest and maintains the lowest

\Delta_{\mathrm{stat}}

throughout training. Removing AutoAdapter leads to the highest mismatch, while removing either distribution matching or the token-wise gate gives intermediate performance.

Panel of Figure 9 — Figure 9: User study. Our FlashControl achieves higher perceptual quality and $50\times$ faster.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Stage 1 DeltaControl
Backbone	Stable Diffusion 2.1	PixArt- $\alpha$	FLUX.2-klein-4B
# of training data	2M supervised images
Condition types	Canny, Depth, Segmentation, Hough, Pose, Inpainting
# of LoRA parameters	412M	370M	190M
# of AutoAdapter parameters	4M	4M	30M
# Total parameter	614M	864M	3.9B

Appendix

Table 7: Training hyperparameters and compute settings for Stage 1 DeltaControl and Stage 2 FlashControl . For the conditioning projectors, we randomly initialize these weights and also train them using LoRA.

Figure 10: Training Difficulty . With more control types and under the same budget, FID and convergence rate are slightly worse.

	Canny	Depth	Seg	Pose	Hough	Inpainting
DMD PixArt- $\alpha$ ( Ours )	17.83	20.10	22.64	26.62	28.16	18.95
DMD2	16.45	19.29	21.29	26.38	28.95	19.02

Appendix

Table 8: Quantitative results (FID) on different distillation methods across 6 tasks. FlashControl leverages DMD.

Figure 11: Qualitative results of DeltaControl PixArt- $\alpha$ (20 steps). Zoom in for a better view.

Figure 12: Qualitative results of FlashControl PixArt- $\alpha$ (1 step). Zoom in for a better view.

Figure 13: Qualitative results of DeltaControl FLUX.2-klein-4B (20 steps). Zoom in for a better view.

Figure 14: Qualitative results of DeltaControl FLUX.2-klein-4B (20 steps). Zoom in for a better view.

Figure 15: Qualitative results of FlashControl FLUX.2-klein-4B (1 step). Zoom in for a better view.

Figure 16: Qualitative results of FlashControl FLUX.2-klein-4B (1 step). Zoom in for a better view.

Controllable Image Generation

Controllable image generation steers images using spatial, visual, semantic, stylistic, or physical constraints beyond ordinary text prompts.

See all 210 papers

Improving Controllable Generation: Faster Training and Better Performance via $x_0$ -Supervision
Apr 7, 2026Amadou S. Sangare, Adrien Maglo, Mohamed Chaouch +1Controllable Image Generation Conditional Diffusion Models
Université Paris-Saclay, CEA, List, F-91120, Palaiseau, France
PixelControl: Fine-Grained Condition Fidelity in Text-to-Image Diffusion
Aug 16, 2026Xin Lin, Haodong Li, Zhifei Zhang +5Controllable Image Generation T2I Generation
University of California, San Diego · Adobe Research · Nanyang Technological University
Concept Guidance: Precise, Training-Free Latent Control for Text-to-Image Generation
Aug 14, 2026Nikolai Röhrich, Isabell Hans, Felix Krause +1Diffusion Model Guidance T2I Generation
LMU Munich, Germany · Konrad Zuse School of Excellence in Reliable AI (relAI), Germany · Munich Center for Machine Learning (MCML), Germany

Apr 7, 2026J/K move · Enter open · S save

Improving Controllable Generation: Faster Training and Better Performance via $x_0$ -Supervision

Amadou S. Sangare, Adrien Maglo, Mohamed Chaouch, Bertrand Luvison

Université Paris-Saclay, CEA, List, F-91120, Palaiseau, France

Controllable Image Generation Conditional Diffusion Models T2I Diffusion Models Diffusion Models

Open paper AskPDF arXiv

Text-to-Image (T2I) diffusion/flow models have recently achieved remarkable progress in visual fidelity and text alignment. However, they remain limited when users need to precisely control image layouts, something that natural language alone cannot reliably express. Controllable generation methods augment the initial T2I model with additional conditions that more easily describe the scene. Prior works straightforwardly train the augmented network with the same loss as the initial network. Although natural at first glance, this can lead to very long training times in some cases before convergence. In this work, we revisit the training objective of controllable diffusion models through a detailed analysis of their denoising dynamics. We show that direct supervision on the clean target image, dubbed

x_0

-supervision, or an equivalent re-weighting of the diffusion loss, yields faster convergence. Experiments on multiple control settings demonstrate that our formulation accelerates convergence by up to 2

\times

according to our novel metric (mean Area Under the Convergence Curve - mAUCC), while also improving both visual quality and conditioning accuracy. Our code is available at https://github.com/CEA-LIST/x0-supervision

Abstract

Controllable Image Generation

Explore similar work

Improving Controllable Generation: Faster Training and Better Performance via x0x_0x0​-Supervision

PixelControl: Fine-Grained Condition Fidelity in Text-to-Image Diffusion

Concept Guidance: Precise, Training-Free Latent Control for Text-to-Image Generation

Improving Controllable Generation: Faster Training and Better Performance via $x_0$ -Supervision