A Lightweight Velocity Correction Framework for Improving Instruction Following in Few-Step Text-to-Image Generation

Yating Gao; Zhenlong Zhu; Wen Zihao; Lenan Zhang; Jiexiong Wang; Yang Yang; Guanjun Jiang; Yixiong Zou

cs.CVOct 9, 2026

A Lightweight Velocity Correction Framework for Improving Instruction Following in Few-Step Text-to-Image Generation

Authors: Yating Gao, Zhenlong Zhu, Wen Zihao, Lenan Zhang, Jiexiong Wang, Yang Yang, Guanjun Jiang, Yixiong Zou,

Organizations: The University of Osaka · Alibaba Group · Peking University · Huazhong University of Science and Technology

Abstract

Few-step text-to-image generators produce high-quality images with few network forward passes, yet struggle with complex prompts involving multiple subjects, interactions, spatial relations, counting, and attribute binding. Can these errors be reduced without changing the generator's parameters or increasing its forward-pass count? We propose a lightweight correction framework for a frozen 6B-parameter generator. The generator predicts velocities that determine how its intermediate image representation is updated; a corrector with fewer than 20M parameters learns residual adjustments at selected sampling steps. Because earlier corrections change the states encountered at later steps, our On-Trajectory Flow-Bridge Supervision derives residual targets at the states actually reached during corrected generation. A single corrector is shared across selected steps and runs in parallel with the generator. Only the corrector is trained, requiring neither reward optimization nor re-distillation. To our knowledge, this is the first framework for lightweight velocity-residual adaptation of fully frozen few-step text-to-image generators that preserves their original sampling schedule and forward-pass count. On our target complex-scene distribution, the framework achieves a +17.60-percentage-point NetWin over frozen Z-Image-Turbo in pairwise instruction-following evaluation, with only 26.9 ms (0.84%) additional end-to-end latency per image and no increase in backbone evaluations.

Figures & tables

Figure 1: Few-step and multi-step generation: intermediate updates and instruction following. (a) Intermediate update directions differ substantially, with angles corresponding to mean cosine similarities increasing from $37.1^{\circ}$ to $54.5^{\circ}$ . (b) Relative update magnitudes remain broadly comparable, with Turbo-to-Z-Image ratios ranging from $0.85$ to $1.46$ . (c) On 574 complex-scene prompts, 50-step Z-Image outperforms eight-step Z-Image-Turbo by +19.5 pp NetWin in pairwise instruction-following evaluation. The example highlights differences in satisfying the specified fox placement and horse stride.

Figure 2: Sparse velocity correction for few-step generation. From shared noise, Turbo (orange) and Ours (teal) generate images in eight steps, alongside a multi-step reference (gray). Ours freezes the Turbo backbone and adds velocity residuals $r_{\phi,k}$ only at steps $k\in\{2,3,4\}$ . Purple arrows $\delta_{k}$ indicate the resulting trajectory displacements; markers link intermediate states to image thumbnails. The highlighted animal in Ours better satisfies “its front and hind legs spread wide apart.” Trajectories and endpoint gaps are schematic.

Figure 3: Overview of velocity residual correction. A lightweight corrector augments a frozen few-step generator without changing its sampling schedule or forward-pass count. (a) Parallel Velocity Residual Correction. Using the current latent and cached previous-step velocity, the corrector runs alongside the generator and adds a residual to its predicted velocity. (b) On-Trajectory Flow-Bridge Supervision. A noise-to-target reference bridge defines residual targets at corrected states, combining reference-velocity correction with trajectory recovery. (c) Shared Multi-Step Learning. One corrector is trained across selected steps with local supervision and no backpropagation through sampling updates. $r_{\phi,k}$ and $r_{k}^{\star}$ denote predicted and target residuals.

Figure 4: On-Trajectory Flow-Bridge Supervision. (a) Reusing a residual target from

z_{k}^{B}

z_{k}^{C}

would cause a mismatch (red dashed). We instead recompute targets at states visited by the corrected sampler during training. (b) The total update (teal in both panels) sums the generator update (blue) and residual update (orange), which decomposes into reference-velocity correction (gold) and trajectory recovery (purple). The black dot denotes the generator-only next state

\widetilde{z}_{k+1}=z_{k}^{C}+h_{k}v_{\theta,k}^{C}

. Arrows denote state displacements.

		Target-domain	General-purpose
Method	Trainable	NetWin (pp)	Color	Shape	Texture	2D Spatial	3D Spatial	Numeracy	Non-spatial	Complex
Frozen baseline	–	0.00	0.7919	0.5731	0.7350	0.3349	0.4158	0.6738	0.3121	0.3927
SFT + LoRA	19.85M	3.83	0.7962	0.5706	0.7351	0.3359	0.4156	0.6704	0.3125	0.3947
DPO + LoRA	19.85M	1.05	0.7929	0.5722	0.7347	0.3375	0.4148	0.6757	0.3122	0.3928
D-OPSD	19.85M	-12.37	0.7597	0.5631	0.7206	0.3286	0.4147	0.6382	0.3109	0.3880
DiffCon adaptation	20.21M	1.74	0.7827	0.5843	0.7495	0.3305	0.4133	0.6838	0.3119	0.3938

Table 1: Results on Z-Image-Turbo. NetWin is measured against the frozen baseline on 574 held-out target-domain prompts. All metrics are higher-is-better; best scores are bold, and second-best scores achieved by our method are underlined.

Figure 5: Compositional errors in few-step generation and qualitative comparisons. Our method better satisfies the highlighted requirements than competing methods.

Figure 6: Step-wise qualitative comparison with frozen Z-Image-Turbo. The upper row shows the baseline and the lower row shows our method across eight sampling steps.

Method	NetWin (pp) $\uparrow$	Latency overhead (%) $\downarrow$	Added latency (ms) $\downarrow$	Memory increase (GiB) $\downarrow$
Frozen Turbo	0.00	0.00	0.0	0.00
DiffCon	+1.74	3.24	103.9	0.02
Ours	+17.60	0.84	26.9	0.04

Table 2: Target-domain NetWin and inference overhead relative to frozen Z-Image-Turbo. Bold values indicate the best results among adaptation methods.

Figure 7: Ablations on Z-Image-Turbo: (a) state–target pairing, (b) Flow-Bridge target decomposition, and (c) velocity conditioning. We report GSB proportions (%, left) and NetWin (pp, right) against frozen Z-Image-Turbo.

\dagger

marks the full-method result repeated for reference.

Panel of Figure 8 — Figure 8: Active-step ablations under eight-step Z-Image-Turbo sampling: (a) single-step and (b) multi-step correction. NetWin is evaluated against the frozen baseline on 574 held-out prompts with matched seeds. The orange dashed line denotes the default $\mathcal{S}=\{2,3,4\}$ configuration (+17.60 pp).

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Additional qualitative comparisons of compositional instruction following in few-step text-to-image generation, supplementing Fig. 5 .

T2I Generation

Text-to-image generation creates images from natural-language prompts using generative models, with research focused on fidelity, control, efficiency, and safety.

See all 299 papers

IB-Flow: Information Bottleneck-Guided CFG Distillation for Few-Step Text-to-Image Generation
Jul 10, 2026Yiting Wang, Jingyi Zhang, Wenhu Zhang +4Diffusion Model Distillation Few-Step Diffusion Sampling
Tsinghua University · Wan Team, Alibaba Group · HKUST +1
Self-Adversarial One Step Generation via Condition Shifting
Apr 14, 2026Deyuan Liu, Peng Sun, Yansen Han +3One-Step Generative Modeling T2I Diffusion Models
Westlake University · Zhejiang University · Shanghai Innovation Institute +1
Qwen-Image-Flash: Rethinking the Training Recipe for Few-Step Distillation
Jun 2, 2026Tianhe Wu, Zikai Zhou, Kun Yan +23Diffusion Model Distillation Few-Step Diffusion Sampling

Jul 10, 2026J/K move · Enter open · S save

IB-Flow: Information Bottleneck-Guided CFG Distillation for Few-Step Text-to-Image Generation

Yiting Wang, Jingyi Zhang, Wenhu Zhang, Ke Chao+3

Tsinghua University · Wan Team, Alibaba Group · HKUST+1

Diffusion Model Distillation Few-Step Diffusion Sampling T2I Diffusion Models

Open paper AskPDF arXiv

While large-scale text-to-image generative models have achieved unprecedented visual performance, their inherent reliance on multi-step iterative solvers incurs severe inference latency. Few-step distillation targeting the Classifier-Free Guidance (CFG) trajectory has emerged as the prevalent dual-dimensional compression paradigm. However, existing frameworks remain subjugated by a coarse-grained blind injection paradigm that perpetually enforces a globally static guidance strength while indiscriminately sampling the supervisor timestep. This state-agnostic design completely disregards the intrinsic nature of image generation as a dynamic evolutionary process characterized by progressive entropy reduction, which not only restricts the performance boundary of few-step compression but also precipitates severe CFG over-conditioning artifacts. To transcend these limitations, we re-examine the distillation procedure through the theoretical lens of Information Theory, formally modeling it as a dynamic mutual information game constrained by the Information Bottleneck (IB) principle. Specifically, we dismantle traditional blind assumptions via a dual-track adaptive framework. To determine the injection target, we propose an instance-aware selection mechanism that transmutes the intractable KL divergence constraint into a zero-overhead closed-form solution predicated on the local vector field norm. To regulate the injection strength, we introduce an entropy-aware schedule that dynamically decays alongside the SNR, applying maximal thrust for initial structural anchoring before smoothly reverting to the natural manifold to refine micro-details. Extensive empirical evaluations corroborate that our framework fundamentally eradicates over-conditioning artifacts, shattering the performance ceiling to achieve SOTA generative fidelity under extremely stringent 2-step configurations.

Abstract

T2I Generation

Explore similar work

IB-Flow: Information Bottleneck-Guided CFG Distillation for Few-Step Text-to-Image Generation

Self-Adversarial One Step Generation via Condition Shifting

Qwen-Image-Flash: Rethinking the Training Recipe for Few-Step Distillation