AViTS: Adaptive Spatiotemporal Token Selection for Efficient Dynamic-Resolution Generation

Haoran Qin; Zhengan Yan; Shikang Zheng; Xiaobing Tu; Jiacheng Liu; Yuqi Lin; Chang Zou; JinShan Liu

cs.CVAug 18, 2026

AViTS: Adaptive Spatiotemporal Token Selection for Efficient Dynamic-Resolution Generation

Authors: Haoran Qin, Zhengan Yan, Shikang Zheng, Xiaobing Tu, Jiacheng Liu, Yuqi Lin, Chang Zou, JinShan Liu, +4 more

Organizations: Shanghai Jiao Tong University, China · Shandong University, China · Terminal Intelligent Computing Division, Alibaba Cloud, China · Jilin University, China · Xi’an Jiaotong University, China

Abstract

Diffusion Transformers (DiTs) achieve high-quality generation but are costly due to iterative sampling. Dynamic-resolution sampling reduces early-stage cost by denoising at low resolution; however, uniformly upsampling all latent tokens at resolution transitions incurs redundant computation and may degrade fine-detail consistency. Existing partial upsampling strategies typically rely on local latent structure cues or single-step statistics, making it difficult to jointly capture token-text semantic relevance and token-wise representation dynamics across diffusion steps. We propose AViTS, an adaptive spatiotemporal token selection framework for dynamic-resolution DiTs. AViTS models spatial importance via latent-text attention and temporal importance via token-level feature variation across diffusion timesteps, and fuses them to enable spatiotemporal importance-aware selective upsampling: it prioritizes resolution refinement for critical tokens while deferring less important ones, thereby reducing redundant high-resolution computation and improving the quality-efficiency trade-off. AViTS achieves up to 6.34x on FLUX and nearly 9x FLOPs reduction on Qwen-Image-Edit and FLUX.1-Kontext-dev, orthogonal to distillation, quantization, and feature caching, and reaching 14.76x with distilled models. Code: https://github.com/QHR69/AViTS

Figures & tables

Figure 1: Images sampled by FLUX.1-dev with AViTS with 6.34× acceleration.

Figure 2: Token-importance distributions. (a) Distributions of attention and step variance on FLUX.1-dev. (b) Distributions of attention and step variance on Qwen-Image-Edit.

Figure 3: Heatmaps of token importance on two text-to-image examples. Attention-based selection covers instruction-relevant semantic regions, whereas step-variance-based selection focuses on regions requiring fine-grained synthesis.

Figure 4: Overview of AViTS and spatiotemporal importance estimation. (a) Three-stage dynamic-resolution sampling: Stage 1 low-res denoising with signal collection over the last $N_{T}$ steps, Stage 2 selective upsampling of top- $K$ tokens, and Stage 3 full-res refinement. (b) Temporal importance from token-wise step variance across the collected snapshots. (c) Spatial importance from aggregated latent–text cross-attention (averaged over heads/blocks and collection steps); fused scores guide prioritization.

Figure 5: Token-importance maps for editing. RALU/Fresco tend to allocate importance to irrelevant regions (edges or diffuse background), which leads to semantic misalignment (Row 1), degraded color/style consistency (Row 2), and inconsistency in non-edited regions (Row 3), whereas AViTS concentrates tokens on the instruction-relevant edit areas.

Method	NFE	Acceleration				Image Reward $\uparrow$	CLIP Score $\uparrow$
Method	NFE	Latency(s) $\downarrow$	Speed $\uparrow$	FLOPs(T) $\downarrow$	Speed $\uparrow$	Image Reward $\uparrow$	CLIP Score $\uparrow$
FLUX.1-dev	50	25.78	1.00 $\times$	3719.50	1.00 $\times$	0.9719 (+0.00%)	32.325 (+0.00%)
$60\%$ steps	30	16.63	1.55 $\times$	2231.70	1.67 $\times$	0.9646 (-0.75%)	32.232 (-0.28%)
SpargeAttention	50	15.44	1.67 $\times$	2150.02	1.73 $\times$	0.9795 (+0.78%)	32.312 (-0.04%)
TeaCache $({l}=0.25)$	50	14.09	1.83 $\times$	1937.24	1.92 $\times$	0.9442 (-2.85%)	32.167 (-0.49%)
TaylorSeer $(\mathcal{N}=3)$	50	9.88	2.61 $\times$	1320.07	2.82 $\times$	0.9861 (+1.46%)	32.313 (-0.04%)

Table 1: Quantitative comparison of text-to-image results on FLUX.1-dev.

Figure 6: Visualization of the image generated by different methods on FLUX.1-dev . AViTS achieves the best semantic fidelity and the fastest speed (4.80×), surpassing all dynamic-resolution and feature-caching baselines.

Method	NFE	Acceleration				GEdit-EN(FULL)
Method	NFE	Latency ( s ) $\downarrow$	Speed $\uparrow$	FLOPs ( T ) $\downarrow$	Speed $\uparrow$	SC $\uparrow$	PQ $\uparrow$	OS $\uparrow$
$100\%$ steps	50	50.20	1.00 $\times$	8299.54	1.00 $\times$	6.80	7.26	6.51
$60\%$ steps	30	32.23	1.56 $\times$	4979.72	1.67 $\times$	6.54	7.28	6.25
$20\%$ steps	10	10.47	4.79 $\times$	1659.91	5.00 $\times$	6.60	7.18	6.28
SpargeAttention	50	46.05	1.09 $\times$	5603.68	1.48 $\times$	6.46	7.25	6.21
RALU	30	23.34	2.15 $\times$	2730.53	3.04 $\times$	7.06	7.20	6.70

Table 2: Quantitative evaluation results of image editing on the GEdit-bench with FLUX.1-Kontext-dev .

Method	NFE	Acceleration				GEdit-CN(FULL)			GEdit-EN(FULL)
Method	NFE	Latency ( $s$ ) $\downarrow$	Speed $\uparrow$	FLOPs (T) $\downarrow$	Speed $\uparrow$	SC $\uparrow$	PQ $\uparrow$	OS $\uparrow$	SC $\uparrow$	PQ $\uparrow$	OS $\uparrow$
$100\%$ steps	50	284.51	1.00 $\times$	28219.71	1.00 $\times$	7.68	7.51	7.41	7.82	7.54	7.54
$60\%$ steps	30	172.43	1.65 $\times$	16931.83	1.67 $\times$	7.70	7.53	7.44	7.77	7.52	7.47
$20\%$ steps	10	58.66	4.85 $\times$	5638.18	5.01 $\times$	7.65	7.42	7.35	7.73	7.46	7.44
SpargeAttention	50	231.30	1.23 $\times$	16846.78	1.67 $\times$	7.87	7.57	7.56	7.81	7.53	7.50
Bottleneck	30	109.38	2.60 $\times$	9954.36	2.83 $\times$	7.44	7.59	7.28	7.62	7.40	7.24

Table 3: Quantitative evaluation results of image editing on the GEdit-bench with Qwen-Image-Edit .

Figure 7: Qualitative comparison on GEdit-Bench with Qwen-Image-Edit. AViTS achieves superior semantic preservation and visual quality at a higher acceleration ratio ( 5.24 $\times$ ) compared to state-of-the-art baselines.

Method	NFE	Acceleration				CLIP-IQA $\uparrow$	CLIP Score $\uparrow$
Method	NFE	Latency(s) $\downarrow$	Speed $\uparrow$	FLOPs(T) $\downarrow$	Speed $\uparrow$	CLIP-IQA $\uparrow$	CLIP Score $\uparrow$
FLUX.1-dev [ 9 ]	50	25.78	1.00 $\times$	3719.50	1.00 $\times$	0.8494 (+0.00%)	33.402 (+0.00%)
FreqCa(N=4) [ 13 ]	50	8.38	$3.08\times$	1116.32	$3.33\times$	0.8472 (-0.26%)	32.255 (-3.43%)
AViTS +Feature Caching	18	3.02	8.54 $\times$	412.26	9.02 $\times$	0.8217 (-3.26%)	32.47 (-2.79%)
FLUX.1-lite-8B [ 24 ]	28	9.16	2.81 $\times$	1291.49	2.88 $\times$	0.8617 (+1.45%)	32.52 (-2.64%)
AViTS +Model Distillation	14	3.09	8.34 $\times$	438.76	8.48 $\times$	0.8484 (-0.12%)	31.87 (-4.59%)

Table 4: Quantitative comparison of text-to-image generation with other accleration methods.

Panel of Table 5 — Table 5: Quantitative comparison of text-to-image gen. on FLUX.1-dev.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Ablation on fusion ratio and upsampling ratio. (a–c) Impact of the fusion weight $\alpha$ (spatial attention vs. temporal step-variance) on three models: FLUX.1-dev (ImageReward), Qwen-Image-Edit (Overall score), and FLUX.1-Kontext-dev (Overall score). $\alpha{=}0$ uses only step-variance, $\alpha{=}1$ uses only attention. AViTS remains strong across a wide range of $\alpha$ and outperforms TaylorSeer at comparable speedups (shown in legends), indicating complementarity between semantic alignment and cross-step dynamics. (d) Speed–quality trade-off on FLUX.1-dev by varying the upsampling ratio $\rho$ : increasing $\rho$ improves ImageReward but reduces latency speedup, revealing a clear Pareto frontier for selecting the high-resolution budget.

Resolution	Method	Latency(s) $\downarrow$	Speed $\uparrow$	ImageReward $\uparrow$
1024 $\times$ 1024	FLUX	25.78	1.00 $\times$	0.9719
1024 $\times$ 1024	AViTS	4.73	5.45 $\times$	0.9959
1440 $\times$ 1440	FLUX	52.32	1.00 $\times$	0.7385
1440 $\times$ 1440	AViTS	9.72	5.38 $\times$	0.9008
2048 $\times$ 2048	FLUX	120.03	1.00 $\times$	0.8371
2048 $\times$ 2048	AViTS	22.87	5.25 $\times$	0.8905

Appendix

Table 6: Comparison at different resolutions.

Figure 10: Qualitative comparison for high-resolution generation (2048 $\times$ 2048). Compared with the original FLUX sampling at the target resolution, AViTS produces more faithful and coherent results, with better color consistency and fewer artifacts, especially for prompts involving text rendering and multi-object relations.

Figure 11: Additional visualizations of AViTS token selection for text-to-image generation. For each prompt, we show the generated result and the corresponding AViTS-focused token regions (highlighted overlays) selected by the final spatiotemporal importance score. AViTS consistently assigns more high-resolution budget to semantically critical regions (e.g., object parts/attributes, text-bearing areas, and salient regions), showing strong correlation between token prioritization and prompt intent.

Figure 12: Additional editing examples and token-allocation comparison. We compare AViTS with Fresco and RALU on diverse editing instructions. Along with the edited outputs, we visualize the tokens prioritized for high-resolution refinement. AViTS focuses on instruction-relevant regions (e.g., the target object/attribute to be modified) and assigns fewer tokens to non-edited areas, whereas Fresco/RALU often over-allocate tokens to edges or diffuse background regions, which can lead to weaker edit fidelity or reduced consistency outside the edited region.

Explore similar work

DDiT: Dynamic Patch Scheduling for Efficient Diffusion Transformers
Feb 19, 2026Dahye Kim, Deepti Ghadiyaram, Raghudeep GaddeDiffusion Model Inference Acceleration Diffusion-Based Image Generation
Boston University · Amazon
Level-of-Token Diffusion
Oct 5, 2026Kiyohiro Nakayama, Brian Chao, Jan Ackermann +5Video Diffusion Models Diffusion Transformer
Stanford University · Google
Not All Tokens Need 40 Steps: Heterogeneous Step Allocation in Diffusion Transformers for Efficient Video Generation
May 7, 2026Ernie Chu, Vishal M. PatelVideo Diffusion Models Diffusion Transformer
Johns Hopkins University Baltimore, MD 21218

Feb 19, 2026J/K move · Enter open · S save

DDiT: Dynamic Patch Scheduling for Efficient Diffusion Transformers

Dahye Kim, Deepti Ghadiyaram, Raghudeep Gadde

Boston University · Amazon

Diffusion Model Inference Acceleration Diffusion-Based Image Generation Diffusion Transformer Video Diffusion Models

Open paper AskPDF arXiv

Diffusion Transformers (DiTs) have achieved state-of-the-art performance in image and video generation, but their success comes at the cost of heavy computation. This inefficiency is largely due to the fixed tokenization process, which uses constant-sized patches throughout the entire denoising phase, regardless of the content's complexity. We propose dynamic tokenization, an efficient test-time strategy that varies patch sizes based on content complexity and the denoising timestep. Our key insight is that early timesteps only require coarser patches to model global structure, while later iterations demand finer (smaller-sized) patches to refine local details. During inference, our method dynamically reallocates patch sizes across denoising steps for image and video generation and substantially reduces cost while preserving perceptual generation quality. Extensive experiments demonstrate the effectiveness of our approach: it achieves up to

3.52\times

and

3.2\times

speedup on FLUX-1.Dev and Wan

2.1

, respectively, without compromising the generation quality and prompt adherence.

Abstract

Explore similar work

DDiT: Dynamic Patch Scheduling for Efficient Diffusion Transformers

Level-of-Token Diffusion

Not All Tokens Need 40 Steps: Heterogeneous Step Allocation in Diffusion Transformers for Efficient Video Generation