Organizations: Peking University, School of Computer Science, Beijing, China · Peking University, School of Electronics Engineering and Computer Science, Beijing, China · Beijing Institute of Technology, School of Computer Science & Technology, Beijing, China · Peking University, Beijing, China
Diffusion Transformers (DiTs) enable on-device image generation and editing but incur high attention costs at high resolutions. Existing sparse attention reduces computation, yet dynamic sparsity introduces preprocessing and irregular data accesses that poorly fit mobile NPUs. We present DiT.XPU, a sparse DiT inference framework for heterogeneous mobile SoCs. DiT.XPU combines tile-aligned sparse attention with anchor-based error compensation by exploiting slowly varying dense-sparse output residuals across denoising steps. Heterogeneous XPU collaboration overlaps NPU dense attention with GPU sparse calibration, while mask-guided dataflow reduces data transfers and layout conversion by overlapping data movement with computation. Across five DiT models on two Snapdragon smartphones, DiT.XPU achieves up to 1.56x end-to-end denoising speedup over the Hexagon-NPU backend and 11.5x attention-kernel speedup over the fastest alternative, while preserving generation quality close to dense attention. Code is available at https://github.com/IF-LAB-PKU/dit.xpu.
Figures & tables
图 1. Attention dominates high-resolution DiT latency. It accounts for 32%, 60%, and 86% of the SD3.5 Medium DiT runtime at 512, 1024, and 2048 resolution, respectively, and 64% for Z-Image Turbo at 2048.
图 2. Attention patterns across DiT models. The first column shows the full Q–K attention map of a representative layer, while the remaining columns visualize the spatial attention distributions of selected query tokens.
图 3. (a) Dynamic sparse attention workflow. (b) Static sparse attention workflow. The image K/V blocks are requested from the dynamically selected blocks in (a), and from the fixed mask in (b).
图 4. Architecture of a mobile SoC. Left: the CPU, GPU, and Hexagon NPU with their compute and memory resources. Right: the tensor stored in tile-first order, and the storage order within a 32×32 tile.
图 5. Dynamic and static sparse attention kernel breakdown for SD3.5 Medium at 1024*1024 on 8Gen4 NPU.
图 6. Calibration coefficient ρ and residual MSE across denoising steps for SD3.5 Medium at 1024×1024 resolution.
GenEval (%) ↑
HPSv2.1 ↑
ImageReward ↑
Model
Dense
Static
Dense
Static
Dense
Static
SD3.5 Medium
68.37
26.83
29.18
20.46
0.8774
−0.7486
SD3.5 Large
71.31
23.66
29.91
19.35
0.8781
−1.3234
Z-Image
75.35
65.88
27.74
25.50
0.6936
0.2371
Z-Image-Turbo
75.49
63.14
29.83
28.59
0.6960
0.4741
FLUX.2 [klein]
82.66
70.60
30.09
28.39
0.9108
0.5522
表 1. Static block-sparse attention substantially degrades generation quality across five DiT models.
Measurement
Per-query loading
Cross-query reuse
dit.xpu
DRAM read volume (MB)
456.3
216.4
51.36
Mean DMA request size (KB)
15.33
5.32
7.42
Data-wait time (ms)
4.52
2.11
0.02
K/V repacking time (ms)
5.45
3.40
0
Kernel latency (ms)
24.96
21.35
13.30
表 2. K/V data movement and kernel latency for SD3.5 Medium static sparse attention on the 8Gen4 NPU.
图 7. Overview of dit.xpu .
图 8. An illustration of the heterogeneous XPU collaboration.(1) sequential execution with CPU fallback, (2) NPU–GPU co-execution with explicit data copies, and (3) NPU–GPU co-execution through shared memory mappings.
图 9. (a) Naive Dataflow with repeated K/V loading and layout conversion; (b) Mask-Guided Dataflow with K/V reuse, prefetching, and fused QKV packing.
图 10. End-to-end denoising latency of dit.xpu and baseline backends across DiT models and mobile SoCs.
图 11. Attention kernel latency of dit.xpu and the baseline backends across DiT models and mobile SoCs.
Model
Architecture
Params
Heads
Head Dim
Layers
Steps
SD3.5 Medium
MMDiT-X
2.5B
24
64
24
20
SD3.5 Large
MMDiT
8.1B
38
64
38
20
Z-Image
S3-DiT
6B
30
128
32
15
Z-Image Turbo
S3-DiT
6B
30
128
32
8
FLUX.2 [klein]
RF Transformer
4B
24
128
25
4
表 3. Architecture and sampling configuration of the evaluated DiT models.
Latency (s) ↓
Dense
PISA
Sparge
Static
Ours
Speedup
SD3.5 Medium
579
1446
876
325
371
1.56×
SD3.5 Large
1579
3183
2185
1385
1206
1.31×
Z-Image
919
3030
1144
536
631
1.46×
Z-Image-Turbo
483
1239
616
292
326
1.48×
FLUX.2-klein
186
433
295
147
154
1.21×
Mean Speedup
1.00×
0.40×
0.72×
1.51×
1.40×
表 4. End-to-end denoising latency (s) of dense attention and three sparse-attention strategies at 1024×1024 resolution on the Snapdragon 8 Gen 4.
Model
CPU Dense
GPU Dense
GPU Sparse
NPU Dense
GPU/NPU Anchor
NPU Sparse
SD3.5 Medium
18.32
2.35
0.48
0.45
0.76
0.05
SD3.5 Large
12.88
3.77
0.77
0.79
1.18
0.08
FLUX.2-klein
12.17
4.20
0.23
0.53
0.72
0.07
Z-Image
15.16
5.37
0.28
0.73
0.90
0.09
Z-Image-Turbo
14.98
5.32
0.27
0.71
0.85
0.08
表 5. Single-kernel energy consumption across DiT models and execution configurations.
图 12. Qualitative comparison of dit.xpu and the baselines on three prompts, one per row: SD3.5-Medium, SD3.5-Large and Z-Image.
GenEval ↑
Dense
PISA
Sparge
Static
Ours
Δ
SD3.5 Medium
68.37
64.55
68.19
26.83
68.19
-0.18
SD3.5 Large
71.31
70.06
70.54
23.66
71.88
+0.57
Z-Image
75.35
73.30
75.13
65.88
76.26
+0.91
Z-Image-Turbo
75.49
72.49
75.47
63.14
76.54
+1.05
FLUX.2-klein
82.66
84.87
82.40
70.60
83.01
+0.35
Avg. Δ
–
-1.58
-0.29
-24.61
+0.54
表 6. Generation quality comparison across attention strategies. FID and CLIP are averaged over Landscape, People, and Food. Δ denotes the absolute change relative to Dense. For ↑ metrics, positive values indicate improvement; for FID ↓ , negative values indicate improvement.
图 13. Component ablation. Each build adds one technique to the previous one.
p
GenEval ↑
HPSv2.1 ↑
ImageReward ↑
Latency (s) ↓
2
68.36
29.63
0.8813
412.32
3
68.31
29.60
0.8725
394.88
4
68.24
29.55
0.8604
386.40
5
68.19
29.53
0.8575
371.11
6
66.07
29.42
0.8219
367.09
表 7. Generation quality and latency of dit.xpu under different anchor step intervals p on SD3.5 Medium at 1024×1024 resolution and 20 steps.
图 14. Heterogeneous-strategy ablation for a single denoising step on the Snapdragon 8 Gen 4 device.
图 15. KV load ablation of the attention kernel on the Snapdragon 8 Gen 4 device.
1920×1080
1080×1920
896×1152
1152×896
Dense
1565.88
1556.36
493.32
495.24
dit.xpu
786.14
803.97
324.48
317.02
表 8. Latency of SD3.5 Medium at different resolutions.
Method
GenEval ↑
Latency (s) ↓
Speedup ↑
Dense
0.6837
579
1.00 ×
ours
0.6819
371.11
1.56 ×
w/ TeaCache
0.6451
361.57
1.60 ×
w/ DBCache
0.6399
333.89
1.73 ×
w/ TaylorSeer
0.6423
333.14
1.74 ×
表 9. GenEval scores, inference latency and speedup of orthogonal caching on the DiT.xPU backend for the SD3.5-medium model at 1024×1024 resolution.