Diffusion Transformers (DiTs) have achieved strong performance in generation based video editing, but suffer from high inference latency due to the long token sequences formed by conditioning inputs and generated videos. Although training-free sparse attention reduces this cost, existing methods typically apply a fixed retention ratio or a single threshold shared across all tokens. In this work, we view video editing as conditional generation and identify a key insight: attention demand varies significantly both across different key sources (e.g., source videos, reference images, audio, text, and generated videos) and across queries within each source. Motivated by this, we propose QASK-Edit (Query-Adaptive Sparsity over Key Sources for Video Editing), a training-free sparse attention framework that allocates computation independently for each query and key source. Specifically, for each query, QASK-Edit considers both the attention mass and approximation risk of each block. It then evaluates candidate routing thresholds against each key source's risk budget, choosing exact computation or approximations to dynamically adjust sparsity for each query. Extensive experiments on three video editing models demonstrate that QASK-Edit achieves up to 1.46× end-to-end speedup over dense attention pipeline while preserving both editing quality and unedited region fidelity. Furthermore, QASK-Edit generalizes directly to Reference-to-Video generation, delivering up to 1.36× speedup while maintaining generation quality.
Figures & tables
Figure 1: Qualitative video editing results of QASK-Edit on MiniMax-H3. Three sampled frames are shown for each source–edited video pair, covering diverse editing tasks such as clothing modification, object replacement, and scene recoloring. The results demonstrate high-quality edits with preserved temporal consistency.
Figure 2: Overview of QASK-Edit. The process begins with the generation of summaries for blocks, sub-blocks, and second-level sub-blocks. For each query tile, we independently select a routing threshold for each key source within the risk budget, based on the residual risk introduced by approximate calculations. This routing threshold then determines whether to perform exact or approximate calculations. Finally, we aggregate the contributions of all key sources into a single softmax and output the final result.
Figure 3: Attention requirements vary across key sources and within each source. For each source, we report the mean minimum block count and percentage required to cover 90% of its dense attention mass, along with their average within-task standard deviations.
Method
FiVE-Bench
IVE-Bench
Efficiency
FiVE-Acc ↑
PSNR ↑
SSIM ↑
LPIPS ↓
IVE-short ↑
IVE-long ↑
Speedup ↑
StreamEdit
Dense
0.5826
23.42
0.7446
0.1199
0.6741
0.6840
1.00 ×
SpargeAttention
0.5725
23.38
0.7433
0.1205
0.6675
0.6864
1.31 ×
SVG2
0.5267
23.31
0.7407
0.1222
0.6603
0.6832
1.12 ×
SVOO
0.5593
23.36
0.7425
0.1210
0.6676
0.6828
1.29 ×
Table 1: Quantitative comparison of video editing quality and efficiency across three video editing models.
Model
Routing
Speed ↑
FiVE ↑
IVE-S ↑
IVE-L ↑
StreamEdit
Shared- β
1.30 ×
58.31
67.62
68.49
Full
1.46 ×
58.19
66.13
68.61
UniEdit-Flow
Shared- β
1.19 ×
24.70
64.44
65.59
Full
1.26 ×
24.70
64.44
65.48
MiniMax-H3
Shared- β
1.29 ×
41.54
60.13
61.24
Full
1.33 ×
45.72
60.76
62.36
Table 2: Ablation of source-specific and query-adaptive routing.
Figure 4: Sensitivity to the risk budget. Default budgets are scaled by c , and T(c) denotes generation latency. Quality changes are measured in score points relative to c=1 .
Q→K
Low
Mid
High
Qo→Ko
13.9% / 2.4%
15.2% / 0.3%
17.3% / -2.5%
Qe→Ke
11.3% / 2.3%
13.8% / 0.1%
15.7% / -2.0%
Qe→Ko
3.2% / 6.5%
15.7% / -2.0%
15.6% / -6.2%
Table 3: Relative reductions compared with the offline-calibrated source-fixed β baseline. Each entry reports attention output error reduction / exact block ratio reduction (%). Negative exact block ratio reductions indicate increased allocation. o and e denote original and edited streams.
Method
Speedup ↑
OpenS2V-Eval ↑
VACE-R2V ↑
Wan-AI/Wan2.1-VACE-1.3B
Dense
1.00 ×
45.51
75.06
SVG2
1.22 ×
45.14
74.73
SpargeAttn
1.44 ×
43.46
75.22
Sol-Attn
1.32 ×
45.18
75.10
QASK-Edit
1.36 × +36.00%
46.52 +2.22%
75.17 +0.13%
Table 4: Generalization to reference-conditioned video generation.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Layout
Block
Sub-block
Second level
Spatial
1×8×16
1×2×16
1×1×16
Temporal
8×1×16
2×1×16
1×1×16
Mixed
4×2×16
1×2×16
1×1×16
Appendix
Table S1 : Hierarchical KV layouts in (T,H,W) order. Each complete block, sub-block, and second-level sub-block contains 128 , 32 , and 16 tokens, respectively.
Backbone
Reference video
Generated video
Audio
StreamEdit
0.10
0.10
—
UniEdit-Flow
0.02
0.02
—
MiniMax-H3
0.30
0.10
0.10
Appendix
Table S2 : Default source-specific risk budgets εg for the editing experiments. A dash indicates that no audio-source budget is used in the corresponding configuration.
Benchmark
Total tasks
Proportional quota
Selected
FiVE-Bench
419
82.237
82
IVE Short
400
78.508
79
IVE Long
200
39.254
39
Total
1019
200
200
Appendix
Table S3 : Benchmark-level allocation of the 200 -task risk-budget ablation subset. Proportional quotas are shown to three decimals; allocation uses their unrounded values.
Method
Speedup
OpenS2V Open
OpenS2V Single
OpenS2V Human
VACE-R2V Norm. Avg.
Wan2.1-VACE-1.3B
Dense
1.00 ×
45.51
49.1354
49.7747
75.0641
SVG2
1.22 ×
45.14
49.7207
49.3312
74.7346
SpargeAttn
1.44 ×
43.46
44.7765
46.6589
75.2248
Sol-Attn
1.32 ×
45.18
47.6974
46.9560
75.0960
QASK-Edit
1.36 ×
46.52
49.1497
50.1084
75.1665
Appendix
Table S4 : Aggregate generalization scores and overall end-to-end speedups relative to Dense. Quality scores use a 0 – 100 scale; the VACE-R2V column retains additional precision from the normalized-score output.
Method
Subject consistency
Background consistency
Aesthetic quality
Imaging quality
Motion smoothness
Dynamic degree
Wan2.1-VACE-1.3B
Dense
0.9657
0.9581
0.5963
0.7004
0.9908
0.3152
SVG2
0.9620
0.9514
0.5920
0.6966
0.9902
0.3061
SpargeAttn
0.9619
0.9554
0.5885
0.6853
0.9904
0.3303
Sol-Attn
0.9640
0.9564
0.5924
0.6973
0.9906
0.3424
QASK-Edit
0.9637
0.9536
0.5936
0.6990
0.9908
0.3242
Appendix
Table S5 : Raw VBench metrics on OpenS2V-Eval, rounded to four decimals without percentage scaling. Subject and background denote subject consistency and background consistency, respectively.
Method
Subject consist.
Background consist.
Aesthetic quality
Imaging quality
Motion smooth.
Dynamic degree
Temporal flickering
Overall consist.
Wan2.1-VACE-1.3B
Dense
0.9869
0.9809
0.6018
0.6915
0.9957
0.0750
0.9957
0.2670
SVG2
0.9807
0.9745
0.6077
0.6915
0.9956
0.0500
0.9953
0.2661
SpargeAttn
0.9822
0.9774
0.6090
0.6771
0.9954
0.1000
0.9946
0.2749
Sol-Attn
0.9862
0.9790
0.6072
0.6915
0.9956
0.0750
0.9955
0.2674
QASK-Edit
0.9853
0.9799
0.6080
0.6950
0.9956
0.0750
0.9955
0.2679
Appendix
Table S6 : Raw VBench metrics on VACE-R2V, rounded to four decimals without percentage scaling. These per-dimension values precede the normalization used to compute the aggregate VACE-R2V score.
OpenS2V-Eval
VACE-R2V
Method
CLIP-I
DINO-I
CLIP-T
CLIP-I
DINO-I
CLIP-T
Wan2.1-VACE-1.3B
Dense
0.6130
0.3611
0.3507
0.8057
0.5616
0.3475
SVG2
0.6064
0.3524
0.3513
0.8028
0.5494
0.3491
SpargeAttn
0.5700
0.3173
0.3570
0.7522
0.5268
0.3538
Sol-Attn
0.6034
0.3485
0.3520
0.8009
0.5486
0.3496
Appendix
Table S7 : Auxiliary reference-assisted cosine similarities on OpenS2V-Eval and VACE-R2V. CLIP-I, DINO-I, and CLIP-T denote the CLIP image, DINO image, and CLIP text cosine-similarity fields, respectively. Values retain their original scale and are rounded to four decimals.