In
DiT-based video generation models equipped with 3D Rotary Position Embeddings (3D RoPE), the attention mechanism remains a primary computational bottleneck due to its quadratic complexity with respect to sequence length. While quantized
FlashAttention offers a promising path toward hardware acceleration, existing low-bit quantization methods overlook two critical challenges in this setting:
1) applying online rotation matrices -- a widely used technique for mitigating outliers in Queries (
Q) and Keys (
K) -- is difficult to reconcile with
RoPE; and
2) the non-negative attention matrix
P=exp(QK−max(QK)) makes symmetric quantization waste half of the 4-bit dynamic range. In this work, we observe that the outlier distributions of
Q and
K are strongly affected by the dimensional partitioning of
3D RoPE. Based on this finding, we propose
RotateAttention, an efficient
mixed-precision INT4 FlashAttention framework tailored for
DiT-based video generation models with 3D RoPE, using selective
FP16 fallback for accuracy-sensitive attention blocks and denoising steps. RotateAttention introduces two core techniques:
1) RoPE-aware Rotation, which employs either mergeable rotation matrices that can be fused into RoPE or negligible-overhead matrices to mitigate RoPE-induced outliers in
Q and
K; and
\textbf{2) Range-optimized P
Quantization}, which uses fixed scales and zero-points to fully exploit the
INT4 numerical range with minimal computational overhead. Experiments show that
RotateAttention preserves video generation quality nearly identical to full-precision baselines while achieving up to 1.68
× end-to-end speedup and 2.2
× kernel-level acceleration.