QK-Edit: Revisiting Attention-based Injection in MM-DiT for Image and Video Editing

Tiancheng Shen; Zilong Huang; Xiangtai Li; Zhijie Lin; Jiyang Liu; Yitong Wang; Jiashi Feng; Ming-Hsuan Yang

cs.CVOct 13, 2026

QK-Edit: Revisiting Attention-based Injection in MM-DiT for Image and Video Editing

Authors: Tiancheng Shen, Zilong Huang, Xiangtai Li, Zhijie Lin, Jiyang Liu, Yitong Wang, Jiashi Feng, Ming-Hsuan Yang,

Organizations: UC Merced · ByteDance Seed · ByteDance Intelligent Creation · CUHK

Abstract

Multimodal Diffusion Transformers (MM-DiTs) have recently emerged as a powerful framework for unified text-vision synthesis, surpassing traditional U-Net architectures in generative tasks. One key innovation lies in its Multimodal Self-Attention (MM-SA) interaction, where image and text tokens are concatenated and processed via self-attention. However, this mechanism poses significant challenges for editing, rendering conventional U-Net-based attention manipulation methods ineffective. To address this limitation, we propose QK-Edit, a training-free framework that exploits the unique attention dynamics of MM-DiTs for precise text-guided image and video editing. By introducing a novel query-key manipulation strategy, our method isolates and adjusts critical attention components to achieve an optimal balance between prompt fidelity and structural consistency. This enables seamless edits across various tasks, including object addition, object removal, object replacement, changing background, changing material, changing color, and style transformation. Notably, it can be easily implemented with feature replacement in inference. QK-Edit demonstrates superior editing performance on state-of-the-art models, such as FLUX and HunyuanVideo, effectively bridging the gap between generative power and editable flexibility in MM-DiTs, and paving the way for scalable multimodal content creation.

Figures & tables

Figure 2: Directly applying CA-based and SA-based editing techniques designed for U-Net diffusion models to MM-DiTs can lead to undesirable results.

Figure 3: Removing individual blocks in the MM-DiT architecture affects different aspects of the generated video.

Figure 4: We study the role of the Multimodal Self-Attention (MM-SA) and Feed-Forward Network (FFN) components in MM-DiT blocks by bypassing one at the time. The results underscore that MM-SA predominantly determines spatial and temporal consistency, while FFN has a negligible impact.

Figure 5: Pilot experiments. (left) MM-SA attention map. (middle) prompting the influence of each attention quadrant by setting it to $\mathbf{0}$ . (right) visualization of attention map of the 1 st and 3 rd quadrants and PCA feature of the 4 th quadrant.

Figure 6: Query-key manipulation strategy. By substituting the vision components in Q and K, the approach effectively guides the editing process to balance prompt fidelity with preservation of the source structure.

Method	CLIP-T↑	DINO↑	PickScore↑
LEDITS++ [ 8 ]	0.207	0.571	19.88
MasaCtrl [ 11 ]	0.235	0.696	20.68
RF-Inversion [ 56 ]	0.252	0.523	21.31
RF-Solver [ 62 ]	0.257	0.769	21.98
QK-Edit	0.264	0.776	22.33

Table 1: Real image editing evaluation on PIE-Bench .

Figure 7: Visualizations showcasing QK-Edit’s capabilities in various video editing tasks.

Method	CLIP ${}_{\texttt{frames}}$ ↑	PickScore↑	ViCLIP ${}_{\texttt{dir}}$ ↑	ViCLIP ${}_{\texttt{out}}$ ↑
DiTCtrl [ 10 ]	0.904	20.01	0.141	0.242
QK-Edit	0.951	21.24	0.217	0.268

Table 2: Generative video editing performance on TGVE+ dataset.

Method	ViCLIP ${}_{\texttt{dir}}$ ↑	ViCLIP ${}_{\texttt{out}}$ ↑	CLIP ${}_{\texttt{frames}}$ ↑	PickScore↑
CCEdit [ 18 ]	0.159	0.244	0.877	19.73
Flatten [ 15 ]	0.204	0.236	0.887	19.91
AnyV2V [ 35 ]	0.117	0.228	0.906	20.28
QK-Edit	0.196	0.258	0.921	20.52

Table 3: Real video editing performance on TGVE+ dataset.

Figure 8: Manipulation of different QKV combinations. Replacing $KV$ or $V$ preserves original structure better, but sometimes fails to follow the target prompt; replacing $Q$ or $K$ , on the hand, adheres to the editing prompt but often fail to preserve source video structure; replacing $QK$ achieves the best trade-off.

Method	KV ${}_{\text{concat}}$	V ${}_{\text{replace}}$	KV ${}_{\text{replace}}$	QK ${}_{\text{replace}}$
CLIP ${}_{\texttt{frames}}$ ↑	0.896	0.900	0.901	0.908
PickScore↑	19.87	19.94	19.92	20.08

Table 4: Quantitative results on HiLo-Benchmark . Our proposed QK-Edit via QK

{}_{\text{replace}}

method (last column) works the best.

Figure 9: The pseudo code of QK-Edit.

Figure 11: Additional image editing results.

Diffusion-Based Image Editing

Diffusion-based image editing changes image content, style, or attributes while aiming to preserve the parts users want to keep.

See all 225 papers

UniVideo: Unified Understanding, Generation, and Editing for Videos
Oct 9, 2025Cong Wei, Quande Liu, Zixuan Ye +5Video Generation Unified Multimodal Models
University of Waterloo · Kling Team, Kuaishou Technology
AttnRouter: Per-Category Attention Routing for Training-Free Image Editing on MMDiT
May 2, 2026Guandong Li, Mengxia YeDiffusion Transformer Text-Guided Image Editing
iFLYTEK · Aegon THTF
MMCORE: MultiModal COnnection with Representation Aligned Latent Embeddings
Apr 21, 2026Zijie Li, Yichun Shi, Jingxiang Sun +8Unified Multimodal Models Conditional Image Generation
ByteDance Seed

Oct 9, 2025J/K move · Enter open · S save

UniVideo: Unified Understanding, Generation, and Editing for Videos

Cong Wei, Quande Liu, Zixuan Ye, Qiulin Wang+4

University of Waterloo · Kling Team, Kuaishou Technology

Video Generation Unified Multimodal Models Video Editing Image-to-Video Generation Multimodal Diffusion Models Instruction-Based Video Editing+1

Open paper AskPDF arXiv

Unified multimodal models have shown promising results in multimodal content generation and editing but remain largely limited to the image domain. In this work, we present UniVideo, a versatile framework that extends unified modeling to the video domain. UniVideo adopts a dual-stream design, combining a Multimodal Large Language Model (MLLM) for instruction understanding with a Multimodal DiT (MMDiT) for video generation. This design preserves the MLLM's original text generation capabilities, enables accurate interpretation of complex multimodal instructions, and maintains visual consistency in the generated content. Built on this architecture, UniVideo unifies diverse video generation and editing tasks under a single multimodal instruction paradigm and is jointly trained across them. Extensive experiments demonstrate that UniVideo matches or surpasses state-of-the-art task-specific baselines in text/image-to-video generation, in-context video generation and in-context video editing. Notably, the unified design of UniVideo enables two forms of generalization. First, UniVideo supports task composition, such as combining editing with style transfer, by integrating multiple capabilities within a single instruction. Second, even without explicit training on free-form video editing, UniVideo transfers its editing capability from large-scale image editing data to this setting, handling unseen instructions such as changing the environment or altering materials within a video. Beyond these core capabilities, UniVideo also supports visual-prompt-based video generation, where the MLLM interprets visual prompts and guides the MMDiT during synthesis. To foster future research, we released our model and code.

Abstract

Diffusion-Based Image Editing

Explore similar work

UniVideo: Unified Understanding, Generation, and Editing for Videos

AttnRouter: Per-Category Attention Routing for Training-Free Image Editing on MMDiT

MMCORE: MultiModal COnnection with Representation Aligned Latent Embeddings