Text-to-image diffusion models have advanced video editing. However, existing video editing methods are limited by their high computational overhead and memory consumption. Furthermore, these approaches often sacrifice visual fidelity, leading to temporal inconsistencies and artifacts such as blurring and mosaic-like patterns. To address these dual challenges and strike a balance between computational efficiency and visual fidelity, we propose Edit-Your-Interest, a lightweight, text-driven, zero-shot video editing method. Edit-Your-Interest introduces a Spatio-Temporal Feature Memory bank (SFM) to cache spatial-attention features from previous frames under a fixed memory budget. The SFM maintains a representative candidate pool, reducing the computational overhead of full-sequence spatio-temporal modeling. To query this memory, we propose Feature Most-Similar Propagation (FMP), a similarity-based retrieval rule that propagates relevant cached tokens to the current frame. Together, SFM and FMP form a bounded-memory retrieval-based propagation design that improves temporal consistency while reducing blurring and mosaic-like artifacts. Finally, we introduce a feature-diversity-based SFM update algorithm to keep the memory representative and non-redundant throughout the video sequence. Furthermore, to enable precise object editing, we leverage cross-attention maps to automatically extract masks for the instances of interest. These masks are seamlessly integrated into the diffusion denoising process, enabling fine-grained control over target objects and allowing Edit-Your-Interest to perform highly accurate edits while robustly preserving the background integrity. Extensive experiments demonstrate that Edit-Your-Interest outperforms state-of-the-art methods in both efficiency and visual fidelity. Project page: https://edit-your-interest.github.io/.
Figures & tables
Figure 1: We propose Edit-Your-Interest, a zero-shot video editing method that supports both low-cost global editing (left) and precise instance local editing (right), while effectively preserving the entire background.
Figure 2: The pipeline of Edit-Your-Interest. (Top) We employ DDIM inversion to obtain the initial latent noise and then denoise the sequence via DDIM sampling. (Bottom left) We construct a SFM to cache frame feature tokens. Subsequently, FMP propagates them to the current frame. (Bottom right) We introduce an Automatic Mask Extraction and Injection Strategy. This in-diffusion injection effectively suppresses boundary artifacts between foreground and background regions.
Figure 3: Examples of our method’s results in instance object editing and global editing. As shown, our approach enables not only precise local object editing but also global editing.
Figure 4: Comparison results between our method and direct replacement using semantic masks. It can be seen that our method mitigates boundary artifacts between foreground and background.
Figure 5: Qualitative Comparisons. The results of our proposed method and other state-of-the-art video editing methods are shown. It can be observed that our method not only successfully edits the target but also effectively preserves the background regions.
Method
Text Alignment
Foreground
Background preservation
Global
Fidelity
CLIP-T
CLIP-F
Warp-err
SSIM
PSNR
CLIP-F
Warp-err
FID
FateZero [ 29 ]
31.05
95.98
5.77
81.91
21.49
94.76
6.80
289.92
DMT [ 47 ]
30.78
98.50
1.21
52.35
16.67
98.39
1.15
214.99
GAV [ 15 ]
27.82
96.45
2.11
68.96
17.51
96.44
4.91
243.28
TokenFlow [ 7 ]
31.32
97.03
1.24
77.65
21.31
98.51
1.38
162.59
STEM [ 21 ]
29.89
97.89
1.48
71.79
17.52
98.48
3.47
170.46
Table 1: Quantitative comparison of automatic metrics. Bold indicates the best result. Background preservation denotes background preservation, Foreground denotes foreground coherence, and Global denotes overall video quality.
Method
8 frames
16 frames
32 frames
GPU
RAM
Runtime
GPU
RAM
Runtime
GPU
RAM
Runtime
FateZero [ 29 ]
18.57
71.13
154
27.34 ∗
144.21 ∗
517 ∗
-
-
-
GAV [ 15 ]
17.87
6.87
93
25.40
6.99
242
29.41
7.27
721
DMT [ 47 ]
19.96
3.06
316
30.99
3.07
521
51.97
3.08
935
TokenFlow [ 7 ]
9.64
2.59
102
11.33
2.78
195
11.42
2.82
403
STEM [ 21 ]
9.78
2.84
52
11.46
2.90
97
11.57
2.93
198
Table 2: Comparison of Runtime and Memory Consumption. GPU denotes the GPU memory usage (in GB), RAM denotes the system memory usage (in GB), and Runtime indicates the time required to edite a video (in seconds). Values marked with an asterisk ( ∗ ) are adapted from [ 46 ]
Figure 6: The example of the scoring interface for the user study. Participants are required to select a specific score from the provided table for each video.
Method
LV (mean)
LV (std)
HFR
PSD
GE
DCT-E
Source video
413.50
32.30
4.54×10−3
6.45
3.74
0.88
TokenFlow
596.53
132.99
7.23×10−3
9.73
3.56
0.89
τ=1.0
377.05
57.95
5.24×10−3
6.75
3.61
0.84
τ=0.5
367.94
57.08
5.15×10−3
6.74
3.58
0.82
τ=0.1
378.89
57.28
5.33×10−3
6.75
3.60
0.85
τ=0.01
379.58
58.06
5.45×10−3
6.72
3.61
0.86
Table 3: Representation-level analysis with temperature sweep. For all metrics except LV mean and GE, values closer to source indicate less artifact-induced distortion.
Figure 7: Comparison between FMP and interpolation-like weighted propagation methods. It can be observed that FMP mitigates blurring and mosaic-like artifacts, thereby enhancing the fidelity of edited videos.
Method
Text
Foreground
Fidelity
Override
CLIP-T ↑
CLIP-F ↑
Warp-err ↓
FID ↓
Keyframe candidates + weighted
31.32
97.03
1.24
162.59
N/A
Keyframe candidates + hard
31.46
97.15
1.22
144.75
0.6832
SFM candidates + softmax ( τ=0.5 )
32.20
98.14
1.18
124.03
0.8050
SFM candidates + hard
32.29
97.92
1.19
120.12
0.8031
Table 4: Quantitative comparison of candidate-pool construction and propagation strategies. Override denotes the fraction of tokens propagated from the candidate pool after passing the confidence threshold.
Figure 8: Visualization of temporal mask overlap strategy. This demonstrates that the temporal mask overlap strategy can effectively generate robust instance masks.
λ
Warp-err
τ
PSNR b
SFM-L
Warp-err
0.7
3.92
0.2
28.17
1
1.57
0.8
2.42
0.3
29.26
3
1.33
0.9
1.19
0.4
29.03
5
1.19
0.95
1.44
0.5
27.04
7
1.21
Table 5: Hyperparameters Analysis: the similarity threshold λ , the mask threshold τ , and the length of SFM (SFM-L).
Method
TC
TA
Quality
FateZero
67.50
73.5
71.75
DMT
72.75
69.75
79.75
GAV
71.75
81.50
69.50
TokenFlow
88.75
88.00
80.95
STEM
87.25
84.00
84.00
OURS
90.25
95.5
90.75
Table 6: The results of the user study. Bold indicates the best result.
Figure 9: The computational overhead of our proposed Edit-Your-Interest when processing videos with varying numbers of frames.
Method
Warp-err ↓
PSNR b ↑
Baseline
10.87
21.79
w/o AMEIS
3.12
21.89
w/o FMP
8.53
26.71
OURS
1.19
29.26
Table 7: Ablation Study Results. PSNR b denotes PSNR computed on the background region.
Figure 10: Our proposed Edit-Your-Interest achieves precise editing in multi-instance scenarios by integrating video instance segmentation model. As demonstrated, this method accurately edites specified target objects while preserving the background in complex scenes containing multiple instances of the same class.
Current video editing models often rely on expensive paired video data, which limits their practical scalability. In essence, most video editing tasks can be formulated as a decoupled spatiotemporal process, where the temporal dynamics of the pretrained model are preserved while spatial content is selectively and precisely modified. Based on this insight, we propose ImVideoEdit, an efficient framework that learns video editing capabilities entirely from image pairs. By freezing the pre-trained 3D attention modules and treating images as single-frame videos, we decouple the 2D spatial learning process to help preserve the original temporal dynamics. The core of our approach is a Predict-Update Spatial Difference Attention module that progressively extracts and injects spatial differences. Rather than relying on rigid external masks, we incorporate a Text-Guided Dynamic Semantic Gating mechanism for adaptive and implicit text-driven modifications. Despite training on only 13K image pairs for 5 epochs with exceptionally low computational overhead, ImVideoEdit achieves editing fidelity and temporal consistency comparable to larger models trained on extensive video datasets.
Diffusion-based video editing has made significant progress; however, achieving precise and temporally consistent object-level control, especially in multi-object scenarios, remains challenging due to attention leakage, identity drift, and unstable temporal dynamics. In this work, we propose IDAGEdit, a training-free framework for fine-grained multi-object video editing with strong temporal consistency. The framework adopts Layout-guided Attention Modulation to facilitate coherent multi-object editing, while Instance-level Masks are introduced to preserve individual object identity and enforce localized attention within each object region, thereby enabling fine-grained, object-level editing. Extensive qualitative and quantitative evaluations demonstrate that our method improves temporal stability and multi-object controllability over state-of-the-art video editing approaches.
Yuan-Zhih Lin, Huu-Thang Nguyen, Huu-Phu Do +2
Department of Computer Science, National Yang Ming Chiao Tung University, Taiwan
Recently, diffusion models have achieved considerable success in the text-guided video editing domain. However, existing works often struggle to balance the trade-off between temporal consistency and editability in video editing, with consistency and editability typically being inversely related. To address this, we propose a high-quality video editing framework enhanced for consistency and editability, named EquiEdit, which improves coordinatively the temporal consistency and editability of the edited videos while achieving a balance between the two. In terms of temporal consistency, the proposed temporal Mamba module with a tailored temporal-aware scanning scans fused video sequences following four designed directions, effectively enhancing the inter-frame consistency of edited videos. For editability, we design a noise injection strategy based on the spectral transformation to increase editing flexibility, where the Fourier transform is used to preserve the hidden structure in the initial latent noise used for editing, ensuring inter-frame consistency of the edited video and fidelity to the input video. Extensive qualitative and quantitative experiments demonstrate the effectiveness of our method in terms of temporal consistency and editability, as well as its great fidelity to the input video itself.