cs.CVApr 9, 2026
SaveImVideoEdit: Image-learning Video Editing via 2D Spatial Difference Attention Blocks
Organizations: Zhejiang University
Abstract
Current video editing models often rely on expensive paired video data, which limits their practical scalability. In essence, most video editing tasks can be formulated as a decoupled spatiotemporal process, where the temporal dynamics of the pretrained model are preserved while spatial content is selectively and precisely modified. Based on this insight, we propose ImVideoEdit, an efficient framework that learns video editing capabilities entirely from image pairs. By freezing the pre-trained 3D attention modules and treating images as single-frame videos, we decouple the 2D spatial learning process to help preserve the original temporal dynamics. The core of our approach is a Predict-Update Spatial Difference Attention module that progressively extracts and injects spatial differences. Rather than relying on rigid external masks, we incorporate a Text-Guided Dynamic Semantic Gating mechanism for adaptive and implicit text-driven modifications. Despite training on only 13K image pairs for 5 epochs with exceptionally low computational overhead, ImVideoEdit achieves editing fidelity and temporal consistency comparable to larger models trained on extensive video datasets.
Figures & tables
Figure 1: Illustration of four of ImVideoEdit ’s basic editing task. Zoom in for best viewing.
Figure 2: Overview of ImVideoEdit . Left: The overall pipeline processes latents from single image through a frozen 3D DiT, featuring a Predict-Update module parallel to each attention block. Right: Detailed design of the Predict & Update Module. The frozen 3D self-attention safeguards spatiotemporal priors, while the parallel 2D branch extracts spatial features from the reference latent. The Predict Module generates a coarse alignment ( ) via a Zero-Linear layer. The spatial difference ( ) is then processed by the Update Module and a Text Gate, which implicitly modulates the editing intensity to yield . The final output ( ) achieves precise text-driven spatial reconstruction. During training stage, the numbers of frame maintains .
Figure 3: Overview of the dataset construction pipeline.
Figure 4: Dataset statistics.
| Method | Bg. Rep. | Cam. Trans. | Color/Texture Trans. | Style Trans. | Relight | Local Edit | Rigid/Non Rep. | Obj. Add./Rem. | AVG. |
|---|---|---|---|---|---|---|---|---|---|
| 13B & 14B Parameter Models | |||||||||
| Vace (14B) | 52.3 | 58.6 | 58.0 | 69.7 | 79.7 | 56.0 | 60.8 | 50.5 | 59.68 |
| DITTO (14B) | 57.7 | 61.8 | 60.6 | 74.7 | 78.6 | 50.1 | 62.0 | 55.4 | 61.82 |
| ICVE (13B) | 55.7 | 59.5 | 75.6 | 69.0 | 67.7 | 57.6 | 77.1 | 55.7 | 65.04 |
| 5B Parameter Models | |||||||||
| Kiwi-Edit (5B) | 46.8 | 64.4 | 81.9 | 69.4 | 87.4 | 67.4 | 81.3 | 65.3 | 71.13 |
Table 1: Quantitative Results of VLM across all subtasks. The best are highlighted in bold .
| Method | Subject Consist. | Background Consist. | Motion Smooth. | Dynamic Deg. | Aesthetic Qual. | Imaging Qual. |
|---|---|---|---|---|---|---|
| 13B & 14B Parameter Models | ||||||
| Vace(14B) | 0.973 | 0.973 | 0.990 | 0.296 | 0.685 | 0.715 |
| DITTO(14B) | 0.979 | 0.968 | 0.994 | 0.140 | 0.655 | 0.670 |
| ICVE(13B) | 0.972 | 0.959 | 0.991 | 0.404 | 0.629 | 0.697 |
| 5B Parameter Models | ||||||
| Kiwi-Edit(5B) | 0.976 | 0.959 | 0.993 | 0.140 | 0.616 | 0.716 |
Table 2: Quantitative Results on VBench. The best are highlighted in bold .
Figure 5: Qualitative Results of ImVideoEdit and baselines.
| Method | IA | TC | VF | AA | Total |
|---|---|---|---|---|---|
| 13B & 14B Parameter Models | |||||
| Vace (14B) | 10.67 | 23.99 | 14.41 | 10.62 | 59.68 |
| DITTO (14B) | 14.14 | 23.40 | 13.88 | 10.40 | 61.82 |
| ICVE (13B) | 15.64 | 22.85 | 16.04 | 10.51 | 65.04 |
| 5B Parameter Models | |||||
| Kiwi-Edit (5B) | 19.15 | 23.87 | 16.86 | 11.26 | 71.13 |
Table 3: Quantitative Results of VLM. Metrics: IA (Instruction Adherence), TC (Temporal Consistency), VF (Visual Fidelity) and AA (Artifact Absence).
| Method | IA | TC | VF | AA | Total |
|---|---|---|---|---|---|
| w/o Text Gate | 15.29 | 20.37 | 14.31 | 9.76 | 59.73 |
| w/o Update Module | 9.71 | 18.10 | 11.09 | 7.59 | 47.29 |
| Naive Parallel 2D (ViFedit [ 43 ] ) | 12.04 | 16.82 | 12.24 | 7.72 | 48.81 |
| ImVideoEdit (Ours) | 16.21 | 21.93 | 16.32 | 10.78 | 65.24 |
Table 4: Quantitative Ablation Results.
Figure 6: Qualitative Ablation Results.
| Method | Overall Editing Quality |
|---|---|
| VACE (1.3B) | |
| Kiwi-Edit (5B) | |
| ImVideoEdit (1.3B Based) |
Table 5: User Study Results. denotes the standard deviation.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
| Evaluator Model | Pearson | MAE | Variance |
|---|---|---|---|
| GPT-5.4 | 0.720 | 0.185 | 0.113 |
| Gemini-2.5-Pro | 0.843 | 0.137 | 0.068 |
Table 6: Quantitative Cross-Validation of VLM Evaluators. We report the linear correlation (Pearson ), Mean Absolute Error (MAE), and Inter-model Variance of various models against Gemini-3.1-Pro (the default evaluator in our pipeline).
| Prompt Template |
| # Role You are an expert video quality assessor and professional video editor. Your task is to evaluate the quality of an AI-edited video based on a specific user instruction, comparing it to the original unedited video. # Inputs Original Video: [Insert pre-edited video] Edited Video: [Insert post-edited video] Editing Instruction: ”{Insert the editing instruction here}” # Evaluation Criteria Please evaluate the edited video rigorously based on the following four dimensions. Provide a distinct score for each dimension based on its maximum point value. 1. Instruction Adherence (Max: 30 points): Did the editing strictly follow the given instruction? Are the requested changes accurately reflected without altering unintended elements? 2. Temporal Consistency & Micro-Stability (Max: 30 points) - STRICT DEDUCTION RULES : VLM Warning: Do not just look at the overall subject. You must zoom in on high-frequency details. Scoring Anchor: - 28-30 pts: Perfect stability, identical to the physics of a real camera. - 20-27 pts: Overall stable, but minor ”AI boiling” (micro-flickering of pixels) on edges or complex patterns during movement. - 10-19 pts: Noticeable morphing, shifting of textures (e.g., embroidery changing shape frame-by-frame), or jittery outlines. - 0-9 pts: Severe flickering or structural collapse. 3. Texture Sharpness & Anti-Smoothing (Max: 25 points) - CALIBRATED FOR AI : Do not compare this to an 8K cinema camera. Evaluate the AI rendering quality. We are looking for SHARPNESS vs. PLASTICITY. Focus on the materials: the velvet texture of the blue dress, the individual threads of the embroidery, and the natural pores/lighting on the skin. Scoring Anchor: - 21-25 pts: Excellent sharpness. Fabrics look like real cloth with distinct threads. Lighting has depth and natural contrast. - 15-20 pts: Good, but slightly soft. - 10-14 pts: The ”Plastic AI” look. Textures are overly smoothed, faces look like wax or airbrushed, and fine details (like embroidery) look like flat paint rather than raised threads. (DEDUCT HEAVILY HERE). - 0-9 pts: Extremely blurry or washed out. 4. Artifact Absence (Max: 15 points): Are there any visible AI generation artifacts (e.g., floating pixels, anatomical distortions, weird edge blending)? The edited areas should blend seamlessly with the original unedited parts. # Output Format Provide your response strictly in the following JSON format. Do not include a total score, only the sub-scores for each dimension. { ”Reasoning”: ”Provide a concise step-by-step analysis evaluating the video against the 4 criteria. Explicitly mention your observations on temporal stability and texture details.”, ”Scores”: { ”Instruction_Adherence”: <int between 0 and 30>, ”Temporal_Consistency”: <int between 0 and 30>, ”Visual_Fidelity”: <int between 0 and 25>, ”Artifact_Absence”: <int between 0 and 15> } } |
Table 7: System Prompt for VLM Evaluation.
| Task | Method | IA | TC | VF | AA | Total |
|---|---|---|---|---|---|---|
| Background Replacement | VACE (14B) | 4.4 | 25.6 | 12.2 | 10.0 | 52.3 |
| DITTO (14B) | 10.1 | 24.3 | 12.6 | 10.7 | 57.7 | |
| ICVE (13B) | 7.3 | 22.9 | 15.7 | 9.8 | 55.7 | |
| Kiwi-Edit (5B) | 8.0 | 18.4 | 12.8 | 7.5 | 46.8 | |
| Lucy-Edit-Dev (5B) | 5.1 | 15.5 | 8.7 | 5.4 | 34.8 | |
| OmniVideo2 (1.3B) | 4.4 | 12.0 | 9.1 | 5.0 | 30.5 |
Table 8: Fine-grained VLM evaluation results for the first five task categories. IA, TC, VF, and AA denote Instruction Adherence, Temporal Consistency, Visual Fidelity, and Artifact Absence, respectively.
| Task | Method | IA | TC | VF | AA | Total |
|---|---|---|---|---|---|---|
| Local Attribute/State Editing | VACE (14B) | 7.2 | 25.4 | 13.0 | 10.5 | 56.0 |
| DITTO (14B) | 9.8 | 20.8 | 11.6 | 7.9 | 50.1 | |
| ICVE (13B) | 9.4 | 23.8 | 14.6 | 9.8 | 57.6 | |
| Kiwi-Edit (5B) | 15.8 | 24.6 | 15.8 | 11.2 | 67.4 | |
| Lucy-Edit-Dev (5B) | 8.5 | 18.3 | 11.3 | 6.6 | 44.8 | |
| OmniVideo2 (1.3B) | 10.0 | 16.9 | 12.3 | 7.5 | 46.7 |
Table 9: Fine-grained VLM evaluation results for the remaining five task categories and the overall average. IA, TC, VF, and AA denote Instruction Adherence, Temporal Consistency, Visual Fidelity, and Artifact Absence, respectively.
Figure 7: visualizations of datasets (Part 1).
Figure 8: visualizations of datasets (Part 2).
Figure 9: Additional qualitative result (Part 1).
Figure 10: Additional qualitative result (Part 2).
More in this topic
Video Editing
Video editing uses AI to alter, compose, relight, or reorganize moving footage while controlling how changes persist across frames.
See all 179 papers