With large pretrained models, existing methods have effectively improved instruction-based video editing. However, most of them rely on an in-place editing assumption. They align the edited video with the given source clip frame by frame over a fixed time span. This pattern fails for open-ended streams, e.g., restyling a live game or applying a camera move to an ongoing shot. In such cases, edits must extend to future frames as they arrive, rather than be applied to a static input clip. In this paper, we study this setting and name it infinite video editing: given a preceding segment and an edit request, a model must generate the next segment that continues the stream while applying the requested edit. This process repeats as an unbounded sequence of edit instructions arrives. This task brings two challenges: the edit must be a faithful continuation rather than a frame-wise rewrite, and generation quality must remain stable as edits accumulate. To address them, we first design a data-collection pipeline for infinite video editing. Based on the collected data, we propose InfinityEdit, a lightweight edit adapter that equips a streaming video generator with unbounded editing ability. The adapter contains three attention modules. History cross-attention guides the denoising frames using the input frames. Temporal causal self-attention keeps temporal cues flowing only from earlier frames to later ones. Edit cross-attention injects the edit request into generation. During inference, the adapter is activated only in the chunk where an edit request arrives. Subsequent chunks are generated by the original model with a reset anchor frame. This scheme applies the edit while preserving the original model's infinite generation ability. Extensive experiments show that InfinityEdit faithfully continues the stream under each edit, and stays stable over unbounded edit sequences.
Figures & tables
Figure 1: Our data collection pipeline.
Figure 2: Our adapter’s architecture. It is composed of three attention modules.
Figure 3: Inference pipeline of our method. Each edit round starts with an ignition chunk, after which subsequent chunks continue the stream.
Category
Method
Camera Motion ↑
Motion Smoothness ↑
Temporal Flickering ↑
Dynamic Degree
Pure Backbone
Helios-Base (w/o Adapter) [ 51 ]
0.5494
0.9869
0.9641
0.6383
In-Place Editing
Lucy-Edit [ 33 ]
0.5062
0.9808
0.9616
0.6067
SANA-Streaming [ 56 ]
0.5432
0.9858
0.9631
0.4750
Prompt Switching
Anchor-Forcing [ 47 ]
0.4815
0.9775
0.9536
0.7917
Infinity-RoPE [ 49 ]
0.3580
0.9671
0.9344
0.8550
–
Ours
0.7654
0.9833
0.9660
0.7400
Table 1: Quantitative comparison on our sequential editing task (200 samples, 3 editing rounds each). All source-conditioned methods receive the same source video for fairness. Best results are in bold .
Category
Method
Edit Faithfulness ↑
Visual Quality ↑
Preservation ↑
Coherence ↑
Pure Backbone
Helios-Base (w/o Adapter) [ 51 ]
2.637
2.850
2.790
2.885
In-Place Editing
Lucy-Edit [ 33 ]
2.863
2.723
2.665
2.560
SANA-Streaming [ 56 ]
3.303
3.093
3.060
3.030
Prompt Switching
Anchor-Forcing [ 47 ]
2.545
2.620
1.830
3.480
Infinity-RoPE [ 49 ]
2.658
2.675
1.865
3.265
–
Ours
3.828
3.765
3.815
3.840
Table 2: VLM-as-Judge evaluation on our infinite video editing task (3 editing rounds). Best results are in bold .
Method
Edit 1
Edit 2
Edit 3
Std ↓
Helios-Base (w/o Adapter) [ 51 ]
2.590
2.740
2.580
0.072
Lucy-Edit [ 33 ]
2.490
2.980
3.120
0.268
SANA-Streaming [ 56 ]
3.260
3.295
3.355
0.039
Anchor-Forcing † [ 47 ]
1.530
2.955
3.150
0.715
Infinity-RoPE † [ 49 ]
1.655
3.215
3.105
0.694
Ours
3.860
3.805
3.820
0.023
Table 3: Edit faithfulness across sequential editing rounds. Our method maintains stable performance ( Δ<0.06 between rounds), demonstrating robust sequential editing capability without degradation.
Figure 4: The aesthetic quality score recorded with different editing round.
Figure 5: Qualitative comparison with various methods on the streaming video editing task, where a sequence of editing instructions (here camera-movement and style edits) is applied continuously to a scenery source video. All methods share the same source video and editing instructions. Methods marked with † take no source video as input, while all other conditions are kept identical for fairness.
Figure 6: Qualitative comparison with various methods on the streaming video editing task, where a sequence of editing instructions (here entity-transformation edits) is applied continuously to a human-centric source video. All methods share the same source video and editing instructions. Methods marked with † take no source video as input, while all other conditions are kept identical for fairness.
Figure 7: Long-video streaming editing. We lengthen each editing segment so that the full sequence exceeds 1000 frames. Our method stays visually stable and keeps realizing each instruction. The edited attributes also persist across segments.
Real-time video editing requires low-latency causal generation with bounded computational resources while preserving source fidelity and long-term temporal consistency. We present JoyAI-Video-Edit, a 16B-parameter autoregressive diffusion framework for real-time, open-ended video editing without access to future frames or a predefined video duration. Our method combines chunk-wise autoregressive adaptation, Source-Anchored Distribution Matching Distillation (SA-DMD), and Long-Horizon Autoregressive Distillation to reduce train--inference mismatch, preserve source fidelity during two-step generation, and mitigate accumulated temporal drift. Extensive automatic and human evaluations show that JoyAI-Video-Edit substantially outperforms existing streaming editors and remains competitive with strong offline systems on both short and long videos. The complete system achieves end-to-end 720p video editing at approximately 30 FPS on a single Nvidia B200 GPU. Code is available at https://github.com/jd-opensource/JoyAI-Video-Edit.
Although existing video editing methods are generally feasible, they often require many costly iterations and still struggle to deliver high-quality yet satisfying editing results. We attribute this limitation to the prevalent data-to-data paradigm, which is less compatible with modern generative models than noise-to-data generation. To address this gap, we revisit video editing from a noise-to-data perspective and propose Streaming-Generation-based Video Editing (StreamEdit), which preserves few-step sampling while seamlessly injecting source-video conditions. Built on pre-trained streaming generation models, StreamEdit introduces dual-branch fast sampling with a self-attention bridge and cross-attention grounding/boosting to satisfy both sampling and conditioning requirements. We further propose source-oriented guidance to improve target-generation quality, and a visual prompting strategy to enhance editing flexibility and practicality. The method is effective, robust, and generalizable across different models. Extensive experiments on diverse video editing tasks show that StreamEdit consistently outperforms existing approaches, even in few-step settings with minimal time cost. Code and results are available at: https://dsl-lab.github.io/StreamEdit/.
Guanlong Jiao, Chenyangguang Zhang, Jia Jun Cheng Xian +2
The University of British Columbia · Vector Institute · ETH Zürich +2
Streaming video editing has made rapid progress, yet practical deployment is still limited by two core issues: maintaining stable backgrounds and non-edited regions over time, and achieving the low latency required for real-time interactive scenarios. Meanwhile, recent streaming video generation methods are mostly developed for synthesis and cannot be directly applied to editing due to the strict preservation requirement and region-specific control. In this work, we present a novel streaming video editing framework that performs causal, frame-by-frame editing with strong content preservation and real-time responsiveness. Our key design is a three-stage distillation pipeline that progressively transfers editing capability from a powerful bidirectional foundation model to an efficient unidirectional streaming editor, enabling stable long-horizon edits without sacrificing visual fidelity. To further support real-time deployment, we introduce an AR-oriented mask cache that reuses region-related computation across frames, substantially reducing redundant processing and accelerating inference. Finally, we establish a dedicated benchmark for streaming video editing. Extensive evaluations demonstrate that our method achieves state-of-the-art visual quality among streaming baselines while drastically boosting inference speed to 12.66 FPS, making it suitable for interactive and augmented reality applications.