cs.CVMar 18, 2026
SaveVersatile Editing of Video Content, Actions, and Dynamics without Training
Organizations: Google DeepMind · Technion – Israel Institute of Technology · The Weizmann Institute of Science
Abstract
Controlled video generation has seen drastic improvements in recent years. However, editing actions and dynamic events, or inserting contents that should affect the behaviors of other objects in real-world videos, remains a major challenge. Existing trained models struggle with complex edits, likely due to the difficulty of collecting relevant training data. Similarly, existing training-free methods are inherently restricted to structure- and motion-preserving edits and do not support modification of motion or interactions. Here, we introduce DynaEdit, a training-free editing method that unlocks versatile video editing capabilities with pretrained text-to-video flow models. Our method relies on the recently introduced inversion-free approach, which does not intervene in the model internals, and is thus model-agnostic. We show that naively attempting to adapt this approach to general unconstrained editing results in severe low-frequency misalignment and high-frequency jitter. We explain the sources for these phenomena and introduce novel mechanisms for overcoming them. Through extensive experiments, we show that DynaEdit achieves state-of-the-art results on complex text-based video editing tasks, including modifying actions, inserting objects that interact with the scene, and introducing global effects.
Figures & tables
Figure 1: Training-free versatile editing of actions and dynamics in videos. We present DynaEdit, a training-free flow-based method for video editing, which is the first to enable manipulation of dynamics and contents in videos using textual descriptions. DynaEdit supports the modification of actions and the insertion of objects that interact with the scene ( e.g. causing a horse to jump due to a newly inserted obstacle, a cat to run off due to a toy edited to become a burning marshmallow, or a billiard ball to enter the pocket). It also allows global and stylistic modifications, like changing daytime to nighttime, all while avoiding unnecessary changes to the video (see SM for the videos).
Figure 2: Limitations of state-of-the-art inversion-free editing methods. Existing inversion-free methods [ 25 , 26 , 22 ] suffer from a tradeoff between edit expressivity and visual quality, illustrated here with FlowEdit [ 25 ] using an I2V model. When starting the generation at timestep , the method struggles to modify motion (second row). Using allows making more significant spatio-temporal modifications and thus to better adhere to the prompt, but results in severe jitter artifacts and illogical motions (third row). Attempting to reduce artifacts by averaging over edit directions in each step leads to blur (fourth row).
Figure 3: Effects of noise in inversion-free editing. (a) A source video (three frames and a spatio-temporal slice corresponding to the dashed line). (b) Three inversion-free editing results (FlowEdit) differing only in the noise sample at timestep . As seen, the initial noise strongly affects coarse spatio-temporal features, e.g. modifying the camera motion and train position across edits, although those features are not required to change to adhere to the prompt. (c) Using independent noise samples across timesteps (top) leads to high-frequency jitter, e.g. the blurry bucket and paint drops. This can be alleviated by using the same noise sample for all timesteps (bottom) but at the cost of worse alignment with the source video, e.g. causing the bucket to levitate.
Figure 4: DynaEdit. Our method constructs a noise free path from the source video to the edited one (top pane). The middle pane shows one step in this process, with our key contributions colored. Our SGA module (bottom left) aggregates several noise-free velocities based on the similarities between the edits they induce and the source video. Our ANC mechanism (bottom right) induces gradually increasing correlations between the noises of consecutive timesteps.
Figure 5: DynaEdit Results. Our method supports a wide range of edits, including motion manipulation (swans), interactive object addition (horse, barrier, dinosaur), and global style changes (magma, Manga).
Figure 6: Qualitative comparison. A video depicting two strawberries falling into a water tank is edited such that the right strawberry is replaced by a feather. In contrast to competing methods, DynaEdit generates a physically plausible video, where the feather slowly descends and then floats on the water, while keeping the dynamics of the left strawberry unchanged (see App. 10 for more comparisons).
Figure 7: Quantitative comparison. We compare DynaEdit to existing methods on content preservation, text adherence and visual quality. (a) VLM ratings of these criteria show that DynaEdit ranks best in content preservation, while achieving comparable text adherence and visual quality to the trained Aleph model. (b) A user study comparing DynaEdit against the top three competitors shows higher preference rates for DynaEdit, with the trained Aleph model being the closest competitor.
Appendix figures & tables22 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Additional Results. DynaEdit is capable of diverse dynamic editing effects. For example, on the top left we insert a mirror into the forest video, showcasing reflections reactive to the scene’s content. In the fourth row (left), we prompt the leftmost swan to dive, while preserving the other swans’ motions. On the bottom left row, a global earthquake effect is added to the bridge scene causing the moving train to collapse, all while preserving the original camera motion.
Figure 9: Additional results with the Hunyuan I2V model. DynaEdit is model agnostic and can thus potentially leverage any I2V model. The figure illustrates diverse results obtained by DynaEdit with the Hunyuan I2V model, like interactive insertion with of a Mars flag and reactive material swap (strawberries fall into gel vs. water).
| Method | CFG , SGA Temp. | Text Adherence | Content Pres. | Visual Quality |
|---|---|---|---|---|
| DynaEdit | 4.08 | 4.50 | 3.83 | |
| 4.04 | 4.38 | 3.60 | ||
| 4.21 | 4.36 | 3.35 | ||
| 4.14 | 4.25 | 3.54 | ||
| FlowAlign | 3.76 | 3.40 | 1.95 | |
| 3.84 | 3.59 | 2.47 |
Table 1: Comparison vs. competing methods (with reported hyperparameters) using VLM evaluation. Best scores are highlighted in bold, second best are underlined.
Figure 10: Additional comparisons against the competing methods. On the left, a cat is inserted into a zooming-in video of a living room. It is prompted to jump on the sofa. The DynaEdit result is the only one to simultaneously adhere to the original camera motion, to add a cat that performs the proper action, and to exhibit no visible artifacts. On the right, an orange in a market stand is swapped into a red apple, and a person walks in to grab it. DynaEdit performs the required edit without visible artifacts while preserving the background (people passing by). In both examples, the other edits do not exhibit all three properties simultaneously.
Figure 11: Qualitative comparison against DynVFX on object insertion subset. Our method meaninfgully integrates inserted objects into the scene. As visible by the inserted helicopter that shines light on the the city at night, or the kite and umbrella that react to the wind in the scene. This is in contrast to DynVFX, which showcases limited dynamical interactions between the inserted objects and the scene.
Figure 12: Comparison against DynVFX on object insertion subset. While DynVFX achieves a good visual quality score, the inserted objects are not able to affect the scene’s outcomes. Due to this, they receive a lower text adherence score.
Figure 13: Qualitative comparison against FlowDirector on object swap subset. On the left example, the source video depicts a person squeezing a lemon into a pitcher with red juice. The edit requires changing the juice’s color to green and adding a sudden chemical reaction with the lemon. Our method performs the edit successfully, depicting a realistic reaction, while FlowDirector fails to add a meaningful reaction effect due to the localized nature of it’s edits. On the right, a train is edited into a giant snail. The edit prompt requires integrating an object that is not strictly bound to the source object’s geometry (the train), which our method successfully follows. However, FlowDirector struggles in performing the required edit, due to the limitations imposed by it’s locality.
Figure 14: Comparison against FlowDirector on object swap subset. Due to the local nature of FlowDirector’s edits, it struggles to follow prompts that require dynamic interaction with the scene, and thus generates unnatural videos. This is evident by the low text and visual quality scores. FlowDirector also performs worse in terms of content preservation, as oftentimes it struggles to maintain the swapped object’s identity, or the background in close proximity to the object (see qualitative comparisons in Fig. 13 ).
Figure 15: VLM Evaluation and user study results per-edit-category. (a) Per-category VLM evaluations reveal that DynaEdit achieves a favorable balance between source content preservation, adherence to target text, and visual quality compared to training-free methods. Against the trained Aleph model, we achieve comparable results on most categories, with a disadvantage only in visual quality for the global effects category. (b) Per-category user study results show that on the insertion and swap categories DynaEdit is favorable compared to the leading competing methods, including Aleph. For the action and global effects categories, DynaEdit is preferred over the training-free approaches, and is mostly comparable to Runway Aleph, losing only in text adherence on the global effects category.
Figure 16: Ablation of the SGA mechanism. On the left, we edit a source video depicting synchronous motion of a sea turtle and a fish under it. The edit prompts the fish to leave the scene without changing the turtle’s motion. When using the SGA mechanism, the motion of the turtle stays the same as in the source, while with regular averaging the motion changes. On the right, the edit requires adding a fire-tornado to a video of a forest fire. While strong changes are required to achieve this manipulation, the SGA mechanism keeps the original camera motion intact, as opposed to the naive averaging approach (white arrows).
Figure 17: Ablation of the ANC mechanism. On the left, a source video of a train passing through a forest is manipulated by adding a yellow paint bucket to the train’s path. In the i.i.d. noise case, while the result adheres to the original motion (due to the SGA mechanism) it exhibits severe high-frequency jitter artifacts. In contrast, with ANC the edited video exhibits good fidelity in the high frequency details, evident by the visible paint particles. On the right, a flag is inserted into a video depicting astronauts walking on Mars. The edit prompt requires one astronaut to pick up the flag, while the other one to continue walking. In the i.i.d. noise case, the flag dissolves towards the end of the video. On the other hand, our method maintains the high-fidelity dynamics of the flag throughout the whole video.
Figure 18: Effect of noise correlations on edit velocity. The left pane shows the effect of using i.i.d. noise on the correlations between consecutive edit velocities. On the right, our annealed noise correlation schedule (ANC) is employed, which induces stronger correlations between consecutive noises (orange), which leads to higher correlations between consecutive edit velocities (blue) compared to the i.i.d. case.
Figure 19: Ablation of different annealing noise correlation schedules. Three different noise correlation schedulers are compared on a video of a camel in a desert (first row). The edit prompt requires inserting a giraffe that interacts with the camel during its meal. The second row shows the result of employing non-Markovian correlations, with the same increasing coefficients as used in our method (see Sec. 6.1 ). The result showcases visible ghosting artifacts, as seen by the two giraffe necks. The third row shows the result of Markovian correlations (as in our method) but with a mirrored correlation schedule (lower correlations at later timesteps). The result exhibits high-frequency jitter artifacts. The fourth row shows Markovian correlations with an increasing schedule, which is the one employed by our method. As seen, the results are artifact- and jitter-free.
Figure 20: Ablation of SGA similarity function choices. Two SGA similarity functions are compared: negative MSE loss and our cosine similarity loss (see right pane). Edits are performed on a video of a paintbrush painting edited into a pencil sketch, and a video of a bird swapped into a flaming phoenix that sets fire to a branch. For the painting example, the cosine similarity function performs better, capturing the fine spatio-temporal motion of the paintbrush, as evident by the aligned motion of the pencil in the edit. This is in contrast to the MSE variant, where the pencil’s location is not aligned with the paintbrush. For the phoenix example, there is little difference between the two loss functions.
Figure 21: Ablation of prompt robustness. Each row depicts the editing result obtained with a different variant of the text prompts (see Sec. 0.C.5 ). As can be seen, the precise phrasing of the prompt has little effect on the result.
| Source Prompt | Target Prompt |
|---|---|
| Scenario 1: Astronauts | |
| Tracking shot of two astronauts traversing the Martian landscape. The terrain is a desert with mountains in the distance. | Tracking shot of two astronauts traversing the Martian landscape. The terrain is a desert with mountains in the distance. The astronaut on the right hand side grabs a flag labeled ’MARS’ mid-stride. The second astronaut remains oblivious to this action. |
| Show two astronauts walking together on Mars. There are mountains and red sand in the background, and the camera is following behind them. | Show two astronauts walking together on Mars. There are mountains and red sand in the background, and the camera is following behind them. Suddenly, the astronaut on the right spots a flag that says ’MARS’ and picks it up without stopping. The other astronaut doesn’t notice it happening. |
| Two astronauts walking on Mars. Desert setting, mountain backdrop. Camera follows subjects. | Two astronauts walking on Mars. Desert setting, mountain backdrop. Camera follows subjects. Right astronaut picks up a ’MARS’ flag while walking. Left astronaut is unaware. |
| A pair of space travelers stroll across the dusty red dunes of Mars, flanked by towering mountains. The viewpoint moves with them as they walk. | A pair of space travelers stroll across the dusty red dunes of Mars, flanked by towering mountains. The viewpoint moves with them. The traveler on the right reaches out and snatches a flag reading ’MARS’ from the ground without halting. Their companion continues walking, completely unaware. |
| Generate a clip of two astronauts on Mars. Background: Mountains/Desert. Camera: Follow movement. | Generate a clip of two astronauts on Mars. Background: Mountains/Desert. Camera: Follow movement. Action: The astronaut on the right must pick up a ’MARS’ flag mid-walk. Condition: The other astronaut must not react to the flag. |
Table 2: Prompts used for prompt robustness ablation
Figure 22: Importance of I2V conditioning. Performing strong edits ( ) without an image condition for the source and target branches results in random color changes, which are evident in the edit result in the middle row. Using an Image-to-Video model conditioned on the first frame of the input video results in preserved colors, as seen in the bottom row.
Figure 23: Effect of Hyperparameters. Depending on the prompt requirement, hyperparameter configurations can sometimes be crucial to strike a favorable balance between edit adherence and loyalty to the source. On the left, the horse is prompted to jump over an obstacle. Since the horse takes up many pixels in each frame, a strong CFG is preferred (parameter sets (3),(4)). In the other cases, the edit is too weak to properly edit the horse, resulting in visual artifacts. Among parameter sets (3),(4), it can be seen that parameter set (4) (with the lower temperature) induces a strong adherence to the input video’s coarse features, hindering the ability to change the horse’s original trajectory. So in this case parameter set (3) (marked with green frame), which uses a higher temperature, is preferable. For the turtle and fish example, where the prompt requests to change the action of the smaller fish, a weaker edit is enough to adhere to the target. So here parameter set (2) (marked with green frame), which uses lower CFG and lower SGA temperature, suffices. In the other cases, the motion of the turtle changes as well. While these options are plausible in terms of text adherence, they needlessly deviate from the input video compared to the low CFG settings (parameter sets (2),(3)).
| Location | Video description | Parameter group |
|---|---|---|
| Fig. 1 | “Horse jumping” | large object, strict motion (3) |
| Fig. 1 | “Cat playing” | action change, large motion change (3) |
| Fig. 1 | “Billiard ball entering” | action change, limited motion change (4) |
| Fig. 1 | “Nighttime beach” | global effect, limited motion change (4) |
| Fig. 5 | “Mars flag” | small object, limited motion change (2) |
| Fig. 5 | “Heli with light” | small object, limited motion change (2) |
Table 3: Hyperparameter groups for qualitative results.
| Source prompt | Target prompt | Instruction prompt |
|---|---|---|
| “Two astronauts are walking on Mars. Desert, mountains in the background. The camera follows them.” | “Two astronauts are walking on Mars. Desert, mountains in the background. The camera follows them. The astonaut on the right approaches a flag spelling MARS and picks it up mid walk. The other astonaut is unaware of the flag.” | “Make the astronaut on the right pick up a flag spelling MARS while walking. Do not change the other astronaut.” |
| “A tracking camera shot of a forest filled with trees. The camera is moving to the right.” | “A tracking camera shot of a forest filled with trees. The camera is moving to the right. A big mirror is in the middle of the forest.” | “Place a large mirror in the middle of the forest.” |
| “A scenic shot of a stone train bridge in the mountains. A red train is crossing the bridge from left to right. The camera is slowly moving forward.” | “A scenic shot of a stone train bridge in the mountains. A giant snail is crossing the bridge from left to right, leaving a trail of ooze. The camera is slowly moving forward.” | “Replace the train with a giant snail crossing the bridge and leaving a trail of ooze.” |
| “A static camera shot of yellow tulips swaying in the wind. Close-up shot.” | “A static camera shot of yellow tulips and a dandelion swaying in the wind. Close-up shot. Suddenly a hand reaches out and picks the middle tulip, leaving the others intact.” | “Have a hand appear and pick the middle tulip.” |
| “A static shot of a Ferris wheel in motion. The wheel is spinning, and the spokes are moving. Sky in the background.” | “A static shot of a Ferris wheel in motion. The wheel is spinning, and the spokes are moving. Sky in the background. Suddenly, gray clouds begin to form in the sky, turning the sky gray.” | “Make gray storm clouds gather in the sky.” |
Table 4: Examples for source-target prompt to instruction prompt conversion.
Figure 24: User study snippet. The first-page instruction is given at the top. The videos and following question structure are given in the bottom.
Figure 25: Limitations arising from the base I2V model. Top row examples showcase limited visual quality. For instance, on the right, the person’s face takes up a small region in the video, so the I2V model struggles to properly generate such small details. On the top right, a video of a waterfall is edited to have a monster suddenly emerge from the water. The monster’s details are not of high quality due to the model’s generative capacity. The bottom examples show limitations due to the I2V model’s limits in visual reasoning. For example, on the left, the man is prompted to remove his hat due to the sandstorm. After the man successfully removes the hat, another hat appears under it, which is illogical. In the bottom right, a video of palm trees is edited such that a sudden storm begins. However, the trees remain static, showcasing the model’s limited understanding of wind dynamics.
More in this topic
Video Editing
Video editing uses AI to alter, compose, relight, or reorganize moving footage while controlling how changes persist across frames.
See all 176 papers