We study object motion path editing in videos, where the goal is to alter a target object's trajectory while preserving the original scene content. Unlike prior video editing methods that primarily manipulate appearance or rely on point-track-based trajectory control, which is often challenging for users to provide during inference, especially in videos with camera motion, we offer a practical, easy-to-use approach to controllable object-centric motion editing. We present Trace, a framework that enables users to design the desired trajectory in a single anchor frame and then synthesizes a temporally consistent edited video. Our approach addresses this task with a two-stage pipeline: a cross-view motion transformation module that maps first-frame path design to frame-aligned box trajectories under camera motion, and a motion-conditioned video re-synthesis module that follows these trajectories to regenerate the object while preserving the remaining content of the input video. Experiments on diverse real-world videos show that our method produces more coherent, realistic, and controllable motion edits than recent image-to-video and video-to-video methods.
Figures & tables
Figure 2: Trace Overview. Our system consists of two modules. First, the Cross-View Motion Transformation Module converts a user’s 2D input path drawn on the first-frame into a scene- and camera-aware bounding-box trajectory in the video view. Second, the Video Re-Synthesis Module uses this transformed box sequence to guide the generation of a new video where the object follows the desired path while inpainting the original path of the object and preserving the other parts of the input video. Both models in the figure are diffusion models.
Figure 3: Video Re-Synthesis Module. Our model employs a Diffusion Transformer (DiT) backbone conditioned on the first frame, masked video, and binary masks (object and inpainting).
Figure 4: Cross-View Motion Transformation Comparison. We compare our cross-view motion transformation against three baselines: (1) simple Interpolation of bounding boxes, (2) and 3D warping using MegaSAM [ megasam ] to estimate depth and camera pose. Left: Shows the first frame path and generated video-view bounding boxes on our Cross-View Motion Transformation Module evaluation set. Only the one generated with our method Trace accurately translates the first-frame view bounding boxes to the video-view. Right: The user’s intended path moves the fish towards the static red coral (used as an anchor). Interpolation produces an off-track path to the right. Existing 3D warping methods yield incorrect paths due to noisy depth and pose estimation. Our cross-view transformation generates a smooth, stable, and accurate path that delivers the fish box into the coral as intended.
Figure 5: Comparison of Video Re-Synthesis Baselines .Given an input video with a region (the original object) masked out and conditioned on the appearance of a reference object, the model must regenerate the object within the masked region. The goal is to produce a high-fidelity video that accurately restores the object while maintaining its original identity and temporal consistency.
Figure 6: Full pipeline comparison. Our Cross-View Motion Transformation module converts a user’s path (defined by the first and last boxes in the first frame) into a sequence of video boxes. This sequence is then used to guide our re-synthesis model and three baselines (which first require a separate inpainting step). For Image-to-Video Baselines, we only use a sequence of video boxes and background point tracks (only for Motion Canvas).
Figure 7: Additoinal Application. Applications for diverse video editing tasks. We leverage motion flexibility for various inference-time edits: (1) Object insertion : defining a first-frame layout and path, then using Qwen [ qwen ] to inpaint first frame and leveraging our model to propagate a new object; (2) Object editing : modifying first-frame attributes using Qwen and consistently propagating them along the trajectory; (3) Object replacement : substituting an object category in the initial frame using Qwen and re-synthesizing the sequence following original temporal dynamics.
Figure 8: Multiple object editing. Even though our method is trained to edit one object, it enables simultaneous zero-shot motion editing for multiple objects by processing parallel trajectories while maintaining scene consistency.
Video (camera) trajectory editing aims to synthesize new videos that follow user-defined camera paths while preserving scene content and plausibly inpainting previously unseen regions, upgrading amateur footage into professionally styled videos. Existing VTE methods struggle with precise camera control and long-range consistency because they either inject target poses through a limited-capacity embedding or rely on single-frame warping with only implicit cross-frame aggregation in video diffusion models. To address these issues, we introduce a new VTE framework that 1) explicitly aggregates information across the entire source video via a hybrid warping scheme. Specifically, static regions are progressively fused into a world cache then rendered to target camera poses, while dynamic regions are directly warped; their fusion yields globally consistent coarse frames that guide refinement. 2) processes video segments jointly with their history via a history-guided autoregressive diffusion model, while the world cache is incrementally updated to reinforce already inpainted content, enabling long-term temporal coherence. Finally, we present iPhone-PTZ, a new VTE benchmark with diverse camera motions and large trajectory variations, and achieve state-of-the-art performance with fewer parameters.
Zhihao Shi, Kejia Yin, Weilin Wan +5
McMaster University · University of Toronto · The University of Hong Kong +3
We present Search2Motion, a training-free framework for object-level motion editing in image-to-video generation. Unlike prior methods requiring trajectories, bounding boxes, masks, or motion fields, Search2Motion adopts target-frame-based control, leveraging first-last-frame motion priors to realize object relocation while preserving scene stability without fine-tuning. Reliable target-frame construction is achieved through semantic-guided object insertion and robust background inpainting. We further show that early-step self-attention maps predict object and camera dynamics, offering interpretable user feedback and motivating ACE-Seed (Attention Consensus for Early-step Seed selection), a lightweight search strategy that improves motion fidelity without look-ahead sampling or external evaluators. Noting that existing benchmarks conflate object and camera motion, we introduce S2M-DAVIS and S2M-OMB for stable-camera, object-only evaluation, alongside FLF2V-obj metrics that isolate object artifacts without requiring ground-truth trajectories. Search2Motion consistently outperforms baselines on FLF2V-obj and VBench.
Generative video editing has enabled creative control over an object's position in a video by prescribing an object's 3D or 2D motion trajectory, while preserving both video plausibility and identity. However, manually specifying a plausible full motion trajectory, like the arcs of a bouncing ball, requires time and expertise, and may therefore not be a suitable editing task for non-experts or quick edits. In contrast, in the image domain, generative object translation has been established as a simple editing task that requires only a single drag to move an object to a new location; yet an equivalent method is still missing for videos. We propose an analogous new editing task for videos that translates an object's 3D motion trajectory in a video while preserving identity and plausibility of appearance and motion, for example, translating the trajectory of a bouncing ball while preserving its relative motion. The main challenge in training this task lies in obtaining paired video data for this scenario. We introduce TrajectorySynth, a new data generation strategy for large-scale synthetic paired video data and a video generator TrajectoryMover fine-tuned with this data. We show that this enables generative movement of object trajectories. Project Page: https://chhatrekiran.github.io/trajectorymover