Generating motion-controlled videos--where user-specified actions drive physically plausible scene dynamics under freely chosen viewpoints--demands two capabilities: (1) disentangled motion control, allowing users to separately control the object motion and adjust camera viewpoint; and (2) motion causality, ensuring that user-driven actions trigger coherent reactions from other objects rather than merely displacing pixels. Existing methods fall short on both fronts: they entangle camera and object motion into a single tracking signal and treat motion as kinematic displacement without modeling causal relationships between object motion. We introduce MoRight, a unified framework that addresses both limitations through disentangled motion modeling. Object motion is specified in a canonical static-view and transferred to an arbitrary target camera viewpoint via temporal cross-view attention, enabling disentangled camera and object control. We further decompose motion into active (user-driven) and passive (consequence) components, training the model to learn motion causality from data. At inference, users can either supply active motion and MoRight predicts consequences (forward reasoning), or specify desired passive outcomes and MoRight recovers plausible driving actions (inverse reasoning), all while freely adjusting the camera viewpoint. Experiments on three benchmarks demonstrate state-of-the-art performance in generation quality, motion controllability, and interaction awareness.
Figures & tables
Figure 2: Model architecture. Our model adopts a dual-stream architecture with shared weights to disentangle object motion from camera motion. The canonical stream encodes motion trajectories using a track encoder and learns motion in a fixed canonical view. The target stream encodes camera pose signals through a camera encoder. The resulting motion and camera conditions are injected into every attention block of the network. Cross-view self-attention connects the two streams, transferring motion learned in the canonical view to the target view and enabling disentangled camera–object motion generation.
Figure 3: Active vs. passive motion. The active object (hand) initiates the action, while the passive object (cloth) responds.
Figure 4: Data curation pipeline. Foundation models [ 30 , 36 , 59 ] extract depth, camera poses, and tracks from raw videos. A VLM [ 3 ] segments tracks into active/passive regions. We further optionally use a video-to-video model [ 21 ] to generate paired videos with the same object motion but different camera motions.
Figure 5: Disentangled camera–object control. \ours enables independent control of object motion and camera viewpoint. Rows 1-3 fix the camera and vary object motion (rows 1-2: forward reasoning; row 3: inverse reasoning), while rows 4-6 fix object motion and vary camera motion.
Figure 6: Qualitative comparison of ATI [ 68 ] , WanMove [ 15 ] , and \ours on interactive motion generation with camera control. All methods use the same input image. ATI and WanMove rely on pixel-aligned per-frame tracks (top-left), which entangle camera and object motion and require privileged future tracks. In contrast, \ours uses only reprojected first-frame tracks. The first two rows show active motion reasoning, and the third row shows passive motion reasoning. Our model disentangles camera and object control and produces more coherent interactions.
Figure 7: Causal interaction reasoning. In the first 3 rows, we provide active motion ( e.g . , hand movement) as input, and the model infers the resulting passive motion ( e.g . , cloth movement). In the last 3 rows, we provide passive motion ( e.g . , ball movement), and the model infers the corresponding active motion ( e.g . , human movement).
Figure 8: Human perceptual evaluation. From 330 responses by 11 participants, our method is preferred across controllability, motion realism, and photorealism, outperforming ATI [ 68 ] and WanMove [ 15 ] , which rely on privileged 3D tracks but lack interaction reasoning.
Figure 9: Limitation analysis. Input tracks are overlaid on the first frame as in previous figures. (1) Incorrect interaction reasoning may lead to implausible outcomes (two kabobs merging). (2) Unnatural motion can occur when input tracks become temporally sparse due to occlusion (hand example). (3) Physically unrealistic dynamics may appear, such as objects disappearing during motion (soccer ball). (4) Hallucinated content may emerge in later frames (extra hand).
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 10: Prompt used for active and passive object identification for Qwen3 [ 3 ] in data curation pipeline.
Figure 11: Interactive demo interface. Our system enables users to control both object and camera motion from a single image. Users draw trajectories on the first frame to specify object motion (active or passive), either by moving a selected region using keypoint trajectories or by defining fine-grained motion paths for detailed control.
Figure 12: Human perceptual evaluation interface. Given an input image, object trajectories, and a target camera motion, participants evaluate generated videos under three criteria: Controllability (matching object tracks and camera motion), Motion Realism (physically plausible interactions and scene responses), and Photorealism (overall visual quality). For each criterion, participants select the best video (multiple selections allowed for ties, or None if none satisfy it). The study contains 30 randomly selected video sets with shuffled candidate order.
Figure 13: Causal interaction reasoning. Input tracks are shown in color and overlaid on the generated static reference-view video. The tracks represent user actions (active) or passive trajectories. Given these inputs, our model either predicts plausible consequences (forward reasoning) or recovers feasible driving actions that produce the desired outcomes (inverse reasoning, last row).
Figure 14: Additional controllable generation-1. Object motion trajectories are overlaid on the input image. For each video, we show different camera and object motion control. Each group shares the same object motion but uses different camera motions. Minor variations under the same object motion but different camera motions arise from the stochastic nature of interaction generation.
Figure 15: Additional qualitative comparison with ATI [ 68 ] , WanMove [ 15 ] , and \ours . ATI and WanMove rely on privileged 3D trajectories (with depth) projected to pixel-aligned per-frame tracks and take full interaction trajectories (active and passive) as input. In contrast, \ours uses only first-frame active tracks without privileged information and infers plausible interactions.