Diffusion models for video generation have achieved remarkable success, yet precise motion control under explicit user guidance remains challenging. Recent noise-warping methods introduce temporal correlations by warping the noise according to the user-provided motion signal. However, standard spatial warping generally disrupts the noise's i.i.d. Gaussian structure, requiring distribution correction, such as Jacobian-based correction, but at a substantial computational overhead. We propose Householder Motion Embedding (HME), which encodes the motion signal into the noise through a sequence of Householder reflections that together form an orthogonal transformation applied to each location's channel vector. HME exactly preserves the spatial Gaussian structure within each frame while introducing cross-frame correlations that promote temporally consistent video. HME is agnostic to the network architecture and integrates directly into standard video diffusion fine-tuning pipelines. Experiments on different scenarios show that HME matches or outperforms the state of the art, while achieving a 2.7x speedup over noise warping. These results demonstrate that channel-space Householder transformations offer a principled and efficient alternative to noise warping for motion-controllable video generation.
Figures & tables
Figure 2: HME generates motion controlled videos in 3 stages: 1) optical flow estimation and spatial resampling, 2) temporally correlated noise construction via Householder Reflectors (see Section 3.1 for details), 3) diffusion model fine-tuning and inference. At inference time, our method enables motion- and camera-controlled video generation given a text prompt, an initial frame, and a reference video as the motion signal.
Figure 3: Qualitative comparison of (a) motion-transfer, (b) fine-grained motion control, and (c) local object motion control I2V generation, evaluated on DAVIS and MC-Bench. We present the intermediate frame and last frame of generated videos by each method. Our method demonstrates strong appearance consistency and high generation fidelity while faithfully adhering to the provided motion control signals. Best viewed zoomed in; videos are provided in the Supplementary Material.
Method
Subject
BG
Temporal
Motion
FVD
Optical Flow
CoTracker
PSNR
CLIP
Consist. ↑
Consist. ↑
Flicker. ↑
Smooth. ↑
FID ↓
( ×103 ) ↓
(EPE) ↓
(mIoU) ↑
(dB) ↑
Image ↑
MotionClone [ 32 ]
0.754
0.862
0.874
0.928
133.852
1.771
26.451
0.373
10.032
0.814
ImageConductor [ 30 ]
0.791
0.891
0.913
0.939
128.087
1.830
20.382
0.372
12.117
0.811
GWTF γ=0.5 [ 7 ]
0.888
0.920
0.942
0.978
68.385
0.987
5.644
0.464
14.748
0.912
MotionPro [ 65 ]
0.873
0.913
0.932
0.964
103.859
1.253
19.692
0.473
14.257
0.879
DisMo [ 43 ]
0.851
0.903
0.920
0.956
85.201
1.156
11.189
0.428
11.475
0.882
Table 1: Quantitative comparison of motion controlled I2V generation on DAVIS. Best performance in Bold , best baseline performance in Underline .
Method
Subject Cons. ↑
Background Cons. ↑
Motion Smooth. ↑
Aesthetic Qual. ↑
Imaging Qual. ↑
CTD ↓
Bg-Obj CTD ↑
MotionPro [ 65 ]
0.980
0.976
0.995
0.586
0.610
8.019
23.30
TTM [ 46 ]
0.984
0.981
0.995
0.601
0.645
8.432
36.32
DisMo [ 43 ]
0.985
0.973
0.991
0.608
0.601
8.954
36.16
Noise-based methods
GWTF γ=0.5 [ 7 ]
0.975
0.974
0.993
0.589
0.645
15.747
42.44
Ours γ=0.3,K=2
0.976
0.977
0.995
0.594
0.645
13.879
39.53
Table 2: Quantitative comparison of Fine-grained and Object-level motion control on MC-Bench.
Figure 4: Qualitative Comparison of camera motion controlled I2V generation, evaluated on DL3DV. We present the intermediate frames and last frames of generated videos by each method. Best viewed zoomed in; videos are provided in the Supplementary Material.
Method
FID ↓
FVD ↓
PSNR (dB) ↑
CLIP Image ↑
LPIPS ↓
SSIM ↑
ATE ↓
MotionPro [ 65 ]
37.195
291.510
12.701
0.814
0.545
0.470
0.825
DisMo [ 43 ]
22.458
187.134
12.255
0.876
0.506
0.446
1.155
TTM [ 46 ]
21.367
196.533
12.764
0.881
0.443
0.467
0.464
GWTF γ=0.5 [ 7 ]
23.394
235.260
12.626
0.880
0.431
0.469
0.473
Ours γ=0.3,K=2
20.338
175.103
13.656
0.885
0.402
0.496
0.421
Table 3: Quantitative comparison of DL3DV camera motion control.
Method
Subject
BG
Temporal
Motion
FVD
Optical Flow
CoTracker
PSNR
CLIP
Consist. ↑
Consist. ↑
Flicker. ↑
Smooth. ↑
FID ↓
( ×103 ) ↓
(EPE) ↓
(mIoU) ↑
(dB) ↑
Image ↑
GWTF γ=0.5 [ 7 ]
0.882
0.918
0.945
0.979
73.348
1.180
5.857
0.472
14.437
0.901
Ours γ=0.3,K=2
0.895
0.926
0.948
0.979
63.619
0.837
4.707
0.541
15.418
0.915
Table 4: Robustness to the flow estimator: DAVIS results with UniMatch [ 56 ] flows at evaluation time.
Method
Subject ↑
Background ↑
Temporal ↑
Motion ↑
FID ↓
FVD ↓
Flow EPE ↓
CoTracker mIoU ↑
PSNR ↑
CLIP Image ↑
Ours K=1
0.898
0.925
0.946
0.978
63.992
861.121
4.868
0.543
15.300
0.918
Ours K=2
0.898
0.925
0.946
0.979
61.780
861.740
4.433
0.546
15.399
0.923
Ours K=4
0.898
0.926
0.948
0.979
64.579
821.749
4.358
0.555
15.424
0.922
Ours K=8
0.898
0.927
0.948
0.979
63.314
827.664
4.144
0.550
15.512
0.923
Ours K=12
0.898
0.927
0.950
0.979
64.699
831.586
4.197
0.558
15.581
0.923
Table 5: Performance comparison across different numbers of Householder reflections K (all with γ=0.3 ), on DAVIS.
Method
HIWYN [ 8 ]
GWTF [ 7 ]
InfRes [ 10 ]
Ours K=1
Ours K=2
Ours K=4
Ours K=6
Ours K=8
Ours K=12
GPU Time (ms/Latent) ↓
38.00
7.63
23.94
2.27
2.80
5.18
7.59
10.01
14.78
Memory Overhead (MB) ↓
49,384
76
285
0.5
1
2
3
4
6
Table 6: Computational overhead of noise-based methods: GPU time and approximate peak additional GPU memory on an NVIDIA RTX PRO 6000 Blackwell GPU (1024 × 1024 resolution, C=16 ; HIWYN with L=3 ).
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Moran’s I
Moran’s I
K-S Test
K-S Test
Std.
Skewness
Ex. Kurtosis
GPU Time
( ≈0 )
p-value ↑
(index) ↓
p-value ↑
( ≈1.0 )
( ≈0 )
( ≈0 )
ms/Latent ↓
Warping + Interpolation Operation
Nearest
0.1150
0.012
0.0107
0.115
1.0015
0.0016
-0.0116
-
Bilinear
0.6218
0.008
0.3318
0.000
0.2179
0.0720
0.4753
-
Bicubic
-0.0671
0.020
0.0670
0.000
1.0489
0.0075
0.2194
-
PYoCo [ 13 ]
0.0002
0.478
0.0033
0.512
1.000
0.0005
0.0000
-
Appendix
Table 7: Comparison of Gaussianity preservation and computational efficiency between methods that produce temporally correlated noise (1024 × 1024 resolution, C=16 ) driven by optical flow; statistics for our method are computed on Z~ (i.e., γ=0 ), and all GPU times are measured on an NVIDIA RTX PRO 6000 Blackwell GPU.
Figure 5: Motion transfer example: we demonstrate strong motion adherence to the reference video while maintaining high generation quality and fidelity. Prompt used: "Cyberpunk rooftop at night, wet reflective ground, neon city skyline, three humanoid robots standing slightly left of center in a triangular group pose, aligned like three girls in an open field. White-blue robot, black-orange robot, white-red-green robot. Wide composition, large empty foreground and right-side space, glowing Japanese signs, holographic billboards, photorealistic, cinematic, ultra detailed."
Figure 6: Example from MC-Bench: we highlight the disentanglement between the moving subject (masked planet) and the background in our method.
Figure 7: Example from MC-Bench: we highlight the disentanglement between the moving astronaut and the background in our method.
Figure 8: Example from DAVIS: our method shows better motion adherence.
Figure 9: Example from DAVIS: our method shows better motion adherence.
Figure 10: Example from DL3DV: our method shows better camera motion adherence under optical flow control.
Figure 11: Example from DL3DV: our method shows better camera motion adherence under optical flow control.
Figure 12: The effect of noise degradation level γ on generated videos. As degradation decreases, the video adheres more to the input flow.