Synthesizing a monocular video from a new camera trajectory requires preserving the observed scene geometry and motion while completing regions that become newly visible. Feed-forward 4D reconstructors recover the structure of the observed scene, whereas video diffusion models provide a generative prior for completing unseen content. We introduce Holo4D, a video reshooting framework that uses reconstructed 3D trajectories as a holistic source-view 4D scaffold. We project these trajectories into the requested views and encode their geometry and source appearance to guide a pretrained video diffusion model. Together with the source video and target cameras, the scaffold grounds generation in reconstructed observations while leaving unsupported regions to the generative prior. Experiments demonstrate the value of trajectory-based conditioning for camera control and novel-view synthesis. The framework accommodates different 4D reconstructors without retraining the video generator, allowing it to benefit from future advances in source reconstruction. More details can be found in our project page.
Figures & tables
Figure 1: 4D reconstruction as a scaffold for video reshooting. Top: input video with its 3D trajectories (projected onto the image plane for visualization). Middle: reconstruction with the source camera (red) and target cameras (blue) for camera paths from Vista4D’s benchmark. Bottom: Holo4D outputs for each path; the middle frame overlays the projected trajectories (subset shown).
Figure 2: Holo4D overview. Top (4D scaffold): a 4D reconstructor R recovers persistent 3D trajectories, paired with sampled source colors, which are projected into the target cameras and encoded into track features T . Bottom (video diffusion): source and noisy target latents are concatenated along width; each DiT block concatenates T with its hidden features along channels (null on the source half) and adds camera features C encoded from Plücker rays. Flames and snowflakes mark trained and frozen modules ( Appendix B ). Trajectory panels show a subset of the dense tracks; the top-right overlay is for visualization only. The output is obtained by iterative denoising; Figure 11 details the DiT block.
Figure 3: Video reshooting on Vista4D’s benchmark trajectories. Top: dense source geometry, source camera (red), and requested camera path (blue; circle: j=0 , frusta: j=24,48 ). The track condition projects source trajectories into the target view; dark regions have no support. The Holo4D frame at j=24 overlays a subset of the tracks. Red boxes (with magnified insets) mark Vista4D’s errors: the diner is turned toward the camera, and the car loses its rear window. Holo4D uses 50 steps with the two-stage backend (VGGT- Ω + CoWTracker; left) or 4RC (right); baseline inputs and released settings are described in Appendix E .
Figure 4: Novel-view synthesis on iPhone. Rows show spin, block, and teddy at j=24 , with ground-truth target views (GT) on the right. Holo4D uses the two-stage backend and 50 steps.
Method
mPSNR ↑
mSSIM ↑
mLPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
ReCamMaster
10.34
0.478
0.628
10.56
0.326
0.689
TrajectoryCrafter
11.56
0.438
0.569
11.14
0.288
0.631
EX-4D
13.31
0.520
0.464
12.91
0.366
0.532
Gen3C
12.42
0.469
0.489
12.20
0.312
0.550
Vista4D
12.84
0.487
0.484
12.46
0.328
0.535
[0pt][0pt] Holo4D -14B α
13.22
0.484
0.435
12.68
0.326
0.509
Table 1: Novel-view synthesis on iPhone [ 81 ] . α : 4RC; β : two-stage backend; same generator, no GMOS. “m”: masked metrics. Bold/underline: best/second best.
Camera control
VBench [ 84 ]
Method
Control
ATE ↓
RPE-T ↓
RPE-R ↓
Aest. ↑
Imag. ↑
Subj. ↑
Backg. ↑
Temp. ↑
ReCamMaster
Extrinsics
9.547
15.37
0.099
0.493
0.503
0.922
0.942
0.256
TrajectoryCrafter
Point cloud
3.533
4.817
0.207
0.500
0.548
0.894
0.923
0.253
EX-4D
Point cloud
2.903
3.560
0.587
0.442
0.377
0.827
0.899
0.238
Gen3C
Point cloud
5.168
7.581
0.386
0.454
0.450
0.856
0.911
0.248
Vista4D
Point cloud
3.576
3.608
0.132
0.478
0.534
0.904
0.926
0.256
Table 2: Camera control and video quality on DAVIS. Holo4D outperforms rendered-geometry baselines in camera control, imaging quality, and subject and background consistency. ATE: ×102 ; RPE-T: ×103 ; RPE-R: degrees. Bold and underline mark best and second-best results.
Figure 5: Zero-shot robot reshooting. Two DROID [ 85 ] episodes (top: source) reshot along a crane-above path and a near orbit (bottom), at j=0,24,48 . Holo4D -14B uses no robot training data; inference uses 4RC and 50 steps. More examples and track conditions are in Figure 17 .
Method
ATE ↓
RPE-T ↓
RPE-R ↓
PSNR ↑
SSIM ↑
LPIPS ↓
FVD ↓
ReCamMaster
12.68
62.20
0.749
13.66
0.118
0.236
1849
TrajectoryCrafter
5.018
20.61
0.225
13.64
0.256
0.178
1878
EX-4D
8.163
57.89
0.563
12.69
0.273
0.271
1916
Gen3C
5.724
40.03
0.469
13.10
0.103
0.211
1633
Vista4D
5.953
25.19
0.255
14.28
0.285
0.151
1214
[0pt][0pt] Holo4D -14B
1.065
6.154
0.073
16.52
0.441
0.119
973
Table 3: Novel-view synthesis on Syn4D. Ground-truth source geometry at inference removes upstream reconstruction error. ATE: ×102 ; RPE-T: ×103 ; RPE-R: degrees. Bold and underline mark best and second-best results.
Figure 6: Long-video rollout. A 196 -frame DROID [ 85 ] episode reshot to a static target view in five 49 -frame windows (4RC, 50 steps): Holo4D -14B for window 1, the I2V continuation model for windows 2–5 ( Appendix B ). Brackets mark the window boundaries shown.
Condition
ATE ↓
RPE-T ↓
RPE-R ↓
w/o source color
2.327
2.125
0.094
w/o target depth
1.882
2.135
0.068
+ rendered RGB/masks
1.83
2.48
0.098
[0pt][0pt] Full ( Holo4D )
1.621
2.077
0.065
Table 4: Stage-1 conditioning design on DAVIS. All variants condition on trajectories. ATE: ×102 ; RPE-T: ×103 ; RPE-R: degrees.
Source geometry
ATE ↓
RPE-R ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Open-D4RT ∗
1.858
0.151
13.10
0.158
0.169
V-DPM
1.660
0.079
14.96
0.268
0.151
4RC
1.353
0.087
14.98
0.278
0.131
CoWTracker ∗
1.205
0.107
15.09
0.294
0.137
GT geometry
1.065
0.073
16.52
0.441
0.119
α=0.1
3.033
0.221
13.44
0.171
0.172
Table 5: Reconstruction backends and track noise on Syn4D. Top: backends ( ∗ VGGT- Ω depth and cameras); bottom: GT tracks with relative noise α . ATE ×102 ; RPE-R in degrees. No GMOS. Bold/underline: best/second best per block.
Method
VGGT- Ω
Pi3
ATE ↓
RPE-T ↓
RPE-R ↓
TransErr 2 ↓
RotErr ↓
Vista4D
46.04
36.81
0.255
4.573
4.169
Holo4D -14B α
46.31
21.33
1.142
2.164
14.98
Holo4D -14B β
13.76
9.064
0.301
2.367
6.098
Table 6: Camera control on Vista4D’s 110 trajectories. α : 4RC; β : two-stage. ATE ×102 ; RPE-T ×103 ; rotations in degrees.
Overall
LIBERO-plus perturbation category
Initialization
LIBERO
LIB.-plus
Background
Camera
Language
Light
Layout
Robot
Noise
Official release
96.80
50.91
50.09
16.82
69.94
81.79
61.97
46.00
39.41
Control (Wan2.2)
95.60
50.81
56.32
9.01
70.79
85.20
66.62
49.10
31.73
[0pt][0pt] Holo4D init.
96.45
52.10
63.38
8.01
71.57
89.14
66.16
49.16
32.92
Table 7: Policy learning with Fast-WAM. Success (%) after 20 k LIBERO steps. Control and Holo4D differ only in video self-attention initialization; the official release is a separate reference. Bold/underline: best/second best.
Appendix figures & tables29 assets
Supplementary material from the paper’s appendix.
Appendix
Submitted version
This version
Source video
VAE latents, width-concatenated with target latents
Unchanged
RGB-D branch
Target-view RGB, normalized depth, and validity rendered from a static/dynamic point cache; Conv3d geometry tokens
Removed
Track branch
Tracked positions; z-buffer aggregation in the described design; fused with RGB-D tokens
Positions, source color, target-camera depth, and validity; channel-wise max with occupancy; injected along channels in every DiT block
Target cameras
Plücker rays
Plücker rays, encoded into camera features
Backbone
Wan2.1-I2V-14B; Wan2.1-T2V-1.3B
Wan2.1-T2V-14B
Resolution
144×256 , then 288×512
144×256 , then 384×672
Appendix
Table 8: Submitted version versus this version. The submitted column summarizes the described design; the rendering used for the submitted results is distinguished below.
Figure 7: Submitted pipeline (reproduced from the submitted version). Static and dynamic source pixels were rendered into a hybrid target-view RGB-D cache (projected RGB, depth, and visibility), which conditioned the video diffusion model together with Plücker rays and projected 3D tracks. This version removes the rendered RGB-D branch and keeps the track branch ( Figure 2 ).
Figure 9: Submitted qualitative comparison on DAVIS (frames from the submitted version). “Holo4D (submitted)” is the submitted 14B checkpoint with rendered RGB-D and track conditioning; the second row shows source pixels reprojected into the target view (labelled GT in the submission). See Figures 3 and 16 for the final model.
Figure 11: Trajectory conditioning. (a) Each trajectory is projected into target frame j ; its observation zn,j (frame time, normalized position, validity, target-camera depth, and source color) is encoded, scattered to the target grid with a channel-wise maximum and an occupancy channel, and brought to the DiT token grid by a causal 3D adapter, giving T . (b) Source and noisy target latents are concatenated along width once, at the input; in every block, T occupies the target half (null on the source half) and the camera features C from one shared encoder are repeated on both halves. (c) Each of the 40 DiT blocks concatenates T with the normalized hidden features, projects back to width D with W(ℓ) initialized as [I∣0] , and adds C before self-attention; the camera projector is identity-initialized and gated. Flames and snowflakes mark trained and frozen modules (network icons: dark = trained, light = frozen); token shading runs from the first to the last frame. Standard cross-attention/FFN residuals and the auxiliary query-frame input are omitted.
Index
Dataset
Real / Syn.
Dyn. / Static
Depth, cameras
3D tracks
Stage 1
Stage 2
1
Syn4D [ 73 ]
Synthetic
Dynamic
Ground truth
Ground truth
31.65
40.00
2
MultiCamVideo [ 1 ]
Synthetic
Dynamic
VGGT- Ω
CoWTracker
0.00
30.00
3
DL3DV [ 75 ]
Real
Static
DA3 + dataset
Fixed points
32.91
20.00
4
MVS-Synth [ 77 ]
Synthetic
Static
Ground truth
Fixed points
6.33
10.00
5
Kubric [ 74 ]
Synthetic
Dynamic
Ground truth
Box motion
12.66
0.00
6
TartanAir [ 78 ]
Synthetic
Static
Ground truth
Fixed points
12.66
0.00
Appendix
Table 9: Training data. Sampling weights (%) per stage, not proportions of distinct videos. Static scenes need no tracker: backprojected source points stay fixed in 3D. Dynamic pairs come from synchronized multiview videos; static pairs from different chunks of the same video. DA3: DepthAnything 3 [ 4 ] .
Method
Backbone
Res.
ReCamMaster
Wan2.1-T2V-1.3B
480 × 832
TrajectoryCrafter
CogVideoX-5B
384 × 672
EX-4D
Wan2.1-I2V-14B
512 × 512
Gen3C
Cosmos-7B
704 × 1280
Vista4D
Wan2.1-T2V-14B
384 × 672
[0pt][0pt] Holo4D -14B
Wan2.1-T2V-14B
384 × 672
Appendix
Table 10: Backbones and native resolutions.
Method
Steps
Aest. ↑
Imag. ↑
Subj. ↑
Backg. ↑
Temp. ↑
Vista4D α
50
0.478
0.534
0.904
0.926
0.256
Holo4D -14B α
50
0.493
0.601
0.910
0.931
0.253
Holo4D -14B β
50
0.478
0.589
0.891
0.919
0.252
Holo4D -14B β
25
0.471
0.580
0.883
0.910
0.249
Holo4D -5B α
50
0.501
0.559
0.894
0.928
0.253
Holo4D -5B β
50
0.489
0.554
0.881
0.920
0.252
Appendix
Table 11: VBench across reconstruction backends on DAVIS. α : 4RC; β : two-stage backend with GMOS. 5B rows use Wan2.2-TI2V-5B at 704×1280 . Columns follow Table 2 .
Method
Steps
VGGT- Ω
Pi3
ATE ↓
RPE-T ↓
RPE-R ↓
TransErr 2 ↓
RotErr ↓
Vista4D α
50
3.576
3.608
0.132
6.201
2.070
Holo4D -14B α
50
1.657
2.415
0.102
3.206
2.332
Holo4D -14B β
50
0.719
1.299
0.101
3.359
2.284
Holo4D -14B β
25
0.729
1.095
0.077
3.625
1.785
Holo4D -5B α
50
1.464
2.952
0.116
3.195
2.275
Appendix
Table 12: Source geometry and sampling steps on DAVIS. α : 4RC; β : two-stage backend with GMOS. 5B rows use Wan2.2-TI2V-5B at 704×1280 ( 384×672 for † ). † Trained without Syn4D, MVS-Synth, and DL3DV, with a dense track condition without persistent identities. ATE is scaled by 102 ; RPE-T and TransErr 2 by 103 . Rotations are in degrees.
Mask
ATE ↓
RPE-T ↓
RPE-R ↓
TransErr 2 ↓
RotErr ↓
Motion-derived
0.984
1.527
0.114
3.351
2.364
[0pt][0pt] GMOS
0.719
1.299
0.101
3.359
2.284
Appendix
Table 13: Segmentation for static/dynamic track fusion on DAVIS. Two-stage backend with the same first-frame tracks, 384×672 , 50 steps; only the foreground mask differs. ATE: ×102 ; RPE-T and TransErr 2 : ×103 ; rotations: degrees.
Figure 12: GMOS in the two-stage backend. VGGT- Ω supplies depth and cameras and CoWTracker supplies tracks. GMOS segments the moving regions in the query frame; trajectories outside the mask are held at their query-frame 3D positions, and trajectories inside keep their tracked motion. Both sets form one track condition, shown projected into the target view at j=24 . Background regions occluded by the foreground in the query frame have no trajectories. DAVIS blackswan , final checkpoint, 50 denoising steps.
Method
Geometry
ATE ↓
RPE-T ↓
RPE-R ↓
TransErr 2 ↓
RotErr ↓
Vista4D
GT
2.233
12.12
0.079
13.36
0.372
[0pt][0pt] Holo4D -14B
GT
1.299
8.121
0.051
1.881
0.242
[0pt][0pt] Holo4D -5B
GT
1.957
11.19
0.080
5.023
0.515
Vista4D
Pi3
2.348
10.15
0.067
19.26
0.651
[0pt][0pt] Holo4D -14B
Pi3 + GT 2D
1.330
8.158
0.052
5.715
0.534
[0pt][0pt] Holo4D -5B
Pi3 + GT 2D
1.996
10.40
0.084
13.50
0.740
Appendix
Table 14: Camera control with GT and estimated source geometry on Syn4D. Nine held-out clips; Pi3 camera recovery. ATE: ×102 ; RPE-T and TransErr 2 : ×103 ; rotations: degrees. Bold/underline: best/second best per geometry group. Protocols: Appendix E .
Condition
VGGT- Ω
Pi3, source-aligned
Rot. gain ↑
ATE ↓
RPE-T ↓
RPE-R ↓
TransErr 2 ↓
RotErr ↓
Plücker only
15.03
60.92
1.423
0.9866
31.26
0.0218
+ rendered RGB-D
16.21
41.69
1.565
0.7625
33.59
0.0027
+ rendered RGB-D + tracks
16.14
45.22
1.642
0.7117
33.25
0.0096
+ multi-frame RGB-D + tracks
16.39
34.69
1.635
0.6854
33.29
0.0149
Appendix
Table 15: Conditioning in early 1.3B checkpoints. Rotation gain is the fitted ratio of generated to requested rotation. ATE is scaled by 102 ; RPE-T by 103 . Rotations are in degrees.
Condition
ATE ↓
RPE-T ↓
RPE-R ↓
Conditioning variants (training not matched)
Plücker rays only
10.65
7.32
0.373
PCD(RGB-D) + valid mask, no tracks
10.36
9.16
0.236
PCD(RGB-D) + 3D tracks
2.35
2.38
0.086
Tracks without per-track color
2.33
2.13
0.094
[0pt][0pt] 3D tracks only ( Holo4D )
1.62
2.08
0.065
Appendix
Table 16: Track and rendered conditioning on DAVIS. PCD denotes rendered point clouds. The unmatched upper block is not ranked; lower-block bold and underline mark best and second-best results. ATE and RPE-T are scaled by 102 and 103 ; rotations are in degrees.
Configuration
mPSNR ↑
mSSIM ↑
mLPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Plücker rays only
11.23
0.442
0.630
11.36
0.281
0.717
+ RGB render
11.91
0.466
0.611
11.95
0.314
0.725
+ depth + valid mask
11.93
0.453
0.640
11.93
0.296
0.758
+ 3D tracks (single-frame)
13.58
0.477
0.490
13.20
0.319
0.568
+ multi-frame static cache
13.65
0.479
0.497
13.28
0.322
0.581
Appendix
Table 17: Earlier conditioning variants on DyCheck. Source geometry: (a) 4RC; (b) two-stage backend. “m” denotes masked metrics; bold and underline mark best and second-best results per panel.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
mPSNR ↑
mSSIM ↑
mLPIPS ↓
ViDAR [ 26 ]
12.90
0.385
0.734
13.01
0.525
0.653
[0pt][0pt] Holo4D -14B α
12.68
0.326
0.509
13.22
0.484
0.435
[0pt][0pt] Holo4D -14B β
13.20
0.348
0.465
14.03
0.507
0.379
Appendix
Table 18: Novel-view synthesis compared with ViDAR. Under Vista4D’s iPhone protocol, ViDAR is optimized per scene on the first 49 source frames, center-cropped and resized to 288×512 . We evaluate 49 target frames, unlike ViDAR’s full-sequence benchmark. α : 4RC; β : two-stage backend. “m” denotes masked metrics. Bold and underline mark best and second-best results.
Figure 13: Syn4D with ground-truth source geometry. Two held-out pairs with large viewpoint changes; the right column is the rendered target view. Rendered-geometry baselines reproject ground-truth source depth, Holo4D receives ground-truth 3D trajectories, and ReCamMaster uses only the cameras ( Table 3 ). EX-4D generates square frames, shown as a central crop. Holo4D places the moving objects as in the ground truth; all methods differ from it in unobserved regions.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
ATE ↓
RPE-T ↓
RPE-R ↓
FVD ↓
ReCamMaster
12.40
0.277
0.761
11.69
63.79
0.776
1908
TrajectoryCrafter
9.418
0.176
0.725
9.636
39.98
0.522
3073
EX-4D
9.865
0.163
0.863
18.76
89.83
2.438
2625
Gen3C
11.72
0.216
0.686
9.336
52.28
0.395
1693
Vista4D
10.98
0.158
0.668
9.972
41.56
0.419
1525
[0pt][0pt] Holo4D -14B
11.70
0.180
0.667
3.267
32.09
0.272
1711
Appendix
Table 19: Novel-view synthesis on Syn4D with 4RC source geometry. Geometry-conditioned methods receive 4RC source geometry in their native form; ReCamMaster uses no geometry. FVD uses the paired target videos. ATE and RPE-T are scaled by 102 and 103 ; rotations are in degrees. Bold/underline: best/second best.
Syn4D (exact depth)
iPhone (pseudo-reference)
VGGT- Ω
4RC
VGGT- Ω
4RC
Method
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
ReCamMaster
0.762
0.311
0.738
0.298
0.152
0.765
0.121
0.835
TrajectoryCrafter
0.673
0.367
0.713
0.337
0.109
0.848
0.098
0.881
EX-4D
0.673
0.298
0.766
0.232
0.109
0.823
0.112
0.877
Gen3C
0.740
0.337
0.736
0.326
0.098
0.862
0.112
0.895
Appendix
Table 20: Depth reconstructed from generated videos. Syn4D uses exact target depth; iPhone uses reconstructed target-video depth as a pseudo-reference. α : 4RC; β : two-stage source backend, without GMOS. Evaluation and aggregation are specified in Appendices F and I .
Source geometry
ATE ↓
RPE-T ↓
RPE-R ↓
PSNR ↑
SSIM ↑
LPIPS ↓
FVD ↓
VGGT- Ω + Open-D4RT
1.858
11.22
0.151
13.10
0.158
0.169
1337
V-DPM
1.660
8.904
0.079
14.96
0.268
0.151
1252
4RC
1.353
7.148
0.087
14.98
0.278
0.131
1168
VGGT- Ω + CoWTracker
1.205
7.688
0.107
15.09
0.294
0.137
1216
Ground-truth tracks
1.065
6.154
0.073
16.52
0.441
0.119
1098
GT tracks, α=0.1
3.033
22.53
0.221
13.44
0.171
0.172
1375
Appendix
Table 21: Source geometry and track noise on Syn4D. The same generator receives estimated source geometry from four reconstruction backends (top; no GMOS), ground-truth tracks (middle), or ground-truth tracks with Gaussian noise whose standard deviation is α times that of the source tracks (bottom). 32 held-out pairs, 25 denoising steps. ATE and RPE-T are scaled by 102 and 103 ; rotations are in degrees. FVD uses an earlier 256×256 scoring and is comparable only within this table. Bold/underline rank estimated backends.
Figure 14: Changing the source reconstructor without retraining. The same checkpoint receives ground-truth trajectories or trajectories from four reconstruction backends, with target cameras, sampler, and seed fixed ( Table 21 ). Geometric errors change the generated layout and appearance, as seen with V-DPM and Open-D4RT.
Stage
Time (s)
Share
4RC feed-forward
1.46±0.02
1.5%
Static cache + rendering
7.54±0.30
7.8%
Dynamic render + fusion
1.43±0.31
1.5%
3D-track encoding
0.06±0.00
0.1%
Denoising + decoding
86.50±0.26
89.2%
Warm-pipeline total
96.99±0.67
100%
Appendix
Table 22: Warm inference runtime on one H200. Times are mean ± standard deviation over four videos, including reconstruction, proxy rendering, and generation but excluding model loading and video writing.
Table 35
Figure 15: Additional video reshooting examples. Source and generated frames for six trajectories from Vista4D’s benchmark (top) and six from our DAVIS evaluation set (bottom), named above each example. The middle generated frame ( j=24 ) overlays a subset of the projected trajectories, with trails covering the previous 12 frames. All examples use the two-stage backend and 50 denoising steps; none appears in the other figures.
Figure 16: Comparison with all baselines on two further Vista4D trajectories. Left: a dolly-out from a golfer that reveals a golf cart; right: a rising, zooming-out crane shot of a man on a hillside. Layout as in Figure 3 . TrajectoryCrafter, EX-4D, and Gen3C render the cart incomplete or distort the people, EX-4D leaves large holes, and ReCamMaster stays close to the source framing; Vista4D and Holo4D both follow the requested paths. Holo4D -14B uses 4RC source reconstruction and 50 denoising steps; baselines use the same settings as in Figure 3 .
Figure 17: Zero-shot novel-view synthesis of robot videos. Each example shows the source video (top), projected 3D trajectories with source colors (middle; dark regions have no support, and trails show a subset over the preceding 12 frames), and the generated view (bottom). Top: DROID [ 85 ] episodes with 4RC reconstruction; bottom: humanoid clips with the two-stage backend. Holo4D -14B uses no robot training data and no fine-tuning; 50 denoising steps.
Figure 18: More long-video rollouts. From top: three DROID [ 85 ] episodes of 196 , 214 , and 246 frames ( 5 , 5 , and 6 windows) reshot to static target views, and a 94 -frame humanoid clip ( 2 windows) reshot along a close crane-above path. Six evenly spaced frames per video. Window 1 uses Holo4D -14B with 4RC (DROID) or the two-stage backend (humanoid); later windows use the I2V continuation model, trained with OmniWorld-DROID, and 4RC; 50 steps ( Appendix B ).
Figure 19: Content outside the initial scaffold. The marked person enters the source view late and has no first-frame query points, so generated views must complete them without trajectory support (no target-view ground truth is available).
LIBERO
LIBERO-plus
Suite
Official
Ctrl.
Ours
Tasks
Official
Ctrl.
Ours
Spatial
97.40
97.00
97.60
2,402
54.58
59.87
55.08
Object
99.40
99.20
99.20
2,518
69.46
62.43
65.45
Goal
96.40
95.20
96.00
2,591
37.32
38.67
44.89
LIBERO-10
94.00
91.00
93.00
2,519
42.83
43.03
43.35
All
96.80
95.60
96.45
10,030
50.91
50.81
52.10
Appendix
Table 25: Fast-WAM details. Closed-loop success rates (%) per suite on LIBERO ( 500 trials per suite) and LIBERO-plus (one trial per task); All : suite mean (LIBERO), task-weighted mean (LIBERO-plus). Official: released Fast-WAM checkpoint; Ctrl.: Wan2.2-initialized control; Ours: Holo4D self-attention initialization.
Asset
License and scope
Training and evaluation data
Syn4D
CC BY 4.0; underlying assets retain their individual terms.
MultiCamVideo , DyCheck iPhone
Apache 2.0 (dataset releases).
DL3DV-10K
CC BY-NC 4.0 and DL3DV-10K Terms of Use.
MVS-Synth
Custom terms: research and educational use only.
Kubric , RealEstate10K
Apache 2.0 (code); rendered assets/source videos have separate terms.
Appendix
Table 26: External asset licenses. Asset names link to official license notices or release cards. Code licenses do not automatically cover separate weights or source footage.
We present Vista4D, a robust and flexible video reshooting framework that grounds the input video and target cameras in a 4D point cloud. Specifically, given an input video, our method re-synthesizes the scene with the same dynamics from a different camera trajectory and viewpoint. Existing video reshooting methods often struggle with depth estimation artifacts of real-world dynamic videos, while also failing to preserve content appearance and failing to maintain precise camera control for challenging new trajectories. We build a 4D-grounded point cloud representation with static pixel segmentation and 4D reconstruction to explicitly preserve seen content and provide rich camera signals, and we train with reconstructed multiview dynamic data for robustness against point cloud artifacts during real-world inference. Our results demonstrate improved 4D consistency, camera control, and visual quality compared to state-of-the-art baselines under a variety of videos and camera paths. Moreover, our method generalizes to real-world applications such as dynamic scene expansion and 4D scene recomposition. See our project page for results, code, and models: https://eyeline-labs.github.io/Vista4D
Precise camera control for reshooting dynamic videos is bottlenecked by the severe scarcity of paired multi-view data for non-rigid scenes. We overcome this limitation with a highly scalable self-supervised framework capable of leveraging internet-scale monocular videos. Our core contribution is the generation of pseudo multi-view training triplets, consisting of a source video, a geometric anchor, and a target video. We achieve this by extracting distinct smooth random-walk crop trajectories from a single input video to serve as the source and target views. The anchor is synthetically generated by forward-warping the first frame of the source with a dense tracking field, which effectively simulates the distorted point-cloud inputs expected at inference. Because our independent cropping strategy introduces spatial misalignment and artificial occlusions, the model cannot simply copy information from the current source frame. Instead, it is forced to implicitly learn 4D spatiotemporal structures by actively routing and re-projecting missing high-fidelity textures across distinct times and viewpoints from the source video to reconstruct the target. At inference, our minimally adapted diffusion transformer utilizes a 4D point-cloud derived anchor to achieve state-of-the-art temporal consistency, robust camera control, and high-fidelity novel view synthesis on complex dynamic scenes.
Recent advances in video diffusion enable novel-view video synthesis from a single input video, supporting world modeling, embodied intelligence, and robotic simulation. However, existing methods process camera trajectories independently, lacking the cross-trajectory consistency needed for coherent, navigable 4D worlds. We introduce Demiurge, an autoregressive video rerendering pipeline built on a dynamic 4D point-cloud memory cache. It promotes cross-trajectory consistency through four components: (1) an appearance video diffusion model that synthesizes photorealistic novel-view videos conditioned on the 4D memory; (2) a memory-consistent video depth completion model that aligns known-region depths with cached geometry and completes missing regions, mitigating scale ambiguity and discontinuities during memory updates; (3) memory-aware dual-stream warping that provides high-fidelity visual guidance and occlusion cues for novel-view synthesis; and (4) geometry-aware video refinement that enhances perceptual quality. Extensive experiments demonstrate state-of-the-art performance and three key capabilities: coherent spatiotemporal lattice construction over dense multi-view trajectories; large-parallax video rerendering under challenging camera motions, including 180-degree rotations with no input-view overlap; and geometry-faithful outputs that can be directly optimized into high-quality 3D Gaussian Splatting representations with minimal artifacts. Project page: https://hanxinzhu-lab.github.io/demiurge_project_page/.