Video generation and feed-forward 3D reconstruction typically use separate representations, requiring RGB decoding and visual re-encoding to recover geometry from generated latents. We propose Latent Bridge, which connects video generative latent spaces with geometric feature spaces to directly predict 3D geometry from video latents. We develop representation-specific bridges for Wan2.1 and V-RAE, enabling feed-forward prediction of cameras, depth, and 3D point clouds without the intermediate RGB pathway. Each bridge is trained on RGB videos through feature distillation, while the video encoder and geometry model remain frozen. Training requires neither ground-truth geometry nor execution of the downstream geometric aggregator or prediction heads. At inference, the same video latents support both RGB decoding and geometry prediction. The same trained bridge can be reused for image-to-video and text-to-video generation within the same video latent space, without bridge retraining. Experiments on Co3Dv2 and WildRGB-D demonstrate competitive geometric reconstruction quality, efficient end-to-end generation and reconstruction, and faster latent-to-geometry decoding, highlighting latent alignment as a practical and reusable interface between video generation and 3D reconstruction. Our code is available at: https://github.com/hustvl/Latent-Bridge.
Figures & tables
Figure 1: Bypass the video decoder and directly align latent spaces to generate 3D outputs.
Figure 2: Overview of Latent Bridge, showing (a) feature alignment training, (b) video generation and geometry prediction from a shared latent, and (c) the architectures of bridges.
Method
Co3Dv2
WildRGB-D
Latency ↓
Accuracy ↓
Completeness ↓
CD ↓
Accuracy ↓
Completeness ↓
CD ↓
Aether ( Zhu et al., 2025 )
0.6982
0.5437
0.6209
0.0591
0.1340
0.0965
181.33
FantasyWorld ( Dai et al., 2026 )
0.5939
0.3698
0.4818
0.0440
0.1258
0.0849
448.63
Gen3R ( Huang et al., 2026 )
0.3820
0.1796
0.2808
0.0247
0.1129
0.0688
189.57
Ours (Wan2.1)
0.3536
0.1833
0.2685
0.0189
0.1209
0.0699
70.09
Ours (V-RAE)
0.3231
0.2093
0.2662
0.0237
0.1076
0.0657
165.57
Table 1: Geometry reconstruction results on Co3Dv2 and WildRGB-D. Geometry metrics are measured in meters and latency in seconds. The best and second-best results are highlighted in pink and yellow, respectively. Aether uses 41 frames, whereas the other methods use 49.
Figure 3: Visual comparison of generation and reconstruction results, including the generated videos, depth maps, and point clouds produced by each method.
Method
Co3Dv2
WildRGB-D
PSNR ↑
SSIM ↑
LPIPS ↓
I2V Subj. ↑
I2V BG ↑
I.Q. ↑
PSNR ↑
SSIM ↑
LPIPS ↓
I2V Subj. ↑
I2V BG ↑
I.Q. ↑
Aether
13.9595
0.5689
0.5942
90.64
92.13
38.46
12.4790
0.4466
0.5660
95.82
95.94
52.60
FantasyWorld
12.0307
0.4807
0.6741
89.08
91.27
46.20
12.9733
0.4385
0.5116
97.07
96.72
59.41
Gen3R
16.5289
0.6027
0.4405
94.79
96.05
45.20
15.2252
0.5075
0.4025
98.38
98.20
56.58
Ours (V-RAE)
16.3780
0.6025
0.4303
94.91
95.99
47.84
16.5769
0.5516
0.3217
97.87
97.60
56.85
Ours (Wan2.1)
17.0388
0.6166
0.4121
95.42
96.41
46.73
16.1703
0.5412
0.3394
98.44
98.24
59.67
Table 2: Video Generation Quality Comparison. All results are evaluated at 560×560 resolution. I2V Subj., I2V BG, and I.Q. are multiplied by 100.
Method
CO3Dv2
WildRGB-D
Time (s) ↓
Accuracy ↓
Completeness ↓
CD ↓
Accuracy ↓
Completeness ↓
CD ↓
Latent → RGB → VGGT- Ω (Wan2.1)
0.3559
0.1754
0.2657
0.0174
0.1227
0.0700
7.92
Latent → Bridge → VGGT- Ω (Wan2.1)
0.3536
0.1833
0.2685
0.0189
0.1209
0.0699
4.92
Latent → RGB → VGGT- Ω (VRAE)
0.3241
0.1925
0.2583
0.0209
0.1114
0.0661
5.76
Latent → Bridge → VGGT- Ω (VRAE)
0.3231
0.2093
0.2662
0.0237
0.1076
0.0657
4.64
Table 3: Comparison of RGB Decoding Solutions. Latency covers latent-to-geometry inference and excludes diffusion sampling. RGB decoding is included in the RGB baseline but bypassed by the Bridge.
Figure 4: Ablation study on the number of parameters.
Structural ablation
Norm. L1 ↓
Depth L1 ↓
AbsRel ↓
Loss ablation
Norm. L1 ↓
Depth L1 ↓
AbsRel ↓
w/o 2×2 PixelUnshuffle
0.18057
0.15849
0.10628
w/o Cosine loss
0.18823
0.16459
0.10851
w/o Causal Conv3D + phase rearrangement
0.19177
0.16593
0.10766
w/o Delta loss
0.18780
0.16235
0.10461
w/o Spatiotemporal mixing blocks
0.25196
0.22100
0.15228
w/o Relation loss
0.18412
0.15863
0.10313
Simple baseline (joint replacements)
0.31666
0.27490
0.19346
Full Bridge & loss
0.18133
0.15644
0.10163
Table 4: Ablations of the Wan Bridge at 50K optimization steps. All variants are evaluated on the same 64 clips of 49 frames.
Figure 5: The Bridge can adapt to generative models for different tasks that use the same VAE, without retraining.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Pretrained Checkpoint Reuse without Further Training
Trainable Modules and Token Overhead.
Cross-DiT checkpoint
T2V/I2V checkpoint
Pretrained task-head
Trainable components
Additional Geometry Tokens
FantasyWorld
No
No
No
Geometry branch & adapters
Yes
Gen3R
No
No
Yes
DiT & geometry adapter
Yes
Beyond Pixels
Yes
Yes
No
Alignment module, LoRA & heads
No
VIST3A
Yes
-
No
Stitching layer & geometry/DiT LoRA
No
Latent Bridge
Yes
Yes
Yes
Bridge only
No
Appendix
Table 5: Differences Between Our Method and Existing Approaches.
Supervision
Trainable modules
Params (M) ↓
Time (s) ↓
Memory (GiB) ↓
Feature (ours)
Bridge
35.65
0.166
3.17
Geometry
Bridge
35.65
5.896
31.27
Geometry
Bridge + geometry model
876.31
6.375
35.48
Appendix
Table 6: Computational cost of different supervision strategies. All settings use the same 6000 Pro GPU with a batch size of 1 and 49 frames at 560×560 resolution.
Figure 6: Downstream reuse with a frozen VGGT-S module. Given the same first-frame mask prompt, the RGB and Bridge routes propagate object masks across generated frames. Predicted geometry enables object-level 3D labeling.
Figure 7: Additional downstream reuse examples showing generated RGB frames, propagated object masks, and the corresponding label-colored 3D point clouds.