Organizations: East China Normal University, Shanghai, China. · King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia. · Qwen Team, Alibaba Group, Hangzhou, China. · The Chinese University of Hong Kong, Hong Kong SAR, China.
Controllable video generation has recently achieved impressive visual quality, yet existing control paradigms remain poorly suited to embodied navigation. Text prompts are spatially ambiguous, while camera-trajectory control relies on low-level motion specifications that are unintuitive for human users and difficult to manipulate. In this work, we introduce Map2Vid, a novel task that reformulates embodied navigation video generation as a BEV-map-to-video synthesis problem, together with a large-scale benchmark for systematic evaluation. By using BEV maps, which provide a visual representation of global scene layout and intended navigation routes, Map2Vid enables human-friendly navigation video generation. To support this task, we construct a dedicated dataset with over 100K navigation sequences across 61 indoor scenes, each paired with egocentric videos, BEV maps, camera trajectories and language instructions. We further develop a unified framework that integrates multiple control signals, together with a curriculum training strategy that progressively weakens non-map inputs, enabling flexible inference under both full conditioning and map-only input. Extensive experiments across multiple foundation generation models demonstrate that controllable navigation video generation via BEV map is a viable and promising direction.
Figures & tables
Fig. 2 : Overview of the benchmark. It consists of three components: (1) Data Construction : synchronized multimodal data collection in the Habitat simulator. (2) Dataset Statistics : covering diverse scene semantics and a wide range of navigation distances, highlighting both the diversity and scale of the benchmark. (3) Evaluation Metrics : comprehensive assessment along two complementary dimensions, visual fidelity and geometric consistency.
Fig. 3 : Overview of the framework and training strategy. (Top) Architecture: We adopt a hybrid condition injection on a DiT-based backbone. Instruction q and the first frame x0 are injected through the model’s native pathways, while camera trajectory τ and BEV map m are encoded via Plücker embeddings and temporal broadcasting with a VAE Encoder, respectively, and then fused via a fusion layer. (Bottom) Training: A progressive masking schedule gradually weakens non-map conditions, with modality-specific designs.
Method
Condition
FID ↓
FVD ↓
LPIPS ↓
SFM-SR ↑
APEtrans↓
APErot↓
Wan2.2-Camera ∘
ALL
47.64
336.02
0.538
58.7
1.93
112.36
BEV
66.37
457.32
0.568
51.3
1.96
116.29
HunyuanVideo V1.5 ∘
ALL
42.78
516.00
0.589
56.6
1.73
108.79
BEV
196.27
1255.75
0.652
23.7
2.26
122.32
Open-Sora V1.3 ∘
ALL
57.08
441.69
0.539
46.3
2.11
120.68
BEV
161.05
1129.96
0.612
27.0
2.46
124.84
TABLE I : Quantitative comparison of our framework built upon different base models for the Map2Vid task. The symbol ∘ denotes methods adapted by us. We report performance under two inference settings: “All” (utilizing the full condition set) and “BEV” (conditioned exclusively on the BEV map). The best and second-best results are highlighted with darker and lighter shades, respectively, using purple for visual quality and blue for geometric consistency.
Instruction
First Frame
Trajectory
BEV Map
FID ↓
FVD ↓
LPIPS ↓
SfM-SR ↑
APEtrans↓
APErot↓
-
-
✓
-
86.38
856.80
0.568
33.3
2.10
120.28
-
-
-
✓
66.37
457.32
0.568
51.3
1.96
116.29
-
-
✓
✓
91.31
948.52
0.593
43.3
2.17
119.28
✓
✓
-
-
61.22
673.08
0.530
35.0
2.37
123.80
✓
✓
✓
-
62.92
650.05
0.532
41.6
2.25
112.70
✓
✓
-
✓
60.41
663.08
0.530
50.0
2.37
121.58
TABLE II : Ablation on inference conditions. The best and second-best results are highlighted with darker and lighter shades, respectively, using blue for visual quality and cyan for geometric consistency.
Fig. 4 : Quantitative comparison of training strategies. We compare our method with four ablation variants: w/o Aggressive Masking, Constant Masking, w/o Warmup, and w/o Fusion. The results demonstrate that our proposed curriculum strategy significantly improves geometric alignment without compromising visual quality compared to the baselines.
Metric
Original
Cross-Scene
Sim2Real
FID ↓
66.37
160.35
188.32
FVD ↓
457.32
2980.71
2319.96
LPIPS ↓
0.568
0.594
0.612
SFM-SR ↑
51.3
44.3
39.2
APEtrans↓
1.96
4.36
5.25
APErot↓
116.29
114.43
126.92
TABLE III : Quantitative evaluation of zero-shot generalization capabilities for the Map2Vid task. All evaluations are conducted using the fine-tuned Wan2.2-Camera model under the BEV-only conditioning setting.
Metric
Camera Trajectory
BEV Map
Understandability
0%
100%
Editability
0%
100%
Overall Preference
0%
100%
TABLE IV : User study on the human-friendliness of navigation control representations. Results report the percentage of participants preferring each representation across three usability criteria.
VLM
Top-1 Acc. (%) ↑
Gain (pp)
Qwen3
50.0
+16.7
ChatGPT-5.5
70.0
+36.7
Gemini-3.1 Pro
60.0
+26.7
Random Guess
33.3
–
TABLE V : Top-1 accuracy of VLM-based route verification using Map2Vid-generated videos. Each instruction has three candidate BEV routes: one ground-truth route and two perturbations. Gain is measured relative to random selection (33.3%).
Fig. 5 : Qualitative results. From top to bottom: ground-truth video, and generated results via full conditions and BEV map, respectively.
Fig. 6 : Qualitative comparison between the ground-truth BEV route(left) and the trajectory reconstructed from a Map2Vid-generated video using VGGSfM(right). The recovered route follows the overall navigation structure with local deviations.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Condition
FID ↓
FVD ↓
LPIPS ↓
SFM-SR ↑
APEtrans↓
APErot↓
Constant Masking
ALL
69.38
737.83
0.566
61.6
2.32
121.48
BEV
79.65
885.15
0.583
53.3
2.11
116.92
w/o Aggressive Masking
ALL
63.04
660.11
0.559
58.7
2.16
119.34
BEV
78.62
850.32
0.592
51.3
2.01
113.69
w/o warmup
ALL
68.32
701.85
0.578
56.6
2.11
117.42
BEV
81.22
902.52
0.623
46.6
2.18
120.46
Appendix
TABLE VI : Quantitative evaluation of ablation studies on model architecture and training strategies. Complementing the qualitative comparison in Fig. 4 of the main text, this table details the performance of different variants under both full control (ALL) and map-only (BEV) settings. The results demonstrate that our proposed geometric fusion and curriculum learning strategy achieve superior geometric consistency (SFM-SR, APE) while maintaining high visual quality, validating the effectiveness of these components.
Fig. 7 : Qualitative comparison under different control conditions. We visualize results for three different scenes. In each group, the first column displays the input Conditions , and the second column shows the Ground Truth frames. The third column ( All ) presents the generation results conditioned on both the BEV Map and camera trajectory, while the fourth column ( BEV Map ) shows the results conditioned only on the BEV Map.
Fig. 8 : Qualitative comparison under different control conditions.
The authors are with the Intelligent Space Robotics Laboratory, Skolkovo Institute of Science and Technology Moscow, Bolshoy Boulevard 30, bld. 1, 121205, Moscow, Russia.