Deploying generative world models for end-to-end closed-loop evaluation and policy learning requires precise scenario control, consistent observations, and sustained generation quality during extended interactions. This report presents X-World~1.6, advancing X-World~1.0 toward a more reliable and versatile driving simulator. The upgraded base model offers finer-grained control over traffic semantics, including traffic lights and road markings, together with improved spatial consistency across cameras. Virtual City capabilities broaden scenario construction through a family of models conditioned on diverse 2D and 3D references, including support for generating multi-camera future videos from a single image. An enhanced post-training framework enables efficient causal, few-step generation with stronger temporal coherence, better quality retention over longer rollouts, and improved alignment with human preferences. We also introduce a dedicated evaluation suite to support systematic assessment and guide model iteration. Together, these upgrades expand the scenarios X-World can simulate and improve the efficiency and reliability of its generated observations, supporting broader use in autonomous-driving development.
Figures & tables
Figure 1: (a) Data curation through quality filtering, behavior tagging, and scenario selection; flow widths are illustrative. (b) Tag distribution across ego behavior, environment, and conditions; bar lengths indicate within-category percentages.
Figure 2: Base-model conditioning paths. Separate models use visual references or 3D scene representations to guide seven-camera generation. Dashcam conditioning is a related synchronized-completion capability.
Figure 3: Generation without and with 3DGS absolute static scene (ASC) conditioning, alongside ground truth. Each row shows a different scene.
Figure 4: Two-stage training pipeline. Block 01 develops the controllable base model. Blocks 02–06 adapt it for causal few-step generation, restore cross-chunk feedback, extend the training horizon, and apply reward-guided refinement.
Obs/s ↑
Denoiser (s) ↓
Step latency (s) ↓
Configuration
PPU
Node
P50
Mean
P50
Mean
P99
Speedup ↑
Reference kernels
0.156
0.625
5.96
5.92
19.04
18.68
23.80
1.00 ×
+ Channels-last VAE with fused causal input
0.201
0.803
4.48
4.55
14.70
14.46
18.94
1.28 ×
+ Fused DiT kernels
0.218
0.873
3.95
4.16
13.42
13.25
18.36
1.40 ×
+ AIU convolutions
0.237
0.949
3.69
3.80
12.49
12.22
15.23
1.52 ×
+ X-Cache
0.320
1.281
2.10
2.62
8.69
8.94
17.82
2.05 ×
Table 1: Steady-state closed-loop serving throughput, server denoiser forward time, and per-session end-to-end step latency on a 4 × PPU M810E node (12 concurrent sessions) with optimizations enabled cumulatively. Best and second-best values in each column are bold and underlined, respectively.
Metric
Our evaluator
Qwen3-VL-8B (zero-shot)
Evaluation set
Same-scene pairwise accuracy (%)
71.4 [53.6, 88.5]
21.4 [7.4, 37.9]
28 scenes
Video-penalty Spearman ρ
0.671 [0.476, 0.815]
0.097 [ −0.181 , 0.378]
60 videos
Video-penalty Pearson r
0.867 [0.540, 0.943]
−0.074 [ −0.238 , 0.207]
60 videos
Table 2: Agreement with human evaluation on paired X-World 1.0 and X-World 1.6 generations. Brackets give 95% confidence intervals from a scene-level cluster bootstrap. The zero-shot baseline uses Qwen3-VL-8B-Instruct with the same inputs and prompts. The three metrics capture complementary aspects of agreement. Same-scene pairwise accuracy measures how often the VLM selects the video preferred by humans within a scene. Across all 60 videos, Spearman ρ measures agreement in quality ordering using penalty ranks, while Pearson r measures the strength of the linear relationship between the actual penalty values. Spearman therefore focuses on ordering, whereas Pearson also reflects relative score differences; high Pearson correlation does not require identical score scales or absolute values.
Tracking
Compression
Reconstruction
Overall
Model / Variant
App. ↑
Decay ↑
Ray ↓
Δ SSIM ↑
Δ PSNR ↑
Bits ↓
SSIM ↑
PSNR ↑
MAE ↓
Avg. Rank ↓
X-World 1.0
0.8832
-0.0257
0.1836
0.0333
0.8587
0.6305
0.7238
21.4834
14.3622
3.83
SF
0.8856
-0.0241
0.1813
0.0336
0.8947
0.6277
0.7279
21.2349
14.6864
3.56
CSF
0.8873
-0.0234
0.1686
0.0339
0.9183
0.6305
0.7272
21.2381
14.7831
2.94
SF+LongTuning
0.8857
-0.0230
0.1680
0.0322
0.8160
0.6445
0.7300
21.5293
14.4123
2.78
CSF+LongTuning
0.8868
-0.0228
0.1691
0.0353
1.0039
0.6244
0.7286
21.3137
14.6838
1.89
Table 3: Consistency comparison with X-World 1.0 and training variants. SF denotes Self Forcing, CSF denotes Connected Self Forcing, and LongTuning denotes long-horizon tuning. Best and second-best results in each column are shown in bold and underlined, respectively; missing values are excluded. App. denotes Mean Appearance Consistency, Decay denotes Mean Appearance Decay Rate, Ray denotes Mean Ray Residual (m), and Bits denotes the inter/intra encoded-size ratio. A less negative decay slope indicates slower loss of appearance similarity with temporal separation. Both PSNR quantities are measured in dB, and reconstruction MAE uses pixel intensity units (0–255). Avg. Rank is the mean rank across the nine metrics among the five models, with average ranks assigned to ties; lower is better.
Figure 5: Qualitative comparison of temporal consistency between Self Forcing (SF) and Connected Self Forcing (CSF). Each pair of rows shows synchronized frames from SF (top) and CSF (bottom). From top to bottom, the examples compare vehicle appearance, sky color, gate details, and road structure. Blue and orange boxes mark selected earlier and later regions, respectively, with corresponding enlarged views and timestamps shown in the rightmost column.
Figure 6: Shared interaction pipeline for simulation and RL. Conceptual illustration of scene-conditioned generation and the policy–environment loop. Interaction records support behavioral evaluation; the RL branch adds training feedback and policy updates.
Metric
Change
Mean FDE (relative)
−20.9%
Share with FDE <1 m
+5.6 pp
Share with FDE <2 m
+9.4 pp
Share with FDE <5 m
+10.6 pp
Table 4: Trajectory consistency and target-behavior reproduction: X-World 1.6 versus 1.0.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Compression-based temporal consistency. (a) Inter-frame and all-intra coding of the same video provide quality gains and an encoded-size ratio. (b) Conceptual illustration of larger prediction residuals from irregular appearance changes under the same motion.
Figure 9: One Packed NFT. Sequential evaluation interleaves target prediction and clean-cache extension; packed evaluation uses a detached clean KV bank while preserving the same causal visibility. Both use frozen old-model trajectories.
Scalable and reliable evaluation is increasingly critical in the end-to-end era of autonomous driving, where vision--language--action (VLA) policies directly map raw sensor streams to driving actions. Yet, current evaluation pipelines still rely heavily on real-world road testing, which is costly, biased toward limited scenario coverage, and difficult to reproduce. These challenges motivate a real-world simulator that can generate realistic future observations under proposed actions, while remaining controllable and stable over long horizons. We present X-World, an action-conditioned multi-camera generative world model that simulates future observations directly in video space. Given synchronized multi-view camera history and a future action sequence, X-World generates future multi-camera video streams that follow the commanded actions. To ensure reproducible and editable scene rollouts, X-World further supports optional controls over dynamic traffic agents and static road elements, and retains a text-prompt interface for appearance-level control (e.g., weather and time of day). Beyond world simulation, X-World also enables video style transfer by conditioning on appearance prompts while preserving the underlying action and scene dynamics. At the core of X-World is a multi-view latent video generator designed to explicitly encourage cross-view geometric consistency and temporal coherence under diverse control signals. Experiments show that X-World achieves high-quality multi-view video generation with (i) strong view consistency across cameras, (ii) stable temporal dynamics over long rollouts, and (iii) high controllability with strict action following and faithful adherence to optional scene controls. These properties make X-World a practical foundation for scalable and reproducible evaluation.
Driving world models provide a promising route toward scalable counterfactual data generation and interactive simulation beyond recorded driving logs. Realizing this potential requires a system that can generalize across diverse scenes, respond faithfully to prescribed controls, generate coherent multi-sensor observations, and operate efficiently under repeated inference. We present HelloWorld, a 2B driving world model system designed around these requirements. HelloWorld progressively specializes broad visual and motion priors from heterogeneous video data into controllable driving generation using ego pose, HD maps, and 3D boxes. A block-causal generation interface, together with adaptation to self-generated context, aligns the model with sequential simulation. The system further supports synchronized seven-camera RGB generation and conditional LiDAR synthesis, and is distilled toward few-step inference for efficient deployment. Experiments evaluate visual quality, control fidelity, cross-view consistency, robustness under repeated generation, inference efficiency, and LiDAR synthesis. Together, HelloWorld provides a unified framework for scalable driving data generation and interactive simulation.
Driving-world generation has emerged as a core capability for scalable autonomous-driving simulation, yet existing methods remain limited in object-level controllability and long-horizon stability. We present M4World, a Multi-view and Multimodal generative driving world model that synthesizes future surround-view video streams and synchronized LiDAR scans while supporting interactive object Manipulation and stable Minute-long streaming. Fine-grained object manipulation is realized through a flexible conditioning interface that supports explicit control over both the spatial layout and visual appearance of individual objects. Stable minute-long streaming, on the other hand, is achieved through a multi-stage training framework that enables online causal generation in only four denoising steps while maintaining coherent world dynamics throughout extended rollouts. Building on these components, we introduce an efficient few-clip post-training as well as a suite of visual reference-conditioned generation models, preserving general generation ability while allowing rare-case customization for long-tail controllability. To assess controllability beyond realism, we further introduce an automated VLM-based judging pipeline that evaluates scene-level condition adherence, view-wise object controllability, and cross-view object consistency. Comprehensive experiments show that M4World consistently delivers high generation quality, precise controllability, and stable minute-long streaming. Together with downstream long-tail augmentation and scene editing, these results demonstrate the potential of M4World for controllable, scalable driving simulation.
Ke Cheng, Hanqiao Ye, Lei Shi +8
1Meituan · Institute of Automation, Chinese Academy of Sciences · 3Beijing Institute of Technology