cs.CVNov 5, 2026

WAM-Sim0.1: World Anchored Memory Meets Online RL for Consistent and Realistic Driving Simulation

Authors: Jiahao Cui, Jiaye Li, Wenhao Zhang, Yaoyi Li, Shan Luan, Quanhui Tang, Jianhua Han, Lin Cheng, 

Organizations: Fudan University · Shenzhen Yinwang Intelligent Technology Co., Ltd

Abstract

Autoregressive driving world models systematically suffer from structural drift, identity drift, and quality decay over long rollout horizons due to transient conditioning signals that lack persistent state. We present WAM-Sim, a driving world model anchored by a persistent hierarchical memory and optimized via online flow-matching RL. WAM-Sim structures persistent context across three temporal and semantic scales: pose-aligned structural tokens encoding persistent road topology, read-only scene tokens supplying geolocated background appearance, and write-once instance tokens preserving individual traffic participant identities across occlusions and chunk boundaries. To restore visual sharpness and physical plausibility degraded by multi-scale conditioning, we introduce an online post-training framework that optimizes self-generated rollouts via flow-matching RL guided by hierarchy-aligned visual rewards and a vision-language model physical judge. The invariant memory empirically confines policy optimization within the anchored context, helping suppress semantic collapse and reward hacking, while online RL systematically enforces physical commonsense and fine-grained visual fidelity. On the nuScenes benchmark, WAM-Sim achieves a superior generation fidelity (FID=3.47, FVD=25.72) and strong downstream perception performance (24.16 mAP, 71.23 road mIoU). On minute-scale rollouts of self-collected data, WAM-Sim substantially mitigates structural and identity drift, preserving high scene similarity and subject identity coherence across extended horizons. Project page: https://fudan-generative-vision.github.io/WAM-Sim.

Figures & tables

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. SimVTA: Simulation-Grounded Video-Trajectory Alignment for Driving World Action Models

    Oct 9, 2026Changhui Jing, Haochen Tian, Hongchen Li +8World Models for Autonomous DrivingAutonomous Driving Planning

  2. Orbis 2: A Hierarchical World Model for Driving

    Jul 17, 2026Sudhanshu Mittal, Arian Mousakhan, Silvio Galesso +5World ModelsHierarchical World Models

  3. HorizonDrive: Self-Corrective Autoregressive World Model for Long-horizon Driving Simulation

    May 12, 2026Conglang Zhang, Yifan Zhan, Qingjie Wang +10World Model LearningLong-Horizon Video Generation