DynFlowDrive: Flow-Based Dynamic World Modeling for Autonomous Driving

Xiaolu Liu; Yicong Li; Song Wang; Junbo Chen; Angela Yao; Jianke Zhu

cs.CVMar 20, 2026

DynFlowDrive: Flow-Based Dynamic World Modeling for Autonomous Driving

Authors: Xiaolu Liu, Yicong Li, Song Wang, Junbo Chen, Angela Yao, Jianke Zhu

Organizations: Zhejiang University · National University of Singapore

Abstract

Recently, world models have been incorporated into the autonomous driving systems to improve the planning reliability. Existing approaches typically predict future states through appearance generation or deterministic regression, which limits their ability to capture trajectory-conditioned scene evolution and leads to unreliable action planning. To address this, we propose DynFlowDrive, a latent world model that leverages flow-based dynamics to model the transition of world states under different driving actions. By adopting the rectifiedflow formulation, the model learns a velocity field that describes how the scene state changes under different driving actions, enabling progressive prediction of future latent states. Building upon this, we further introduce a stability-aware multi-mode trajectory selection strategy that evaluates candidate trajectories according to the stability of the induced scene transitions. Extensive experiments on the nuScenes and NavSim benchmarks demonstrate consistent improvements across diverse driving frameworks without introducing additional inference overhead. Source code will be abaliable at https://github.com/xiaolul2/DynFlowDrive.

Figures & tables

Figure 1: (a) Comparisons of perception-based and latent world model-based approaches on nuScenes and NavSim benchmarks. (b) Planning visualization on the front view and bird’s-eye-view (BEV) space. Our DynFlowDrive achieves comparable performance.

Figure 2: Comparison between (a) the existing static world model and (b) the dynamic latent world model of our DynFlowDrive . Instead of the static regression of next-frame latents, we propose the dynamic modeling that learns a continuous velocity field $v_{\theta}$ to capture the evolution of world transitions.

Figure 3: Overview of DynFlowDrive. Given current observations, multi-mode trajectories are firstly generated by the standard planning module. A flow-based dynamic latent world model is incorporated to simulate the progressive future evolution in latent space. The resulting dynamics are used by a stability-aware multi-mode selection module, which assess the trajectory based on reconstruction quality and flow-based stability, enabling reliable supervision and improved planning robustness.

Figure 4: The architecture of our dynamic latent world model design , in which the velocity field $v_{\theta}$ is learnt to capture the trajectory-conditioned dynamics transitions in the latent space.

Figure 5: Stability-aware Multi-mode Selection. For training, the score head is supervised by the stable criterion. For Inference, the best mode trajectory is selected according to the highest score index.

Method	Pub.	BEV	$\mathbf{L_{2}}$ (m) $\downarrow$				Collision Rate (%) $\downarrow$
Method	Pub.	BEV	1s	2s	3s	Avg.	1s	2s	3s	Avg.
Perception-based Approaches
ST-P3 [ 17 ]	ECCV 2022	✓	1.33	2.11	2.90	2.11	0.23	0.62	1.27	0.71
UniAD [ 19 ]	CVPR 2023	✓	0.48	0.96	1.65	1.03	0.05	0.17	0.71	0.31
VAD-Tiny [ 20 ]	ICCV 2023	✓	0.46	0.76	1.12	0.78	0.21	0.35	0.58	0.38
VAD-Base [ 20 ]	ICCV 2023	✓	0.41	0.70	1.05	0.72	0.07	0.17	0.41	0.22

Table 1: Comparison of the SOTA methods on the nuScenes dataset. “

\ast

” denotes the re-implemented result from SSR [ 23 ] by official public codes. “

\diamond

” denotes using ego status in the planning module following BEVPlanner++ [ 30 ] . Evaluations are conducted in the same way as the settings in VAD [ 20 ] . Our DynFLowDrive are implemented based on LAW [ 24 ] and SSR. LAW adopts Swin-Tiny [ 38 ] as image backbone, while others adopt ResNet-50 [ 15 ] .

Figure 6: Visualization of Planning Results on nuScenese dataset. The perception representations are rendered from annotations. Surrounding light lines denote the maps of the scene, while black boxes represent the object detection. The ego vehicle is drawn by the green box in the center.

Method	Input	#Traj.	Traj. Eval.	NC $\uparrow$	DAC $\uparrow$	EP $\uparrow$	TTC $\uparrow$	Comf. $\uparrow$	PDMS $\uparrow$
Human	-	-	-	100	100	87.5	100	99.9	94.8
Constant Velocity	-	1	✗	69.9	58.8	49.3	49.3	100.0	21.6
Ego Status MLP	-	1	✗	93.0	77.3	62.8	83.6	100.0	65.6
VADv2 [ 4 ]	C	8192	Rule-based	97.9	91.7	77.6	92.9	100.0	83.0
UniAD [ 19 ]	C	1	Rule-based	97.8	91.9	78.8	92.9	100.0	83.4
LTF [ 41 ]	C	1	✗	97.4	92.8	79.0	92.4	100.0	83.8

Table 2: Comparison with the SOTA approaches on the NavSim test set. C: Camera. L: LiDAR. Rule-based means that trajectory evaluation follows specific rules. Model-free represents the evaluation without world modeling. Model-based denotes evaluating the trajectory with the assistance of the world model.

Modules			$L_{2}$ (m) $\downarrow$				CR (%) $\downarrow$				FPS $\uparrow$
Static-WM	Dyn-WM	MS	1s	2s	3s	Avg.	1s	2s	3s	Avg.	FPS $\uparrow$
-	-	-	0.31	0.63	1.12	0.69	0.19	0.27	0.64	0.37	13.8
$\checkmark$	-	-	0.26	0.57	1.01	0.61	0.14	0.21	0.54	0.30	13.8
-	$\checkmark$	-	0.26	0.55	0.94	0.59	0.11	0.16	0.51	0.26	13.8
-	-	$\checkmark$	0.25	0.55	0.93	0.58	0.11	0.15	0.49	0.25	13.6
-	$\checkmark$	$\checkmark$	0.24	0.54	0.92	0.57	0.10	0.12	0.45	0.22	13.6

Table 3: Component-wise Ablation. Static-WM and Dyn-WM mean the existing stable world model design and our flow-based dynamic modeling. MS denotes the strategy of stability-aware multi-mode selection. FPS are measured on one RTX 4090.

World Model Design	$L_{2}$ (m) $\downarrow$				CR (%) $\downarrow$
World Model Design	1s	2s	3s	Avg.	1s	2s	3s	Avg.
Static WM	0.26	0.57	1.01	0.61	0.14	0.21	0.54	0.30
Flow WM	0.26	0.55	0.94	0.59	0.11	0.16	0.51	0.26
+ World Feat. Design	0.24	0.54	0.92	0.57	0.10	0.12	0.45	0.22

Table 4: Ablations on the Dynamic Latent World Model Design.

Table 5: Ablations on Flow Steps and Multi-Mode Selection Strategies.

Explore similar work

Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving
Sep 3, 2026Zhaoxin Fan, Tianbao Zhang, Wenjun Wu +5World Models World Models for Autonomous Driving
School of Artificial Intelligence, Beihang University, Beijing, China · Shanghai Jiao Tong University, Shanghai, China · Dim12 AI +3
DriveFuture: Future-Aware Latent World Models for Autonomous Driving
May 10, 2026Yufeng Hong, Xiaotian Zhou, Yingyan Li +6World Models for Autonomous Driving Latent World Models
Beijing Institute of Technology · Institute of Automation, Chinese Academy of Sciences · Beihang University +5
DriveVA: Video Action Models are Zero-Shot Drivers
Apr 5, 2026Mengmeng Liu, Diankun Zhang, Jiuming Liu +7Action-Conditioned Video Generation World Models
University of Twente, The Netherlands · Xiaomi EV, China · University of Cambridge, United Kingdom +1

Sep 3, 2026J/K move · Enter open · S save

Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving

Zhaoxin Fan, Tianbao Zhang, Wenjun Wu, Xiaofeng Wang+4

School of Artificial Intelligence, Beihang University, Beijing, China · Shanghai Jiao Tong University, Shanghai, China · Dim12 AI+3

World Models World Models for Autonomous Driving Latent World Models Autonomous Driving Planning

Open paper AskPDF arXiv

World models offer a promising paradigm for autonomous driving by predicting how traffic scenes may evolve and using such predictions to support action generation. However, existing approaches either separate future prediction from action generation or jointly predict them at the same temporal scale, making it difficult to simultaneously achieve long-horizon anticipation and responsive, observation-grounded decision making. We present Drive-HWM, a hierarchical slow--fast world modeling framework that organizes future representation prediction and action generation at complementary temporal scales. The slow world model predicts multi-step future representations to capture extended scene evolution. To explicitly model the abundant motion dynamics in driving environments, we introduce Dynamic-Aware Latents learned through optical-flow prediction. Guided by these future representations, the fast model uses a lightweight multimodal backbone and an autoregressive expert to jointly predict the next frame and the immediate action from the latest observation. Next-frame prediction encourages the fast model to capture imminent scene evolution, while one-step action generation allows decisions to be continuously updated as new observations arrive. Extensive experiments on NAVSIM v1 and v2 demonstrate the strong driving performance of Drive-HWM. Comprehensive ablation studies further validate the effectiveness of the hierarchical slow--fast design, dynamics-aware future representations, and joint next-frame and action prediction.

Abstract

Explore similar work

Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving

DriveFuture: Future-Aware Latent World Models for Autonomous Driving

DriveVA: Video Action Models are Zero-Shot Drivers