DynVLA: Learning World Dynamics for Action Reasoning in Autonomous Driving

Shuyao Shang; Bing Zhan; Yunfei Yan; Yuqi Wang; Yingyan Li; Yasong An; Xiaoman Wang; Jierui Liu

cs.CVMar 11, 2026

DynVLA: Learning World Dynamics for Action Reasoning in Autonomous Driving

Authors: Shuyao Shang, Bing Zhan, Yunfei Yan, Yuqi Wang, Yingyan Li, Yasong An, Xiaoman Wang, Jierui Liu, +4 more

Organizations: NLPR, Institute of Automation, Chinese Academy of Sciences (CASIA) · Yinwang Intelligent Technology Co. Ltd.

Abstract

We propose DynVLA, a driving VLA model that introduces a new CoT paradigm termed Dynamics CoT. DynVLA forecasts compact world dynamics before action generation, enabling more informed and physically grounded decision-making. To obtain compact dynamics representations, DynVLA introduces a Dynamics Tokenizer that compresses future evolution into a small set of dynamics tokens. Considering the rich environment dynamics in interaction-intensive driving scenarios, DynVLA decouples ego-centric and environment-centric dynamics, yielding more accurate world dynamics modeling. We then train DynVLA to generate dynamics tokens before actions through SFT and RFT, improving decision quality while maintaining latency-efficient inference. Compared to Textual CoT, which lacks fine-grained spatiotemporal understanding, and Visual CoT, which introduces substantial redundancy due to dense image prediction, Dynamics CoT captures the evolution of the world in a compact, interpretable, and efficient form. Extensive experiments on NAVSIM, Bench2Drive, and a large-scale in-house dataset demonstrate that DynVLA consistently outperforms Textual CoT and Visual CoT methods, validating the effectiveness and practical value of Dynamics CoT. Project Page: https://yaoyao-jpg.github.io/dynvla.

Figures & tables

Figure 1: Comparison of different CoT paradigms in autonomous driving VLA models. (a) Textual CoT suffers from limited spatiotemporal understanding and high inference latency due to long textual reasoning traces. (b) Visual CoT introduces substantial redundancy and computational overhead from pixel-level generation. (c) Dynamics CoT compresses future dynamics into a small set of tokens, achieving latency-efficient inference with compact reasoning and accurate spatiotemporal modeling.

Figure 2: Overview of the DynVLA. (a) Given adjacent image observations, a dynamics encoder extracts ego-centric and environment-centric dynamics, which are discretized via VQ codebooks. Then, the ego-centric dynamics are regularized by the GT ego action, and the combined dynamics are decoded to reconstruct the future image and BEV map conditioned on each current state. (b) DynVLA is supervised to first generate discrete dynamics tokens followed by action tokens, forming structured Dynamics CoT modeling.

Figure 3: Overview of the training pipeline for DynVLA . DynVLA first learns a Dynamics Tokenizer by reconstructing future states from adjacent frames, producing discrete dynamics tokens. It then performs SFT on Dynamics CoT, training the model to generate dynamics tokens before action tokens. Finally, the policy is optimized via RFT with trajectory-level reward and KL regularization.

Method	NC $\uparrow$	DAC $\uparrow$	TTC $\uparrow$	C $\uparrow$	EP $\uparrow$	PDMS $\uparrow$
Human	100.0	100.0	100.0	99.9	87.5	94.8
Traditional End-to-End Methods
VADv2 ( Chen et al., 2024 )	97.2	89.1	91.6	100	76.0	80.9
UniAD ( Hu et al., 2023 )	97.8	91.9	92.9	100	78.8	83.4
TransFuser ( Chitta et al., 2022 )	97.7	92.8	92.8	100	79.2	84.0
PARA-Drive ( Weng et al., 2024 )	97.9	92.4	93.0	99.8	79.3	84.0

Table 1: Comparison on NAVSIM Benchmark. The best results are denoted by bold and the second best are denoted by underline .

Method	DS $\uparrow$	SR $\uparrow$	Mean Multi-Ability $\uparrow$
Think2Drive † ( Li et al., 2024a )	91.85	85.41	86.26
PDM-Lite † ( Sima et al., 2024 )	97.02	92.27	92.82
AD-MLP ( Zhai et al., 2023 )	18.05	0.00	0.87
TCP ( Wu et al., 2022 )	40.70	15.00	14.63
VAD ( Jiang et al., 2023 )	42.35	15.00	18.07
UniAD ( Hu et al., 2023 )	45.81	16.36	15.55

Table 2: Comparison on Bench2Drive Benchmark. The best results are denoted by bold and the second best are denoted by underline . † denotes using privileged perceptual information.

Figure 4: Transferability of learned dynamics. Dynamics tokens extracted from one scenario are injected into a new scene and decoded into the future image and the BEV map. We contrast the current states, the future states decoded with transferred dynamics, and the original future states. The results show that both ego-centric and environment-centric dynamics are transferable across scenarios.

Figure 5: Codebook collapse without dynamics decouple.

Figure 6: Dynamics CoT improves planning by reasoning over future dynamics. The first two columns show the current observation and the future decoded by reasoned dynamics. The third and last columns compare planning results with and without Dynamics CoT. Compared to direct action prediction, Dynamics CoT provides intent-aware, foresighted, and constraint-compliant future dynamics, enabling safer and more feasible planning in challenging scenarios.

Model	ADE (m) $\downarrow$	Collision Rate (‱) $\downarrow$
Transfuser ( Chitta et al., 2022 )	1.746	5.63
DriveVLA-W0 (VQ) ( Li et al., 2025a )	1.599	5.20
DriveVLA-W0 (ViT) ( Li et al., 2025a )	1.344	5.13
DynVLA (Ours)	1.215	4.04

Table 3: Comparison on a large-scale In-house Dataset. The best results are denoted by bold .

CoT Content	Latency(s) $\downarrow$	NC $\uparrow$	DAC $\uparrow$	TTC $\uparrow$	C $\uparrow$	EP $\uparrow$	PDMS $\uparrow$
None (w/o CoT)	0.20	98.3	93.8	94.6	99.9	79.5	85.6
Scene Description	3.04	98.4	93.4	94.4	99.9	79.3	85.3
Meta Action	0.43	98.3	94.3	94.6	100	79.8	86.0
Future Image	2.29	98.7	94.4	95.0	99.9	80.0	86.3
Optical Flow	2.29	98.6	94.4	95.3	100	80.0	86.4
Dynamics (Ours)	0.37	98.6	95.3	95.5	100	80.6	87.2

Table 4: Analysis on CoT Design and Latency. All inference latencies are measured on a single NVIDIA H800 GPU. The best results are denoted by bold .

Base Model	Dyn CoT	SFT	RFT	NC $\uparrow$	DAC $\uparrow$	TTC $\uparrow$	C $\uparrow$	EP $\uparrow$	PDMS $\uparrow$
EMU3 ( Wang et al., 2024 )		✓		98.3	93.8	94.6	99.9	79.5	85.6
	✓	✓		98.6	95.3	95.5	100	80.6	87.2
		✓	✓	98.6	96.7	95.6	100	82.3	88.7
	✓	✓	✓	98.6	98.7	95.5	100	86.8	91.7
Qwen2.5-VL ( Bai et al., 2025 )		✓		98.3	93.5	94.8	100	79.1	85.3
	✓	✓		98.8	94.4	95.8	100	79.9	86.6

Table 5: Ablation on Training Stages. Dyn CoT denotes Dynamics CoT. The best results are denoted by bold .

Model	Decouple	Image	BEV	NC $\uparrow$	DAC $\uparrow$	TTC $\uparrow$	C $\uparrow$	EP $\uparrow$	PDMS $\uparrow$
w/o CoT	–	–	–	98.3	93.8	94.6	99.9	79.5	85.6
w/ Dyn CoT		✓	✓	98.5	94.0	94.9	100	79.5	85.8
	✓		✓	98.4	94.5	94.8	100	79.8	86.2
	✓	✓		98.6	94.8	95.0	100	80.6	86.7
	✓	✓	✓	98.6	95.3	95.5	100	80.6	87.2

Table 6: Ablation on Dynamics Tokenizer Designs. Dyn CoT denotes Dynamics CoT. The best results are denoted by bold .

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Visualization of the decoded image and BEV from Dynamics Tokenizer. For each row, we show the current image and BEV map, the decoded future image and BEV map, and the ground-truth future image and BEV map. Across diverse driving scenarios, the decoded futures faithfully capture ego motion and the dynamics of surrounding agents.

Figure 8: Additional visualizations of the transferability of learned dynamics.

Pred Horizon	Latency(s) $\downarrow$	NC $\uparrow$	DAC $\uparrow$	TTC $\uparrow$	C $\uparrow$	EP $\uparrow$	PDMS $\uparrow$
w/o CoT	0.20	98.3	93.8	94.6	99.9	79.5	85.6
$K=1$	0.27	98.6	94.6	95.0	100	80.1	86.5
$K=2$	0.37	98.6	95.3	95.5	100	80.6	87.2
$K=3$	0.49	98.6	94.7	95.2	100	80.3	86.7
$K=4$	0.61	98.5	94.7	95.3	99.9	80.2	86.6

Appendix

Table 7: Ablation on the prediction horizon of Dynamics CoT. The prediction horizon

K

denotes the number of reasoned future dynamics. The best results are denoted by bold .

Dyn Num	${N}_{\text{ego}}$	${N}_{\text{env}}$	NC $\uparrow$	DAC $\uparrow$	TTC $\uparrow$	C $\uparrow$	EP $\uparrow$	PDMS $\uparrow$
8	4	4	98.6	95.3	95.5	100	80.6	87.2
8	2	6	98.7	94.9	95.3	100	80.5	86.9
8	6	2	98.6	94.5	95.0	100	80.2	86.4
4	2	2	98.5	94.6	95.1	100	80.1	86.4
16	8	8	98.5	94.7	94.9	100	80.3	86.5

Appendix

Table 8: Ablation on the number and allocation of dynamics tokens. Dyn Num denotes the total number of dynamics tokens,

N_{\text{ego}}

denotes the number of ego-centric tokens, and

N_{\text{env}}

denotes the number of environment-centric tokens. The best results are denoted by bold .

Figure 9: Additional qualitative comparisons of planning behavior with and without Dynamics CoT.

Figure 10: Representative failure cases. From left to right, we show the planning result, the current observation, the future decoded from inferred dynamics, and the ground-truth future. These cases include incorrect inference of surrounding vehicles’ intentions, misidentification of drivable areas during large turning maneuvers, and ambiguous dynamics reasoning due to degraded visual observations.

Explore similar work

CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Driving
May 11, 2026Minqing Huang, Yujiao Xiang, Zihan Liang +7VLMs for Autonomous Driving Diffusion-Based Planning
Afari Intelligent Drive · University of Electronic Science and Technology of China · Shanghai Jiao Tong University +2
XCoT-VLA: Executable Chain-of-Thought for Vision-Language-Action Driving
Aug 11, 2026Foundation Model Team, XPeng IncVLMs for Autonomous Driving Vision-Language-Action Models
Foundation Model Team, XPeng Inc.
Xiaomi OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation
Apr 20, 2026Jinghui Lu, Jiayi Guan, Zhijian Huang +47Trajectory Generation VLMs for Autonomous Driving

May 11, 2026J/K move · Enter open · S save

CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Driving

Minqing Huang, Yujiao Xiang, Zihan Liang, Jiajie Huang+6

Afari Intelligent Drive · University of Electronic Science and Technology of China · Shanghai Jiao Tong University+2

VLMs for Autonomous Driving Diffusion-Based Planning World Models Vision-Language-Action Models World Models for Autonomous Driving Autonomous Driving Planning

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models have emerged as a promising paradigm for end-to-end autonomous driving. However, existing reasoning mechanisms still struggle to provide planning-oriented intermediate representations: textual Chain-of-Thought (CoT) fails to preserve continuous spatiotemporal structure, while latent world reasoning remains difficult to use as a direct condition for action generation. In this paper, we propose CoWorld-VLA, a multi-expert world reasoning framework for autonomous driving, where world representations serve as explicit conditions to guide action planning. CoWorld-VLA extracts complementary world information through multi-source supervision and encodes it into expert tokens within the VLA, thereby providing planner-accessible conditioning signals. Specifically, we construct four types of tokens: semantic interaction, geometric structure, dynamic evolution, and ego trajectory tokens, which respectively model interaction intent, spatial structure, future temporal dynamics, and behavioral goals. During action generation, CoWorld-VLA employs a diffusion-based hierarchical multi-expert fusion planner, which is coupled with scene context throughout the joint denoising process to generate continuous ego trajectories. Experiments show that CoWorld-VLA achieves competitive results in both future scene generation and planning on the NAVSIM v1 benchmark, demonstrating strong performance in collision avoidance and trajectory accuracy. Ablation studies further validate the complementarity of expert tokens and their effectiveness as planning conditions for action generation. Code will be available at https://github.com/AFARI-Research/CoWorld-VLA.

Abstract

Explore similar work

CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Driving

XCoT-VLA: Executable Chain-of-Thought for Vision-Language-Action Driving

Xiaomi OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation