Scene-Adaptive VLA: Efficient Autonomous Driving via Dynamic Layer Routing

Yujia Yang; Quan Yuan; Jiawei Fan; Yang Li; Tiange Fu; Xiaoyuan Fu; Guiyang Luo; Jinglin Li

cs.CVOct 17, 2026

Scene-Adaptive VLA: Efficient Autonomous Driving via Dynamic Layer Routing

Authors: Yujia Yang, Quan Yuan, Jiawei Fan, Yang Li, Tiange Fu, Xiaoyuan Fu, Guiyang Luo, Jinglin Li

Organizations: State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications

Abstract

Vision-Language-Action (VLA) models offer a promising paradigm for autonomous driving. However, the massive parameter scale of VLAs strains onboard computational resources during real-time inference. This directly increases energy consumption and limits vehicle battery range, making it critical to minimize computational overhead without compromising driving capability and explainability. Current research on efficient VLA-based driving focuses on sequential adaptation (e.g., Chain-of-Thought and token adjustment), leaving structural adaptation underexplored despite parameter redundancy across dynamic driving scenes. To address this gap, we propose Scene-Adaptive VLA, an efficient framework leveraging dynamic layer routing to adjust active parameters based on real-time scenes. Inspired by the temporal continuity of scenes and model predictive uncertainty, our approach characterizes model-perceived scene complexity via frame-level temporal scene variations and decision deviations. To uncover latent scene-to-parameter correspondences, we encode this characterization into scene-aware tokens, alongside two specialized queries. A budget-conditioned layer routing mechanism evaluates these refined queries to allocate a budget that determines the active parameter ratio and subsequently selects appropriate layer combinations. The entire framework is optimized end-to-end. Extensive evaluations on the Bench2Drive benchmark show that when adapted to a state-of-the-art VLA model, our approach significantly reduces inference-time computational overhead by 53.5% with negligible degradation in driving and language capabilities. Code is available at https:github.com/yang-luvu/Scene-Adaptive-VLA.

Figures & tables

Figure 1: Driving performance under varying capacities of a state-of-the-art model [ 3 ] on Bench2Drive. (a) Fixed Ratio Across Routes: The model with 70% active parameters completely fails on routes featuring more complex driving scenes, yet completes routes with only a 20% score drop on routes dominated by routine scenes. (b) Activation Ratios (Complex Route): Reducing parameters severely degrades performance, with lower ratios causing critical local failures (e.g., lane deviation at 40%, blockage at 70%). (c) Layer Selection Strategies (Complex Route): At 70% activation, an early-exit strategy significantly underperforms random layer dropping, proving that active parameter combinations are critical.

Figure 2: Overview of the Scene-Adaptive VLA, featuring Model-Perceived Scene Complexity Characterization and Budget-Conditioned Layer Routing. Adjacent icons denote paramter status (left/top: Stage 1; right/bottom: Stage 2). Left : The characterization module encodes temporal visual variations and inter-frame decision deviations into scene-aware tokens ( $\mathbf{e}_{s}$ , $\mathbf{e}_{pw}$ , $\mathbf{e}_{sw}$ ). Right : The LLM prefix layers process these tokens alongside budget ( $\mathbf{q}_{b}$ ) and routing ( $\mathbf{q}_{lr}$ ) queries. A budget scheduler allocates a real-time computational budget $b$ based on intermediate $\mathbf{q}_{b}$ features from the middle prefix layers. At the final prefix layer, a routing scheduler utilizes $\mathbf{q}_{lr}$ to generate a binary mask $\mathbf{M}$ , dynamically activating specific layers and bypassing others via residual connections to yield final predictions.

Method	Avg. $P$ (B)	$N_{\text{in}}$	$N_{\text{out}}$	TFLOPs $\downarrow$	Latency (ms) $\downarrow$	Mem. (GB) $\downarrow$	Language
DriveMoE [ 6 ]	3.00	527	1	3.17	319.54	14.85	$\times$
AutoVLA [ 5 ]	3.00	1736	18	10.52	1322.85	7.18	$\checkmark$
ORION [ 4 ]	7.00	538	110	9.07	5440.70	17.83	$\checkmark$
ORION+SA	3.68	538	110	4.77	2971.60	17.83	$\checkmark$
SimLingo [ 3 ]	0.50	569	22	0.59	410.79	2.03	$\checkmark$
SimLingo+SA	0.23	569–576	22	0.27	214.01	2.01	$\checkmark$

Table 1: LLM computation overhead per driving decision.

Method	DS $\uparrow$	SR( $\%$ ) $\uparrow$	Efficiency $\uparrow$	Comfortness $\uparrow$	TFLOPs $\downarrow$
UniAD-Base [ 15 ]	45.81	16.36	129.21	43.58	3.42
VAD [ 16 ]	42.35	15.00	157.94	46.01	1.14
TCP-traj [ 32 ]	59.90	30.00	76.54	18.08	0.04
AutoVLA [ 5 ]	78.84	57.73	146.93	39.33	10.52
DriveMoE [ 6 ]	74.22	48.64	175.96	15.31	3.17
ORION [ 4 ]	77.74	54.62	151.48	17.38	9.07

Table 2: Comparison with SOTA driving methods on the closed-loop benchmark.

Figure 3: Visualization of scene-adaptive parameter activation. Left: driving scenes at selected steps. Right: corresponding budget and sampled layer combinations.

Budget	Scene	Speed	Path	DS $\uparrow$	SR(%) $\uparrow$	Efficiency $\uparrow$	Comfortness $\uparrow$	Avg. $B$ (%) $\downarrow$
$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	84.87	65.45	266.19	54.36	44.64
$\times$	$\checkmark$	$\checkmark$	$\checkmark$	83.81	62.55	259.18	51.49	60.03
$\checkmark$	$\checkmark$	$\times$	$\checkmark$	84.60	64.09	249.68	37.63	71.89
$\checkmark$	$\checkmark$	$\checkmark$	$\times$	83.88	63.63	247.17	30.75	66.70
$\checkmark$	$\checkmark$	$\times$	$\times$	84.12	63.18	241.10	34.24	68.05
$\checkmark$	$\times$	$\checkmark$	$\checkmark$	83.37	64.09	258.74	56.74	54.37

Table 4: Ablation study on the closed-loop benchmark.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

	Visual Tokens	Prompt Tokens	Specific Tokens
ORION	512	10	16
SimLingo	512	25	32
AutoVLA	1536	200	0
DriveMoE	512	10	5

Appendix

Table 6: Token numbers across different models.

Method	Ability (%) $\uparrow$					Mean
Method	Merging	Overtaking	Emergency Brake	Give Way	Traffic Sign	Mean
UniAD-Base	14.10	17.78	21.67	10.00	14.21	15.55
VAD	8.11	24.44	18.64	20.00	19.15	18.07
TCP-traj	8.89	24.29	51.67	40.00	46.28	34.22
Orion	25.00	71.11	78.33	30.00	69.15	54.72
DriveMoE	34.67	40.00	65.45	40.00	59.44	47.91

Appendix

Table 7: Multi-ability evaluation on Bench2Drive.

Efficient VLA Model Inference

Efficient vision-language-action (VLA) model inference reduces robot-control latency, memory use, and energy while preserving task performance.

See all 190 papers

Rethinking Language's Role in Efficient VLA for Autonomous Vehicles: Toward Smarter, Trustworthy Driving
Aug 31, 2026Tongfei Guo, Lili SuVLMs for Autonomous Driving Efficient VLA Model Inference
Department of Electrical and Computer Engineering, Northeastern University, Boston, MA, USA.
ETA-VLA: Efficient Token Adaptation via Temporal Fusion and Intra-LLM Sparsification for Vision-Language-Action Models
Mar 26, 2026Yiru Wang, Anqing Jiang, Shuo Wang +3NavSim Efficient VLA Models
Bosch Corporate Research, Bosch (China) Investment Ltd., Shanghai, China. · School of Communication and Information Engineering, Shanghai University, Shanghai, China
FlashDrive: Flash Vision-Language-Action Inference for Autonomous Driving
Aug 13, 2026Zekai Li, Yihao Liang, Hongfei Zhang +3Efficient VLA Model Inference Autonomous Driving
1UC San Diego · 2Princeton · 3Independent Researcher

Aug 31, 2026J/K move · Enter open · S save

Rethinking Language's Role in Efficient VLA for Autonomous Vehicles: Toward Smarter, Trustworthy Driving

Tongfei Guo, Lili Su

Department of Electrical and Computer Engineering, Northeastern University, Boston, MA, USA.

VLMs for Autonomous Driving Efficient VLA Model Inference Vision-Language-Action Models

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models are reshaping autonomous driving (AD) by unifying perception, reasoning, and control through language, enabling semantic grounding, interpretable decisions, and better long-tail generalization. But language is expensive onboard: latency and memory budgets are tight, and autoregressive decoding is inherently sequential. This work reframes the central question as when and where language should act at inference, since inference cost recurs at every deployed frame while training cost is paid once. We introduce the Language Residue taxonomy to organize methods by their inference-time use of language: train-time-only supervision (L1), latent non-textual reasoning (L2), conditional invocation (L3), and full per-frame generation (L4). We review representative methods and tag each across five deployment axes (latency, parameters, memory, FLOPs, tokens), analyzing them on major open- and closed-loop driving benchmarks (e.g., nuScenes, NAVSIM, Bench2Drive). We further trace how efficient methods from NLP/LLM are adapted in AD, identifying the constraints and motivations driving these adaptations. A continuously updated repository will be available at Github.

Abstract

Efficient VLA Model Inference

Explore similar work

Rethinking Language's Role in Efficient VLA for Autonomous Vehicles: Toward Smarter, Trustworthy Driving

ETA-VLA: Efficient Token Adaptation via Temporal Fusion and Intra-LLM Sparsification for Vision-Language-Action Models

FlashDrive: Flash Vision-Language-Action Inference for Autonomous Driving