VLALight: Lightweight Vision-Language-Action Models for Emergency-Aware Traffic Signal Control

Kemou Jiang; Maonan Wang; Xingchen Zou; Jiayue Zhu; Yuhang Fu; Sicheng Wang; Xi Chen; Yirong Chen

cs.CVSep 25, 2026

VLALight: Lightweight Vision-Language-Action Models for Emergency-Aware Traffic Signal Control

Authors: Kemou Jiang, Maonan Wang, Xingchen Zou, Jiayue Zhu, Yuhang Fu, Sicheng Wang, Xi Chen, Yirong Chen,

Organizations: School of Transportation Science and Engineering, Beihang University, Beijing, China · The Chinese University of Hong Kong (Shenzhen), Shenzhen, China · The Chinese University of Hong Kong, Hong Kong SAR, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China ·

Abstract

Traffic signal control (TSC) is essential for mitigating urban congestion. Recent advances in vision-language models (VLMs) enable richer interpretation of intersection scenes, opening new opportunities for visual-context-aware TSC. However, the loose coupling and repeated information conversion between modules can lead to the loss of fine-grained visual details, while sequential inference introduces substantial latency. To address these limitations, we propose VLALight, a lightweight end-to-end vision-language-action framework that directly maps intersection observations and signal-phase information to discrete signal actions. To handle the multi-view nature of TSC, VLALight combines multiple directional camera views into a unified visual input and uses textual instructions to establish their correspondence with traffic movements and signal phases. This design enables direct action prediction with a compact 0.5 B-parameter model, without intermediate image-to-text descriptions or handcrafted traffic-state representations. Experiments show that VLALight delivers the best emergency-vehicle service of all compared methods, reducing pooled emergency waiting time by 21.1% over the cascaded VLMLight while running in real time on local hardware and generalizing to unseen intersection topologies and traffic-flow patterns.

Figures & tables

Figure 2: Architecture of VLALight. (a) The simulation-grounded traffic data-generation pipeline: road network topology extraction, traffic flow design and expert label collection, and realistic intersection image rendering. (b) Post-training and inference: the directional camera images and the text instruction are encoded by the frozen DINOv2 and SigLIP visual encoders and the compact VLM backbone, which directly outputs the next signal phase.

Type	Method	Beihuan		Beishahe		Changjianglu		Gaojiaoyuan		Pinganli		Zhijingdao
Type	Method	ETT $\downarrow$	EWT $\downarrow$	ETT $\downarrow$	EWT $\downarrow$	ETT $\downarrow$	EWT $\downarrow$	ETT $\downarrow$	EWT $\downarrow$	ETT $\downarrow$	EWT $\downarrow$	ETT $\downarrow$	EWT $\downarrow$
Trad.	FixTime	59 $\pm$ 19 $\ddagger$	19 $\pm$ 12 $\dagger$	44 $\pm$ 2.9	12 $\pm$ 2.6	37 $\pm$ 4.1 $\dagger$	1.6 $\pm$ 2.8 $\dagger$	57 $\pm$ 6.7	22 $\pm$ 7.2	51 $\pm$ 7.6	16 $\pm$ 7.8	63 $\pm$ 24	28 $\pm$ 20
	Webster	62 $\pm$ 17	21 $\pm$ 12	43 $\pm$ 2.6	11 $\pm$ 2.6	37 $\pm$ 4.0 $\star$	1.4 $\pm$ 2.6 $\star$	57 $\pm$ 7.0	22 $\pm$ 7.1	51 $\pm$ 7.8	16 $\pm$ 7.8	63 $\pm$ 24	28 $\pm$ 20
	SOTL	59 $\pm$ 9.5 $\dagger$	21 $\pm$ 9.5 $\ddagger$	44 $\pm$ 3.8	11 $\pm$ 3.5	41 $\pm$ 4.3	5.0 $\pm$ 4.4	54 $\pm$ 4.6	19 $\pm$ 3.3	46 $\pm$ 15	11 $\pm$ 12	61 $\pm$ 15	26 $\pm$ 14
	MaxPressure	77 $\pm$ 21	35 $\pm$ 17	42 $\pm$ 5.9	9.3 $\pm$ 5.1	45 $\pm$ 10	9.4 $\pm$ 8.6	59 $\pm$ 10	25 $\pm$ 10	45 $\pm$ 8.3	9.7 $\pm$ 8.0	67 $\pm$ 20	33 $\pm$ 17
RL	IntelliLight	124 $\pm$ 25	69 $\pm$ 24	41 $\pm$ 5.1	8.8 $\pm$ 3.9	40 $\pm$ 3.2	3.6 $\pm$ 2.5	52 $\pm$ 6.9 $\ddagger$	16 $\pm$ 5.9 $\ddagger$	43 $\pm$ 5.3 $\ddagger$	8.0 $\pm$ 4.3 $\ddagger$	65 $\pm$ 18	29 $\pm$ 16

Table 1: Per-city comparison on the regular flow (mean

\pm

std over 4 seeds, seconds). The top three per column are boldfaced and ranked ⋆ / † / ‡ .

Figure 4: An emergency-priority example in TransimHub. At time $T$ an emergency vehicle approaches the intersection in one camera view; VLALight switches to the phase 3, and at time $T{+}1$ the vehicle has crossed the intersection.

Figure 5: Ablation of the visual-input design: stitched vs. unstitched visual input. (a) Pooled ATT and ETT under the regular flow; (b) inference latency per decision.

Figure 6: Generalization to unseen traffic flows: ETT and EWT on the regular flow and the three test flows.

Figure 7: Generalization to unseen intersection topologies: transfer to two unseen intersections with few-shot and zero-shot adaptation.

Traffic Signal Control

Traffic signal control studies how to set intersection phases and green times to reduce congestion while balancing safety, emissions, and priority needs.

See all 39 papers

VLALight: A Vision-Language-Action Model for Traffic Signal Control
Sep 29, 2026Pan Zhang, Siqi Lai, Kemu Dong +1Intelligent Transportation Systems Vision-Language-Action Models
The Hong Kong University of Science and Technology (Guangzhou) · Dalian University of Technology
DGLight: DQN-Guided GRPO Fine-Tuning of Large Language Models for Traffic Signal Control
Apr 28, 2026Chenbo YuLanguage Model-Based Control Intelligent Transportation Systems
National University of Singapore Singapore
ReasonLight: A Multimodal Foundation Model-Enhanced Reinforcement Learning Framework for Zero-Shot Traffic Signal Control
May 28, 2026Aoyu Pang, Maonan Wang, Yuejiao Xie +3RL Control Intelligent Transportation Systems
School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, China · Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Hong Kong · Shanghai AI Laboratory, Shanghai, China +1

Sep 29, 2026J/K move · Enter open · S save

VLALight: A Vision-Language-Action Model for Traffic Signal Control

Pan Zhang, Siqi Lai, Kemu Dong, Hao Liu

The Hong Kong University of Science and Technology (Guangzhou) · Dalian University of Technology

Intelligent Transportation Systems Vision-Language-Action Models Traffic Signal Control

Open paper AskPDF arXiv

Traffic signal control (TSC) is essential for improving urban mobility and reducing congestion. Although roadside cameras are widely deployed at signalized intersections and provide rich visual observations of evolving traffic, existing TSC methods typically rely on manually engineered traffic states or separate perception modules, creating a gap between physical observations and control decisions. We present VLALight, the first vision-language-action (VLA) model for end-to-end traffic signal control from multi-view roadside videos. VLALight directly maps visual observations to coordinated signal actions through multi-target spatiotemporal traffic reasoning and topology-aware cooperative perception across intersections. To establish this capability, we develop a two-stage supervised cold-start training strategy for visual traffic understanding and signal decision-making, followed by cooperative agentic reinforcement learning that jointly optimizes local control and network-wide traffic efficiency. Furthermore, VLALight introduces adaptive fast and slow reasoning modes, enabling the policy to allocate deeper reasoning only when additional deliberation provides sufficient control benefits. Through balanced mode-aware rollouts and relative advantage optimization, VLALight learns to trade off decision quality and inference cost. Extensive experiments on seven real-world traffic-flow datasets across three urban networks demonstrate that VLALight consistently outperforms transportation-based, RL-based, and LLM/VLM-based baselines. Ablation studies validate the effectiveness of cooperative perception, network-level optimization, and adaptive reasoning. These results demonstrate the potential of VLA models for real-world physical traffic control. Our project is available at https://github.com/usail-hkust/VLALight.git.

Abstract

Traffic Signal Control

Explore similar work

VLALight: A Vision-Language-Action Model for Traffic Signal Control

DGLight: DQN-Guided GRPO Fine-Tuning of Large Language Models for Traffic Signal Control

ReasonLight: A Multimodal Foundation Model-Enhanced Reinforcement Learning Framework for Zero-Shot Traffic Signal Control