cs.AISep 29, 2026

VLALight: A Vision-Language-Action Model for Traffic Signal Control

Authors: Pan Zhang, Siqi Lai, Kemu Dong, Hao Liu

Organizations: The Hong Kong University of Science and Technology (Guangzhou) · Dalian University of Technology

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Traffic signal control (TSC) is essential for improving urban mobility and reducing congestion. Although roadside cameras are widely deployed at signalized intersections and provide rich visual observations of evolving traffic, existing TSC methods typically rely on manually engineered traffic states or separate perception modules, creating a gap between physical observations and control decisions. We present VLALight, the first vision-language-action (VLA) model for end-to-end traffic signal control from multi-view roadside videos. VLALight directly maps visual observations to coordinated signal actions through multi-target spatiotemporal traffic reasoning and topology-aware cooperative perception across intersections. To establish this capability, we develop a two-stage supervised cold-start training strategy for visual traffic understanding and signal decision-making, followed by cooperative agentic reinforcement learning that jointly optimizes local control and network-wide traffic efficiency. Furthermore, VLALight introduces adaptive fast and slow reasoning modes, enabling the policy to allocate deeper reasoning only when additional deliberation provides sufficient control benefits. Through balanced mode-aware rollouts and relative advantage optimization, VLALight learns to trade off decision quality and inference cost. Extensive experiments on seven real-world traffic-flow datasets across three urban networks demonstrate that VLALight consistently outperforms transportation-based, RL-based, and LLM/VLM-based baselines. Ablation studies validate the effectiveness of cooperative perception, network-level optimization, and adaptive reasoning. These results demonstrate the potential of VLA models for real-world physical traffic control. Our project is available at https://github.com/usail-hkust/VLALight.git.

Figures & tables

Figure 1: The overview of VLALight.
MethodJinanHangzhouNew York
1231212
 ATTAQLAWTATTAQLAWTATTAQLAWTATTAQLAWTATTAQLAWTATTAQLAWTATTAQLAWT
Transportation-Engineering Methods
FixedTime458.95383.91237.36366.59185.60152.92394.99206.57136.14544.46207.24269.66464.08274.97229.641464.242716.681171.091666.713940.701387.68
MaxPressure270.97163.7779.79267.68109.8274.35260.02135.3073.35306.4565.2263.13304.98120.8873.961215.432395.16894.251467.474066.031202.50
RL-Based Methods
Table 1: Traffic-control performance on the Jinan, Hangzhou, and New York datasets. Lower values are better. The best, second-best, and third-best results are highlighted through boldface, double underline, and underline, respectively. E- and A- denote Efficient and Advanced, respectively; Ge-4-31B-IT denotes Gemma-4-31B-IT.
Figure 3
TraceMethodReasoning statisticsControl performance
Fast (%)Slow (%)Avg. Tokens (Slow)Min–max Tokens (Slow)ATT
New York 1VLALight51.9248.08220.4754–545975.80
New York 1w/o balanced rollouts27.5672.44209.4262–649984.76
New York 1SFT59.7240.28233.1562–1,2301171.08
New York 2VLALight51.2448.76208.2658–6821240.11
New York 2w/o balanced rollouts28.3871.62204.9262–6091296.10
Table 3: Adaptive reasoning token statistics and ATT on New York datasets. Fast and Slow report the percentages of decisions assigned to each mode.
Table 5
Fast 1 ⋅\cdot clear local winner intersection 3–1, step 99 Local perception ETWT: vv =19, qq =19, Δv\Delta v =0, Δq\Delta q =0 NTST : vv = 39 , qq = 39 , Δv\Delta v = 0 , Δq\Delta q = 0 ELWL: vv =8, qq =8, Δv\Delta v =0, Δq\Delta q =0 NLSL: vv =0, qq =0, Δv\Delta v =0, Δq\Delta q =0 Cooperative context Neighbor movement cues: ETWT: 19 vehicles ⋅\cdot NTST: 19 vehicles Neighbor directional pressure: N: 20 queued ⋅\cdot E: 19 queued ⋅\cdot ETA: 27 s FAST signal NTST Why: The local queue maximum is unambiguous.Fast 2 ⋅\cdot clear local winner intersection 5–1, step 88 Local perception ETWT: vv =14, qq =0, Δv\Delta v = −6-6 , Δq\Delta q = −20-20 NTST : vv = 21 , qq = 21 , Δv\Delta v = 0 , Δq\Delta q = 0 ELWL: vv =1, qq =1, Δv\Delta v =0, Δq\Delta q =0 NLSL: vv =0, qq =0, Δv\Delta v =0, Δq\Delta q =0 Cooperative context Neighbor movement cues: ETWT: 1 vehicle Neighbor directional pressure: E: 1 queued ⋅\cdot ETA: 27 s FAST signal NTST Why: One phase dominates visible and stopped vehicles.
Slow 1 ⋅\cdot competing local queues intersection 2–1, step 40 Local perception ETWT : vv = 20 , qq = 19 , Δv\Delta v = +4+4 , Δq\Delta q = +10+10 NTST: vv =21, qq =19, Δv\Delta v = −1-1 , Δq\Delta q = −1-1 ELWL: vv =5, qq =5, Δv\Delta v =0, Δq\Delta q =0 NLSL: vv =0, qq =0, Δv\Delta v =0, Δq\Delta q =0 Cooperative context Neighbor movement cues: ETWT: 11 vehicles ⋅\cdot NTST: 1 vehicle Neighbor directional pressure: E: 20 queued ⋅\cdot N: 12 queued ⋅\cdot ETA: 27 s SLOW signal ETWT Key reasoning: Given ETWT has the largest visible demand, worsening trend, and strong local coordination plus east neighbor pressure, it is clearly the most effective choice.Slow 2 ⋅\cdot close local competition intersection 7–10, step 62 Local perception ETWT: vv =4, qq =3, Δv\Delta v = +1+1 , Δq\Delta q = +1+1 NTST: vv =0, qq =0, Δv\Delta v = −5-5 , Δq\Delta q = −2-2 ELWL : vv = 5 , qq = 3 , Δv\Delta v = +2+2 , Δq\Delta q = +1+1 NLSL: vv =2, qq =2, Δv\Delta v =0, Δq\Delta q = +1+1 Cooperative context Neighbor movement cues: ETWT: 1 vehicle ⋅\cdot ELWL: 1 vehicle Neighbor directional pressure: N: 2 queued ⋅\cdot S: 2 queued ⋅\cdot ETA: 27 s SLOW signal ELWL Key reasoning: Given ELWL’s high demand, queue, positive trend, and long waiting time, it is the most effective choice.
Figure 3: Case study on adapative reasoning.
Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Traffic-signal-control setting used in our experiments. (A) A four-arm intersection. (B) Twelve incoming lanes and their permitted movements; right turns remain permissive. (C) Four controlled signal phases, each pairing non-conflicting movements: ETWT and NTST serve east–west and north–south through movements, respectively, whereas ELWL and NLSL serve the corresponding left-turn movements.
CityTraceGridVehicles5-min arrivals ( μ±σ\mu\pm\sigma )Min–max
Jinan13×43\times 46295524.58±98.53524.58\pm 98.53256–672
Jinan23×43\times 44365363.75±74.66363.75\pm 74.66237–493
Jinan33×43\times 45494457.83±46.22457.83\pm 46.22363–544
Hangzhou14×44\times 42983248.58±40.45248.58\pm 40.45212–333
Hangzhou24×44\times 46984582.00±318.52582.00\pm 318.52203–1146
New York128×728\times 711058850.62±174.20850.62\pm 174.20383–965
Appendix
Table 6: Traffic-flow trace statistics. Arrival rates are vehicles per 5 min; values are reported as mean ±\pm standard deviation, followed by the observed range.
Figure 5: Topology-routed cooperative information: (A) downstream-linked frames estimate potential arrivals; (B) upstream vehicle and queue counts with link travel time summarize directional demand.
Figure 6: Fast case. A clear local-demand winner leads to fast routing.
Figure 7: Slow case. Competing local and cooperative evidence triggers slow reasoning.

Explore similar work

CardsList
Apr 28, 2026cs.LG

DGLight: DQN-Guided GRPO Fine-Tuning of Large Language Models for Traffic Signal Control

Traffic signal control (TSC) plays a central role in reducing congestion and maintaining urban mobility. This dissertation introduces DGLight, a critic-guided reinforcement-learning framework for adapting a pretrained large language model to TSC. DGLight first trains a CoLight-based Deep Q-Network critic to estimate traffic-aware action values from structured intersection states, then uses the frozen critic to score candidate language-model actions and optimize the policy with Group Relative Policy Optimization (GRPO). The resulting controller maps traffic states to interpretable reasoning traces and signal decisions while learning from dense per-state supervision rather than raw cumulative environment rewards. Experiments on TSC benchmarks covering Jinan and Hangzhou show that DGLight is the strongest overall method among the compared LLM-based controllers, remains competitive with strong RL baselines, and transfers well to city datasets not used to fit the critic. Qualitative examples further show that the model's generated reasoning is interpretable and aligned with the chosen signal phase. The project code is available [here](https://github.com/yyccbb/FYPLLMTSC)[here](https://github.com/yyccbb/FYP_LLMTSC).

Chenbo Yu

National University of Singapore Singapore

Traffic Signal ControlReinforcement Learning ControlTrajectory-Level CreditDeep Q-NetworksModel Fine-TuningValue Functions
AskPDF
May 28, 2026cs.AI

ReasonLight: A Multimodal Foundation Model-Enhanced Reinforcement Learning Framework for Zero-Shot Traffic Signal Control

Reinforcement learning (RL) has shown promise in traffic signal control (TSC). However, its reliance on predefined states limits responsiveness to observable open-world events that are absent from training data. IoT-enabled intersections provide heterogeneous observations from roadside sensors and cameras, creating opportunities to improve RL adaptability to such events. To this end, we propose ReasonLight, a multimodal foundation model-enhanced RL framework for zero-shot TSC. ReasonLight integrates three sources of information: structured traffic measurements, multi-view camera observations, and candidate phase decisions from a pre-trained RL controller. Given an RL-proposed phase, ReasonLight extracts visual semantics from multi-view images and aligns them with compact sensor-derived scene descriptions. This alignment enables a semantic-guided refinement module to either preserve or adjust the proposed action according to traffic rules and event semantics. To ensure operational reliability, refined actions are constrained by the set of available phases. Any invalid decision is rejected, and the system falls back to the original RL action. We evaluate ReasonLight on two types of rare events not seen during RL training: emergency vehicle priority and temporary traffic regulation. Experimental results show that ReasonLight achieves zero-shot adaptation without retraining. It reduces emergency vehicle waiting time by up to 88.7% compared with the RL-only backbone while preserving comparable routine traffic performance.

Aoyu Pang, Maonan Wang, Yuejiao Xie +3

School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, China · Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Hong Kong · Shanghai AI Laboratory, Shanghai, China +1

Traffic Signal ControlReinforcement Learning ControlOffline Reinforcement LearningMultimodal Foundation ModelEmergency Assistance
AskPDF
Apr 26, 2026cs.AI

LLM-Augmented Traffic Signal Control with LSTM-Based Traffic State Prediction and Safety-Constrained Decision Support

Traffic signal control is a critical task in intelligent transportation systems, yet conventional fixed-time and rule-based methods often struggle to adapt to dynamic traffic demand and provide limited decision interpretability. This study proposes an LLM-augmented traffic signal control framework that integrates LSTM-based short-term traffic state prediction, predictive phase selection, structured large language model reasoning, and safety-constrained action filtering. The LSTM module forecasts future queue length, waiting time, vehicle count, and lane occupancy based on recent intersection-level observations. A predictive controller then generates candidate signal actions, while the LLM module evaluates these actions using structured traffic-state inputs and produces congestion diagnoses, phase adjustment recommendations, and natural-language explanations. To ensure operational reliability, all LLM-generated recommendations are validated by a safety filter before execution. Simulation-based experiments in SUMO compare the proposed method with fixed-time control, rule-based control, and an LSTM-based predictive baseline under balanced demand, directional peak demand, and sudden surge scenarios. The results indicate that the proposed framework improves traffic efficiency, especially under dynamic and non-recurrent traffic conditions, while maintaining zero constraint violations after safety filtering. Overall, this study demonstrates that LLMs can enhance traffic signal control when used as constrained reasoning and decision-support modules rather than direct low-level controllers. Keywords: Intelligent Transportation Systems; Traffic Signal Control; Large Language Models; LSTM; Traffic State Prediction; Decision Support; Safety-Constrained Control; SUMO Simulation.

Jiazhao Shi

Tandon School of Engineering, New York University, 6 MetroTech Center, Brooklyn, NY 11201, USA

Traffic Signal ControlAccurate Traffic Forecasting
AskPDF