C$^2$T: Captioning-Structure and LLM-Aligned Common-Sense Reward Learning for Traffic--Vehicle Coordination

Yuyang Chen; Kaiyan Zhao; Yiming Wang; Ming Yang; Bin Rao; Zhenning Li

cs.MAApr 10, 2026

C $^2$ T: Captioning-Structure and LLM-Aligned Common-Sense Reward Learning for Traffic--Vehicle Coordination

Authors: Yuyang Chen, Kaiyan Zhao, Yiming Wang, Ming Yang, Bin Rao, Zhenning Li

Organizations: The State Key Laboratory of Internet of Things for Smart City, University of Macau · Wuhan University · Computer and Information Science, University of Macau · Hong Kong Polytechnic University

Abstract

State-of-the-art (SOTA) urban traffic control increasingly employs Multi-Agent Reinforcement Learning (MARL) to coordinate Traffic Light Controllers (TLCs) and Connected Autonomous Vehicles (CAVs). However, the performance of these systems is fundamentally capped by their hand-crafted, myopic rewards (e.g., intersection pressure), which fail to capture high-level, human-centric goals like safety, flow stability, and comfort. To overcome this limitation, we introduce C2T, a novel framework that learns a common-sense coordination model from traffic-vehicle dynamics. C2T distills "common-sense" knowledge from a Large Language Model (LLM) into a learned intrinsic reward function. This new reward is then used to guide the coordination policy of a cooperative multi-intersection TLC MARL system on CityFlow-based multi-intersection benchmarks. Our framework significantly outperforms strong MARL baselines in traffic efficiency, safety, and an energy-related proxy. We further highlight C2T's flexibility in principle, allowing distinct "efficiency-focused" versus "safety-focused" policies by modifying the LLM prompt.

Figures & tables

Figure 1: C 2 T pipeline. Stage 1 converts raw observations into schema-constrained captions and samples high-contrast/safety-contrast pairs. Stage 2 queries an LLM to label pairwise preferences and trains a scalar scorer $r_{\phi}$ with a Bradley–Terry loss (with frequency reweighting and score centering). Stage 3 freezes $r_{\phi}$ and injects it as an intrinsic component for TLC steps under a safety mask, mixing with external rewards; all streams are normalized before GAE+PPO. In our CityFlow instantiation, only TLCs are learning agents and vehicles remain part of the environment dynamics.

Dataset/Method	ATT $\downarrow$	AQL $\downarrow$	AWT $\downarrow$
Jinan-1 (Baseline)	62.30	14.80	21.50
Jinan-1 (C 2 T)	56.10	13.10	19.40
Jinan-2 (Baseline)	58.90	13.10	19.70
Jinan-2 (C 2 T)	53.00	11.80	17.90
Hangzhou-1 (Baseline)	71.40	16.30	24.20
Hangzhou-1 (C 2 T)	65.10	14.90	22.00

Table 1: Jinan/Hangzhou overall performance. Lower is better.

Method	ATT $\downarrow$	AQL $\downarrow$	AWT $\downarrow$
Adv-CoLight (baseline)	95.0	22.6	38.4
C 2 T (ours)	87.9	20.3	34.5

Table 2: Large-scale New York (196 intersections).

Figure 2: Learning curves on CityFlow. C 2 T steadily improves ATT (↓) while also increasing TTC P10 (↑), showing that efficiency does not come at the cost of safety.

Method	ATT $\downarrow$	AQL $\downarrow$	AWT $\downarrow$	Steps $\downarrow$
External-only	72.5	17.2	25.8	100k
C 2 T zero-shot	68.9	16.4	24.6	80k
C 2 T (1% prefs)	67.3	16.0	23.9	70k
Fusion (50/50)	66.1	15.6	23.5	65k

Table 3: City transfer: train on Jinan, evaluate on Hangzhou.

Setting / Method	ATT $\downarrow$	AWT $\downarrow$	Oscillation $\downarrow$
Extreme (baseline)	105.0	44.2	1.00
C 2 T	96.8	40.1	0.86
24h (baseline)	78.5	29.4	0.76
C 2 T	72.2	26.9	0.65

Table 4: Stress test results. Lower is better unless noted.

Figure 3: Ablation on Jinan-1. Removing mask/norm/schedule harms performance, while the full C 2 T achieves the best ATT/AWT.

Model	AUC $\uparrow$	Spearman $\uparrow$	ATT $\downarrow$	AWT $\downarrow$
Numeric-only	0.67	0.41	64.8	22.4
Caption-only	0.73	0.49	63.1	22.1
Fusion	0.78	0.56	61.9	21.6
Shuffled / No-units	0.60	0.30	66.1	23.9
C 2 T (full)	0.78	0.56	59.3	20.8

Table 5: Caption-structure ablation . Higher AUC/Spearman is better; lower ATT/AWT is better.

Traffic Signal Control

Traffic signal control studies how to set intersection phases and green times to reduce congestion while balancing safety, emissions, and priority needs.

See all 39 papers

LATS: Large Language Model Assisted Teacher-Student Framework for Multi-Agent Reinforcement Learning in Traffic Signal Control
Mar 25, 2026Yifeng Zhang, Peizhuo Li, Tingguang Zhou +2Traffic Signal Control LLM-Guided RL
Department of Mechanical Engineering, National University of Singapore
TrafficClaw: A Generalizable LLM Agent in the Unified Physical Environment for Urban Traffic Control
Apr 19, 2026Siqi Lai, Pan Zhang, Yuping Zhou +3Agentic RL Language Model-Based Control
The Hong Kong University of Science and Technology (Guangzhou) Guangzhou, Guangdong, China · Shandong University, Jinan, Shandong, China
CoordLight: Learning Decentralized Coordination for Network-Wide Traffic Signal Control
Mar 25, 2026Yifeng Zhang, Harsh Goel, Peizhuo Li +3Traffic Signal Control Multi-Agent Coordination
Department of Mechanical Engineering, National University of Singapore · Chandra Department of Electrical and Computer Engineering, The University of Texas at Austin · Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology

Mar 25, 2026J/K move · Enter open · S save

LATS: Large Language Model Assisted Teacher-Student Framework for Multi-Agent Reinforcement Learning in Traffic Signal Control

Yifeng Zhang, Peizhuo Li, Tingguang Zhou, Mingfeng Fan+1

Department of Mechanical Engineering, National University of Singapore

Traffic Signal Control LLM-Guided RL Multi-Agent Reinforcement Learning

Open paper AskPDF arXiv

Adaptive Traffic Signal Control (ATSC) aims to optimize traffic flow and minimize delays by adjusting traffic lights in real time. Recent advances in Multi-agent Reinforcement Learning (MARL) have shown promise for ATSC, yet existing approaches still suffer from limited representational capacity, often leading to suboptimal performance and poor generalization in complex and dynamic traffic environments. On the other hand, Large Language Models (LLMs) excel at semantic representation, reasoning, and analysis, yet their propensity for hallucination and slow inference speeds often hinder their direct application to decision-making tasks. To address these challenges, we propose a novel learning paradigm named LATS that integrates LLMs and MARL, leveraging the former's strong prior knowledge and inductive abilities to enhance the latter's decision-making process. Specifically, we introduce a plug-and-play teacher-student learning module, where a trained embedding LLM serves as a teacher to generate rich semantic features that capture each intersection's topology structures and traffic dynamics. A much simpler (student) neural network then learns to emulate these features through knowledge distillation in the latent space, enabling the final model to operate independently from the LLM for downstream use in the RL decision-making process. This integration significantly enhances the overall model's representational capacity across diverse traffic scenarios, thus leading to more efficient and generalizable control strategies. Extensive experiments across diverse traffic datasets empirically demonstrate that our method enhances the representation learning capability of RL models, thereby leading to improved overall performance and generalization over both traditional RL and LLM-only approaches. [...]

Abstract

Traffic Signal Control

Explore similar work

LATS: Large Language Model Assisted Teacher-Student Framework for Multi-Agent Reinforcement Learning in Traffic Signal Control

TrafficClaw: A Generalizable LLM Agent in the Unified Physical Environment for Urban Traffic Control

CoordLight: Learning Decentralized Coordination for Network-Wide Traffic Signal Control