Spatio-temporal dual-stage hypergraph MARL for human-centric multimodal corridor traffic signal control

Xiaocai Zhang; Neema Nassir; Milad Haghani

cs.LGFeb 19, 2026

Spatio-temporal dual-stage hypergraph MARL for human-centric multimodal corridor traffic signal control

Authors: Xiaocai Zhang, Neema Nassir, Milad Haghani

Organizations: Department of Infrastructure Engineering, Faculty of Engineering and Information Technology, The University of Melbourne, VIC 3010, Australia

Abstract

Human-centric traffic signal control in corridor networks must increasingly account for multimodal travelers, particularly high-occupancy public transportation, rather than focusing solely on vehicle-centric performance. This paper proposes STDSH-MARL (Spatio-Temporal Dual-Stage Hypergraph based Multi-Agent Reinforcement Learning), a multi-agent deep reinforcement learning framework that follows a centralized training and decentralized execution paradigm. The proposed method captures spatio-temporal dependencies through a novel dual-stage hypergraph attention mechanism that models interactions across both spatial and temporal hyperedges. In addition, a hybrid discrete action space is introduced to jointly determine the next signal phase configuration and its corresponding green duration, enabling more adaptive signal timing decisions. Experiments conducted on a corridor network under five traffic scenarios demonstrate that STDSH-MARL achieves strong overall multimodal performance, with substantial and relatively consistent reductions in tram waiting time, while improvements in bus waiting time are more variable across traffic scenarios. These results highlight the trade-off among overall network efficiency, tram priority, and bus service quality. Compared with state-of-the-art baseline methods, the proposed approach achieves superior overall performance. Further ablation studies confirm the contribution of each component of STDSH-MARL, with temporal hyperedges identified as the most influential factor driving the observed performance gains.

Figures & tables

Figure 1: Overview of the proposed STDSH-MARL framework for multimodal corridor traffic signal control.

Traffic Simulation	References	Quantity
SUMO	Wang et al. (2021a) ; Li et al. (2021) ; Su et al. (2023) ; Song et al. (2024) ; Wang et al. (2024b) ; Bie et al. (2024) ; Yang (2023) ; Yang et al. (2021) ; Ren et al. (2024) ; Xu et al. (2024) ; Zhang et al. (2022b) ; Hu and Li (2024) ; Tan et al. (2020) ; Chu et al. (2019) ; Wu et al. (2020b) ; Bokade et al. (2023) ; Ge et al. (2021) ; Luo et al. (2024) ; Liu and Li (2023) ; Nie et al. (2025) ; Jia and Ji (2025) ; Fereidooni et al. (2025) ; Shen (2025) ; Chu et al. (2016) ; Ma and Wu (2020) ; Tao et al. (2023)	26
CityFlow	Zhu et al. (2023) ; Liu et al. (2023a) ; Wu et al. (2022) ; Liu et al. (2023b) ; Zhang et al. (2022a) ; Wang et al. (2020b) ; Liu et al. (2025) ; Lai et al. (2025) ; Satheesh and Powell (2025) ; Wang et al. (2025)	10
VISSIM	Jiang et al. (2021) ; Wang et al. (2021b) ; Wu et al. (2020a) ; Lee et al. (2019) ; Zhang et al. (2025)	5
Aimsun	Abdoos (2020) ; Yoon et al. (2025)	2
Paramics	El-Tantawy et al. (2013)	1
Not mentioned	Wang et al. (2020a)	1

Table 1: Summary of traffic simulation environments

Reward Components	References	Quantity
Queue length (veh)	Zhu et al. (2023) ; Liu et al. (2023a) ; Wang et al. (2020a) ; Liu et al. (2023b) ; Tan et al. (2020) ; Bokade et al. (2023) ; Zhang et al. (2022a) ; Wang et al. (2020b) ; Fereidooni et al. (2025) ; Liu et al. (2025) ; Lai et al. (2025) ; Chu et al. (2016) ; Wang et al. (2025)	13
Waiting time (veh)	Wang et al. (2024b) ; Yang (2023) ; Yang et al. (2021) ;	5
Queue length (veh)	Chu et al. (2019) ; Liu and Li (2023)
Difference of queue lengths between two consecutive timesteps (veh)	Jiang et al. (2021) ; Luo et al. (2024)	2
Waiting time (veh)	Wu et al. (2022) ; Xu et al. (2024)	2
Throughput (veh)	Hu and Li (2024)	1

Table 2: Summary of factors considered in reward mechanism for each reference (i)

Reward Components	References	Quantity
Travel time delay (veh)	Song et al. (2024)	1
Queue length (veh)
Throughput (veh)
Waiting time (veh)	Bie et al. (2024)	1
Queue length (veh)
Throughput (veh)

Table 3: Summary of factors considering for reward mechanism for each reference (ii)

DRL Framework	References	Quantity
Q-learning	Zhu et al. (2023) ; Abdoos (2020) ; El-Tantawy et al. (2013) ; Chu et al. (2016)	4
DQN	Wang et al. (2021a) ; Wang et al. (2024b) ; Bie et al. (2024) ; Liu et al. (2023a) ; Xu et al. (2024) ; Wang et al. (2021b) ; Tan et al. (2020) ; Wu et al. (2020a) ; Zhang et al. (2022a) ; Lee et al. (2019) ; Liu and Li (2023) ; Wang et al. (2020b) ; Fereidooni et al. (2025) ; Lai et al. (2025) ; Tao et al. (2023)	15
QMIX	Liu et al. (2023b) ; Bokade et al. (2023) ; Liu et al. (2025)	3
DDQN	Hu and Li (2024) ; Wang et al. (2020a)	2
DDDQN	Jiang et al. (2021)	1

Table 4: Summary of value-based DRL frameworks

DRL Framework	References	Quantity
PPO	Ren et al. (2024) ; Zhang et al. (2022b) ; Luo et al. (2024) ; Nie et al. (2025) ; Fereidooni et al. (2025) ; Satheesh and Powell (2025)	6
COMA Policy Gradient	Song et al. (2024)	1

Table 5: Summary of policy-based DRL frameworks

DRL Framework	References	Quantity
A2C	Su et al. (2023) ; Wu et al. (2022) ; Chu et al. (2019) ; Ma and Wu (2020)	4
A3C	Wu et al. (2022)	1
Actor-Critic	Yang (2023) ; Yang et al. (2021)	2
DDPG	Li et al. (2021) ; Wu et al. (2020b) ; Jia and Ji (2025) ; Shen (2025) ; Yoon et al. (2025)	5
SAC	Ge et al. (2021) ; Zhang et al. (2025) ; Wang et al. (2025)	3

Table 6: Summary of hybrid-based DRL frameworks

Figure 2: Overview of STDSH-MARL: DSHA learns a hypergraph-level embedding from spatial and temporal hyperedges for a centralized critic, while decentralized agents act from local state representations in a VISSIM-simulated corridor.

Type	Features
Vehicle number	Total vehicle number, number of buses, number of trams, number of private vehicles.
Passenger number	Total passenger number, passenger number for buses, passenger number for trams, passenger number for private vehicles.
Queue length	Total queue length, queue length for buses, queue length for trams, queue length for private vehicles.
Speed	Average speed, average speed for buses, average speed for trams, average speed for private vehicles.
Current signal phase	One-hot embedding of current signal phase for corresponding intersection.

Table 7: Features to define the state representation

Figure 3: Coverage of the approaching lanes for the example intersection B among a corridor network. Green stands for the vehicle lanes and pink denotes tram lanes. The schematic assumes left-hand traffic.

Figure 4: Phase configurations for action selection.

Figure 5: Partial view of the constructed corridor network in VISSIM for experimental evaluation.

Hyperparameter	Value	Hyperparameter	Value
$M$ - Number of spatial hyperedges	5	Batch size	64
$P$ - Number of temporal hyperedges	6	Actor learning rate	0.0001
$K$ - Number of heads	4	Critic learning rate	0.0003
$d_{model}$ - Model dimension	64	Number of episodes	300
Hidden dimension	128	$\epsilon$ - Clipping range	0.1
$\gamma$ - Discount factor	0.99	$\alpha_{\mathrm{entropy}}$ - Entropy coefficient	0.001

Table 8: Hyperparameter settings used in the proposed STDSH-MARL model.

Figure 6: Training reward curve over the episodes. The light blue line represents the original cumulative reward, while the dark orange line represents the 10-episode moving average.

Panel of Figure 7 — Figure 7: Time-series heatmaps of delayed passengers and vehicle queue lengths over the 1,800-s horizon in Scenario 1, comparing all control methods; color intensity indicates magnitude (cooler = lower, warmer = higher).

Model	ANP	AQL	AWT (bus)	AWT (tram)
FS-WF	1401.86	298.88	344.13	112.00
Actuated	1671.05	390.66	350.56	169.75
MP	1385.87	243.63	364.76	581.80
MADQN	1680.78	282.48	944.81	1552.67
MADDQN	1123.50	192.18	687.44	484.40
MAA2C	1679.78	359.62	377.15	321.75

Table 9: Performance comparison for different models (scenario 1)

Panel of Figure 8 — Figure 8: Time-series heatmaps of delayed passengers and vehicle queue lengths over the 1,800-s horizon in Scenario 2, comparing all control methods; color intensity indicates magnitude (cooler = lower, warmer = higher).

Model	ANP	AQL	AWT (bus)	AWT (tram)
FS-WF	1298.39	286.81	313.97	129.50
Actuated	1542.32	378.17	315.94	172.35
MP	1237.46	255.47	350.10	532.55
MADQN	1501.23	289.10	843.04	1594.54
MADDQN	1031.01	214.91	502.11	640.79
MAA2C	1470.48	325.22	314.49	328.85

Table 10: Performance comparison for different models (scenario 2)

Panel of Figure 9 — Figure 9: Time-series heatmaps of delayed passengers and vehicle queue lengths over the 1,800-s horizon in Scenario 3, comparing all control methods; color intensity indicates magnitude (cooler = lower, warmer = higher).

Model	ANP	AQL	AWT (bus)	AWT (tram)
FS-WF	1489.44	337.71	354.82	127.20
Actuated	1747.40	387.99	377.09	205.85
MP	1261.59	245.88	284.35	423.85
MADQN	1842.39	302.48	958.39	1594.54
MADDQN	1143.86	195.24	659.35	538.63
MAA2C	1642.75	349.47	362.77	317.50

Table 11: Performance comparison for different models (scenario 3)

Panel of Figure 10 — Figure 10: Time-series heatmaps of delayed passengers and vehicle queue lengths over the 1,800-s horizon in Scenario 4, comparing all control methods; color intensity indicates magnitude (cooler = lower, warmer = higher).

Model	ANP	AQL	AWT (bus)	AWT (tram)
FS-WF	1452.45	312.61	351.15	124.56
Actuated	1655.16	399.70	354.04	177.85
MP	1267.16	249.59	328.72	377.85
MADQN	1841.18	330.92	769.57	1594.54
MADDQN	1079.67	195.48	559.05	437.10
MAA2C	1649.37	351.65	352.28	336.00

Table 12: Performance comparison for different models (scenario 4)

Panel of Figure 11 — Figure 11: Time-series heatmaps of delayed passengers and vehicle queue lengths over the 1,800-s horizon in Scenario 5, comparing all control methods; color intensity indicates magnitude (cooler = lower, warmer = higher).

Model	ANP	AQL	AWT (bus)	AWT (tram)
FS-WF	1525.12	323.31	367.67	138.05
Actuated	1632.73	385.13	365.03	166.10
MP	1271.78	241.23	356.08	388.10
MADQN	1567.47	292.65	890.94	1594.54
MADDQN	1165.59	206.80	575.79	656.68
MAA2C	1571.48	363.43	342.23	267.75

Table 13: Performance comparison for different models (scenario 5)

DSHA	SHE	THE	Scenario	ANP	AQL	AWT (bus)	AWT (tram)
✓	✓	$\times$	1	1511.44	324.95	357.38	239.70
✓	✓	$\times$	2	1446.86	334.88	297.68	226.30
✓	✓	$\times$	3	1584.63	347.83	354.56	189.65
✓	✓	$\times$	4	1572.83	355.27	353.03	205.25
✓	✓	$\times$	5	1487.27	335.51	337.15	240.05
✓	✓	$\times$	Average	1520.61	339.69	339.96	220.19

Table 14: Ablation study performance of each component across all scenarios

Comparison	Proposed Mean	Baseline Mean	$p$ -value
STDSH-MARL vs MADQN	791.35	1680.78	0.0
STDSH-MARL vs MADDQN	791.35	1123.50	0.0
STDSH-MARL vs MAA2C	791.35	1679.78	0.0
STDSH-MARL vs MAPPO	791.35	1468.63	0.0
STDSH-MARL vs MP	791.35	1385.87	0.0
STDSH-MARL vs CMRM	791.35	822.11	$4.01\times 10^{-125}$

Table 15: Statistical significance testing of number of passengers experiencing delay between the proposed STDSH-MARL and some baseline methods (on Scenario 1).

Figure 12: Training time comparison among different control methods.

Traffic Signal Control

Traffic signal control studies how to set intersection phases and green times to reduce congestion while balancing safety, emissions, and priority needs.

See all 39 papers

A Robust and Efficient Multi-Agent Reinforcement Learning Framework for Traffic Signal Control
Mar 12, 2026Sheng-You Huang, Hsiao-Chuan Chang, Yen-Chi Chen +6Centralized Training with Decentralized Execution Reinforcement Learning
National Yang Ming Chiao Tung University · Academia Sinica · Kochi University of Technology +1
LATS: Large Language Model Assisted Teacher-Student Framework for Multi-Agent Reinforcement Learning in Traffic Signal Control
Mar 25, 2026Yifeng Zhang, Peizhuo Li, Tingguang Zhou +2Traffic Signal Control LLM-Guided RL
Department of Mechanical Engineering, National University of Singapore
CoordLight: Learning Decentralized Coordination for Network-Wide Traffic Signal Control
Mar 25, 2026Yifeng Zhang, Harsh Goel, Peizhuo Li +3Traffic Signal Control Multi-Agent Coordination
Department of Mechanical Engineering, National University of Singapore · Chandra Department of Electrical and Computer Engineering, The University of Texas at Austin · Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology

Mar 12, 2026J/K move · Enter open · S save

A Robust and Efficient Multi-Agent Reinforcement Learning Framework for Traffic Signal Control

Sheng-You Huang, Hsiao-Chuan Chang, Yen-Chi Chen, Ting-Han Wei+5

National Yang Ming Chiao Tung University · Academia Sinica · Kochi University of Technology+1

Centralized Training with Decentralized Execution Reinforcement Learning Multi-Agent Reinforcement Learning Robust RL

Open paper AskPDF arXiv

Reinforcement Learning (RL) in Traffic Signal Control (TSC) faces significant hurdles in real-world deployment due to limited generalization to dynamic traffic flow variations. Existing approaches often overfit static patterns and use action spaces incompatible with driver expectations. This paper proposes a robust Multi-Agent Reinforcement Learning (MARL) framework validated in the Vissim traffic simulator. The framework integrates three mechanisms: (1) Turning Ratio Randomization, a training strategy that exposes agents to dynamic turning probabilities to enhance robustness against unseen scenarios; (2) a stability-oriented Exponential Phase Duration Adjustment action space, which balances responsiveness and precision through cyclical, exponential phase adjustments; and (3) a Neighbor-Based Observation scheme utilizing the MAPPO algorithm with Centralized Training with Decentralized Execution (CTDE). By leveraging centralized updates, this approach approximates the efficacy of global observations while maintaining scalable local communication. Experimental results demonstrate that our framework outperforms standard RL baselines, reducing average waiting time by over 10%. The proposed model exhibits superior generalization in unseen traffic scenarios and maintains high control stability, offering a practical solution for adaptive signal control.

Abstract

Traffic Signal Control

Explore similar work

A Robust and Efficient Multi-Agent Reinforcement Learning Framework for Traffic Signal Control

LATS: Large Language Model Assisted Teacher-Student Framework for Multi-Agent Reinforcement Learning in Traffic Signal Control

CoordLight: Learning Decentralized Coordination for Network-Wide Traffic Signal Control