Learning to Move Cities: Deep Meta-Models and Reinforcement Policies for Calibration and Control in Urban Networks
Authors: Adewumi Augustine Adepitan, Christopher J. Haruna, Oluwasegun Adegoke, Ayooluwatomiwa Ajiboye, Oluwatobi Oluwasakin
Organizations: Dept. of Civil, Environmental and Infrastructure Engineering George Mason University Fairfax, VA, USA · Dept. of Sustainability University of South Dakota Vermillion, SD, USA · School of Information Studies Syracuse University Syracuse, NY, USA · Department of Computer Science George Mason University Fairfax, VA, USA · Dept. of Transportation and Computing Federal University of Technology Akure, Nigeria
Urban transportation networks present complex optimization challenges spanning calibration of high-fidelity simulators and real-time operational control. This paper presents a shared latent-space framework that connects simulator calibration and reinforcement learning control through a common learned representation of urban traffic dynamics. First, we develop a combinatorial MLP-autoencoder architecture that learns low-dimensional manifolds linking simulator inputs (origin-destination demand, network parameters) to outputs (travel times, congestion patterns), enabling efficient Bayesian optimization for calibration. This approach demonstrates superior sample efficiency compared to traditional dimension reduction methods, achieving better fit to observational data within fixed computational budgets. Second, we implement a deep Q-learning agent with experience replay and target networks to optimize dynamic traffic assignment through scheduling and routing adjustments. In empirical evaluations on benchmark networks, our approach reduces system-wide travel times by up to 51% compared to baseline operations. The learned latent representation is not only used to reduce the dimensionality of Bayesian calibration, but is also incorporated into the reinforcement learning state representation, allowing the control policy to operate on compressed and calibrated traffic dynamics. This shared latent-space formulation provides a unified pathway from simulator calibration to adaptive operational control within intelligent transportation systems. Our results highlight the transformative potential of deep learning methods in urban mobility planning and management, particularly for large-scale networks where traditional optimization approaches face computational bottlenecks.
Figures & tables
Fig. 1: Combinatorial neural network architecture integrating autoencoder for dimension reduction with MLP output prediction
Method
Dim
Samples
Error
Time (h)
Bayesian Opt.
50
500
0.152
48.2
Active Subspaces+BO
8
300
0.098
28.7
MLP-AE (Ours)
6
200
0.064
18.3
TABLE I: Calibration methods comparison on benchmark network
Configuration
NRMSE
Travel Time Reduction
Full Framework
0.064
51%
Without Latent Compression
0.089
38%
Without Shared Latent RL State
0.081
41%
Without Prioritized Replay
0.074
46%
TABLE II: Ablation analysis of framework components
Component
Value
Component
Value
Latent dimension
6
Batch size
64
Encoder structure
50-32-16-6
Replay memory
50,000
Decoder structure
6-16-32-50
Discount factor
0.90
MLP predictor
6-32-16- m
Exploration decay
1.0→0.1
DQN structure
64 - 128 - 64 - ∣A∣
Training epochs
200
Optimizer
Adam
Hardware
NVIDIA V100 GPU
TABLE III: Implementation and training configuration
Fig. 2: Learning curve showing system travel time reduction achieved by DQN controller compared to baseline operations.
This paper focuses on dynamic origin-destination matrix estimation (DODE), a crucial calibration process necessary for the effective application of microscopic traffic simulations. The fundamental challenge of the DODE problem in microscopic simulations stems from the complex temporal dynamics and inherent uncertainty of individual vehicle dynamics. This makes it highly challenging to precisely determine which vehicle traverses which link at any given moment, resulting in intricate and often ambiguous relationships between origin-destination (OD) matrices and their contributions to resultant link flows. This phenomenon constitutes the credit assignment problem, a central challenge addressed in this study. We formulate the DODE problem as a Markov Decision Process (MDP) and propose a novel framework that applies model-free deep reinforcement learning (DRL). Within our proposed framework, the agent learns an optimal policy to sequentially generate OD matrices, refining its strategy through direct interaction with the simulation environment. This approach was evaluated through a toy experiment on the Nguyen-Dupuis network and a case study utilizing an actual highway subnetwork spanning Santa Clara and San Jose. Experimental results show that the proposed method consistently improves calibration performance relative to the strongest conventional baseline, reducing link-flow MSE by 23.7% in the toy experiment and by 59.2-88.3% in the real-world case study. By reframing DODE as a sequential decision-making problem, our approach addresses the credit assignment challenge through a learned policy and provides a novel framework for calibration of microscopic traffic simulations.
Donggyu Min, Seongjin Choi, Dong-Kyu Kim
Seoul National University · Seoul, 08826 · University of Minnesota +1
Urban traffic congestion is a growing global issue contributing significantly to long commute times and environmental pollution. Traditional traffic signal control systems often fail to adapt to dynamic traffic conditions. Adaptive traffic signal control can improve urban traffic without changing road infrastructure. Deep Reinforcement Learning (DRL) has shown strong performance for this task, but existing delay and queue-based rewards often produce short-sighted or unstable policies. This paper proposes a Momentum-Based Reward Function (MBRF) that encourages vehicles to keep moving rather than penalizing congestion alone. The method is evaluated in SUMO (Simulation of Urban MObility) using standard traffic metrics such as waiting time, queue length, throughput, and CO2 emissions. Results show that the proposed reward produces better throughput-emission trade-offs and more stable learning behavior than delay or queue-based rewards, as well as classical controllers such as Max Pressure and LQF.
Urban traffic congestion presents a significant challenge for modern cities, which impacts mobility and sustainability. Traditional traffic light control systems often fail to adapt to dynamic conditions, leading to inefficiencies. This paper proposes a novel deep reinforcement learning agent for traffic light control that addresses this limitation by explicitly integrating fairness considerations for both vehicular and pedestrian traffic. Unlike prior work, our approach dynamically balances these flows based on real-time demand, moving beyond systems focused solely on vehicles. Experimental results demonstrate that our agent effectively reduces congestion while ensuring equitable service for both the categories of road users. This research contributes to a practical and adaptable solution for intelligent traffic management within the framework of smart cities, paving the way for more efficient and inclusive urban mobility.
Matteo Cederle, Giacomo Scatto, Gian Antonio Susto