OpenHail: An Event-Driven Gymnasium Environment for Electric Ride-Hailing Fleet Control

Tommaso Schettini; Nicholas D. Kullman; Jorge E. Mendoza

cs.LGSep 24, 2026

OpenHail: An Event-Driven Gymnasium Environment for Electric Ride-Hailing Fleet Control

Authors: Tommaso Schettini, Nicholas D. Kullman, Jorge E. Mendoza

Organizations: Concordia University · Université de Tours · HEC Montréal

Abstract

Machine-learning policies have attracted increasing interest for ride-hailing fleet control in recent years. Reinforcement learning, in particular, requires a structured simulation environment that specifies observations, actions, rewards, and decision epochs for training and evaluation. For electric fleets, this environment must also capture the interaction among stochastic demand, vehicle operations, and capacitated charging infrastructure. We present OpenHail, an open-source Gymnasium environment for joint control of electric ride-hailing fleets. Its fixed-size observation--action interface exposes request assignment, repositioning, and charging to a single policy. The event-driven simulator represents requests with pickup deadlines, vehicle job queues, battery dynamics, and finite-capacity charging facilities with first-in--first-out queues. A configurable decision-epoch mechanism separates internal simulator events from policy interactions, supporting event-driven, periodic, hybrid, and policy-requested control within the same operational model. The software provides seeded instances, feasible-action utilities, evaluation tools, operational metrics, and baseline policies. The source code is available at https://github.com/tommaso-schettini/openhail.

Figures & tables

Figure 1: The OpenHail architecture. Configuration supplies instance data and decision-epoch settings; policies interact with OpenhailEnv , which coordinates the five supporting components.

Figure 2: Post-hoc simulation time by controller, fleet size, and location count. Each marker is the median duration of five paired one-day rollouts; error bars span the interquartile range. The vertical axes are logarithmic. Policy optimization, model loading, and reset are excluded.

	Reposition locations
Vehicles	5	10	20	40
100	0.54	0.60	0.82	0.75
500	2.74	2.85	3.27	10.44
1,000	6.56	6.56	13.85	8.44
2,000	16.29	16.55	18.76	21.03

Table 4: Elapsed time (hours) to train each PPO agent for 500 episodes, including periodic evaluation. Columns give the number of reposition locations.

Figure 3: Post-hoc simulation time per exposed decision epoch. Each marker is the median of the five rollout-specific ratios of episode duration to decision count; error bars span the interquartile range. All panels use the same linear vertical scale.

RL Control

Reinforcement learning control uses reward-driven learning to make decisions in dynamic systems, from robots and vehicles to energy and infrastructure.

See all 275 papers

A Distributionally Robust Reinforcement Learning Framework for Constrained Urban EV Dispatch
Apr 28, 2026An Nguyen, Hoang Nguyen, Phuong Le +3Electric Vehicle Charging Constrained RL
College of Engineering and Computer Science, VinUniversity, Hanoi, Vietnam · Center for Environmental Intelligence, VinUniversity, Hanoi, Vietnam
Emergent Charging Coordination in Electric Delivery Fleets
Sep 7, 2026Javier Vales-Alonso, Juan J. AlcarazMulti-Agent Coordination Electric Vehicle Charging
Department of Information and Communication Technologies, Universidad Politécnica de Cartagena, Cartagena, Spain
EXHOLD: Experience-Aware Real-Time Hold Control for Large-Scale Ride-Hailing Matching at DiDi
Jul 10, 2026Xu Liu, Kai Wan, Zihao LuMulti-Objective Optimization
Didichuxing Co. Ltd · Beijing, China

Apr 28, 2026J/K move · Enter open · S save

A Distributionally Robust Reinforcement Learning Framework for Constrained Urban EV Dispatch

An Nguyen, Hoang Nguyen, Phuong Le, Hung Pham+2

College of Engineering and Computer Science, VinUniversity, Hanoi, Vietnam · Center for Environmental Intelligence, VinUniversity, Hanoi, Vietnam

Electric Vehicle Charging Constrained RL Distributionally Robust Optimization Distributionally Robust RL Robust RL

Open paper AskPDF arXiv

We study city-scale control of electric-vehicle (EV) ride-hailing fleets where dispatch, repositioning, and charging decisions must respect charger and feeder limits under uncertain, spatially correlated demand and travel times. We formulate the problem as a hex-grid semi-Markov decision process (semi-MDP) with mixed actions -- discrete actions for serving, repositioning, and charging, together with continuous charging power -- and variable action durations. To guarantee physical feasibility during both training and deployment, the policy learns over high-level intentions produced by a masked, temperature-annealed actor. These intentions are projected at every decision step through a time-limited rolling mixed-integer linear program (MILP) that strictly enforces state-of-charge, port, and feeder constraints. To mitigate distributional shifts, we optimize a Soft Actor-Critic (SAC) agent against a Wasserstein-1 ambiguity set with a graph-aligned Mahalanobis ground metric that captures spatial correlations. The robust backup uses the Kantorovich-Rubinstein dual, a projected subgradient inner loop, and a primal-dual risk-budget update. Our architecture combines a two-layer Graph Convolutional Network (GCN) encoder, twin critics, and a value network that drives the adversary. Experiments on a large-scale EV fleet simulator built from NYC taxi data show that PD-RSAC achieves the highest net profit, reaching $1.22M, compared with $0.58M-$0.70M for strong heuristic, single-agent RL, and multi-agent RL baselines, including Greedy, SAC, MAPPO, and MADDPG, while maintaining zero feeder-limit violations.

Abstract

RL Control

Explore similar work

A Distributionally Robust Reinforcement Learning Framework for Constrained Urban EV Dispatch

Emergent Charging Coordination in Electric Delivery Fleets

EXHOLD: Experience-Aware Real-Time Hold Control for Large-Scale Ride-Hailing Matching at DiDi