LLM-Enhanced Multi-Agent Reinforcement Learning for Unified Electric Vehicles-Charging Station-Grid Optimization in Public Charging Systems

Yang Zhang; Lindong Xie; Chongyu Wang; Gaojunjie Li; Siqi Bu; Edward Chung

cs.AISep 12, 2026

LLM-Enhanced Multi-Agent Reinforcement Learning for Unified Electric Vehicles-Charging Station-Grid Optimization in Public Charging Systems

Authors: Yang Zhang, Lindong Xie, Chongyu Wang, Gaojunjie Li, Siqi Bu, Edward Chung

Organizations: Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong, China · Galvin Center for Electricity Innovation, Illinois Institute of Technology, Chicago, IL 60616, USA

Abstract

In the era of the Internet of Things (IoT), coordinating connected electric vehicle (EV) charging scheduling to balance EV charging satisfaction, station profitability, and smart grid stability presents a complex multi-objective challenge. Existing Multi-Agent Reinforcement Learning (MARL) approaches often struggle with high-dimensional state spaces generated by massive IoT sensing data and conflicting stakeholder interests. This paper proposes a novel LLM-enhanced MARL framework that, for the first time, simultaneously optimizes the Grid, EVs, and Stations within a unified loop. By integrating Large Language Model (LLM), we address two critical bottlenecks: interpretable feature selection and adaptive multi-objective balancing. The LLM analyzes real-time IoT-collected environmental states to extract physically significant features and dynamically assigns weights to conflicting objectives-including profit, user satisfaction, and grid load-using semantic reasoning instead of complex manual tuning. Extensive experiments demonstrate that our framework significantly outperforms state-of-the-art baselines, achieving superior market efficiency while reducing training time by over 70%. This approach offers a scalable, transparent solution for efficient and sustainable IoT-enabled urban charging infrastructure management.

Figures & tables

Fig. 1: Unified EV charging scheduling framework among three perspectives: pricing by the charging station, charging power decisions by EVs, and transmission power control by the grid operator.

Fig. 2: Overview of the proposed approach. The framework consists of two LLM-based modules: features selection (LLM-FS) and multi-objective balancing (LLM-MOB). Building upon these modules, we employ MADDPG, a widely-adopted MARL algorithm, to develop the optimal scheduling policy.

Fig. 3: LLM-based interpretable feature selection approach.

Fig. 4: LLM-based adaptive and interpretable multi-objective balancing approach.

Parameter	Value
Minimum charging price ( $p_{t}^{min}$ )	$[7.5,15]$ $/MWh (seasonal)
Maximum charging price ( $p_{t}^{max}$ )	$[35,75]$ $/MWh (seasonal)
Grid price ( $p_{t}^{g}$ )	$[5.5,17.5]$ $/MWh (seasonal)
Minimum charging power ( $w_{min}^{i}$ )	$[7.2,12.5]$ kW (EV-dependent)
Maximum charging power ( $w_{max}^{i}$ )	$[29.0,50.0]$ kW (EV-dependent)
Target SoC ( $SoC_{des}^{i}$ )	$[70\%,95\%]$ (EV-dependent)

TABLE I: Parameter Settings for EV Charging Market

Parameter	Value
Training episodes	10,000
Batch size	128
Replay buffer size	10,000
Learning rate	0.0001
Actor network layers	3
Critic network layers	2

TABLE II: Parameter Settings for Examined Algorithms

Fig. 5: Episodic weighted rewards during training for feature selection-based approaches with five random seeds.

Approach	DEG ( $\downarrow$ )	AQL ( $\downarrow$ )	ECC ( $\downarrow$ )	DPS ( $\uparrow$ )	LPG ( $\downarrow$ )	CMEI ( $\uparrow$ )
Ours (LLM-FS+LLM-MOB)	${0.037\pm 0.014}$	$\mathbf{2.958\pm 1.241}$	$\mathbf{104.2\pm 35.1}$	${86.9\pm 40.2}$	$\mathbf{0.411\pm 0.021}$	$\mathbf{2.195\pm 0.192}$
LassoNet+MARL	${0.039\pm 0.015}$	${3.479\pm 1.435}$	${234.2\pm 151.8}$	${13.6\pm 17.0}$	${5.884\pm 0.489}$	${1.705\pm 0.316}$
LassoNet+LLM-MOB	${0.044\pm 0.012}$	${5.083\pm 1.782}$	${217.2\pm 151.8}$	${28.7\pm 15.6}$	${5.792\pm 0.381}$	${1.672\pm 0.217}$
RFE+MARL	${0.071\pm 0.016}$	${6.987\pm 0.523}$	${365.5\pm 118.4}$	${138.3\pm 45.2}$	${0.524\pm 0.035}$	${1.896\pm 0.185}$
RFE+LLM-MOB	${0.041\pm 0.017}$	${3.979\pm 2.628}$	${268.2\pm 103.1}$	${42.1\pm 162.7}$	${0.426\pm 0.028}$	${2.002\pm 0.372}$
Transformer+MARL	${0.117\pm 0.022}$	${10.251\pm 0.923}$	${333.5\pm 118.4}$	$\mathbf{156.3\pm 37.1}$	${0.416\pm 0.035}$	${1.719\pm 0.212}$

TABLE III: Comparison of feature selection-based approaches evaluated using DPS, DEG, AQL, ECC, LPG, and CMEI metrics during the execution phase (averaged over five seeds). Bold values indicate the best results. The gray-highlighted column displays the CMEI, which measures the overall operational efficiency of the charging market.

Approach	Episodes to converge	Time per episode (sec)	Total time (min)
Ours	${642}$	${3.85}$	$\mathbf{41.16}$
LassoNet+MARL	${557}$	${7.63}$	${70.83}$
LassoNet+LLM-MOB	${536}$	${7.67}$	${68.52}$
RFE+MARL	${718}$	${5.04}$	${60.31}$
RFE+LLM-MOB	${581}$	${4.41}$	${42.70}$
Transformer+MARL	${821}$	${40.41}$	${552.94}$

TABLE IV: Average training efficiency metrics for feature selection-based approaches: episodes to convergence, time per episode (sec), and total training time (min). Bold value indicates the minimum total training time.

Fig. 6: Episodic weighted rewards during training for multi-objective RL-based methods over five random seeds.

Approach	DEG ( $\downarrow$ )	AQL ( $\downarrow$ )	ECC ( $\downarrow$ )	DPS ( $\uparrow$ )	LPG ( $\downarrow$ )	CMEI ( $\uparrow$ )
Ours (LLM-FS+LLM-MOB)	${0.037\pm 0.014}$	$\mathbf{2.958\pm 1.241}$	$\mathbf{104.2\pm 35.1}$	${86.9\pm 40.2}$	$\mathbf{0.411\pm 0.021}$	$\mathbf{2.195\pm 0.192}$
GPI-PD	${0.035\pm 0.016}$	${2.983\pm 0.791}$	${342.6\pm 106.8}$	${4.6\pm 6.1}$	${15.189\pm 1.471}$	${1.153\pm 0.274}$
GPI-PD+LLM-FS	$\mathbf{0.034\pm 0.012}$	${4.104\pm 1.385}$	${281.8\pm 162.3}$	${56.7\pm 12.5}$	${8.983\pm 8.365}$	${1.555\pm 0.480}$
CAPQL	${0.072\pm 0.023}$	${5.625\pm 0.682}$	${146.7\pm 60.2}$	${70.9\pm 39.4}$	${0.425\pm 0.041}$	${1.973\pm 0.145}$
CAPQL+LLM-FS	${0.034\pm 0.011}$	${4.333\pm 0.973}$	${350.1\pm 125.7}$	$\mathbf{165.2\pm 112.4}$	${2.525\pm 1.941}$	${2.012\pm 0.145}$
PGMORL	${0.047\pm 0.014}$	${4.652\pm 0.917}$	${303.4\pm 41.7}$	${89.9\pm 21.1}$	${5.012\pm 1.368}$	${1.756\pm 0.192}$

TABLE V: Comparison between our approach and MORL-based approaches evaluated using DPS, DEG, AQL, ECC, LPG, and CMEI metrics during execution (averaged over five random seeds).

Approach	Episodes to converge	Time per episode (sec)	Total time (min)
Ours	${642}$	${3.85}$	$\mathbf{41.16}$
GPI-PD	${537}$	${16.40}$	${146.93}$
GPI-PD+LLM-FS	${626}$	${9.06}$	${94.53}$
CAPQL	${715}$	${8.95}$	${106.65}$
CAPQL+LLM-FS	${1328}$	${3.86}$	${85.43}$
PGMORL	${3885}$	${2.09}$	${135.33}$

TABLE VI: Training efficiency metrics for multi-objective RL-based approaches (episodes to convergence, time per episode in seconds, and total training time in minutes).

Fig. 7: Pareto fronts for different approaches across daily EVs, grid, and charging station rewards.

Panel of Figure 14 — Fig. 8: Feature importance ranking and rationale as determined by the LLM for EV charging scheduling.

Panel of Figure 15 — Fig. 9: Weights balancing in multi-objective optimization as determined by the LLM for EV charging scheduling.

Scenario Scale		Performance Metrics
EV number	Station number	DEG ( $\downarrow$ )	AQL ( $\downarrow$ )	ECC ( $\downarrow$ )	DPS ( $\uparrow$ )	LPG ( $\downarrow$ )	CMEI ( $\uparrow$ )
200+	1	0.037 ± 0.014	2.958 ± 1.241	104.2 ± 35.1	86.9 ± 40.2	0.411 ± 0.021	2.195 ± 0.192
400+	1	0.228 ± 0.017	45.162 ± 2.226	137.6 ± 14.4	99.4 ± 15.2	0.487 ± 0.034	1.916 ± 0.155
400+	3	0.052 ± 0.012	2.521 ± 1.951	529.2 ± 69.8	292.1 ± 142.1	3.791 ± 0.032	2.045 ± 0.454
800+	3	0.162 ± 0.033	23.547 ± 5.346	545.6 ± 75.4	299.3 ± 164.6	4.093 ± 0.416	2.055 ± 0.441
800+	5	0.067 ± 0.019	5.468 ± 0.832	1115.6 ± 314.1	751.1 ± 343.1	10.651 ± 0.128	2.114 ± 0.552

TABLE VII: Performance Evaluation Across Different Scenario Scales

Approaches	DEG ( $\downarrow$ )	AQL ( $\downarrow$ )	ECC ( $\downarrow$ )	DPS ( $\uparrow$ )	LPG ( $\downarrow$ )	CMEI ( $\uparrow$ )
Ours	$\mathbf{0.037\pm 0.014}$	$\mathbf{2.958\pm 1.241}$	$\mathbf{104.2\pm 35.1}$	${86.9\pm 40.2}$	$\mathbf{0.411\pm 0.021}$	$\mathbf{2.195\pm 0.192}$
w/o LLM-FS	${0.059\pm 0.006}$	${4.931\pm 1.256}$	${230.3\pm 105.2}$	${11.6\pm 76.1}$	${6.428\pm 1.467}$	${1.573\pm 0.274}$
w/o LLM-MOB	${0.071\pm 0.028}$	${7.275\pm 4.142}$	${290.1\pm 201.7}$	$\mathbf{156.7\pm 154.2}$	${3.941\pm 3.500}$	${1.764\pm 0.145}$

TABLE VIII: Ablation studies for component analysis.

LLM	DEG ( $\downarrow$ )	AQL ( $\downarrow$ )	ECC ( $\downarrow$ )	DPS ( $\uparrow$ )	LPG ( $\downarrow$ )	CMEI ( $\uparrow$ )
GPT-4o	$\mathbf{0.037\pm 0.014}$	$\mathbf{2.958\pm 1.241}$	${104.2\pm 35.1}$	${86.9\pm 40.2}$	${0.411\pm 0.021}$	${2.195\pm 0.192}$
o3-mini	${0.060\pm 0.016}$	${4.291\pm 0.582}$	${349.8\pm 43.6}$	$\mathbf{279.4\pm 48.9}$	${0.432\pm 0.004}$	$\mathbf{2.215\pm 0.155}$
DeepSeek-R1	${0.065\pm 0.005}$	${6.625\pm 2.215}$	${115.9\pm 43.2}$	${68.23\pm 27.3}$	${0.411\pm 0.006}$	${1.984\pm 0.148}$
LLaMA3-70B	${0.066\pm 0.019}$	${6.000\pm 1.875}$	${272.9\pm 170.5}$	${199.9\pm 21.1}$	$\mathbf{0.410\pm 0.007}$	${2.081\pm 0.205}$
Gemini-1.5	${0.087\pm 0.018}$	${8.395\pm 1.112}$	$\mathbf{90.3\pm 18.5}$	${32.56\pm 10.3}$	${0.413\pm 0.010}$	${1.824\pm 0.082}$

TABLE IX: Comparison of different LLMs employed in our approach evaluated using DPS, DEG, AQL, ECC, LPG, and CMEI metrics.

Electric Vehicle Charging

Electric vehicle charging research studies how to forecast, plan, control, and secure charging while coordinating vehicles, stations, and power grids.

See all 38 papers

Smart charging of large fleets of Electric Vehicles: Independent Multi-Agent Reinforcement Learning approaches
Jun 30, 2026Xavier Rate, Eloann Le Guern, Raphaël Féraud +8Multi-Agent Coordination Electric Vehicle Charging
Orange · ENS Rennes · IMT Atlantique +1
Forecasting what Matters: Decision-Focused RL for Controlled EV Charging with Unknown Departure Times
Jun 17, 2026Giuseppe Gabriele, Fabio Pavirani, Seyed Soroush Karimi Madahi +1RL Control Electric Vehicle Charging
Ghent University – imec Ghent, Belgium
Emergent Charging Coordination in Electric Delivery Fleets
Sep 7, 2026Javier Vales-Alonso, Juan J. AlcarazMulti-Agent Coordination Electric Vehicle Charging
Department of Information and Communication Technologies, Universidad Politécnica de Cartagena, Cartagena, Spain

Jun 30, 2026J/K move · Enter open · S save

Smart charging of large fleets of Electric Vehicles: Independent Multi-Agent Reinforcement Learning approaches

Xavier Rate, Eloann Le Guern, Raphaël Féraud, Fatma Salem+7

Orange · ENS Rennes · IMT Atlantique+1

Multi-Agent Coordination Electric Vehicle Charging Policy Gradient Methods Contextual Bandits Decentralized MARL Power Systems

Open paper AskPDF arXiv

The electrification of transportation through electric vehicles introduces new challenges for power grid management, such as increased peak demand, voltage fluctuations, line overloads, and the integration of variable renewable energy sources. To enable efficient integration of EVs while minimizing costs for users and avoiding network overloads, implicit coordination between EVs is required. This work compares two independent multi-agent reinforcement learning approaches for optimizing such decentralized EV charging: contextual combinatorial bandits and policy gradient algorithms. Using a realistic simulation environment with autonomous agents making decisions based on local environmental information (including price signals, state-of-charge, and temporal constraints), we evaluate their performance across varying congestion levels, and mixed-strategy configurations with heterogeneous agent groups under dynamic electricity pricing derived from real photovoltaic production data.

Abstract

Electric Vehicle Charging

Explore similar work

Smart charging of large fleets of Electric Vehicles: Independent Multi-Agent Reinforcement Learning approaches

Forecasting what Matters: Decision-Focused RL for Controlled EV Charging with Unknown Departure Times

Emergent Charging Coordination in Electric Delivery Fleets