Energy-Aware Path Following: Comparative Analysis of Reinforcement Learning and NMPC for Electric Vehicles

Mohamed Sabaa; Mostafa Emam

cs.ROOct 6, 2026

Energy-Aware Path Following: Comparative Analysis of Reinforcement Learning and NMPC for Electric Vehicles

Authors: Mohamed Sabaa, Mostafa Emam

Organizations: Computer Science Department, Najran University, Najran, Saudi Arabia · Institute of Applied Mathematics and Scientific Computing, University of the Bundeswehr Munich, Neubiberg, 85579, Germany

Abstract

Path-following control strategies typically follow the bi-objective optimization dilemma: minimizing deviations from a reference path while maintaining smooth speed profiles. The latter objective is especially relevant for Electric Vehicles (EVs), since their limited driving range can be extended by recovering energy through regenerative braking, a feature that has not yet been sufficiently studied in the literature. In this work, we perform a comparative analysis of four controllers under one common Frenet frame-based kinematic vehicle model, utilizing a validated energy model (VT-CPEM) with explicit regenerative braking. Herein, we implement the following controllers: Nonlinear Model Predictive Control (NMPC), Proximal Policy Optimization (PPO), gain-scheduled Ackermann state-feedback baseline (PID-SF), and a Stanley geometric baseline. To satisfy real-time requirements, we implement the NMPC using JIT-compiled CasADi. Moreover, we train the PPO using traditional straight and S-curve tracks, after which we successfully transfer the unmodified policy to unseen tracks, including: an ISO 3888-1 lane-change, a chicane, randomly-generated parameterized-splines, and a $\pm3^\circ$ graded road. In addition, the policy transfers to a dynamic single-track vehicle model with linear tires, zero-shot with an acceptable initial performance, which was optimized after brief fine-tuning. Thereby, we demonstrate that our PPO is readily transferable to more comprehensive vehicle models. We conclude with a performance analysis of developed controllers and discuss ideas for future work.

Figures & tables

Research and Scope	Vehicle Model	Control Strategy	Energy Model	Experiments
Fu et al. (2022) : Path tracking at handling limits	Nonlinear vehicle dynamics with tire-force, load-transfer, and adhesion effects	NMPC	-	HIL and simulation using CarSim.
Domina and Tihanyi (2022) : Automated path following near handling limits	Linear time-varying vehicle model considering steering dynamics	LTV-MPC	-	Simulation-based using MATLAB.
Reiter et al. (2023) : Obstacle avoidance and path following	Kinematic vehicle model in combined Cartesian/Frenet coordinate representation	NMPC	-	Simulation-based using ACADOS.
Belkebir et al. (2026) Curvature-aware path tracking with embedded NMPC	Augmented kinematic model in Frenet frame	NMPC with curvature-aware speed and Lyapunov terminal cost	-	Simulation using CARLA and embedded timing with CasADi on Raspberry Pi 5 and Xavier AGX.
Li et al. (2019) : Vision-based lateral control	TORCS / VTORCS simulator with a perception-to-control pipeline	RL + deep learning perception	-	Simulator-based training and evaluation using VTORCS; compared against LQR, MPC.
Hess and Ljungbergh (2021) : Longitudinal and lateral path following	Custom kinematic bicycle model	DDPG	-	Simulation-based using PyTorch.

Table 1: Comparative overview of representative control and planning methods discussed in this section.

Figure 1: Reference velocity

v_{\mathrm{ref}}(s)

and curvature

\kappa(s)

on an S-curve. The terminal deceleration ramp drives regenerative energy recovery.

Symbol	Value	Description
Vehicle (adopted from VT-CPEM Fiori et al. (2016) )
$m$	$1500~($ \mathrm{k}\mathrm{g} $)$	Mass
$L_{f}$	$4.0~($ \mathrm{m} $)$	Wheelbase
$C_{D}$	$0.30$	Drag coefficient
$A_{f}$	$2.2~($ \mathrm{m} $^{2})$	Frontal area
$C_{r}$	$0.01$	Rolling resistance

Table 2: Complete Parameter List.

Figure 2: PID-SF unit step response on the kinematic model (

d_{0}=1~(

\mathrm{m}

),~v=5~(

\mathrm{m}\mathrm{s}

^{-1})

Phase	Objective	Steps	Description
0	Longitudinal control	$50\times 10^{3}$	Straight line ( $\kappa\approx 0$ ), speed tracking
1	Lateral control	$120\times 10^{3}$	S-curve, lateral introduction $\|d_{0}\|\leq 0.3~($ \mathrm{m} $)$
2	Longitudinal + Lateral	$80\times 10^{3}$	Refining controls of coupled dynamics
3	Disturbance recovery	$80\times 10^{3}$	$\|d_{0}\|\leq 1.5~($ \mathrm{m} $),~\sigma_{\mathrm{obs}}=0.02$ applied
4	Energy optimization	$120\times 10^{3}$	Refining controls with respect to energy $e_{\mathrm{energy}}$
Total: 450 k steps; five seeds (42–46).

Table 3: Five-Phase Curriculum Schedule.

Figure 3: Closing the regenerative-energy gap. Left: regenerated fraction

E_{r}/E_{t}

for the four ablation variants against the NMPC reference; annotations highlight the share of the gap closed. Right: the same variants in the tracking–energy plane, signifying that preview improves both objectives simultaneously.

Variant	Preview observation	Energy reward	$\mathrm{RMSE}_{d}~($ \mathrm{m} $)$	Regen (%)
base	$\times$	$\times$	$0.459$	$9.3$
preview	$\checkmark$	$\times$	$\mathbf{0.310}$	$23.1$
energy-reward	$\times$	$\checkmark$	$0.398$	$32.7$
preview+energy	$\checkmark$	$\checkmark$	$0.314$	$\mathbf{47.9}$
NMPC (reference)	–	–	$0.474$	$59.9$

Table 4:

2\times 2

ablation of the two proposed changes (primary seed, S-curve, benchmark IC). Gap closed is measured against NMPC’s regenerative fraction: preview

27

%, energy-reward

46

%, both

76

%. Energies are at the single benchmark IC;

N=30

means differ compared to Table 8 .

Figure 4: Lateral deviation

d(t)

on the S-curve for the four controllers at the benchmark initial condition. Legend values are the RMSE of the plotted tracks.

Controller	$\mathrm{RMSE}_{d}~($ \mathrm{m} $)$	$E_{t}~($ \mathrm{k}\mathrm{J} $)$	$E_{r}~($ \mathrm{k}\mathrm{J} $)$	$E_{net}~($ \mathrm{k}\mathrm{J} $)$	Solving time $($ \mathrm{ms} $)$	Speedup
NMPC (do-mpc)	$0.474$	$41.69$	$24.95$	$16.79$	$117.6$	–
NMPC (compiled)	$0.453$	–	–	–	$16.3$	$1\times$
PPO (preview+e)	$\mathbf{0.314}$	$49.76$	$23.85$	$25.92$	$0.407$	$\mathbf{40\times}$
PID-SF	$1.155$	$57.76$	$24.74$	$33.02$	$0.065$	$251\times$
Stanley	$1.666$	$57.72$	$25.24$	$32.48$	$0.088$	$185\times$

Table 5: Main Results (S-curve,

\bm{x}_{0}=[0,0.5,0.1,0,3]^{\top}

). Energies in

\mathrm{k}\mathrm{J}

; solve time in

\mathrm{ms}

. Speedups are relative to the JIT-compiled NMPC, cf. Section 5.3 . The do-mpc timings are: mean

117.6

, p95

149.5

, max

164.5

. The compiled solver was executed for timing analysis only; its energy figures are omitted because it solves the identical optimal control problem and its trajectory deviation (

0.453

vs.

0.474

) differs only by solver tolerance.

Figure 5: Heading error

\alpha(t)

on the S-curve for the four controllers at the benchmark initial condition. NMPC and PPO preview+energy maintain smooth orientation tracking, while PID-SF and Stanley exhibit considerable oscillations.

Figure 6: Control inputs on the S-curve: steering rate

u_{1}=\dot{\delta}

(left) and longitudinal acceleration

u_{2}=\dot{v}

(right). Both NMPC and PPO strictly respect actuator bounds without high-frequency oscillations.

Figure 7: Kinematic lateral acceleration on the S-curve. The black dashed line marks the dynamics handling limit

a_{\mathrm{lat},\max}

Figure 8: Energy breakdown per controller. Percentages show regenerative fraction

E_{r}/E_{t}

. NMPC achieves the highest recovery at

59.9

%, while the preview policy only reaches

47.9

Controller	$\eta_{r}=0.50$	$\eta_{r}=0.65$	$\eta_{r}=0.80$
NMPC	$22.55$	$16.79$	$11.04$
PPO	$31.42$	$25.92$	$20.42$
PID-SF	$38.73$	$33.02$	$27.31$
Stanley	$38.30$	$32.48$	$26.65$

Table 6:

E_{\mathrm{net}}~(

\mathrm{k}\mathrm{J}

)

versus regeneration efficiency

\eta_{r}

Configuration	$w_{d}$	$w_{\alpha}$	$w_{v}$	$w_{u_{1}}$	$w_{u_{2}}$	$\mathrm{RMSE}_{d}~($ \mathrm{m} $)$	$E_{\mathrm{net}}~($ \mathrm{k}\mathrm{J} $)$
Best tracking	$76.50$	$100.75$	$6.00$	$0.81$	$0.80$	$\mathbf{0.309}$	$20.59$
Best energy	$4.33$	$239.75$	$0.95$	$4.95$	$3.32$	$0.719$	$\mathbf{15.84}$
Default	$10.0$	$100.0$	$1.0$	$1.0$	$1.0$	$0.474$	$16.79$

Table 7: Extremes of the NMPC weight Pareto front (

N=30

LHS, nine non-dominated configurations).

Figure 9: Pareto-frontier analysis for the NMPC weights with

N=30

LHS. The selected (default) configuration approaches the frontier, exhibiting an acceptable outcome without exhaustive tuning.

Controller	$\mathrm{RMSE}_{d}~($ \mathrm{m} $)$	$E_{\mathrm{net}}~($ \mathrm{k}\mathrm{J} $)$	Speedup	Failures
NMPC	$0.520\pm 0.120$	$10.34\pm 7.35$	$1\times$	$\mathbf{0/30}$
PPO	$\mathbf{0.387\pm 0.190}$	$19.36\pm 7.14$	$250\times$	$\mathbf{0/30}$
PID-SF	$1.337\pm 0.434$	$27.96\pm 7.55$	$1880\times$	$5/30$
Stanley	$1.713\pm 0.127$	$28.31\pm 8.54$	$1535\times$	$4/30$

Table 8: Statistical results over

N=30

LHS initial conditions, which differ from the single-IC values of Table 5 . Wilcoxon signed-rank (paired), PPO

<

NMPC:

p<0.0001

; Cohen’s

d=0.826

. Speedups here are relative to the interpreted do-mpc solver; check Section 5.3 for the compiled comparison.

Figure 10: Tracking accuracy over

N=30

LHS initial conditions (mean

\pm

std, failure counts annotated). Wilcoxon

p<0.0001

; Cohen’s

d=0.826

Controller	S-curve $($ \mathrm{m} $)$	ISO 3888-1 $($ \mathrm{m} $)$	Chicane $($ \mathrm{m} $)$
NMPC	$0.474$	$\mathbf{0.112}$	$\mathbf{0.135}$
PPO (zero-shot)	$\mathbf{0.314}$	$0.197$	$0.189$
PID-SF	$1.155$	$0.198$	$0.451$
Stanley	$1.666$	$0.124$	$0.141$

Table 9: Multi-track evaluation. The policy is not retrained on any evaluation track.

Figure 11: Zero-shot evaluation on the ISO 3888-1 double lane change. The policy was trained only on the S-curve; no retraining was performed.

Controller	Tracks	$\mathrm{RMSE}_{d}~($ \mathrm{m} $)$	Failures
NMPC	$30$	$\mathbf{0.163\pm 0.038}$	$0/30$
PPO	$30$	$0.256\pm 0.032$	$\mathbf{0/30}$
Stanley	$30$	$0.304\pm 0.426$	$1/30$
PID-SF	$30$	$0.375\pm 0.101$	$0/30$

Table 10: Generalization over

30

unseen random splines with

\kappa_{\max}\in[0.021,0.085]~(

\mathrm{m}

^{-1})

. All controllers, including NMPC, are evaluated on all

30

tracks.

$\sigma_{\mathrm{obs}}$	$\mathrm{RMSE}_{d}~($ \mathrm{m} $)$
$0.000$	$0.314\pm 0.000$
$0.010$	$0.308\pm 0.008$
$0.020$	$0.369\pm 0.031$
$0.050$	$1.549\pm 0.546$
$0.100$	$2.015\pm 0.337$

Table 11: Observation-noise robustness. Values are mean

\pm

std over five independent noise realizations of the same trained policy; the zero standard deviation at

\sigma=0

reflects the deterministic policy. This differs from the five training seeds of Section 5.7 .

Var.	Description	$\mathrm{RMSE}_{d}~($ \mathrm{m} $)$	$E_{\mathrm{net}}~($ \mathrm{k}\mathrm{J} $)$
A	Full (Ph. 0 $\to$ 1 $\to$ 2 $\to$ 3 $\to$ 4)	$0.379$	$\mathbf{26.89}$
B	Phase 1 only	$0.332$	$58.10$
C	Phases 1+2 only	$\mathbf{0.312}$	$47.51$
D	Phase 2 only (no curriculum)	$0.323$	$63.65$

Table 12: Curriculum ablation (

200\times 10^{3}

steps each). Ranking is by

E_{\mathrm{net}}

since tracking is effectively unaffected, i.e., tracking spread across variants is

0.067~(

\mathrm{m}

)

, while energy spread is

36.8~(

\mathrm{k}\mathrm{J}

)

Controller	$\mathrm{RMSE}_{d}$ (m)	$E_{n}$ (kJ)	Regen (%)	Status
PPO fine-tuned ( $100$ k)	$0.273$	$25.56$	$48.9$	ok
LQR (dynamic model)	$0.438$	$29.30$	$46.0$	ok
PPO zero-shot	$0.473$	$25.00$	$48.0$	ok
NMPC (kinematic model)	$0.560$	$13.69$	$68.6$	ok
PID-SF (kinematic design)	$0.740$	$30.10$	$44.7$	ok
Stanley (re-tuned)	$2.008$	$25.06$	$52.2$	ok

Table 13: Dynamic single-track validation with linear tires,

C_{f}=C_{r}^{\mathrm{tire}}=80~(

\mathrm{k}\mathrm{N},\mathrm{r}\mathrm{a}\mathrm{d}

^{-1}),~v\leq 8~(

\mathrm{m}\mathrm{s}

^{-1})

. All runs reach the end of path. NMPC retains its kinematic internal model and operates under plant–model mismatch, a nominal case in practice.

Figure 12: Lateral deviation on the dynamic single-track plant with linear tires and

v\leq 8~(

\mathrm{m}\mathrm{s}

^{-1})

. The fine-tuned policy tracks second only to the purposefully-designed dynamic LQR, while remaining within the linear-tire regime

|a_{\mathrm{lat}}|\approx 3.48~(

\mathrm{m}\mathrm{s}

^{-2})

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Phase	$w_{h}$	$w_{d}$	$w_{p}$	$w_{\delta}$	$w_{v}$	$w_{a}$	$w_{e}$
0 (longitudinal, straight)	$0.3$	$0.3$	$1.0$	$0.02$	$0.8$	$0.10$	$0.0$
1 (lateral introduction)	$0.8$	$1.2$	$0.5$	$0.05$	$0.3$	$0.05$	$0.0$
2 (stopping, full path)	$0.8$	$1.2$	$0.8$	$0.05$	$0.3$	$0.05$	$0.0$
3 (disturbance recovery)	$1.0$	$1.5$	$0.5$	$0.10$	$0.4$	$0.05$	$0.0$
4 (energy optimization)	$0.6$	$1.0$	$0.5$	$0.10$	$0.5$	$0.15$	$\mathbf{0.3}$

Appendix

Table 14: Reward weights of ( 22 ) by curriculum phase. An additional penalty of

0.5

is applied whenever the normalized speed error exceeds

0.25

Hess and Ljungbergh (2021) , and

r_{\mathrm{OOB}}=5

is applied once at the terminal step of a failed episode.

Explore similar work

Apr 22, 2026eess.SY

Personalized electric vehicle energy consumption estimation framework that integrates driver behavior with map data

This paper presents a personalized Battery Electric Vehicle (BEV) energy consumption estimation framework that integrates map-based contextual features with driver-specific velocity prediction and physics-based energy consumption modeling. The system combines route selection, detailed road feature processing, a rule-based reference velocity generator, a PID controller-based vehicle dynamics simulator, and a Bidirectional LSTM model trained to reproduce individual driving behavior. The predicted individual-specific velocity profiles are coupled with a quasi-steady backward energy consumption model to compute tractive power, regenerative braking, and State-of-Charge (SOC) evolution. Evaluation across urban, freeway, and hilly routes demonstrates that the proposed approach captures key driver behavioral patterns such as deceleration at intersections, speed-limit tracking, and road grade-dependent responses, while producing accurate power and SOC trajectories. The results highlight the effectiveness of combining learned driver behavior with map-based context and physics-based energy consumption modeling to produce accurate, personalized BEV SOC depletion profiles.

Sreechakra Vasudeva Raju Rachavelpula, Sangwhan Cha

Department of Computer and Information Sciences, Harrisburg University of Science and Technology, Harrisburg, PA 17101, USA

Electric Vehicles Lithium-Ion Batteries Energy Consumption Vehicle Dynamics Human Driving Bidirectional Long Short-Term Memory+1

Ask PDF

Apr 28, 2026cs.AI

A Distributionally Robust Reinforcement Learning Framework for Constrained Urban EV Dispatch

We study city-scale control of electric-vehicle (EV) ride-hailing fleets where dispatch, repositioning, and charging decisions must respect charger and feeder limits under uncertain, spatially correlated demand and travel times. We formulate the problem as a hex-grid semi-Markov decision process (semi-MDP) with mixed actions -- discrete actions for serving, repositioning, and charging, together with continuous charging power -- and variable action durations. To guarantee physical feasibility during both training and deployment, the policy learns over high-level intentions produced by a masked, temperature-annealed actor. These intentions are projected at every decision step through a time-limited rolling mixed-integer linear program (MILP) that strictly enforces state-of-charge, port, and feeder constraints. To mitigate distributional shifts, we optimize a Soft Actor-Critic (SAC) agent against a Wasserstein-1 ambiguity set with a graph-aligned Mahalanobis ground metric that captures spatial correlations. The robust backup uses the Kantorovich-Rubinstein dual, a projected subgradient inner loop, and a primal-dual risk-budget update. Our architecture combines a two-layer Graph Convolutional Network (GCN) encoder, twin critics, and a value network that drives the adversary. Experiments on a large-scale EV fleet simulator built from NYC taxi data show that PD-RSAC achieves the highest net profit, reaching $1.22M, compared with $0.58M-$0.70M for strong heuristic, single-agent RL, and multi-agent RL baselines, including Greedy, SAC, MAPPO, and MADDPG, while maintaining zero feeder-limit violations.

An Nguyen, Hoang Nguyen, Phuong Le +3

College of Engineering and Computer Science, VinUniversity, Hanoi, Vietnam · Center for Environmental Intelligence, VinUniversity, Hanoi, Vietnam

Electric Vehicles Bike-Sharing Systems Distributional Reinforcement Learning Markov Decision Processes Urban Environments

Ask PDF

Jun 17, 2026cs.LG

Forecasting what Matters: Decision-Focused RL for Controlled EV Charging with Unknown Departure Times

The recent growth of EV adoption poses challenges for power systems, including increased peak demand and potential grid instability. Smart control of EV charging -- e.g., based on reinforcement learning (RL) -- can alleviate these issues by learning temporal and contextual patterns from historical data. Yet, in real-world scenarios, key features, such as departure time, often are unavailable. This, in turn, makes it harder for an RL agent to learn and execute an effective charging policy. To mitigate this uncertainty, a trained forecaster can approximate the unknown features from available data. However, since these forecasting models are typically trained for accuracy (rather than their impact on a downstream agent's decision quality), their errors may propagate and hinder the overall performance of a controller that is using the forecasts. To avoid this, we propose a decision-focused RL (DF-RL) framework in which the forecaster is trained end-to-end, i.e., with feedback from the charging policy actions taken by the RL agent. Such joint training of both the forecaster and controller ultimately results in higher-quality actions: our proposed DF-RL method yields superior charging decisions compared to other baselines, achieving up to a 14% improvement in total reward and a 55% reduction of unsupplied energy (i.e., charging that failed to happen because the EV already left), relative to the RL method without departure time forecasting.

Giuseppe Gabriele, Fabio Pavirani, Seyed Soroush Karimi Madahi +1

Ghent University – imec Ghent, Belgium

Electric Vehicles Reinforcement Learning Control Offline Reinforcement Learning Smart Homes

Ask PDF