Physics-informed offline reinforcement learning eliminates catastrophic fuel waste in maritime routing

Aniruddha Bora; Julie Chalfant; Chryssostomos Chryssostomidis

cs.AIMar 18, 2026

Physics-informed offline reinforcement learning eliminates catastrophic fuel waste in maritime routing

Authors: Aniruddha Bora, Julie Chalfant, Chryssostomos Chryssostomidis

Organizations: Department of Computer Science, Texas State University, San Marcos, TX 78666, USA · MIT Sea Grant College Program, Massachusetts Institute of Technology, Cambridge, MA 02139, USA

Abstract

International shipping produces approximately 3% of global greenhouse gas emissions, yet voyage routing remains dominated by heuristic methods. We present PIER (Physics-Informed, Energy-efficient, Risk-aware routing), an offline reinforcement learning framework that learns fuel-efficient, safety-aware routing policies from physics-calibrated environments grounded in historical vessel tracking data and ocean reanalysis products, requiring no online simulator. Validated on one full year (2023) of AIS data across seven Gulf of Mexico routes (840 episodes per method), PIER reduces mean CO2 emissions by 10% relative to great-circle routing. However, PIER's primary contribution is eliminating catastrophic fuel waste: great-circle routing incurs extreme fuel consumption (>1.5x median) in 4.8% of voyages; PIER reduces this to 0.5%, a 9-fold reduction. Per-voyage fuel variance is 3.5x lower (p<0.001), with bootstrap 95% CI for mean savings [2.9%, 15.7%]. Partial validation against observed AIS vessel behavior confirms consistency with the fastest real transits while exhibiting 23.1x lower variance. Crucially, PIER is forecast-independent: unlike A* path optimization whose wave protection degrades 4.5x under realistic forecast uncertainty, PIER maintains constant performance using only local observations. The framework combines physics-informed state construction, demonstration-augmented offline data, and a decoupled post-hoc safety shield, an architecture that transfers to wildfire evacuation, aircraft trajectory optimization, and autonomous navigation in unmapped terrain.

Figures & tables

Figure 1: Pier framework and Gulf of Mexico routing domain. a , The Pier pipeline: AIS vessel tracking data and ocean reanalysis products (Copernicus Marine Service, NOAA CoastWatch ERDDAP) are fused into physics-informed state features (speed-loss model, hull-fatigue risk layer, 15 normalized features). An IQL agent is trained offline on a mixed dataset of A*-optimal teacher demonstrations and stochastic behavioral roll-outs. At inference, a post-hoc safety shield enforces land-avoidance and wave-exposure constraints before each routing decision. b , Gulf of Mexico routing domain with nine key ports (amber dots). Seven port-to-port routes are evaluated: Brownsville $\to$ Key West, Corpus Christi $\to$ Galveston, Corpus Christi $\to$ Tampa, Galveston $\to$ Key West, Mobile $\to$ Tampa, New Orleans $\to$ Tampa, and Port Arthur $\to$ Mobile. c–f , Seasonal mean normalized HF (hull fatigue) exposure from Copernicus wave reanalysis, showing elevated hazard in winter (DJF) and fall (SON) compared with summer (JJA), particularly in the eastern Gulf and Florida Straits. This seasonal variation is the primary driver of Pier ’s fuel savings: weather-aware routing adds the most value when conditions are adverse.

Method	Type	Arrival	Time (h)	Fuel Proxy	HF Exposure
Great Circle	Heuristic	655/840 (78.0%)	49.8 $\pm$ 52.8	89.5 $\pm$ 94.4	55.8 $\pm$ 56.1
Greedy Goal	Heuristic	685/840 (81.5%)	46.6 $\pm$ 42.5	78.8 $\pm$ 74.9	51.7 $\pm$ 48.4
BC (expert)	Supervised	63/840 (7.5%)	101.1 $\pm$ 60.5	117.3 $\pm$ 81.0	83.6 $\pm$ 62.7
DQN	Online RL	3/840 (0.4%)	70.8 $\pm$ 140.0	79.5 $\pm$ 157.0	37.8 $\pm$ 75.8
CQL	Offline RL	643/840 (76.5%)	49.7 $\pm$ 35.0	83.9 $\pm$ 60.9	55.3 $\pm$ 46.5
Pier (IQL)	Offline RL	700/840 (83.3%)	45.6 $\pm$ 26.4	78.5 $\pm$ 44.9	53.2 $\pm$ 45.6

Table 1: Annual baseline comparison (2023, 12 months, all 7 routes). Arrival rate, transit time, estimated fuel proxy, and HF wave exposure (mean

\pm

std). Bold indicates best among methods with

>

50% arrival rate.

Route	Arrival	Time (h)	HF Exposure
Brownsville $\to$ Key West	100%	71.1 $\pm$ 10.5	109.3 $\pm$ 38.7
Galveston $\to$ Key West	100%	62.7 $\pm$ 2.2	85.3 $\pm$ 34.1
Corpus Christi $\to$ Tampa	93%	63.2 $\pm$ 1.2	69.7 $\pm$ 32.7
New Orleans $\to$ Tampa	98%	32.6 $\pm$ 1.1	30.6 $\pm$ 29.7
Mobile $\to$ Tampa	97%	27.7 $\pm$ 5.1	26.9 $\pm$ 17.5
Corpus Christi $\to$ Galveston	84%	13.2 $\pm$ 1.5	10.5 $\pm$ 9.9

Table 2: Per-route annual results (2023, IQL with safety shield). The five cross-Gulf routes achieve 93–100% arrival with consistent transit times. Port Arthur

\to

Mobile (13% arrival) is excluded from the main CO 2 analysis due to grid resolution limitations on this short coastal corridor; full results including this route are in Extended Data Table 1.

Figure 2: Route performance across all 12 months of 2023. a , Arrival rate (percentage of episodes reaching destination) for each route and month. Brownsville $\to$ Key West and Galveston $\to$ Key West maintain 100% arrival year-round; Port Arthur $\to$ Mobile shows persistent failures due to grid resolution limitations. b , Mean HF wave exposure by route and month. Brownsville $\to$ Key West and Galveston $\to$ Key West experience the highest exposure, peaking in winter (February) and fall (October–November), confirming these as the routes where weather-aware routing adds the most value. Missing cells indicate no arrived episodes.

Variant	Arrival	Time (h)	HF Exposure	$\Delta$ Arr
IQL (full)	700/840 (83.3%)	45.9 $\pm$ 21.8	56.3 $\pm$ 45.4	-
No teacher demos	675/840 (80.4%)	46.1 $\pm$ 22.3	58.1 $\pm$ 46.5	$-$ 25
No safety shield	643/840 (76.9%)	47.3 $\pm$ 21.0	59.3 $\pm$ 45.7	$-$ 54
No HF-risk features	659/840 (78.5%)	49.3 $\pm$ 20.7	60.3 $\pm$ 44.4	$-$ 41
No physics features	652/840 (77.6%)	49.0 $\pm$ 20.1	60.2 $\pm$ 45.5	$-$ 48

Table 3: Annual ablation study (2023, 12 months). Each row removes one component.

\Delta

Arr shows the change in arrived episodes relative to the full model.

Figure 3: Ablation study: contribution of each Pier component (2023, 12 months). a , Arrival rate. Removing the safety shield causes the largest drop (

-

6 percentage points), followed by physics features (

-

6 pp) and HF-risk awareness (

-

5 pp). Teacher demonstrations contribute the least (

-

3 pp). b , Mean transit time (

\pm

std). Removing components generally increases both mean and variance. c , Mean HF wave exposure (

\pm

std). The full model achieves the lowest exposure; removing the shield or physics features increases wave encounter risk. Dashed lines indicate full-model performance for reference.

Metric	Pier (IQL)	Great Circle	Saving	%
Mean CO 2 (t/voyage)	171.6	189.8	18.2	9.6%
Median CO 2 (t/voyage)	216.9	218.2	1.3	0.6%
90th percentile CO 2 (t)	240.9	243.2	2.3	0.9%
95th percentile CO 2 (t)	242.9	259.5	16.6	6.4%
Max CO 2 (worst voyage, t)	470.8	1,560.4	1,089.6	69.8%
Std deviation CO 2 (t)	76.4	141.9	-	-

Table 4: CO 2 analysis: mean, median, and tail-risk metrics (5 core routes, arrived voyages, Panamax bulk carrier, VLSFO). Pier ’s advantage grows sharply in the tail of the distribution.

Figure 4: Pier ’s CO 2 advantage is concentrated in the tail of the distribution. a , Per-voyage CO 2 distributions for Pier (teal) and great-circle routing (grey). Medians are nearly identical (

\sim

215–218 t), but great-circle routing has a heavy right tail extending beyond 1,600 t (red shaded region:

>

1.5

\times

median). b , CO 2 at successive quantiles. Savings grow from 1% at the median to 6% at the 95th percentile and 70% at the maximum, demonstrating that Pier ’s value is concentrated in eliminating worst-case fuel events. c , Per-route CO 2 standard deviation. Great-circle variance exceeds Pier by 3–35

\times

depending on route length, with the largest ratios on long cross-Gulf corridors. d , Mean versus median savings by season. Median savings are consistently 1–2% across all seasons, while mean savings vary from 5% to 24%, confirming that elevated means reflect tail-event elimination rather than systematic improvement.

Extended Data Fig. 1 | Binned speed-loss analysis. a , Mean $\Delta U$ increases with $H_{s}$ across all vessel types. b , Head seas ( $\mu<60^{\circ}$ ) cause the largest speed reduction. c , Following current provides $+$ 0.6–0.8 kts per m/s. d , Joint $H_{s}\times\mu$ heatmap for tankers shows strongest losses in head seas at high $H_{s}$ . e , Wave steepness $H_{s}/T_{p}$ is the primary driver of head-sea speed loss. f , Bootstrap significance test: tanker wave effect significant at $p<0.001$ ; cargo at $p\approx 0.05$ . Despite overall $R^{2}=0.02$ , the large sample size yields statistically significant binned trends.

Extended Data Fig. 2 | RL baseline comparison. IQL versus CQL, DQN, and behavioral cloning across all metrics. DQN and BC fail completely (

<

5% arrival) in the offline setting; CQL achieves 72% arrival but with higher transit times and HF exposure than IQL.

Extended Data Fig. 3 | Per-route ablation heatmap. Arrival rate for each ablation variant $\times$ route combination. Brownsville $\to$ Key West achieves 100% arrival for all variants; Corpus Christi $\to$ Galveston is most sensitive to the safety shield (62% without shield versus 84% with shield).

Extended Data Fig. 4

|

Uncertainty analysis. a , Monte Carlo distribution of mean CO 2 savings under speed-loss model coefficient perturbation (1,000 samples); mean 7.8%, 95% CI [

-

37.4%, 41.2%], reflecting sensitivity to physics model parameters. b , Monte Carlo distribution of variance ratio (GC / IQL); 95% CI [1.5

\times

, 8.1

\times

], with the entire distribution above the equal-variance line, confirming that the IQL tail-risk advantage is robust to model uncertainty. c , Bootstrap distribution of mean CO 2 savings (5,000 resamples); 95% CI [2.9%, 15.7%], reflecting sampling variability across routes and months. d , Bootstrap confidence intervals for tail-risk frequency (

>

1.5

\times

median): IQL 0.5% [0.0%, 1.2%] versus great-circle 4.8% [3.2%, 6.7%] (non-overlapping CIs). The wider Monte Carlo interval in a reflects extreme coefficient draws rather than plausible operating conditions; under bootstrap resampling of observed voyages, savings remain strictly positive ( c ).

Extended Data Fig. 5 | Seasonal performance summary. a , Arrival rate aggregated by meteorological season (DJF, MAM, JJA, SON). Spring achieves the highest arrival rate (86%), while summer shows the lowest (76%), reflecting the interaction between seasonal wave patterns and grid resolution limitations on coastal routes. b , Mean transit time (

\pm

std) by season. Transit times are consistent across seasons (47–54 h mean), confirming that Pier maintains routing efficiency year-round; the large standard deviations reflect route-length diversity rather than seasonal instability. c , Mean HF wave exposure (

\pm

std) by season. Winter (DJF) and fall (SON) show the highest exposure (68 and 80, respectively), while summer (JJA) shows the lowest (35), consistent with the seasonal hazard pattern in Fig. 1c–f. This confirms that weather-aware routing adds the most value during adverse seasons.

Route		Jan	Feb	Mar	Apr	May	Jun	Jul	Aug	Sep	Oct	Nov	Dec	Ann.
Bro→Key	Arr%	100	100	100	100	100	100	100	100	100	100	100	100	100 (98)
Bro→Key	HF	135	128	117	143	144	58	70	49	76	98	175	118	109
CC→Gal	Arr%	100	60	70	100	60	100	70	80	80	100	90	100	84 (92)
CC→Gal	HF	4	33	4	4	3	12	2	4	11	25	20	6	11
CC→Tam	Arr%	100	100	100	90	100	100	90	90	70	90	80	100	92 (82)
CC→Tam	HF	56	106	66	74	66	67	26	14	65	134	89	71	70

Extended Data Table 1

|

Monthly Pier (IQL) performance (2023). Arrival rate (%) and mean HF exposure for all seven routes. Annual GC arrival rate shown for reference. Shaded cells: IQL arrival

<

100%.

Vessel	MCR (kW)	$V_{s}$ (kn)	SFOC	VLSFO		HFO		MGO
Vessel	MCR (kW)	$V_{s}$ (kn)	SFOC	IQL	GC	IQL	GC	IQL	GC
Panamax Bulk	10000	14.0	153	0.15	0.17	0.15	0.17	0.16	0.17
			170	0.17	0.19	0.17	0.19	0.17	0.19
			187	0.19	0.21	0.19	0.21	0.19	0.21
Handymax Bulk	7500	13.0	153	0.14	0.16	0.14	0.16	0.15	0.16
			170	0.16	0.18	0.16	0.18	0.16	0.18

Extended Data Table 2

|

Sensitivity analysis. Sensitivity of mean CO 2 emissions (kt) to SFOC and fuel type.

Method	$N$	Mean	Median	Std	P95	Max	Time (h)	$>$ 1.5 $\times$ med
		CO 2 per voyage (tonnes)					mean $\pm$ std
A* (Balanced)	60	175.5	220.7	78.0	251.2	251.5	50.5 $\pm$ 22.5	0.0%
Pier (IQL)	569	171.6	216.9	76.4	242.9	470.8	45.6 $\pm$ 26.4	0.5%
Great Circle	563	189.8	218.2	141.9	259.5	1,560.4	49.8 $\pm$ 52.8	4.8%
Statistical tests (Levene’s test for variance equality):
A* vs Great Circle: $F=1.81$ , $p=0.179$ (ns); A* vs IQL: $F=1.07$ , $p=0.301$ (ns)

Extended Data Table 3

|

A* versus IQL comparison. Three-way comparison of A* (Balanced objective), IQL ( Pier ), and great-circle routing on five core routes (arrived voyages, Panamax bulk carrier, VLSFO). IQL matches A* within 2.2% on mean CO 2 . Both weather-aware methods eliminate tail-risk events; the 4.8% tail-risk rate is specific to great-circle routing.

Method	$N$	Time (h)	CO 2 (t)	HF Exposure
AIS (direct transits)	4	93.0 $\pm$ 78.2	315.0 $\pm$ 348.2	270.5 $\pm$ 149.5
AIS (all voyages)	6	257.2 $\pm$ 257.6	257.2 $\pm$ 301.2	219.8 $\pm$ 141.7
Great Circle	92	52.3 $\pm$ 56.0	185.3 $\pm$ 195.1	36.8 $\pm$ 35.2
CQL	119	27.8 $\pm$ 5.4	96.3 $\pm$ 15.1	28.4 $\pm$ 18.9
Pier (IQL)	116	27.7 $\pm$ 5.1	95.2 $\pm$ 15.1	26.9 $\pm$ 17.5
Variance ratio (AIS direct / IQL):

Extended Data Table 4

|

AIS validation: Mobile

\to

Tampa. Direct comparison of Pier against observed AIS vessel behavior on the only cross-Gulf corridor with sufficient real-voyage data (6 AIS voyages across 5 months). AIS voyages were filtered to direct transits (average speed

>3

kts) to exclude multi-stop itineraries, yielding 4 direct transits. All voyages are shown in the per-voyage breakdown for transparency.

Forecast noise	Approx.	Arrival (%)		Mean HF
$\sigma$	horizon	A*	IQL	A*	IQL
0.00	Perfect	75	93	3.7	22.9
0.10	24 h	72	93	7.9	22.9
0.25	48 h	77	93	8.9	22.9
0.50	72 h	85	93	12.1	22.9
0.75	96 h	88	93	14.1	22.9

Extended Data Table 5

|

Robustness to forecast uncertainty. A* (HF-averse) versus IQL under increasing wave forecast noise. IQL uses only local observations and is forecast-independent.

\sigma

is the standard deviation of multiplicative noise on

H_{s}

;

\sigma=0.25

approximates 48-hour operational forecast error. Results aggregated over 12 months, 3 routes, and 3 random seeds per noise level.

Figure 5: Archipelago Basin simulated environment. a , Domain geometry with three experimental routes: S1 (open water, jet crossing), S2 (constrained corridor, peninsula navigation), S3 (storm crossing, HF-risk optimization). b , Significant wave height $H_{s}$ at $t=0$ h showing the Gaussian storm in the northeast quadrant. c , Ocean current speed with the zonal jet at $27.85^{\circ}$ N. d , Hull-fatigue risk index peaking near the storm centre.

Figure 6: Temporal evolution of the wave field. The storm intensifies from baseline ( $t=0$ h) to peak ( $t=12$ h, $H_{s}\approx 5.5$ m), weakens through the trough ( $t=24$ h), and returns toward baseline ( $t=36$ h). This time-dependence makes optimal routing departure-time sensitive.

Panel of Figure 7 — Figure 7: Synthetic AIS data and speed-loss validation. a , Synthetic AIS tracks showing vessels navigating between random endpoints while avoiding land features. b , Predicted versus observed speed over ground; the cluster near 11–13 kts reflects the base-speed distribution. c , Residual distribution ( $\sigma=1.26$ kts, $R^{2}=0.148$ ).

Route	Arrived	Time (h)	Fuel (t)	HF Exp.	Shield Int.
S1 (Open Water)	✓	16.0	12.3	2.63	0
S2 (Corridor)	✓	16.0	16.1	3.56	0
S3 (Storm)	✓	17.0	16.0	3.73	1

Table 5: Pier main model performance on simulated routes (200-epoch IQL). All three routes are completed successfully.

Figure 8: IQL training convergence. a , Q-network MSE loss. b , Value network expectile loss. c , Policy loss (advantage-weighted regression). d , Mean Q-value and mean advantage. All losses plateau by epoch 150.

	Time (h)			Fuel (t)			HF Exp.			Arrived
Config	S1	S2	S3	S1	S2	S3	S1	S2	S3	S1	S2	S3
Full Pier	16	16	17	12.3	16.1	16.0	2.63	3.56	3.73	✓	✓	✓
Full (100ep)	15	10	9	12.0	8.7	8.0	2.61	1.34	1.28	✓	$\times$	$\times$
$-$ Teacher	15	6	17	11.8	5.5	16.1	2.52	0.81	3.77	✓	$\times$	✓
$-$ Shield	15	19	9	12.0	18.1	8.1	2.63	3.96	1.29	✓	✓	$\times$
$-$ HF Risk	7	24	9	5.1	23.4	8.1	0.98	4.52	1.27	$\times$	✓	$\times$

Table 6: Ablation study on Archipelago Basin. The full Pier system (200 epochs) achieves 3/3 arrivals. Each variant (100 epochs) removes one component. Values for non-arriving routes represent state at the 300-step termination limit.

Figure 9: Ablation study on Archipelago Basin. a , Travel time: the $-$ Physics variant on S3 reaches the 300 h termination limit. b , Fuel consumption: catastrophic waste on S3 without the physics model. c , HF-risk exposure: $-$ Physics on S3 accumulates $18\times$ the full model’s exposure. d , Arrival rate: only the full Pier system achieves 3/3 arrivals.

Explore similar work

Per-Shipment Multi-Agent Reinforcement Learning for Intermodal Freight Routing Under Hurricane Disruption
Aug 7, 2026Aliza Sharmin, Xudong Wang, Mustafa Can Camur +1Vehicle Routing Proximal Policy Optimization
Department of Industrial and Systems Engineering University of Tennessee Knoxville, TN 37996
Does Latent Context Help? A Controlled Evaluation of Inverse Reinforcement Learning in Arctic Shipping
Aug 6, 2026Vaishnav Vaidheeswaran, Dilith Jayakody, Biruk Ambaw +3Reinforcement Learning Meta-Reinforcement Learning
Faculty of Computer Science, Dalhousie University, Halifax, NS, Canada · Department of Industrial Engineering, Faculty of Engineering, Dalhousie University, Halifax, NS, Canada
RL-PaO: Prediction as Action in Decision Making under Uncertainty
Sep 29, 2026Jiahui Feng, Dafang Zhao, Zheng Chen +2Predict-Then-Optimize Decision-Making under Uncertainty
Great Bay University, School of Computing and Information Technology, China · The University of Osaka, School of Information Science and Technology, Japan · The University of Osaka, SANKEN, Japan +1

Aug 7, 2026J/K move · Enter open · S save

Per-Shipment Multi-Agent Reinforcement Learning for Intermodal Freight Routing Under Hurricane Disruption

Aliza Sharmin, Xudong Wang, Mustafa Can Camur, Xueping Li

Department of Industrial and Systems Engineering University of Tennessee Knoxville, TN 37996

Vehicle Routing Proximal Policy Optimization Supply Chain Management Multi-Agent Reinforcement Learning

Open paper AskPDF arXiv

Intermodal freight networks face growing disruption risk from climate extremes that degrade multiple corridors simultaneously. To address this, we formulate freight routing as a Dec-POMDP with per-shipment action granularity and train Independent PPO (IPPO) under Centralized Training with Decentralized Execution, comparing against two heuristic baselines with privileged state access on a 15-hub network under hurricane disruption. Across 30 matched episodes, no single policy dominates: IPPO achieves the highest throughput (

+12.7\%

) and delivery rate while a capacity-aware heuristic leads on Resilience Index (RI) and delay. Under demand surge (2.9:1 capacity ratio), IPPO's RI advantage grows to

+6.4\%

, suggesting learned routing is most valuable when capacity is scarce. A Multi-Agent PPO (MAPPO) variant collapses under train-eval queue mismatch (

\mathrm{RI} = 0.811

); retraining recovers RI to

1.018

but IPPO still leads on throughput, pointing to residual limitations in centralized critics under per-shipment dispatch.

Abstract

Explore similar work

Per-Shipment Multi-Agent Reinforcement Learning for Intermodal Freight Routing Under Hurricane Disruption

Does Latent Context Help? A Controlled Evaluation of Inverse Reinforcement Learning in Arctic Shipping

RL-PaO: Prediction as Action in Decision Making under Uncertainty