Time-Aware Validation of Machine Learning Fuel Consumption Models: Evidence from 1\,Hz Operational Data, CCGS \textit{Sir Wilfrid Laurier}

Samarasimha Reddy Chittamuru; Ayhan Akinturk; Allison Kennedy; Joshua Barnes; Matthew Hamilton

cs.LGAug 17, 2026

Time-Aware Validation of Machine Learning Fuel Consumption Models: Evidence from 1,Hz Operational Data, CCGS \textit{Sir Wilfrid Laurier}

Authors: Samarasimha Reddy Chittamuru, Ayhan Akinturk, Allison Kennedy, Joshua Barnes, Matthew Hamilton

Organizations: National Research Council Canada, Ocean, Coastal and River Engineering, P.O. Box 12093, St. John’s, A1B 3T5, NL, Canada · Department of Computer Science, Memorial University of Newfoundland, 40 Arctic Avenue, St. John’s, A1B 3X5, NL, Canada

Abstract

Ship fuel consumption (SFC) prediction supports vessel operation optimisation, emissions estimation, and decision support systems (DSS) for sustainable maritime transportation. Numerous data-driven fuel models have been developed over the past two decades, but a critical and often overlooked limitation lies in their validation practices: most studies evaluate performance using random train--test splits, which, applied to high-frequency records, admit temporal leakage and yield optimistic results that do not reflect deployment conditions. This paper examines that gap using time-aware evaluation, specifically Time Series Cross-Validation (TSCV) and Blocked TSCV (BTSCV). Using the Canadian Coast Guard Ship (CCGS) \textit{Sir Wilfrid Laurier} as a case study, six regression models and a physics baseline are tuned under three time-aware schemes and three feature configurations, then evaluated on a common chronological hold-out set drawn from approximately 3.88 million steady-state 1,Hz records.

Figures & tables

Literature	Ship Type	Data Source (Size)	Method(s)	Validation	Observations
Coraddu et al. (2017)	Handymax chemical/product tanker	Onboard 15 s data aggregated to 15-min intervals; $\sim$ 100,000 data points	RLS, Lasso, RF	Random – 30 $\times$ random train–test splits across varying data sizes; Bayesian optimization for hyperparameter tuning	Compared WBM, BBM, and GBM for trim optimization. Used 30 $\times$ random splits with Bayesian optimization for tuning.
Bui-Duy et al. (2020)	Container ship	Operational + secondary (wind, TEU); 10,853 records over 2 years	Deep learning neural network (DLNN)	Random – 80:20 train–test split; 7:1 internal validation	Model used in ATSP framework for route optimization; MAPE = 5.89%
Kim et al. (2021)	Container ship (13,000 TEU)	1-min AMS data (6 months); size not specified	ANN, MLR, SVM, Ensemble	Random + voyage-level – 70:30 random split + voyage-level evaluation	Evaluated both randomly and voyage-wise; emphasis on real-world deployment performance
Agand et al. (2022)	Passenger ferry	1-min sensor data (2019–2021); $\sim$ 1M records	MLR, DT, RF, ANN, XGBoost	Random – 70:30 random split + 10-fold CV	XGBoost performed best; consistent route; 34 features used
Xie et al. (2023)	Oil tanker (trial)	1 Hz sensor data; 378,468 records (4.38 days), 496 features	XGBoost, RF, BP-ANN, MLR, polynomial regression	Random – 10 $\times$ random 70:30 splits, 10-fold CV	Applied Kwon method for acceleration/deceleration filtering; $R^{2}$ reached 0.9977 (XGBoost); robust to sensor noise and data quality issues
Hu et al. (2023)	Bulk carrier	Flowmeter, navigation, meteorological sensors (300 s); 24,300 points	Stacking (hybrid ML), trim optimization	Random – random 80:20 split (5 seeds)	Two-voyage real-world case study; hybrid stacking model outperformed all single models; trim optimization reduced fuel/emissions by up to 1.82%

Table 1: Overview of high-frequency sensor-based fuel consumption modeling studies.

Category	Column Name	Description	Unit	Source
Fuel Flow	ME1_Flow	Fuel flow rate, Main Engine 1	L/h	OpDAQ
	ME2_Flow	Fuel flow rate, Main Engine 2	L/h	OpDAQ
	ME3_Flow	Fuel flow rate, Main Engine 3	L/h	OpDAQ
	Aux_Gen_Flow	Fuel flow rate, auxiliary generator	L/h	OpDAQ
Speed	OpDAQ GPS SOG	Speed over ground (SOG)	kn	VTG
Speed	VHW Speed Relative to Water Kn	Speed through water (STW)	kn	VHW

Table 2: Onboard sensor channels integrated for fuel consumption modeling.

Parameter	Variable Code	Description	Unit
Significant wave height	swh	Average height of the highest one-third of ocean waves	m
Mean wave direction	mwd	Mean direction from which the combined wind-sea and swell wave field is propagating	degrees
Mean wave period	mwp	Mean period of the wave spectrum	s
Peak wave period	pp1d	Period corresponding to the maximum of the wave spectrum	s
Significant swell height	shts	Average height of the swell component of waves	m
Mean swell direction	mdts	Direction from which the swell component is propagating	degrees

Table 3: ERA5 metocean parameters used in this study

Feature	Derivation	Unit
Total_ME_Flow	ME1_Flow + ME2_Flow + ME3_Flow	L/h
Relative_wave_direction	( mwd $-$ Heading + 360) mod 360	degrees
Relative_swell_direction	( mdts $-$ Heading + 360) mod 360	degrees
Wind_Direction_Class	$\lfloor(\theta+22.5)/45\rfloor$ mod 8, $\theta=$ MWV Wind Angle	–
Wave_Direction_Class	$\lfloor(\theta+22.5)/45\rfloor$ mod 8, $\theta=$ Relative_wave_direction	–
Swell_Direction_Class	$\lfloor(\theta+22.5)/45\rfloor$ mod 8, $\theta=$ Relative_swell_direction	–

Table 4: Engineered features derived from onboard sensor and ERA5 hindcast channels.

Figure 1: Pearson correlation matrix of engineered features.

Figure 2: Distributions of principal numerical features used for model training.

Figure 3: Directional distributions of relative wind, wave, and swell encounter angles.

Figure 4: Overview of the modeling and validation pipeline, from raw sensor and hindcast inputs through preprocessing and feature engineering to model training, cross-validation, and final hold-out evaluation.

Variant	Speed Channel	Purpose
A	Speed over Ground (SOG)	Available at the voyage-planning stage; the configuration evaluated for deployment.
B	Speed through Water (STW)	Relates fuel consumption directly to hull resistance, independent of surface current and drift.
C	SOG + STW	Tests whether combining both signals improves accuracy beyond either alone, using their difference as an implicit proxy for current and drift.

Table 5: Feature variant definitions. All variants share the same environmental, directional, and draft-related features (Table 4) and differ only in the vessel speed channel used.

Figure 5: Physics baseline model fitted using OLS: predicted fuel flow as a function of speed over ground and mean draft.

Figure 6: Cross-validation fold structure for (a) random $k$ -fold and (b) sequential $k$ -fold, shown for $n=20$ samples and $k=5$ folds.

Figure 7: Time series cross-validation fold structure for (a) the standard expanding-window formulation and (b) the modified scheme adopted in this study, shown for 20 samples divided into 5 partitions.

Figure 8: Blocked time series validation fold structure, shown for 20 samples divided into 5 blocks.

Model	Hyperparameter	Search Grid	Search Iterations
Ridge	Regularization strength ( $\alpha$ )	30-point logarithmic grid, $10^{-3}$ to $10^{3}$	30 a
Lasso	Regularization strength ( $\alpha$ )	20-point logarithmic grid, $10^{-3}$ to $10^{1}$	20 a
ElasticNet	Regularization strength ( $\alpha$ )	15-point logarithmic grid, $10^{-2}$ to $10^{1}$	30
ElasticNet	Mixing parameter ( $\ell_{1}$ ratio)	9-point linear grid, 0.1 to 0.9
Random Forest	Number of trees	50, 100, 150, 200	20
	Maximum tree depth	10, 20, 30

Table 6: Hyperparameter search space and tuning budget for each regression model.

Figure 9: Model performance under random versus chronological 80:20 partitioning of Variant A data, reported as (a) R 2 , (b) RMSE (L h -1 ), and (c) PA15 (%).

Figure 10: Selected hyperparameters for each model under each cross-validation scheme and feature variant. Labels of the form $n\times$ mark identical configurations selected in multiple combinations.

Figure 11: Validation $R^{2}$ across folds for each model, cross-validation scheme and feature variant. Markers show the mean; whiskers show the fold-to-fold range.

Figure 12: Test MAE and test PA15 by feature variant for each model under Sequential K-Fold, Custom TSCV and Blocked TSCV. Percentage changes from Variant A to Variant C are annotated above each model group.

Figure 13: Training and test performance on the chronological hold-out set across all model, cross-validation scheme, and feature variant combinations: R 2 (top), RMSE (L h -1 ) (middle), and PA15 (%) (bottom).

Model Pair	Feature Variant	Cross-Validation Scheme	Model with Lower Error	Mean Squared-Error Reduction	Blocks with Lower Error (of 100)	Wilcoxon Signed-Rank $p$ -value
Ridge vs XGBoost	A	Sequential K-Fold	Ridge	$9921.0$	$71$	< 0.0001
	A	Custom TSCV	Ridge	$4615.7$	$69$	$0.0002$
	A	Blocked TSCV	Ridge	$4897.5$	$68$	< 0.0001
	B	Sequential K-Fold	Ridge	$2353.3$	$67$	$0.0002$
	B	Custom TSCV	Ridge	$1686.1$	$68$	$0.0012$
	B	Blocked TSCV	Ridge	$1190.7$	$63$	$0.0058$

Table 7: Pairwise statistical comparison of model errors on the chronological hold-out set, using a Wilcoxon signed-rank test on 100 block-mean squared-error differentials, with Benjamini–Hochberg correction applied across all 27 tests (

\alpha=0.05

Figure 14: Predicted versus observed fuel consumption for Ridge, XGBoost and Random Forest on the Variant C hold-out set. The dashed line indicates perfect agreement.

Figure 15: Distribution of predicted fuel consumption for Ridge, XGBoost and Random Forest, compared to the observed distribution, on the Variant C hold-out set. Left: kernel density estimate. Right: interquartile range and non-outlier range.

Figure 16: SHAP value distributions for XGBoost and Random Forest on the Variant C hold-out set. Each point represents one observation; colour indicates the corresponding feature value.

Panel of Figure 17 — Figure 17: Left: standardized Ridge coefficients for the Variant C model, signed by direction of effect on predicted fuel consumption. Right: normalized feature importance (0–1 within each model) for Ridge, XGBoost and Random Forest.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Train $R^{2}$	Train RMSE	Train MAE	Train Accuracy (%)	Train PA15 (%)	Test $R^{2}$	Test RMSE	Test MAE	Test Accuracy (%)	Test PA15 (%)
MLR	$0.800$	$77.12$	$57.23$	$88.90$	$74.40$	$0.799$	$77.24$	$57.35$	$88.87$	$74.37$
Ridge	$0.800$	$77.12$	$57.23$	$88.90$	$74.40$	$0.799$	$77.24$	$57.35$	$88.87$	$74.37$
Lasso	$0.790$	$78.97$	$59.18$	$88.49$	$72.66$	$0.789$	$79.07$	$59.27$	$88.47$	$72.62$
ElasticNet	$0.710$	$92.80$	$76.44$	$85.34$	$60.42$	$0.710$	$92.76$	$76.40$	$85.34$	$60.46$
Random Forest	$0.999$	$6.59$	$4.29$	$99.27$	$99.99$	$0.990$	$17.38$	$11.46$	$98.06$	$99.86$
XGBoost	$0.982$	$23.01$	$16.10$	$97.13$	$99.36$	$0.982$	$23.11$	$16.14$	$97.12$	$99.35$

Appendix

Table 8: Model performance under a single random 80:20 train–test split (Variant A), using default (untuned) hyperparameters.

Model	Train $R^{2}$	Train RMSE	Train MAE	Train Accuracy (%)	Train PA15 (%)	Test $R^{2}$	Test RMSE	Test MAE	Test Accuracy (%)	Test PA15 (%)
MLR	$0.800\pm 0$	$77.15\pm 2$	$57.26\pm 1$	$88.89\pm 0$	$74.39\pm 1$	$0.800\pm 0$	$77.14\pm 7$	$57.24\pm 3$	$88.90\pm 1$	$74.41\pm 3$
Ridge	$0.800\pm 0$	$77.15\pm 2$	$57.26\pm 1$	$88.89\pm 0$	$74.39\pm 1$	$0.800\pm 0$	$77.14\pm 7$	$57.24\pm 3$	$88.90\pm 1$	$74.41\pm 3$
Lasso	$0.790\pm 0$	$78.99\pm 2$	$59.21\pm 1$	$88.48\pm 0$	$72.64\pm 1$	$0.790\pm 0$	$78.98\pm 7$	$59.18\pm 3$	$88.49\pm 1$	$72.67\pm 4$
ElasticNet	$0.710\pm 0$	$92.80\pm 1$	$76.45\pm 1$	$85.34\pm 0$	$60.42\pm 1$	$0.710\pm 0$	$92.81\pm 7$	$76.43\pm 5$	$85.34\pm 1$	$60.43\pm 4$
Random Forest	$0.999\pm 0$	$6.59\pm 0$	$4.29\pm 0$	$99.27\pm 0$	$99.99\pm 0$	$0.990\pm 0$	$17.33\pm 4$	$11.45\pm 1$	$98.06\pm 0$	$99.86\pm 0$
XGBoost	$0.982\pm 0$	$23.21\pm 8$	$16.22\pm 5$	$97.10\pm 1$	$99.34\pm 1$	$0.982\pm 0$	$23.26\pm 8$	$16.25\pm 5$	$97.10\pm 1$	$99.33\pm 1$

Appendix

Table 9: Model performance across 10 independently reseeded random 80:20 splits (Variant A), using default (untuned) hyperparameters. Values are mean

\pm

standard deviation across the 10 repetitions. Full metric set.

Model	Train $R^{2}$	Train RMSE	Train MAE	Train Accuracy (%)	Train PA15 (%)	Test $R^{2}$	Test RMSE	Test MAE	Test Accuracy (%)	Test PA15 (%)
Physics	$0.747$	$87.63$	$66.34$	$87.33$	$69.11$	$0.714$	$86.09$	$70.51$	$86.45$	$67.23$
MLR	$0.785$	$80.81$	$60.33$	$88.21$	$72.40$	$0.837$	$65.02$	$47.66$	$91.58$	$82.49$
Ridge	$0.785$	$80.81$	$60.33$	$88.21$	$72.40$	$0.837$	$65.02$	$47.66$	$91.58$	$82.49$
Lasso	$0.778$	$82.05$	$61.95$	$87.89$	$71.23$	$0.820$	$68.35$	$50.57$	$90.89$	$79.51$
ElasticNet	$0.702$	$95.08$	$77.60$	$85.22$	$59.28$	$0.716$	$85.81$	$72.68$	$85.79$	$65.31$
Random Forest	$0.999$	$6.32$	$4.01$	$99.31$	$99.99$	$-0.360$	$187.81$	$131.37$	$69.67$	$58.18$

Appendix

Table 10: Model performance under a chronological hold-out split (Variant A): the earliest 80% of the time-ordered data for training, the most recent 20% for testing, with no reshuffling. Default (untuned) hyperparameters throughout. Full metric set.

Variant	Model	CV Scheme	Train $R^{2}$	Train RMSE	Train MAE	Val $R^{2}$	Val RMSE	Val MAE
A	Physics	Sequential K-Fold	$0.748\pm 16$	$86.70\pm 219$	$65.51\pm 193$	$0.551\pm 166$	$96.18\pm 1046$	$74.26\pm 981$
		Custom TSCV	$0.761\pm 8$	$82.96\pm 60$	$62.87\pm 63$	$0.594\pm 149$	$94.14\pm 1153$	$69.94\pm 839$
		Blocked TSCV	$0.786\pm 27$	$79.89\pm 665$	$59.48\pm 477$	$-2.784\pm 3233$	$107.30\pm 2500$	$91.57\pm 2697$
	Ridge	Sequential K-Fold	$0.790\pm 8$	$79.11\pm 198$	$59.08\pm 131$	$0.533\pm 165$	$98.18\pm 978$	$76.06\pm 785$
		Custom TSCV	$0.820\pm 18$	$71.96\pm 456$	$54.36\pm 353$	$0.451\pm 197$	$109.75\pm 1160$	$85.13\pm 1041$
		Blocked TSCV	$0.849\pm 16$	$67.00\pm 466$	$48.60\pm 153$	$-12.233\pm 14\,828$	$157.02\pm 5101$	$134.17\pm 4927$

Appendix

Table 11: Cross-validation performance across all three feature variants: mean

\pm

standard deviation across folds, training and validation portions, under each cross-validation scheme. The physics baseline is available only for Variant A.

Variant	Model	CV Scheme	Train $R^{2}$	Train RMSE	Train MAE	Train Accuracy (%)	Train PA15 (%)	Test $R^{2}$	Test RMSE	Test MAE	Test Accuracy (%)	Test PA15 (%)
A	Physics	–	$0.747$	$87.63$	$66.34$	$87.33$	$69.11$	$0.714$	$86.09$	$70.51$	$86.45$	$67.23$
	MLR	–	$0.785$	$80.81$	$60.33$	$88.21$	$72.40$	$0.837$	$65.02$	$47.66$	$91.58$	$82.49$
	Ridge	Sequential K-Fold	$0.785$	$80.82$	$60.35$	$88.20$	$72.39$	$0.837$	$65.06$	$47.65$	$91.58$	$82.51$
		Custom TSCV	$0.785$	$80.82$	$60.35$	$88.20$	$72.39$	$0.837$	$65.06$	$47.65$	$91.58$	$82.51$
		Blocked TSCV	$0.785$	$80.82$	$60.35$	$88.20$	$72.39$	$0.837$	$65.06$	$47.65$	$91.58$	$82.51$
	Lasso	Sequential K-Fold	$0.766$	$84.26$	$64.16$	$87.48$	$69.46$	$0.811$	$70.10$	$52.86$	$90.38$	$77.50$

Appendix

Table 12: Final hold-out evaluation across all three feature variants, full metric set, companion to Figure 13. MLR and the physics baseline are reported once per variant; the five tunable models were refit on the full training set using hyperparameters selected under each cross-validation scheme and evaluated on the common chronological hold-out set.

Explore similar work

The Impossible Trinity of Time-Series Validation: A Conservation Law among Training Sufficiency, Test Coverage, and Temporal Causality
Sep 2, 2026Jiayu LiTemporal Data Leakage Statistical Learning Theory
Regularized Machine Learning for System Identification of Ship Free-Running Manoeuvres from CFD-Based Synthetic Data: A Comparative Study
Jun 15, 2026R. F. Suárez, J. C. Berndt, M. Abdel-MaksoudNonlinear System Identification System Identification
Hamburg University of Technology (TUHH), Institute for Fluid Dynamics and Ship Theory (M8), Am Schwarzenberg-Campus 4 (C), 21073, Hamburg, Germany
TS-Fault: Benchmarking Time Series Forecasters Against Structural Faults
Jun 16, 2026Yuyang Zhao, Lian Xu, Hao Miao +2Time Series Forecasting Forecasting Benchmarks
Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · The University of Western Australia, Perth, Australia · Hong Kong Polytechnic University, Hong Kong, China +1

Sep 2, 2026J/K move · Enter open · S save

The Impossible Trinity of Time-Series Validation: A Conservation Law among Training Sufficiency, Test Coverage, and Temporal Causality

Jiayu Li

Temporal Data Leakage Statistical Learning Theory

Open paper AskPDF arXiv

Validating a model on a time series asks for three things at once: each training run should use most of the sample (sufficiency), the test sets should together cover most of the sample (coverage), and training data should come before test data (causality). We prove that the three cannot be had together and price each one. Let

α

be the smallest training fraction over folds,

β

the fraction of the sample covered by tests,

Λ

the fraction of the sample used as training data from the future of a test point, and

δ

the distance from a test point to the nearest training point in its future. Every scheme on a sample of length

T

satisfies

α+β\le 1+Λ

and

α+\min\{β,δ/T\} \le 1

, and under

β

-mixing the leakage bias at a test point is at most

2Mβ_{\mathrm{mix}}(δ)

. In words: going beyond the causal frontier

α+β=1

requires training on the future; that future data must sit within

(1-α)T

of a test point; and its harm depends on its distance, not its amount. Hence expanding walk-forward is exactly the Pareto frontier of causal validation,

k

-fold cross-validation buys the most future data, and purged

k

-fold with an embargo pays in distance instead, which is cheap when the process forgets quickly but cannot repair the part of causality demanded by non-stationarity. On pure noise, shuffled 5-fold reports an information coefficient of

+0.32

, while contiguous 5-fold, using the same amount of future data, reports

+0.004

Abstract

Explore similar work

The Impossible Trinity of Time-Series Validation: A Conservation Law among Training Sufficiency, Test Coverage, and Temporal Causality

Regularized Machine Learning for System Identification of Ship Free-Running Manoeuvres from CFD-Based Synthetic Data: A Comparative Study

TS-Fault: Benchmarking Time Series Forecasters Against Structural Faults