Beyond Accuracy: Evaluating Forecasting Models by Multi-Echelon Inventory Cost

Swata Marik; Swayamjit Saha; Garga Chatterjee

cs.AIMar 17, 2026

Beyond Accuracy: Evaluating Forecasting Models by Multi-Echelon Inventory Cost

Authors: Swata Marik, Swayamjit Saha, Garga Chatterjee

Organizations: Department of Home Science, University of Calcutta, 20 E, Judges Court Road, Alipore, Kolkata, 700027, West Bengal, India. · Department of Computer Science and Engineering, Mississippi State University, 665 George Perry Street, Starkville, 39759, MS, USA. · Psychology Research Unit, Indian Statistical Institute Kolkata, 203, B.T. Road, Kolkata, 700108, West Bengal, India.

Abstract

This study develops a digitalized forecasting-inventory optimization pipeline integrating traditional forecasting models, machine learning regressors, and deep sequence models within a unified inventory simulation framework. Using the M5 Walmart dataset, we evaluate seven forecasting approaches and assess their operational impact under single- and two-echelon newsvendor systems. Results indicate that Temporal CNN and LSTM models significantly reduce inventory costs and improve fill rates compared to statistical baselines. Sensitivity and multi-echelon analyses demonstrate robustness and scalability, offering a data-driven decision-support tool for modern supply chains.

Figures & tables

Model	RMSE	MAE	Avg. Cost/day	Fill rate	Cost $\downarrow$	Fill $\uparrow$ (pp)
Naive (lag-1)	2.909	1.505	4.521	0.534	0.0%	0.0
Holt–Winters ES	2.677	1.487	4.182	0.583	7.5%	4.9
ARIMA(1,1,1)	2.636	1.486	4.258	0.572	5.8%	3.8
GBR (global)	2.296	1.293	3.854	0.605	14.8%	7.1
XGBoost (global)	2.294	1.289	3.839	0.606	15.1%	7.2
LSTM (global seq)	2.207	1.243	3.704	0.620	18.1%	8.6

Table 1: Test-set forecast accuracy and inventory performance (single-echelon newsvendor,

h=1

b=5

). Cost reduction and fill-rate gains are relative to Naive (lag-1).

Model	$b=2$	$b=5$	$b=10$
ARIMA(1,1,1)	2.179	4.258	7.722
GBR (global)	1.933	3.854	7.054
Holt–Winters ES	2.157	4.182	7.558
LSTM (global seq)	1.858	3.704	6.781
Naive (lag-1)	2.259	4.521	8.291
Temporal CNN	1.888	3.674	6.652

Table 2: Average daily inventory cost on the test set for different shortage penalties (

h=1

). Best value per column in bold.

Inventory Control

Inventory control studies how to order, allocate, and position stock to balance service levels, operating costs, uncertainty, and capacity constraints.

See all 32 papers

Bridging Deep Learning and Integer Linear Programming: A Predictive-to-Prescriptive Framework for Supply Chain Analytics
Apr 2, 2026Khai Banh Nghiep, Duc Nguyen Minh, Lan Hoang ThiSupply Chain Optimization Supply Chain Management
Tran Dai Nghia Secondary and High School, Vietnam · AI engineer Vietnam · University of Languages and International studies
Hybrid Deep Learning Approach for Coupled Demand Forecasting and Supply Chain Optimization
Apr 23, 2026Nusrat Yasmin Nadia, Md Habibul Arif, Habibor Rahman Rabby +3Time Series Forecasting Supply Chain Optimization
Department of Information Technology, Washington University of Science and Technology, , Alexandria, 22314, VA, USA · Department of Information Technology, University of the Potomac, , Washington, D.C., 20005, DC, USA · Department of Computer Science, Campbellsville University, , Louisville, 40220, KY, USA +2
ISOMORPH: A Supply Chain Digital Twin for Simulation, Dataset Generation, and Forecasting Benchmarks
May 12, 2026Zhizhen Zhang, Hyemin Gu, Benjamin J. Zhang +6Zero-Shot Time Series Forecasting Forecasting Benchmarks
University of Massachusetts Amherst · University of North Carolina · SRI International +1

Apr 2, 2026J/K move · Enter open · S save

Bridging Deep Learning and Integer Linear Programming: A Predictive-to-Prescriptive Framework for Supply Chain Analytics

Khai Banh Nghiep, Duc Nguyen Minh, Lan Hoang Thi

Tran Dai Nghia Secondary and High School, Vietnam · AI engineer Vietnam · University of Languages and International studies

Supply Chain Optimization Supply Chain Management Time Series Forecasting

Open paper AskPDF arXiv

Although demand forecasting is a critical component of supply chain planning, actual retail data can exhibit irreconcilable seasonality, irregular spikes, and noise, rendering precise projections nearly unattainable. This paper proposes a three-step analytical framework that combines forecasting and operational analytics. The first stage consists of exploratory data analysis, where delivery-tracked data from 180,519 transactions are partitioned, and long-term trends, seasonality, and delivery-related attributes are examined. Secondly, the forecasting performance of a statistical time series decomposition model N-BEATS MSTL and a recent deep learning architecture N-HiTS were compared. N-BEATS and N-HiTS were both statistically, and hence were N-BEATS's and N-HiTS's statistically selected. Most recent time series deep learning models, N-HiTS, N-BEATS. N-HiTS and N-BEATS N-HiTS and N-HiTS outperformed the statistical benchmark to a large extent. N-BEATS was selected to be the most optimized model, as the one with the lowest forecasting error, in the 3rd and final stage forecasting values of the next 4 weeks of 1918 units, and provided those as a model with a set of deterministically integer linear program outcomes that are aimed to minimize the total delivery time with a set of bound budget, capacity, and service constraints. The solution allocation provided a feasible and cost-optimal shipping plan. Overall, the study provides a compelling example of the practical impact of precise forecasting and simple, highly interpretable model optimization in logistics.

Abstract

Inventory Control

Explore similar work

Bridging Deep Learning and Integer Linear Programming: A Predictive-to-Prescriptive Framework for Supply Chain Analytics

Hybrid Deep Learning Approach for Coupled Demand Forecasting and Supply Chain Optimization

ISOMORPH: A Supply Chain Digital Twin for Simulation, Dataset Generation, and Forecasting Benchmarks