Hierarchical Industrial Demand Forecasting with Temporal and Uncertainty Explanations
Authors: Harshavardhan Kamarthi, Shangqing Xu, Xinjie Tong, Xingyu Zhou, James Peters, Joseph Czyzyk, B. Aditya Prakash
Organizations: Georgia Institute of Technology Atlanta, USA · Aspen Technology Houston, USA · The Dow Chemical Company Houston, USA · The Dow Chemical Company Midland, USA
Hierarchical time-series forecasting is essential for demand prediction across various industries. While machine learning models have obtained significant accuracy and scalability on such forecasting tasks, the interpretability of their predictions, informed by application, is still largely unexplored. To bridge this gap, we introduce a novel interpretability method for large hierarchical probabilistic time-series forecasting, adapting generic interpretability techniques while addressing challenges associated with hierarchical structures and uncertainty. Our approach offers valuable interpretative insights in response to real-world industrial supply chain scenarios, including 1) the significance of various time-series within the hierarchy and external variables at specific time points, 2) the impact of different variables on forecast uncertainty, and 3) explanations for forecast changes in response to modifications in the training dataset. To evaluate the explainability method, we generate semi-synthetic datasets based on real-world scenarios of explaining hierarchical demands for over ten thousand products at a large chemical company. The experiments showed that our explainability method successfully explained state-of-the-art industrial forecasting methods with significantly higher explainability accuracy. Furthermore, we provide multiple real-world case studies that show the efficacy of our approach in identifying important patterns and explanations that help stakeholders better understand the forecasts. Additionally, our method facilitates the identification of key drivers behind forecasted demand, enabling more informed decision-making and strategic planning. Our approach helps build trust and confidence among users, ultimately leading to better adoption and utilization of hierarchical forecasting models in practice.
Figures & tables
Fig. 1: In real-world HTSF tasks, stakeholders and planners want to know: RQ1: which input variable(s) contribute most to the prediction; RQ2: which time steps of each variable contribute most to the prediction; RQ3: why prediction results change when input data changes.
Fig. 2: Subtree approximation allows HiereInterpret to accurately capture importance scores of any time-series across the hierarchy
Fig. 4: Adapting to any probabilistic forecast by deriving importance scores from quantiles of the forecasts
Dataset
No. of Nodes
Levels of
Horizon
Obs.
Dow
20022
8
24
1.2 M
M5
3914
12
12
7.7 M
Tourism-L
555
4,5
12
126k
Wiki
207
5
1
696k
TABLE I: Summary of demand datasets used in benchmark establishment. We covered a variety of node capacity, number of hierarchical levels, and horizon sizes in the selection of such datasets.
Dataset
Dow’s Demand
M5
Metric
IAS
EVDA
IAS
EVDA
Int. Method
LIME
IG
FO
SG
LIME
IG
FO
SG
LIME
IG
FO
SG
LIME
IG
FO
SG
FreqShapes
0.39
0.42
0.37
0.22
0.38
0.44
0.48
0.27
0.56
0.62
0.37
0.77
0.52
0.62
0.54
0.77
- w/o subtree
0.11
0.17
0.23
0.17
0.16
0.32
0.39
0.28
0.24
0.36
0.26
0.49
0.21
0.37
0.32
0.46
SeqComb
0.39
0.33
0.38
0.29
0.39
0.42
0.44
0.32
0.47
0.54
0.33
0.73
0.58
0.74
0.37
0.64
- w/o subtree
0.27
0.24
0.35
0.28
0.21
0.24
0.31
0.29
0.22
0.39
0.18
0.53
0.33
0.31
0.24
0.35
TABLE II: Cross-variable deterministic hierarchical evaluation results on Dow’s Demand and M5, averaged across all HTSF models. Subtree approximation improved 34 of 36 Dow experiments with relatively 40.4% in EVDA and 62.0% in IAS, and improved all M5 experiment with relatively 92.5% in EVDA and 78.0% in IAS.
Correlation Type
Same Series
Cross Series
Cross Levels
Int. Method
LIME
IG
FO
SG
LIME
IG
FO
SG
LIME
IG
FO
SG
Dow’s Demand
FreqShapes
0.32
0.35
0.37
0.31
0.15
0.17
0.19
0.18
0.18
0.19
0.17
0.11
- w/o subtree
0.29
0.37
0.34
0.33
0.13
0.18
0.17
0.15
0.15
0.19
0.17
0.09
SeqComb
0.35
0.34
0.39
0.39
0.13
0.16
0.22
0.24
0.16
0.18
0.24
0.25
- w/o subtree
0.25
0.36
0.32
0.39
0.17
0.21
0.19
0.15
0.19
0.24
0.2
0.11
LowVar
0.21
0.24
0.24
0.23
0.11
0.24
0.18
0.19
0.1
0.12
0.12
0.11
TABLE III: Same-variable evaluation results on Dow’s Demand, M5, Tourism-L, and Wiki. Among all the 36 experiments in each dataset, subtree approximation improved 22 Dow experiments with average 12.3% improvement, 24 M5 experiments with average 8.0% improvement, 26 Tourism experiments with average 7.5% improvement, and 26 Wiki experiments with average 10.7% improvement.
Correlation Type
Same Series
Cross Series
Cross Levels
Int. Method
LIME
IG
FO
SG
LIME
IG
FO
SG
LIME
IG
FO
SG
Dow’s Demand
FreqShapes
0.28
0.26
0.34
0.23
0.08
0.19
0.14
0.13
0.08
0.14
0.12
0.07
- w/o subtree
0.22
0.22
0.27
0.17
0.07
0.14
0.10
0.09
0.06
0.14
0.08
0.07
SeqComb
0.25
0.42
0.29
0.43
0.20
0.27
0.18
0.17
0.14
0.15
0.19
0.16
- w/o subtree
0.18
0.33
0.24
0.28
0.16
0.22
0.16
0.16
0.13
0.14
0.17
0.11
LowVar
0.16
0.26
0.21
0.29
0.13
0.22
0.16
0.13
0.11
0.06
0.16
0.13
TABLE IV: Probabilistic hierarchical explanation results on Dow’s Demand, M5, Tourism-L, and Wiki, averaged across all HTSF models and three quantiles. Of all 36 experiments on each dataset, subtree approximation improved 34 Dow experiments with an average 26.0% improvement, 29 M5 experiments with an average 18.1% improvement, 29 Tourism experiments with an average 16.5% improvement, and 31 Wiki experiments with average 25.3% improvement.
Fig. 5: Probabilistic evaluation results of each quantile. While the 70th quantile with subtree has the best absolute performance, the 90th quantile receives the most improvement by introducing subtree approximation.
Fig. 6: Performance improvement of HiereInterpret with Subtree approximation. Point forecasts see up to 16% improvements, whereas probabilistic forecasts show over 25% improvements
Fig. 7: Degradation of performance with increase in hierarchy size with and without subtree approximation
Fig. 8: After accessing new data without periodic spikes, HAILS tend to give high uncertainty in its prediction. HiereInterpret shows that such high uncertainty output comes from the focus on two types of data – with and without periodic spikes.
Fig. 9: As the model anticipates the beginning of the periodic trends, HiereInterpret notes the increased importance of the periodic trends from the past.
Fig. 10: Running time of explanation models on HAILS considering hierarchies up to different levels. Our proposed subtree approximation greatly cuts down the time complexity.
Fig. 11: The downward spike observed in the second forecast causes HiereInterpret to interpret that the important economic variable’s previous downward trend may have a significant impact on the change in forecast trend.
Explainable time series forecasting makes predictions over time interpretable through attributions, structured models, causal reasoning, and counterfactuals.