Daily Emergency Call Volume Forecasting using Univariate and Multivariate TimeGPT
Organizations: Département d’informatique, Université de Sherbrooke, Sherbrooke, QC, Canada, J1K 2R1
Abstract
This study investigates the potential of TimeGPT, a pretrained foundation model for timeseries, for forecasting daily emergency call volumes. Real-world emergency call data, comprising 2,101 daily observations, are used to evaluate both univariate and multivariate forecasting settings under one-step-ahead and seven-day multi-step-ahead scenarios. The multivariate forecasting incorporates temporal, meteorological, demographic, and socioeconomic variables. A chronological training, validation, and test split is adopted to prevent information leakage, while hyperparameters are selected using the validation set through grid search and early stopping when applicable. Forecasting performance is evaluated using multiple error metrics. Its forecasting performance is compared with several established forecasting approaches, including Seasonal Autoregressive Integrated Moving Average with Exogenous Regressors (SARIMAX), Poisson regression (PR), Negative Binomial regression (NBR), Extreme Gradient Boosting (XGBoost), Multilayer Perceptron (MLP), and Long Short-Term Memory (LSTM). The results show that SARIMAX achieves the best overall performance in the univariate setting. In the multivariate setting, NBR and PR provide best performance. These findings indicate that the large-scale pretraining of TimeGPT do not necessarily translate into superior forecasting accuracy for emergency call volumes. Frugal methods trained solely on the available data remain better for this application.
Figures & tables
| Method | Approximate number of trainable parameters |
|---|---|
| PR | |
| NBR | |
| SARIMAX | |
| XGBoost | |
| MLP | |
| LSTM |
| Model | Hyperparameter | Candidate values |
| TimeGPT | Fine-tuning steps | 0,2,5,10,25,100,500 |
| Fine-tuning loss | default, MAE, MAPE | |
| Fine-tuning depth | 1,2,3,4 | |
| LSTM | Number of LSTM units | 32, 64, 128 |
| Dense units | 16, 32 | |
| Dropout | 0.0, 0.2, 0.3 |
| Model | MAE (95% CI) | MAPE (%) (95% CI) | MASE | RMSE | RSD | Min AE | Max AE | ||
|---|---|---|---|---|---|---|---|---|---|
| TimeGPT | 25.380 (23.514, 27.309) | 8.462 (7.869, 9.067) | 1.000 | 31.916 | 31.868 | 0.146 | 90.991 | 32.479 | 7.407 |
| SARIMAX | 19.281 (17.570, 21.157) | 6.251 (5.771, 6.771) | 0.760 | 24.831 | 24.353 | 0.018 | 83.917 | 18.519 | 1.425 |
| NBR | 25.803 (23.882, 27.868) | 8.325 (7.771, 8.902) | 1.017 | 32.372 | 31.102 | 0.404 | 109.563 | 34.188 | 3.989 |
| PR | 25.807 (23.884, 27.869) | 8.326 (7.772, 8.903) | 1.017 | 32.377 | 31.105 | 0.293 | 109.567 | 34.188 | 3.989 |
| LSTM | 27.338 (24.741, 30.341) | 8.648 (7.981, 9.395) | 1.078 | 35.181 | 32.075 | 0.007 | 117.110 | 34.758 | 6.268 |
| MLP | 25.632 (23.743, 27.657) | 8.251 (7.731, 8.814) | 1.010 | 32.418 | 31.112 | 0.181 | 103.951 | 31.909 | 5.413 |
| Model | MAE (95% CI) | MAPE (%) (95% CI) | MASE | RMSE | RSD | Min AE | Max AE | ||
|---|---|---|---|---|---|---|---|---|---|
| TimeGPT | 25.730 (23.619, 27.863) | 8.562 (7.912, 9.243) | 0.969 | 32.504 | 32.481 | 0.308 | 101.898 | 31.909 | 7.407 |
| SARIMAX | 22.602 (20.950, 24.314) | 7.473 (6.960, 7.994) | 0.851 | 28.888 | 28.926 | 0.059 | 94.883 | 26.211 | 4.843 |
| NBR | 27.811 (25.272, 30.518) | 8.827 (8.159, 9.546) | 1.047 | 35.354 | 32.120 | 0.323 | 115.156 | 39.316 | 6.838 |
| PR | 27.813 (25.273, 30.520) | 8.827 (8.160, 9.546) | 1.047 | 35.356 | 32.121 | 0.293 | 115.143 | 39.316 | 6.838 |
| LSTM | 26.943 (25.013, 28.986) | 9.127 (8.421, 9.842) | 1.014 | 33.664 | 32.493 | 0.008 | 100.090 | 35.328 | 8.547 |
| MLP | 27.733 (25.214, 30.383) | 8.809 (8.172, 9.499) | 1.044 | 35.256 | 32.368 | 0.128 | 109.297 | 35.897 | 6.268 |
| Model | MAE (95% CI) | MAPE (%) (95% CI) | MASE | RMSE | RSD | Min AE | Max AE | ||
|---|---|---|---|---|---|---|---|---|---|
| TimeGPT | 24.726 (22.863, 26.669) | 8.111 (7.524, 8.685) | 0.975 | 31.037 | 30.965 | 0.044 | 94.805 | 33.618 | 4.843 |
| SARIMAX | 20.307 (18.684, 22.062) | 6.726 (6.223, 7.270) | 0.800 | 25.508 | 25.535 | 0.114 | 81.018 | 24.217 | 1.994 |
| NBR | 23.146 (21.191, 25.218) | 7.513 (6.973, 8.105) | 0.912 | 30.121 | 29.848 | 0.318 | 119.157 | 28.205 | 3.989 |
| PR | 23.159 (21.214, 25.222) | 7.521 (6.979, 8.106) | 0.913 | 30.122 | 29.865 | 0.239 | 118.446 | 28.490 | 3.989 |
| LSTM | 25.163 (23.086, 27.407) | 8.289 (7.637, 8.927) | 0.992 | 32.311 | 32.336 | 0.046 | 104.507 | 30.199 | 6.553 |
| MLP | 37.078 (33.701, 40.688) | 12.850 (11.607, 14.238) | 1.462 | 44.212 | 33.600 | 0.299 | 119.175 | 53.276 | 18.519 |
| Model | MAE (95% CI) | MAPE (%) (95% CI) | MASE | RMSE | RSD | Min AE | Max AE | ||
|---|---|---|---|---|---|---|---|---|---|
| TimeGPT | 24.930 (23.064, 26.856) | 8.139 (7.618, 8.689) | 0.938 | 31.465 | 31.297 | 0.242 | 107.117 | 34.188 | 5.698 |
| SARIMAX | 24.746 (22.871, 26.890) | 8.202 (7.581, 8.888) | 0.932 | 31.309 | 31.338 | 0.197 | 104.921 | 32.479 | 5.128 |
| NBR | 23.439 (21.180, 25.898) | 7.568 (6.956, 8.223) | 0.882 | 31.037 | 30.521 | 0.181 | 129.741 | 28.490 | 4.843 |
| PR | 23.425 (21.184, 25.874) | 7.567 (6.957, 8.223) | 0.882 | 31.012 | 30.524 | 0.268 | 128.978 | 28.490 | 5.128 |
| LSTM | 43.694 (39.530, 48.106) | 13.559 (12.450, 14.662) | 1.645 | 52.646 | 32.252 | 0.211 | 165.086 | 66.667 | 19.373 |
| MLP | 607.185 (599.292, 614.930) | 201.437 (197.011, 206.048) | 22.857 | 630.168 | 168.875 | 292.804 | 973.597 | 100.000 | 100.000 |
| Forecast | Horizon | TimeGPT vs SARIMAX | TimeGPT vs MA | SARIMAX vs MA |
|---|---|---|---|---|
| Univariate | 0.9817 | |||
| 0.2588 | ||||
| Multivariate | 0.3881 | |||
| 0.8460 | 0.0535 |
Time Series Forecasting
Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.
See all 942 papers