Decomposing Rank Reversals under Temporal Nonstationarity: Alignment, Aggressiveness, and Instability
Organizations: Carnegie Mellon University, Pittsburgh, PA, USA
Abstract
Under temporal nonstationarity, model performance rankings reverse across training horizons due to heterogeneous sensitivities to memory. While existing literature addresses adaptive lookbacks, effective model complexity, and empirical forecast evaluation in isolation, no unified framework explains the structural mechanics driving these rank reversals. We bridge this gap by deriving a model-agnostic decomposition of out-of-sample into target alignment, miscalibrated aggressiveness, and predictive instability. Within this framework, we mathematically characterize complexity phase boundaries, establish closed-form optimal-memory equations for ridge regression under local drift, and extend these guarantees to complex model architectures via surrogate bounds. We validate these theoretical dynamics by benchmarking 14 linear, tree-based, and neural architectures across large-scale financial panels. In doing so, our framework demonstrates that empirical model rank reversals are driven by predictable surges in miscalibrated aggressiveness, providing a rigorous mechanistic diagnosis of generalization under temporal shift.
Figures & tables
| Tabular | Temporal, 60-day context | ||||||||||||
| Ridge, | Tuned ridge | Random forest | XGBoost | Small MLP | Medium MLP | Flattened ridge | Flattened XGBoost | Fixed flattened MLP | Tuned flattened MLP | Vanilla RNN | LSTM | Small Transformer | |
| Mean | |||||||||||||
| 16m | 0.0909 | 0.3515 | 0.3549 | 0.3316 | 0.2869 | 0.2786 | 0.0772 | 0.2812 | 0.8216 | 0.2223 | 0.2776 | 0.1913 | 0.2712 |
| 64m | 0.3256 | 0.3668 | 0.3374 | 0.3419 | 0.3231 | 0.3189 | 0.1078 | 0.3223 | 0.0124 | 0.0002 | 0.2989 | 0.2472 | 0.2969 |
| Exp. | 0.3386 | 0.3602 | 0.2953 | 0.3232 | 0.3301 | 0.3270 | 0.2835 | 0.3148 | 0.0881 | 0.1158 | 0.3169 | 0.2885 | 0.3195 |
| Ensemble | |||||||||||||
| Context | 252d | 756d | Exp. |
|---|---|---|---|
| Tabular | |||
| Temporal 20d | |||
| Temporal 60d |
Appendix figures & tables28 assets
Supplementary material from the paper’s appendix.
Appendix
| Window | Months | Window days | Subsampled days | Network fit days | Validation days |
|---|---|---|---|---|---|
| 16 months | 16 | 328–340 | 256–265 | 192–199 | 79–87 |
| 64 months | 28–64 | 590–1,352 | 460–1,052 | 263–856 | 246–255 |
| Expanding | 28–350 | 590–7,353 | 460–5,718 | 263–5,522 | 246–255 |
| Model | Input | Fixed settings | Candidate grid (eight candidates) |
| Tabular branch | |||
| Ridge, fixed alpha | none | ||
| Tuned ridge | none | ||
| XGBoost | 150 rounds; min. child weight 5; penalty 1; no subsampling | and | |
| Random forest | cuML; 200 bootstrap trees; depth 5; min. leaf 1; all features; 128 bins | none | |
| Small MLP | width (32); 4,193 parameters; batch 256; at most 30 epochs; patience 5 | learning rate , weight decay , dropout | |
| Model | 16 months | 64 months | Expanding | |
|---|---|---|---|---|
| Ridge, fixed alpha | Ensemble | 0.0706 | 0.3481 | 0.3477 |
| Mean [95% CI] | 0.0909 [ 0.1033, 0.0786] | 0.3256 [0.3235, 0.3277] | 0.3386 [0.3374, 0.3397] | |
| Tuned ridge | Ensemble | 0.3671 | 0.3767 | 0.3644 |
| Mean [95% CI] | 0.3515 [0.3487, 0.3543] | 0.3668 [0.3658, 0.3678] | 0.3602 [0.3593, 0.3611] | |
| XGBoost | Ensemble | 0.3621 | 0.3619 | 0.3392 |
| Mean [95% CI] | 0.3316 [0.3284, 0.3348] | 0.3419 [0.3380, 0.3457] | 0.3232 [0.3202, 0.3262] |
| Model | 16 months | 64 months | Expanding | |
|---|---|---|---|---|
| Flattened ridge | Ensemble | 0.1915 | 0.0139 | 0.3308 |
| Mean [95% CI] | 0.1397 [0.1346, 0.1447] | 0.2128 [ 0.2208, 0.2049] | 1.1533 [ 1.1871, 1.1194] | |
| Flattened XGBoost | Ensemble | 0.3341 | 0.3485 | 0.3379 |
| Mean [95% CI] | 0.3026 [0.2992, 0.3061] | 0.3287 [0.3258, 0.3315] | 0.3208 [0.3170, 0.3247] | |
| Small flattened MLP | Ensemble | 0.0200 | 0.1544 | 0.2485 |
| Mean [95% CI] | 0.0841 [ 0.0926, 0.0756] | 0.0574 [0.0532, 0.0617] | 0.1623 [0.1552, 0.1694] |
| Model | 16 months | 64 months | Expanding | |
| Flattened ridge | Ensemble | 0.1092 | 0.1654 | 0.0295 |
| Mean [95% CI] | 0.0772 [0.0712, 0.0831] | 0.1078 [0.1018, 0.1138] | 0.2835 [ 0.2946, 0.2724] | |
| Flattened XGBoost | Ensemble | 0.3161 | 0.3402 | 0.3253 |
| Mean [95% CI] | 0.2812 [0.2779, 0.2845] | 0.3223 [0.3188, 0.3257] | 0.3148 [0.3132, 0.3164] | |
| Small flattened MLP | Ensemble | 0.3891 | 0.0765 | 0.1690 |
| Mean [95% CI] | 0.8216 [ 0.8845, 0.7588] | 0.0124 [ 0.0180, 0.0068] | 0.0881 [0.0846, 0.0916] |
| Model | Window | Mean | SD | CI low | CI high | Ensemble | |||
|---|---|---|---|---|---|---|---|---|---|
| Tabular | |||||||||
| Ridge, fixed alpha | 16m | 0.0909 | 0.0118 | 0.1033 | 0.0786 | 0.0706 | 0.9726 | 0.9019 | 0.1615 |
| Ridge, fixed alpha | 64m | 0.3256 | 0.0020 | 0.3235 | 0.3277 | 0.3481 | 0.9373 | 0.5892 | 0.0225 |
| Ridge, fixed alpha | Exp. | 0.3386 | 0.0011 | 0.3374 | 0.3397 | 0.3477 | 0.7620 | 0.4143 | 0.0091 |
| Tuned ridge | 16m | 0.3515 | 0.0027 | 0.3487 | 0.3543 | 0.3671 | 0.8296 | 0.4625 | 0.0156 |
| Tuned ridge | 64m | 0.3668 | 0.0010 | 0.3658 | 0.3678 | 0.3767 | 0.8571 | 0.4804 | 0.0099 |
Non-Stationary Time Series Forecasting
Non-stationary time series forecasting predicts future values as patterns, distributions, and relationships change over time.
See all 99 papers