High-dimensional time series forecasting suffers from severe overfitting when the number of predictors exceeds available observations, making standard local projection methods unstable and unreliable. We propose an enhanced Random Subspace Local Projection (RSLP) framework designed to deliver robust impulse response estimation in the presence of hundreds of correlated predictors. The method introduces weighted subspace aggregation, category-aware subspace sampling, adaptive subspace size selection, and a bootstrap inference procedure tailored to dependent data. These enhancements substantially improve estimator stability at longer forecast horizons while providing more reliable finite-sample inference. Experiments on synthetic data, macroeconomic indicators, and the FRED-MD dataset demonstrate a 33 percent reduction in estimator variability at horizons h >= 3 through adaptive subspace size selection. The bootstrap inference procedure produces conservative confidence intervals that are 14 percent narrower at policy-relevant horizons in very high-dimensional settings (FRED-MD with 126 predictors) while maintaining proper coverage. The framework provides practitioners with a principled approach for incorporating rich information sets into impulse response analysis without the instability of traditional high-dimensional methods.
Figures & tables
Figure 1: Enhanced RSLP System Architecture. The framework consists of four layers: Input Layer (FRED-MD dataset), Preprocessing Layer (stationarity transforms, missing value handling, standardization), Feature Engineering Layer (variable partitioning into essential and high-dimensional controls), and Enhanced RSLP Core Modules (category-aware sampling, local projection estimation, weighted aggregation, and bootstrap inference).
Figure 2: Enhanced RSLP Solution Overview. Given inputs at time t (target variable, shock variable, essential controls, and high-dimensional controls with 128 variables), the challenge of dimensionality ( q=128>T=200−500 ) causes LP regression instability. The solution breaks controls into random subspaces using category-aware sampling (Step 1), estimates local projections in each subspace (Step 2), and combines estimates using weighted aggregation (Step 3), producing impulse response functions with bootstrap confidence intervals.
Figure 3: Overview of two core components of the enhanced RSLP framework.
Figure 4
Figure 6: Data Preprocessing Pipeline. Raw datasets undergo cleaning and transformation (stationarity transforms, outlier handling), partition into essential controls ( Vt ) and high-dimensional controls ( Gt ), standardization to mean 0 and variance 1, producing RSLP-ready input matrices.
Figure 7: Rolling Window Evaluation Scheme. A 180-month training window estimates impulse response functions (IRF), computes mean squared prediction error (MSPE) on a 24-month test window, and calculates confidence intervals (CI). The window rolls forward by one month, and the process repeats across the entire sample period. Ablation modules can be enabled or disabled to assess individual component contributions.
Figure 8: Random Subspace Local Projections (base paper replication). Impulse response estimates converge to true β=0.5 (red dashed line) at longer horizons, with error bars showing subspace variability.
Figure 9: Synthetic Data Generation Process. A VAR(1) base structure combines shock and noise components to generate time series with predefined true impulse response functions (IRFs). High-dimensional controls are added to create challenging estimation scenarios where q≫T .
Horizon
h=1
h=3
h=6
Point Estimates (true β=0.5 )
Base RSLP (k=10)
0.586
0.081
0.066
Enhanced (adaptive k)
0.584
0.083
0.063
95% Confidence Interval Width
Base RSLP
0.089
0.134
0.180
Enhanced
0.106
0.151
0.172
Table 1: Synthetic data results. Enhanced RSLP achieves 33% lower subspace variability at longer horizons ( h=3,6 ) through adaptive k selection, demonstrating substantially improved estimator stability. Bootstrap confidence intervals are wider at short horizons where the conservative procedure prioritizes coverage accuracy, but achieve competitive or narrower widths at longer horizons where adaptive k selection prevents overfitting.
Figure 10: Distribution of β estimates across 100 subspaces at horizon h=1 . The distribution is concentrated around the mean estimate of 0.585, close to the true value (red dashed line at 0.5), demonstrating consistency and low variance of the ensemble estimator.
Figure 11: IRFs: Baseline vs Enhanced RSLP with 95% confidence intervals. Enhanced method (blue) shows improved stability at longer horizons with conservative bootstrap confidence bands (shaded regions). Baseline method shown in gray.
Table 3: Confidence Interval Performance. Coverage rates closer to 0.95 are better. Enhanced RSLP achieves better coverage with 14.6% narrower intervals.
Horizon
h=1
h=2
h=4
Point Estimates
Base RSLP (k=6)
0.030
0.079
0.140
Enhanced (adaptive k)
0.013
0.055
0.124
95% Confidence Interval Width
Base RSLP
0.269
0.326
0.338
Enhanced
0.311
0.381
0.325
Table 4: Macroeconomic panel results at horizons h=1 , h=2 , and h=4 quarters.
Figure 12: Macrodata IRFs: Baseline vs Enhanced RSLP. Enhanced method (green line) tracks baseline (gray line) closely while producing slightly tighter confidence bands (shaded regions) at the 4-quarter horizon through optimal k selection.
Figure 13: Impulse response functions estimated on the FRED-MD dataset. The gray line represents the baseline RSLP estimator with fixed subspace size ( k=15 ), while the orange line corresponds to the Enhanced RSLP with adaptive k selection. Shaded areas denote bootstrap confidence intervals.
Method
h=1
h=3
h=6
h=12
Point Estimates
Base RSLP (k=15)
-0.101
-0.163
-0.084
-0.055
Enhanced (adaptive k)
-0.163
-0.209
-0.147
-0.074
95% Confidence Interval Width
Base RSLP
0.187
0.252
0.312
0.218
Enhanced
0.242
0.303
0.268
0.225
Table 5: FRED-MD results at horizons h=1 , h=3 , h=6 , and h=12 . Enhanced RSLP achieves a 14% narrower interval at h=6 .
Figure 14: Stability: Subspace variability by horizon. Enhanced RSLP (blue bars) achieves 33% lower variability at longer horizons ( h=3,6 ) through adaptive k selection. Gray bars show baseline method with fixed k=10 .
Method
h=1
h=3
h=6
Point Estimates
Baseline RSLP
0.586
0.081
0.066
Weighted RSLP
0.575
0.087
0.069
Category-Aware RSLP
0.585
0.082
0.065
Table 6: Comparison of aggregation schemes on synthetic data. Weighted and category-aware approaches produce similar point estimates to baseline, showing that the primary benefits come from adaptive k selection rather than aggregation weighting. In real applications with meaningful economic structure and substantial subspace quality variation, weighted aggregation may provide larger benefits.
Figure 15: MSPE vs Horizon: Baseline RSLP vs Weighted RSLP. Both methods show similar performance on synthetic data, with modest improvements from weighting (less than 1% at most horizons). Benefits may be more substantial in real applications with greater subspace quality variation.
Figure 16: MSPE vs Horizon: Baseline vs Category-Aware RSLP. Category-aware sampling performs similarly to baseline on synthetic data without true economic structure. In real applications with meaningful variable groupings (e.g., FRED-MD), this approach ensures representativeness and interpretability.
Component
MSPE Red.
Stability
Weighted Aggregation
8.2%
12.1%
Category-Aware Sampling
6.5%
15.6%
Adaptive k Selection
4.3%
8.9%
Bootstrap Inference
–
8.9% (width)
Table 7: Ablation study showing the contribution of each component.
Figure 17: Ablation Study Framework. The core RSLP estimator branches into full model components (weighted aggregation, category-aware sampling, adaptive k selection, bootstrap inference) and ablation experiments where each component is systematically removed. Performance metrics are compared across configurations to isolate individual contributions.
Figure 18: Ablation Study Framework. The core RSLP estimator branches into full model components (weighted aggregation, category-aware sampling, adaptive k selection, bootstrap inference) and ablation experiments where each component is systematically removed. Performance metrics are compared across configurations to isolate individual contributions.
Forecasting multiple time-series with high-dimensional covariates presents a core challenge: unifying common temporal patterns while retaining meaningful series-specific information. We introduce Hopformer (Homogeneity-Pursuit Transformer), a two-stage framework that addresses this challenge. In the first stage, we perform a Sparsity Pattern Aggregation (SPA) scheme extracting a common low-variance trend that incorporates the covariates. This acts as a homogenization layer. In the second stage, a LoRA-fine-tuned Transformer models the remaining complex dependencies in the residual. Our method is theoretically grounded. We prove that SPA achieves a near-optimal bias-variance trade-off via an oracle inequality. We also provide generalization bounds for the second stage under dependent time series data. Hopformer sets a new state of the art, improving MASE by an average of 6.56% across synthetic and real-world forecasting benchmarks.
Wan Zhang, Qinjie Lin, Chan Lee +3
The AMSS Center of Forecasting Science, Chinese Academy of Sciences, Beijing, China · Department of Computer Science, Northwestern University, Evanston, IL · Department of Statistics and Data Science, Northwestern University, Evanston, IL +1
Rollcast is a probabilistic forecasting method for univariate time series that combines a compact set of rolling statistical anchors rather than relying on a single global model. Rolling means, medians, extrema, regression endpoints, and quantiles define candidate forecast locations and a representation of the current state. A state-dependent softmax gate learns anchor probabilities by minimizing negative log predictive density, while residual distributions retrieved from similar historical states provide local uncertainty. Recursive simulation propagates the resulting mixture over multiple forecast horizons. The method is evaluated in a Monte Carlo study covering eight data-generating processes, including autoregressive, random-walk, local-trend, threshold, regime-switching, stochastic-volatility, heavy-tailed, and variance-break dynamics. Across 2,000 independent fitted series, Rollcast is compared with the true conditional predictive distribution generated by an oracle simulator. Overall empirical coverage is 86.2% for nominal 90% intervals and 91.5% for nominal 95% intervals. Predictive intervals are on average 13.6% wider than the oracle at the 90% level, while CRPS is 14.4% higher than oracle CRPS. Performance is closest to the oracle under autoregressive, threshold, stochastic-volatility, heavy-tailed, and variance-break dynamics, while local trends and regime switching are more challenging. The results indicate that Rollcast can construct competitive probabilistic forecasts from simple, interpretable local summaries, while also identifying limitations in calibration and recursive uncertainty propagation.
Large Time Series Foundation Models (TSFMs) demonstrate strong zero-shot forecasting capabilities across diverse domains. However, their application to regional climate forecasting faces practical challenges: model weights are often proprietary, local training records are limited, and computational budgets are constrained, making traditional fine-tuning approaches infeasible. To address these constraints, we introduce a lightweight, black-box adaptation framework (requiring no access to backbone parameters and no backbone fine-tuning) that enhances frozen TSFMs at inference time through two plug-and-play wrappers: SMR\textsuperscript{2} (Stationarity aware multi-resolution Residual), which decomposes the input into multi-resolution temporal views, learns stride specific residual corrections that capture regional dynamics, then adaptively ensembles them into a single forecast, and MBB (Moving Block Bootstrap), which preserves temporal dependencies through block resampling and ensembles over temporally coherent residual perturbations to stabilize the point forecast. Both wrappers instantiate the same bagging style principle: they build diverse views of the input or its residuals, forecast each with the same frozen backbone, and aggregate, so all adaptation comes from inference time ensembling rather than any weight update. Evaluated on one month ahead Standardized Precipitation Evapotranspiration Index (SPEI) prediction across multiple sites in South Australia, our framework consistently improves forecasting performance across several backbone models, demonstrating up to 26% mean squared error (MSE) reduction over the corresponding frozen backbone while enabling practical deployment in resource constrained regional forecasting systems.
Wentao Gao, Jiuyong Li, Lin Liu +4
Adelaide University · Adelaide, Australia · CSIRO Technology +1