Locally Linear Continual Learning for Time Series based on VC-Theoretical Generalization Bounds

Yan V. G. Ferreira; Igor B. Lima; Pedro H. G. Mapa S.; Felipe V. Campos; Antonio P. Braga

cs.LGMar 14, 2026

Locally Linear Continual Learning for Time Series based on VC-Theoretical Generalization Bounds

Authors: Yan V. G. Ferreira, Igor B. Lima, Pedro H. G. Mapa S., Felipe V. Campos, Antonio P. Braga

Organizations: Department of Electronic Engineering, Universidade Federal de Minas Gerais, UFMG, Belo Horizonte 31270-901, Brazil · Department of Computing Science, University of Alberta, Edmonton, Alberta, Canada · Graduate Program in Electrical Engineering, Universidade Federal de Minas Gerais, UFMG, Belo Horizonte 31270-901, Brazil

Abstract

Most machine learning methods assume fixed probability distributions, limiting their applicability in nonstationary real-world scenarios. While continual learning methods address this issue, current approaches often rely on black-box models or require extensive user intervention for interpretability. We propose SyMPLER (Systems Modeling through Piecewise Linear Evolving Regression), an explainable model for time series forecasting in nonstationary environments based on dynamic piecewise-linear approximations. Unlike other locally linear models, SyMPLER uses generalization bounds from Statistical Learning Theory to automatically determine when to add new local models based on prediction errors, eliminating the need for explicit clustering of the data. Experiments show that SyMPLER can achieve comparable performance to both black-box and existing explainable models while maintaining a human-interpretable structure that reveals insights about the system's behavior. In this sense, our approach conciliates accuracy and interpretability, offering a transparent and adaptive solution for forecasting nonstationary time series.

Figures & tables

Panel of Figure 1 — Fig. 1 : SyMPLER switches between existing local models depending on the value of the input $\mathbf{x}(t)$ . Each local model approximates the desired function linearly around its approximation point $\mathbf{x}(t)=\mathbf{p}_{i}$ . As the domain of the problem grows, more local models are added. In this example, SyMPLER switches to the model colored in red as $\mathbf{x}(t)$ approaches $\mathbf{p}_{4}$ and adds a new model for the new region of the input.

Fig. 2 : Smallest training size

l

for a model with VC-dimension

h

, following [ 20 ] .

Fig. 3 : Continual learning results: (a) prediction squared error, (b) input data and (c) number of members in the network. Data in (a) and (c) start appearing later than in (b) because the network has no member until the first

2(n+1)+10

samples arrive.

Fig. 4 : SyMPLER approximation in feature space. Although each individual model is linear, as shown with the coloured lines in (a), switching between them allows the network to approximate non-linear functions piecewise-linearly, as shown in (b). The zoomed area shows the piecewise-linear behavior of the model.

$\theta_{0}$ (rad)		Slope	Bias
1.55	SyMPLER	-0.44	-18.91
1.55	Taylor	-0.34	-19.07
1.24	SyMPLER	-6.42	-10.54
1.24	Taylor	-6.34	-10.69
0.44	SyMPLER	-17.72	-0.49
0.44	Taylor	-17.78	-0.53

TABLE I : Coefficients of Taylor first-order approximations of the target function and coefficients of SyMPLER members for some of the approximation points.

Fig. 5 : Long-term forecasting of the angular position of the pendulum using SyMPLER and the linearized model for the first cycle of the system (left figure) and its last cycle (right figure).

Model	Searched parameters	Best parameters
SyMPLER (original)	$\lambda=\{10^{-6},10^{-2},0.1,1,5,10,15\}$	$\lambda=5$
SyMPLER (aggr.)	$\lambda=\{10^{-6},10^{-2},0.1,1,5,10,15\}$ $\sigma=\{0.1,0.5,1,10\}$	$\lambda=1$ $\sigma=1$
ICN	$\lambda=\{10^{-6},0.1,1,5\}$ $\epsilon=\{10^{-2},0.5,1,5\}$ $\alpha=\{0.1,1,10\}$	$\lambda=5$ $\epsilon=10^{-2}$ $\alpha=0.1$
eTS	$\omega=\{10^{-6},0.1,1,10,100,1000\}$ $r=\{0.1,0.3,0.5,0.7,0.9\}$	$\omega=0.1$ $r=0.5$
EWC-SWFT	$\beta=\{100,500,1000\}$ $\text{learning\_rate}=\{10^{-4},10^{-3},10^{-2}\}$	$\beta=100$ $\text{learning\_rate}=10^{-3}$
XGBoost	$\text{n\_estimators}=\{10,100,200,500\}$ $\text{max\_depth}=\{3,6,9\}$ $\text{learning\_rate}=\{0.01,0.1,0.3\}$ $\text{colsample\_bytree}=\{0.3,0.6,1\}$ $\text{alpha}=\{0,10,100\}$	$\text{n\_estimators}=200$ $\text{max\_depth}=3$ $\text{learning\_rate}=0.1$ $\text{colsample\_bytree}=0.3$ $\text{alpha}=0$

TABLE II : Hyperparameter search and best parameters for each model.

Model	Fitting error	Prediction error	Forgetting ratio
SyMPLER (original)	21.91	21.06	0.11
SyMPLER (aggr.)	21.09	21.01	0.14
ICN	31.03	39.07	0.15
eTS	26.88	28.87	0.30
EWC-SWFT	49.10 $\pm$ 3.86	55.64 $\pm$ 4.45	0.25 $\pm$ 0.29
XGBoost	16.88	18.56	0.66

TABLE III : Comparison of models based on fitting error (RMSE), prediction error (RMSE), and forgetting ratio. EWC-SWFT performance was evaluated for 10 random initializations, so we present the mean and standard deviation of the results.

Fig. 6 : Predictions of the models for the last month of 2007.

Station	$x_{\text{0}}$ (°F)	$p_{\text{0}}$ (°F)	$w_{\text{0}}$	$x_{\text{1}}$ (°F)	$p_{\text{1}}$ (°F)	$w_{\text{1}}$
14	32.00	37.11	1.48	86.00	83.93	-12.30
6	32.00	30.18	-4.75	86.00	85.64	8.57
1	32.00	30.82	-8.53	86.00	83.82	19.63
25	32.00	28.54	-7.54	86.00	83.18	10.20
21	32.00	31.86	-2.27	86.00	89.29	13.10
20	32.00	29.11	-8.69	86.00	83.93	12.64

TABLE IV : The local approximation points closer to the scenarios

x_{\text{0}}

and

x_{\text{1}}

in which all the weather stations measure 32°F and 86°F, respectively, are

p_{\text{0}}

and

p_{\text{1}}

. The local models associated with these points have weights

w_{\text{0}}

and

w_{\text{1}}

Fig. 7 : Continual learning results for the concept drift experiment: (a) absolute prediction error, (b) input data and (c) number of members in the network.

Fig. 8 : SyMPLER’s average error and number of members for an increasing input dimensionality. The shaded areas represent two standard deviations from the mean.

Fig. 9 : Mean distance between the vector of SyMPLER’s coefficients and the vector of Taylor’s coefficients for different levels of noise and regularization.

Explainable Time Series Forecasting

Explainable time series forecasting makes predictions over time interpretable through attributions, structured models, causal reasoning, and counterfactuals.

See all 61 papers

Mar 4, 2026

PatchDecomp: Interpretable Patch-Based Time Series Forecasting

Time series forecasting, which predicts future values from past observations, plays a central role in many domains and has driven the development of highly accurate neural network models. However, the complexity of these models often limits human understanding of the rationale behind their predictions. We propose PatchDecomp, a neural network-based time series forecasting method that achieves both high accuracy and interpretability. PatchDecomp divides input time series into subsequences (patches) and generates predictions by aggregating the contributions of each patch. This enables clear attribution of each patch, including those from exogenous variables, to the final prediction. Experiments on multiple benchmark datasets demonstrate that PatchDecomp provides predictive performance comparable to recent forecasting methods. Furthermore, we show that the model's explanations not only influence predicted values quantitatively but also offer qualitative interpretability through visualization of patch-wise contributions.

Hiroki Tomioka, Genta Yoshimura

Mitsubishi Electric Corporation, Japan

Time Series Forecasting Explainable Time Series Forecasting

Ask PDF

Aug 25, 2026

NVExplain: Explaining Time Series Forecasting with Latent Trajectory Analysis and Structure-Preserving Surrogates

Time series forecasting models are widely used in high-stakes settings, yet their predictions remain difficult to interpret because existing post-hoc methods often ignore temporal dependence and fail to provide horizon-specific explanations. We propose a model-agnostic explainability framework that explains forecasting predictions by attributing each forecast horizon to temporally relevant historical lags. The framework models forecasting as a latent trajectory and introduces semantic flow to quantify how information evolves across time in the model's internal representations. By aggregating semantic flow, it constructs a lag-horizon attribution matrix that captures horizon-resolved temporal influence. To improve explainability, we further generate structure-preserving perturbations and fit sparse local surrogate models, producing human-readable and temporally coherent explanations. We evaluate the method using faithfulness and stability diagnostics across multiple benchmark datasets. Results show that the semantic-flow variant achieves competitive or superior faithfulness compared to standard post-hoc baselines, while being substantially more computationally efficient. Stability analysis further demonstrates that the explanations are robust and identifies regimes where interpretation should be applied with caution.

Muyan Anna Li, Manikandan Ravikiran, Aditi Gautam

NVIDIA, USA

Explainable Time Series Forecasting Interpretable ML Feature Attribution

Ask PDF

Dec 27, 2024

Surrogate Modeling for Explainable Predictive Time Series Corrections

We introduce a local surrogate approach for explainable time-series forecasting. An initially non-interpretable predictive model to improve the forecast of a classical time-series 'base model' is used. 'Explainability' of the correction is provided by fitting the base model again to the data from which the error prediction is removed (subtracted), yielding a difference in the model parameters which can be interpreted. We provide illustrative examples to demonstrate the potential of the method to discover and explain underlying patterns in the data.

Alfredo Lopez, Florian Sobieczky

Software Competence Center Hagenberg, Hagenberg, Austria.

Surrogate Modeling Explainable Time Series Forecasting Time Series Forecasting

Ask PDF

Abstract

Explainable Time Series Forecasting

Explore similar work

PatchDecomp: Interpretable Patch-Based Time Series Forecasting

NVExplain: Explaining Time Series Forecasting with Latent Trajectory Analysis and Structure-Preserving Surrogates

Surrogate Modeling for Explainable Predictive Time Series Corrections