Locally Linear Continual Learning for Time Series based on VC-Theoretical Generalization Bounds

Yan V. G. Ferreira; Igor B. Lima; Pedro H. G. Mapa S.; Felipe V. Campos; Antonio P. Braga

cs.LGMar 14, 2026

Locally Linear Continual Learning for Time Series based on VC-Theoretical Generalization Bounds

Authors: Yan V. G. Ferreira, Igor B. Lima, Pedro H. G. Mapa S., Felipe V. Campos, Antonio P. Braga

Organizations: Department of Electronic Engineering, Universidade Federal de Minas Gerais, UFMG, Belo Horizonte 31270-901, Brazil · Department of Computing Science, University of Alberta, Edmonton, Alberta, Canada · Graduate Program in Electrical Engineering, Universidade Federal de Minas Gerais, UFMG, Belo Horizonte 31270-901, Brazil

Abstract

Most machine learning methods assume fixed probability distributions, limiting their applicability in nonstationary real-world scenarios. While continual learning methods address this issue, current approaches often rely on black-box models or require extensive user intervention for interpretability. We propose SyMPLER (Systems Modeling through Piecewise Linear Evolving Regression), an explainable model for time series forecasting in nonstationary environments based on dynamic piecewise-linear approximations. Unlike other locally linear models, SyMPLER uses generalization bounds from Statistical Learning Theory to automatically determine when to add new local models based on prediction errors, eliminating the need for explicit clustering of the data. Experiments show that SyMPLER can achieve comparable performance to both black-box and existing explainable models while maintaining a human-interpretable structure that reveals insights about the system's behavior. In this sense, our approach conciliates accuracy and interpretability, offering a transparent and adaptive solution for forecasting nonstationary time series.

Figures & tables

Panel of Figure 1 — Fig. 1 : SyMPLER switches between existing local models depending on the value of the input $\mathbf{x}(t)$ . Each local model approximates the desired function linearly around its approximation point $\mathbf{x}(t)=\mathbf{p}_{i}$ . As the domain of the problem grows, more local models are added. In this example, SyMPLER switches to the model colored in red as $\mathbf{x}(t)$ approaches $\mathbf{p}_{4}$ and adds a new model for the new region of the input.

Fig. 2 : Smallest training size

l

for a model with VC-dimension

h

, following [ 20 ] .

Fig. 3 : Continual learning results: (a) prediction squared error, (b) input data and (c) number of members in the network. Data in (a) and (c) start appearing later than in (b) because the network has no member until the first

2(n+1)+10

samples arrive.

Fig. 4 : SyMPLER approximation in feature space. Although each individual model is linear, as shown with the coloured lines in (a), switching between them allows the network to approximate non-linear functions piecewise-linearly, as shown in (b). The zoomed area shows the piecewise-linear behavior of the model.

$\theta_{0}$ (rad)		Slope	Bias
1.55	SyMPLER	-0.44	-18.91
1.55	Taylor	-0.34	-19.07
1.24	SyMPLER	-6.42	-10.54
1.24	Taylor	-6.34	-10.69
0.44	SyMPLER	-17.72	-0.49
0.44	Taylor	-17.78	-0.53

TABLE I : Coefficients of Taylor first-order approximations of the target function and coefficients of SyMPLER members for some of the approximation points.

Fig. 5 : Long-term forecasting of the angular position of the pendulum using SyMPLER and the linearized model for the first cycle of the system (left figure) and its last cycle (right figure).

Model	Searched parameters	Best parameters
SyMPLER (original)	$\lambda=\{10^{-6},10^{-2},0.1,1,5,10,15\}$	$\lambda=5$
SyMPLER (aggr.)	$\lambda=\{10^{-6},10^{-2},0.1,1,5,10,15\}$ $\sigma=\{0.1,0.5,1,10\}$	$\lambda=1$ $\sigma=1$
ICN	$\lambda=\{10^{-6},0.1,1,5\}$ $\epsilon=\{10^{-2},0.5,1,5\}$ $\alpha=\{0.1,1,10\}$	$\lambda=5$ $\epsilon=10^{-2}$ $\alpha=0.1$
eTS	$\omega=\{10^{-6},0.1,1,10,100,1000\}$ $r=\{0.1,0.3,0.5,0.7,0.9\}$	$\omega=0.1$ $r=0.5$
EWC-SWFT	$\beta=\{100,500,1000\}$ $\text{learning\_rate}=\{10^{-4},10^{-3},10^{-2}\}$	$\beta=100$ $\text{learning\_rate}=10^{-3}$
XGBoost	$\text{n\_estimators}=\{10,100,200,500\}$ $\text{max\_depth}=\{3,6,9\}$ $\text{learning\_rate}=\{0.01,0.1,0.3\}$ $\text{colsample\_bytree}=\{0.3,0.6,1\}$ $\text{alpha}=\{0,10,100\}$	$\text{n\_estimators}=200$ $\text{max\_depth}=3$ $\text{learning\_rate}=0.1$ $\text{colsample\_bytree}=0.3$ $\text{alpha}=0$

TABLE II : Hyperparameter search and best parameters for each model.

Model	Fitting error	Prediction error	Forgetting ratio
SyMPLER (original)	21.91	21.06	0.11
SyMPLER (aggr.)	21.09	21.01	0.14
ICN	31.03	39.07	0.15
eTS	26.88	28.87	0.30
EWC-SWFT	49.10 $\pm$ 3.86	55.64 $\pm$ 4.45	0.25 $\pm$ 0.29
XGBoost	16.88	18.56	0.66

TABLE III : Comparison of models based on fitting error (RMSE), prediction error (RMSE), and forgetting ratio. EWC-SWFT performance was evaluated for 10 random initializations, so we present the mean and standard deviation of the results.

Fig. 6 : Predictions of the models for the last month of 2007.

Station	$x_{\text{0}}$ (°F)	$p_{\text{0}}$ (°F)	$w_{\text{0}}$	$x_{\text{1}}$ (°F)	$p_{\text{1}}$ (°F)	$w_{\text{1}}$
14	32.00	37.11	1.48	86.00	83.93	-12.30
6	32.00	30.18	-4.75	86.00	85.64	8.57
1	32.00	30.82	-8.53	86.00	83.82	19.63
25	32.00	28.54	-7.54	86.00	83.18	10.20
21	32.00	31.86	-2.27	86.00	89.29	13.10
20	32.00	29.11	-8.69	86.00	83.93	12.64

TABLE IV : The local approximation points closer to the scenarios

x_{\text{0}}

and

x_{\text{1}}

in which all the weather stations measure 32°F and 86°F, respectively, are

p_{\text{0}}

and

p_{\text{1}}

. The local models associated with these points have weights

w_{\text{0}}

and

w_{\text{1}}

Fig. 7 : Continual learning results for the concept drift experiment: (a) absolute prediction error, (b) input data and (c) number of members in the network.

Fig. 8 : SyMPLER’s average error and number of members for an increasing input dimensionality. The shaded areas represent two standard deviations from the mean.

Fig. 9 : Mean distance between the vector of SyMPLER’s coefficients and the vector of Taylor’s coefficients for different levels of noise and regularization.

Explainable Time Series Forecasting

Explainable time series forecasting makes predictions over time interpretable through attributions, structured models, causal reasoning, and counterfactuals.

See all 61 papers

PatchDecomp: Interpretable Patch-Based Time Series Forecasting
Mar 4, 2026Hiroki Tomioka, Genta YoshimuraTime Series Forecasting Explainable Time Series Forecasting
Mitsubishi Electric Corporation, Japan
NVExplain: Explaining Time Series Forecasting with Latent Trajectory Analysis and Structure-Preserving Surrogates
Aug 25, 2026Muyan Anna Li, Manikandan Ravikiran, Aditi GautamExplainable Time Series Forecasting Interpretable ML
NVIDIA, USA
Surrogate Modeling for Explainable Predictive Time Series Corrections
Dec 27, 2024Alfredo Lopez, Florian SobieczkySurrogate Modeling Explainable Time Series Forecasting
Software Competence Center Hagenberg, Hagenberg, Austria.

Mar 4, 2026J/K move · Enter open · S save

PatchDecomp: Interpretable Patch-Based Time Series Forecasting

Hiroki Tomioka, Genta Yoshimura

Mitsubishi Electric Corporation, Japan

Time Series Forecasting Explainable Time Series Forecasting

Open paper AskPDF arXiv

Time series forecasting, which predicts future values from past observations, plays a central role in many domains and has driven the development of highly accurate neural network models. However, the complexity of these models often limits human understanding of the rationale behind their predictions. We propose PatchDecomp, a neural network-based time series forecasting method that achieves both high accuracy and interpretability. PatchDecomp divides input time series into subsequences (patches) and generates predictions by aggregating the contributions of each patch. This enables clear attribution of each patch, including those from exogenous variables, to the final prediction. Experiments on multiple benchmark datasets demonstrate that PatchDecomp provides predictive performance comparable to recent forecasting methods. Furthermore, we show that the model's explanations not only influence predicted values quantitatively but also offer qualitative interpretability through visualization of patch-wise contributions.

Abstract

Explainable Time Series Forecasting

Explore similar work

PatchDecomp: Interpretable Patch-Based Time Series Forecasting

NVExplain: Explaining Time Series Forecasting with Latent Trajectory Analysis and Structure-Preserving Surrogates

Surrogate Modeling for Explainable Predictive Time Series Corrections