Pseudo-Label-Triggered Retraining from Forecast Errors for Online Time Series Forecasting

Yeryeong Kwak; Yoo-Min Jung; Jonghun Park

cs.LGSep 30, 2026

Pseudo-Label-Triggered Retraining from Forecast Errors for Online Time Series Forecasting

Authors: Yeryeong Kwak, Yoo-Min Jung, Jonghun Park

Organizations: Seoul National University Seoul, Republic of Korea

Abstract

Real-world time series forecasting systems operate under non-stationary data streams, where forecasting performance may degrade over time. Although retraining can recover the performance, it incurs non-trivial computational and operational costs. Under limited deployment resources, the key challenge is therefore not only how to retrain but also when to retrain. While existing retraining policies often rely on indirect indicators such as drift alarms or model staleness, we instead use realized forecast errors as direct deployment feedback. In this paper, we propose PILOT (Pseudo-label-Informed Learned Online Trigger), an online retraining framework that learns when to retrain from forecast-error dynamics. Since ground-truth retraining labels are unavailable, PILOT constructs a pseudo-label from future increases in forecast error and trains a lightweight scorer to predict it from observed error states. At deployment, PILOT uses only completed forecast errors and serves as a plug-in module for arbitrary forecasting backbones without architectural modification. We evaluate PILOT under standard multivariate forecasting settings across eight benchmarks with three representative backbones---DLinear, iTransformer, and TimesNet. Across all three backbones, PILOT achieves state-of-the-art average-rank performance among retraining policies while maintaining a favorable performance--efficiency trade-off.

Figures & tables

Panel of Figure 1 — Figure 1 . Overall framework of PILOT (Pseudo-label-Informed Learned Online Trigger). Gray snowflake modules are frozen while light-yellow modules are updatable in the corresponding phase. A two-panel schematic of the proposed retraining method. In both panels, modules shown in gray with a snowflake icon are frozen, while modules shown in light yellow have their parameters updated in the corresponding phase: the scorer is trained in panel (a), and the backbone is retrained in panel (b). The top panel illustrates the offline training phase: a training input window spanning from time $t-L$ to $t+H$ is fed into the frozen forecasting backbone $\gF_{\phi}$ , which produces a forecast $\hat{\mY}_t$ . The residual $\mR_t = \mY_t -\hat{\mY}_t$ between the ground-truth future segment $\mY_t$ and the forecast is summarized into a five-dimensional forecast-error state vector $\vc_t$ consisting of the average signed residual, the mean absolute error, the mean squared error, the rolling mean absolute error, and the rolling root mean squared error. The mean squared error component of $\vc_t$ is passed through a pseudo-label construction branch that produces a pseudo-label $g_t$ . The standardized error state sequence $\mC_t$ over the recent $N$ steps is passed to the trainable retraining scorer, which aggregates the sequence via average pooling and max pooling and concatenates the result with the current-step vector before feeding it into a multi-layer perceptron (MLP) that outputs a scalar retraining score $s_t$ , supervised by $g_t$ via the Huber loss. The bottom panel illustrates the online inference phase: the forecasting backbone $\gF_{\phi}$ processes a test input window and produces a forecast whose residual $\mR_t$ forms the current error-state $\vc_t$ , which is passed to the frozen retraining scorer to yield a scalar score $s_t$ . The score is then online-calibrated and compared against a threshold $\theta$ to produce a binary retraining action $a_t = [\tilde{s}_t > \theta]$ . A backbone update equation at the top indicates that when $a_t = 1$ , the backbone parameters are updated on a recent training buffer $\mathcal{B}_t$ via the retraining operator $\mathcal{U}$ . A timeline at the bottom illustrates the triggering schedule: an initial warm-up period of length $N_{\text{wu}}$ , during which triggering is disabled, is followed by an active triggering region, then a cooldown period of length $N_{\text{cd}}$ after the most recent retraining time $t_{\text{last}}$ , during which triggering is again disabled, and finally another active triggering region.

Backbone	Dataset	No Retrain	Periodic	ADWIN ( ( 2007 ) )	KSWIN ( ( 2020 ) )	CARA ( ( 2024 ) )	UPF ( ( 2025 ) )	PILOT (ours)
DLinear	ETTh1	0.4572 $\pm$ 0.0005	0.4514 $\pm$ 0.0003 (16.0)	0.4533 $\pm$ 0.0002 (10.7)	0 .4506 $\pm$ 0.0001 (32.0)	0.4562 $\pm$ 0.0013 (10.3)	0.4536 $\pm$ 0.0007 (15.3)	0.4501 $\pm$ 0.0003 (17.3)
	ETTh2	0.2343 $\pm$ 0.0085	0.2184 $\pm$ 0.0007 (16.0)	0 .2185 $\pm$ 0.0013 (6.7)	0.2188 $\pm$ 0.0009 (31.0)	0.2343 $\pm$ 0.0085 (0.0)	0.2193 $\pm$ 0.0008 (24.7)	0.2185 $\pm$ 0.0009 (14.0)
	ETTm1	0.3651 $\pm$ 0.0027	0.3606 $\pm$ 0.0002 (35.0)	0.3586 $\pm$ 0.0003 (23.3)	0.3618 $\pm$ 0.0009 (34.0)	0 .3594 $\pm$ 0.0006 (23.7)	0.3624 $\pm$ 0.0003 (24.0)	0.3606 $\pm$ 0.0007 (20.7)
	ETTm2	0.1756 $\pm$ 0.0106	0 .1603 $\pm$ 0.0011 (35.0)	0.1625 $\pm$ 0.0006 (17.3)	0.1607 $\pm$ 0.0009 (34.0)	0.1629 $\pm$ 0.0011 (15.3)	0.1610 $\pm$ 0.0039 (13.3)	0.1598 $\pm$ 0.0031 (22.3)
	ECL	0.1963 $\pm$ 0.0000	0.1957 $\pm$ 0.0000 (25.0)	0.1963 $\pm$ 0.0000 (6.0)	0.1963 $\pm$ 0.0000 (50.0)	0.1962 $\pm$ 0.0001 (9.0)	0 .1959 $\pm$ 0.0000 (3.0)	0.1962 $\pm$ 0.0002 (31.7)
	Exchange	0.0888 $\pm$ 0.0000	0.0785 $\pm$ 0.0003 (7.0)	0.0844 $\pm$ 0.0000 (1.0)	0.0785 $\pm$ 0.0004 (13.0)	0 .0782 $\pm$ 0.0001 (4.0)	0.0888 $\pm$ 0.0000 (0.0)	0.0776 $\pm$ 0.0006 (23.0)

Table 1 . Forecasting performance across eight datasets and three forecasting backbones. Each cell reports MSE with its standard deviation, followed by the average number of retraining events (#RT) in parentheses; No Retrain reports MSE only since its #RT is always zero. Bold : best, underline : second-best.

Figure 2 . Retraining performance–efficiency trade-off on ETTm2 across the three backbones. The x-axis shows total retraining runtime including decision overhead, the y-axis shows relative MSE improvement over No Retrain (%), and marker size is proportional to #RT. Methods closer to the upper-left corner achieve greater forecasting improvement at lower retraining cost. Three scatter plots compare retraining methods on ETTm2 using DLinear, iTransformer, and TimesNet. The x-axis shows total retraining runtime, the y-axis shows relative MSE improvement over No Retrain, and marker size is proportional to the number of retraining events. Methods toward the upper-left achieve greater forecasting improvement with lower retraining runtime.

Method	ETTh1	ETTh2	ETTm1	ETTm2	ECL	Exch.	Weat.	Traff.
No RT	0.4572	0.2343	0.3651	0.1756	0.1963	0.0888	0.1911	0.6546
Uniform	0.5619	0.2395	0.3576	0.1782	0.2041	0.0879	0.1853	0.6416
PILOT	0.4501	0.2185	0.3606	0.1598	0.1962	0.0776	0.1819	0.6410

Table 2 . Effect of retraining timing on DLinear. Uniform performs the same number of retraining events as PILOT on each dataset, spacing them uniformly over the test stream. Bold : best.

Figure 3 . Pearson correlation between

s_{t}

and future degradation

g_{t}^{\mathrm{online}}

across the test stream. Lines show means across datasets and seeds; shading denotes the interquartile range.

Method	Trigger Precision ( $\uparrow$ )	Trigger Need ( $\uparrow$ )	Episode Hit Rate ( $\uparrow$ )	Detection Lag Rank ( $\downarrow$ )
Periodic	0.482	0.091	0.417	3.53
ADWIN	0.518	0.113	0.243	3.73
KSWIN	0.514	0.088	0.574	2.66
CARA	0.588	0.255	0.233	4.61
UPF	0.437	0.052	0.353	4.04
PILOT	0.707	0.356	0.468	2.43

Table 3 . Trigger quality averaged across the eight datasets and three backbones. Bold : best, underline : second-best.

Panel of Figure 4 — Figure 4 . Trigger visualization of each retraining method for ( 4(a) ) DLinear on ETTh1 and ( 4(b) ) iTransformer on ETTh2. Two three-panel figures illustrate retraining behavior over time. The first set shows ETTh1 with DLinear: the rolling MSE fluctuates with several distinct surges between steps 500 and 2300; the error trend panel shows positive values during these surges, with shaded regions marking intervals of positive forecast degradation; PILOT concentrates triggers within shaded regions, and it achieves the lowest MSE in this case. The second set shows ETTh2 with iTransformer: the rolling MSE starts high (~1.4) and gradually decreases, with persistent surges around steps 1400 to 1800; the error trend shows alternating positive and negative regions; PILOT achieves the lowest MSE on this seed while triggering substantially fewer times than others around steps 1600 to 2500.

Dataset	PILOT	w/o Scorer †	w/o History	w/o Stacking
ETTh1	0.6406	0.6893 ( $+7.61\%$ )	0.6928 ( $+8.14\%$ )	0.7004 ( $+9.34\%$ )
ETTh2	0.3270	0.3294 ( $+0.74\%$ )	0.3320 ( $+1.54\%$ )	0.3337 ( $+2.04\%$ )
ETTm1	0.3945	0.3973 ( $+0.71\%$ )	0.4034 ( $+2.26\%$ )	0.4024 ( $+2.00\%$ )
ETTm2	0.1817	0.1891 ( $+4.06\%$ )	0.1810 ( $-0.38\%$ )	0.1859 ( $+2.29\%$ )
ECL	0.1618	0.1708 ( $+5.55\%$ )	0.1691 ( $+4.53\%$ )	0.1697 ( $+4.90\%$ )
Exchange	0.1659	0.1624 ( $-2.14\%$ )	0.1678 ( $+1.13\%$ )	0.1729 ( $+4.21\%$ )

Table 4 . Component ablation for PILOT on TimesNet. Parentheses report relative MSE change from PILOT.

Variant	DLinear	iTransformer	TimesNet	Counterfactual RT †
Mean (main)	0.2857	0.2708	0.3157	No
Cumulative	0.2856	0.2730	0 .3164	No
Relative	0.2857	0.2714	0 .3189	No
Binary util.	0 .2872	0.2733	0.3150	Yes
Ordinal util.	0.2878	0.2755	0.3137	Yes
Voting	0 .2873	0 .2932	0.3191	Yes

Table 5 . Descriptive average MSE of alternative pseudo-label designs. Formatting indicates their average-rank positions relative to the baselines. Bold : outperforms the best baseline; underline : outperforms the second-best.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Backbone	Method	#RT	Total RT Time (s)	Time / #RT (s)	GPU Mem. (MB)
DLinear	Periodic	35.0	6.644	0.190	233.4
	ADWIN	17.3	2.944	0.170	233.5
	KSWIN	34.0	6.418	0.189	233.7
	CARA	15.3	2.716	0.177	233.0
	UPF	13.3	2.857	0.214	233.8
	PILOT	22.3	4.349	0.195	232.6

Appendix

Table 6. Detailed retraining efficiency on ETTm2.

Backbone	Dataset	Cum. deg.	Rel. deg.	Binary util.	Ordinal util.	Voting
DLinear	ETTh1	0.4503	0.4503	0.4532	0.4537	0.4510
	ETTh2	0.2170	0.2182	0.2189	0.2240	0.2178
	ETTm1	0.3574	0.3574	0.3601	0.3578	0.3601
	ETTm2	0.1605	0.1605	0.1629	0.1620	0.1618
	ECL	0.1962	0.1962	0.1956	0.1958	0.1960
	Exchange	0.0771	0.0777	0.0790	0.0805	0.0853

Appendix

Table 7 . Dataset-level MSE of the alternative pseudo-label variants, averaged over three seeds.

Explore similar work

RATL: Learning from Retrieved Residuals for Robust Multivariate Time-Series Forecasting
Sep 3, 2026Yuchen He, Yueyang Cang, Zhiyuan Ning +2Residual Learning Multivariate Time-Series
Department of Automation, Tsinghua University · State Key Laboratory of Hydroscience and Engineering, Tsinghua University
Stationarity-Aware Retrieval-Augmented Time Series Forecasting
Jun 2, 2026Shiqiao Zhou, Holger Schöner, Zipeng Wu +3Time Series Forecasting Time Series
University of Birmingham Birmingham, United Kingdom · Siemens AG Munich, Germany
Online Irregular Multivariate Time Series Forecasting via Uncertainty-Driven Dual-Expert Calibration
May 27, 2026Haonan Wen, Hanyang Chen, Songhe FengTime Series Calibrated Uncertainty
Key Laboratory of Big Data & Artificial Intelligence in Transportation (Beijing Jiaotong University), Ministry of Education School of Computer Science and Technology, Beijing Jiaotong University Beijing, China · School of Computer Science and Technology, Beijing Jiaotong University Beijing, China · Tangshan Research Institute, Beijing Jiaotong University Tangshan, China

Sep 3, 2026, cs.LGJ/K move · Enter open · S save

RATL: Learning from Retrieved Residuals for Robust Multivariate Time-Series Forecasting

Yuchen He, Yueyang Cang, Zhiyuan Ning, Ningyu Wang+1

Department of Automation, Tsinghua University · State Key Laboratory of Hydroscience and Engineering, Tsinghua University

Residual Learning Multivariate Time-Series

Open paper AskPDF arXiv

Retrieval-augmented generation (RAG) complements parametric models with retrieved external evidence. The same idea is attractive for continuous-output regression, but directly reusing retrieved target values is often not robust when samples differ in output level, numerical scale, or local dynamics. Moreover, conventional forecasting pipelines generally use residuals for model optimization and error diagnosis, but do not retain individual historical residual examples as memory that can be accessed at inference time.For multivariate time-series forecasting, we propose RATL, a plug-in residual-retrieval and feedback-correction method. RATL freezes a base forecaster to construct retrieval keys and turns its historical forecast residuals into a train-only memory specific to that base model. At inference time, RATL retrieves residual trajectories from similar historical contexts subject to causal availability constraints, then uses a set-aware router operating over forecast blocks and variables to select and combine these trajectories. Experiments show that historical residuals matched to the current context contain reusable forecasting information and that RATL improves frozen base forecasters in most experimental settings. Ablations further show that learned routing strengthens raw residual feedback, while validation-based correction-strength selection limits residual over-injection.On real-world benchmarks, we use iTransformer as the primary frozen base forecaster, compare against multiple strong forecasting baselines, and test transferability across backbones. The results show that RATL can further improve base-forecaster performance in most settings.Overall, RATL shifts the retrieved object from historical target values to base-model-specific historical forecast errors, providing a plug-in, residual-memory-based paradigm for learned feedback correction in continuous-output forecasting.

Abstract

Explore similar work

RATL: Learning from Retrieved Residuals for Robust Multivariate Time-Series Forecasting

Stationarity-Aware Retrieval-Augmented Time Series Forecasting

Online Irregular Multivariate Time Series Forecasting via Uncertainty-Driven Dual-Expert Calibration