TIFO: Time-Invariant Frequency Operator for Stationarity-Aware Representation Learning in Time Series

Xihao Piao; Zheng Chen; Lingwei Zhu; Yushun Dong; Yasuko Matsubara; Yasushi Sakurai

cs.LGFeb 19, 2026

TIFO: Time-Invariant Frequency Operator for Stationarity-Aware Representation Learning in Time Series

Authors: Xihao Piao, Zheng Chen, Lingwei Zhu, Yushun Dong, Yasuko Matsubara, Yasushi Sakurai

Organizations: SANKEN Osaka University Osaka, Japan · School of Information Science and Technology Great Bay University Dongguan, China · Department of Computer Science Florida State University

Abstract

Nonstationary time series forecasting suffers from the distribution shift issue due to the different distributions that produce the training and test data. Existing methods attempt to alleviate the dependence by, e.g., removing low-order moments from each individual sample. These solutions fail to capture the underlying time-evolving structure across samples and do not model the complex time structure. In this paper, we aim to address the distribution shift in the frequency space by considering all possible time structures. To this end, we propose a Time-Invariant Frequency Operator (TIFO), which learns stationarity-aware weights over the frequency spectrum across the entire dataset. The weight representation highlights stationary frequency components while suppressing non-stationary ones, thereby mitigating the distribution shift issue in time series. To justify our method, we show that the Fourier transform of time series data implicitly induces eigen-decomposition in the frequency space. TIFO is a plug-and-play approach that can be seamlessly integrated into various forecasting models. Experiments demonstrate our method achieves 18 top-1 and 6 top-2 results out of 28 forecasting settings. Notably, it yields 33.3% and 55.3% improvements in average MSE on the ETTm2 dataset. In addition, TIFO reduces computational costs by 60% -70% compared to baseline methods, demonstrating strong scalability across diverse forecasting models.

Figures & tables

Figure 1: Overview of TIFO . Before training, we first transfer all samples into the frequency domain and measure their cross-sample stationarity at the dataset level (steps 1 & 2). These features are then used to learn frequency weights that measure frequency stationarity (step 3). During training, each input sample is transformed into the frequency domain and then weighted by the learned stationarity weights. Finally, they are transformed to the time domain to serve as input to the forecasting models (steps 4 & 5). TIFO is optimized using the forecasting loss along with the backbone model.

Models	PatchTST ( Nie et al., 2023 )				iTransformer ( Liu et al., 2024b )
Methods	+ TIFO		Ori		+ TIFO		Ori
Metric	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTh1	0.438 $\pm$ 0.024	0.437 $\pm$ 0.035	0.480 $\pm$ 0.037	0.481 $\pm$ 0.031	0.445 $\pm$ 0.017	0.443 $\pm$ 0.026	0.511 $\pm$ 0.033	0.496 $\pm$ 0.036
ETTh2	0.379 $\pm$ 0.032	0.380 $\pm$ 0.038	0.604 $\pm$ 0.130	0.524 $\pm$ 0.027	0.376 $\pm$ 0.041	0.400 $\pm$ 0.057	0.813 $\pm$ 0.134	0.666 $\pm$ 0.072
ETTm1	0.390 $\pm$ 0.027	0.398 $\pm$ 0.025	0.419 $\pm$ 0.055	0.432 $\pm$ 0.047	0.396 $\pm$ 0.026	0.406 $\pm$ 0.056	0.447 $\pm$ 0.026	0.457 $\pm$ 0.061
ETTm2	0.280 $\pm$ 0.032	0.325 $\pm$ 0.031	0.420 $\pm$ 0.035	0.424 $\pm$ 0.044	0.283 $\pm$ 0.020	0.327 $\pm$ 0.026	0.633 $\pm$ 0.055	0.489 $\pm$ 0.041

Table 1: Multivariate forecasting results (average) with forecasting lengths

H\in\{96,192,336,720\}

for all datasets and fixed input sequence length

L=96

Models	MLP-based (DLinear ( Zeng et al., 2023 ) )										Transformer-based (iTransformer ( Liu et al., 2024b ) )
Methods	+ TIFO *		+ TIFO		+SAN		+ FAN		+ RevIN		+ TIFO *		+ TIFO		+SAN		+ FAN		+ RevIN
Metric	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTh1	0.413	0.424	0.407	0.419	0.421	0.427	0.469	0.473	0.460	0.456	0.455	0.449	0.445	0.443	0.466	0.455	0.504	0.488	0.463	0.452
ETTh2	0.339	0.384	0.337	0.384	0.342	0.387	0.459	0.469	0.561	0.518	0.378	0.408	0.376	0.400	0.392	0.413	0.543	0.515	0.385	0.412
ETTm1	0.341	0.372	0.357	0.375	0.344	0.376	0.367	0.392	0.413	0.407	0.389	0.398	0.396	0.406	0.401	0.406	0.424	0.428	0.406	0.410
ETTm2	0.255	0.316	0.256	0.313	0.260	0.318	0.278	0.339	0.350	0.413	0.285	0.334	0.283	0.327	0.287	0.336	0.337	0.385	0.294	0.337

Table 2: Multivariate forecasting results (average) with

H\in\{96,192,336,720\}

for all datasets and fixed input sequence length

L=96

. The best and second best results are highlighted. TIFO * represents the results where both TIFO and SAN are used in the backbones.

Figure 4: Frequency-domain analysis of Fourier basis Learning. From left to right: unprocessed spectra (Before), after FAN, and after applying TIFO . Each panel is a 3D box with time, frequency, and amplitude axes. We visualize four Fourier basis waves, $\zeta_{\omega_{1:4}}$ , to illustrate how each processing method alters the basis functions. In the frequency–amplitude plane, we plot three forecasting cases: the ground truth in blue and the forecasting results based on the processed input in red. The red diamonds mark key local peak frequencies.

Table 6

Table 5: Ablation study on frequency domain modeling on ETTh1 with iTransformer. (a) Impact of window functions and resolution under the DFT setting. (b) Comparison of reconstruction strategies: DFT vs. Short-Time Fourier Transform (STFT). The default settings with a rectangular window achieves the best performance.

Figure 8

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Illustration of z-score normalization in both time and frequency domains. (Top) Data generating distributions $p(x|t_{i})$ across different temporal structures $t_{i}$ are aligned after $z$ -score, sharing a common location and scale. (Bottom) Frequency-domain power spectra before and after $z$ -score. The frequency-domain distribution remains divergent after normalization.

Metric	Description	MSE	MAE
mu_sigma ( Ours )	Mean/Std (Unsupervised, Cross-sample stability)	0.3904	0.4071
entropy	Spectral Entropy (Intra-sample complexity)	0.3962	0.4106
corr	Correlation $\|r\|$ (Target-aware, MI approx.)	0.3948	0.4077

Appendix

Table 6: Comparison of different stationarity metrics on ETTh1 with iTransformer. mu_sigma denotes our default unsupervised metric; entropy focuses on intra-sample distribution; corr utilizes future labels to approximate Mutual Information.

Dataset	Source	Resolution	Channels	Time Range
Weather	Autoformer ( Wu et al., 2021 )	Every 10 minutes	21 (e.g., temperature, humidity)	2020
ETTh1	Informer ( Zhou et al., 2021 )	Hourly	7 states of a electrical transformer	2016-2017
ETTh2	Informer ( Zhou et al., 2021 )	Hourly	7 states of a electrical transformer	2017-2018
ETTm1	Informer ( Zhou et al., 2021 )	Every 15 minutes	7 states of a electrical transformer	2016-2017
ETTm2	Informer ( Zhou et al., 2021 )	Every 15 minutes	7 states of a electrical transformer	2017-2018
Electricity	UCI ML Repository	Hourly	321 clients’ consumption	2012-2014

Appendix

Table 7: Overview of Datasets

Models		DLinear ( Zeng et al., 2023 )								PatchTST ( Nie et al., 2023 )								iTransformer ( Liu et al., 2024b )
Methods		+ Ours*		+ Ours		+ SAN		+ RevIN		+ Ours*		+ Ours		+ SAN		+ RevIN		+ Ours*		+ Ours		+ SAN		+ RevIN
Metric		MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
Electricity	96	0.135	0.230	0.140	0.237	0.137	0.234	0.210	0.278	0.175	0.266	0.190	0.280	0.182	0.271	0.212	0.297	0.145	0.244	0.143	0.237	0.171	0.262	0.152	0.251
	192	0.149	0.245	0.155	0.249	0.151	0.247	0.210	0.304	0.183	0.273	0.195	0.286	0.186	0.276	0.213	0.300	0.169	0.266	0.159	0.252	0.180	0.270	0.165	0.255
	336	0.165	0.262	0.171	0.267	0.166	0.264	0.223	0.309	0.198	0.289	0.211	0.301	0.200	0.290	0.227	0.314	0.178	0.271	0.172	0.266	0.194	0.284	0.180	0.272
	720	0.198	0.291	0.208	0.298	0.201	0.295	0.257	0.349	0.233	0.317	0.253	0.334	0.237	0.322	0.268	0.344	0.210	0.311	0.205	0.295	0.237	0.319	0.227	0.312

Appendix

Table 8: Detailed results of comparing our proposal and other normalization methods. The best results are highlighted in bold . The second best are underlined .

Models		DLinear ( Zeng et al., 2023 )								iTransformer ( Liu et al., 2024b )
Methods		+ TIFO *		+ TIFO		+ FAN		+ RevIN		+ TIFO *		+ TIFO		+ FAN		+ RevIN
Metric		MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
Electricity	96	0.135	0.230	0.140	0.237	0.146	0.248	0.210	0.278	0.145	0.244	0.143	0.237	0.158	0.254	0.152	0.251
	192	0.149	0.245	0.155	0.249	0.163	0.264	0.210	0.304	0.169	0.266	0.159	0.252	0.170	0.263	0.165	0.255
	336	0.165	0.262	0.171	0.267	0.180	0.282	0.223	0.309	0.178	0.271	0.172	0.266	0.183	0.281	0.180	0.272
	720	0.198	0.291	0.208	0.298	0.216	0.316	0.257	0.349	0.210	0.311	0.205	0.295	0.208	0.306	0.227	0.312

Appendix

Table 9: Detailed results of comparing TIFO and FAN. The best results are highlighted in bold . The second best are underlined .

Figure 6: More results on train-test distance compactness: This figure shows another visualization of the JSD 2 amplitudes distribution distance between the train and test datasets on the electricity data (on a different channel). Each scatter point represents one frequency component. A smaller radius indicates a smaller distributional gap. Green and red colors represent the results before and after applying the learning method, respectively.

Figure 7: Frequency-domain analysis of Fourier basis Learning. ETTh1 dataset, Channel #1.

Figure 8: Frequency-domain analysis of Fourier basis Learning. ETTm1 dataset, Channel #2.

Figure 9: Frequency-domain analysis of Fourier basis Learning. Traffic dataset, Channel #237.

Figure 10: Frequency-domain analysis of Fourier basis Learning. ETTh1 dataset, Channel #6.

Figure 11: Frequency-domain analysis of Fourier basis Learning. Weather dataset, Channel #7.

Model	Electricity		ETTh1		ETTh2		ETTm1		ETTm2		Traffic		Weather
H	96	720	96	720	96	720	96	720	96	720	96	720	96	720
DLinear ( Zeng et al., 2023 )
+ TIFO	16.718	27.545	3.004	3.082	2.820	3.222	13.456	13.124	11.839	12.949	23.217	43.413	13.970	16.536
+ SAN	19.159	34.914	8.688	8.054	8.373	7.811	36.706	36.564	36.620	36.550	31.378	54.518	37.739	40.731
IMP(%)	12.8%	21.1%	65.5%	60.5%	66.6%	58.3%	63.1%	64.2%	67.8%	64.3%	26.0%	20.3%	63.0%	59.9%
PatchTST ( Nie et al., 2023 )

Appendix

Table 10: Running time comparison with forecasting lengths

H\in\{96,720\}

for all datasets and fixed input sequence length

L=96

. The best results are highlighted.

Setting	Run 0		Run 1		Run 2		Average
Setting	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
rect_K48	0.4250	0.4420	0.4020	0.4180	0.4050	0.4210	0.4107	0.4270
rect_K96	0.3902	0.4068	0.3939	0.4105	0.3974	0.4140	0.3938	0.4104
hann_K48	0.4130	0.4260	0.4100	0.4240	0.4120	0.4250	0.4117	0.4250
hann_K96	0.4110	0.4240	0.4090	0.4230	0.4130	0.4260	0.4110	0.4243

Appendix

Table 11: Ablation on window function and FFT resolution on ETTh1 with iTransformer. The best results are highlighted.

$\alpha$	H = 96		H = 192		H = 336		H = 720		Average
$\alpha$	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
0.00	0.4590	0.4740	0.5170	0.5100	0.5620	0.5330	0.5660	0.5520	0.5260	0.5172
0.25	0.4540	0.4690	0.5120	0.5050	0.5570	0.5280	0.5610	0.5470	0.5210	0.5122
0.50	0.4190	0.4340	0.4770	0.4700	0.5220	0.4930	0.5260	0.5120	0.4860	0.4773
0.75	0.3900	0.4050	0.4440	0.4380	0.4940	0.4650	0.4990	0.4840	0.4567	0.4480
1.00	0.3780	0.3930	0.4460	0.4390	0.4810	0.4520	0.4850	0.4710	0.4475	0.4388

Appendix

Table 12: Effect of scaling the learned TIFO weights at test time by

\alpha

on ETTh1 with iTransformer. The best average performance is highlighted.

Setting	Run 0		Run 1		Run 2		Average
Setting	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
Ori	0.3902	0.4068	0.3939	0.4105	0.3974	0.4140	0.3938	0.4104
hann_50_ola	0.5887	0.5148	0.5623	0.5053	0.5951	0.5152	0.5820	0.5118
sqrt_hann_50_wola	0.3999	0.4135	0.3985	0.4133	0.4032	0.4175	0.4005	0.4148
hann_75_ola	0.5872	0.5130	0.5627	0.5050	0.5944	0.5149	0.5814	0.5110

Appendix

Table 13: Comparison of reconstruction strategies (global DFT vs. OLA/WOLA) on ETTh1 with iTransformer. The best results are highlighted.

Setting	Run 0		Run 1		Run 2		Average
Setting	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
no_update	0.3902	0.4068	0.3939	0.4105	0.3974	0.4140	0.3938	0.4104
ema_0.9	0.3916	0.4101	0.3930	0.4105	0.3952	0.4136	0.3932	0.4114
ema_0.99	0.3903	0.4085	0.3938	0.4110	0.3966	0.4146	0.3936	0.4114

Appendix

Table 14: Effect of online EMA refresh of frequency masks during inference on ETTh1 with iTransformer. The best results are highlighted.

Setting	Run 0		Run 1		Run 2		Average
Setting	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
hop0.125_zpad0.0	0.3950	0.4100	0.3980	0.4130	0.3990	0.4140	0.3973	0.4123
hop0.125_zpad0.5	0.3920	0.4070	0.3940	0.4090	0.3950	0.4100	0.3937	0.4087
hop0.125_zpad1.0	0.3930	0.4080	0.3950	0.4100	0.3960	0.4110	0.3947	0.4097
hop0.25_zpad0.0	0.3970	0.4120	0.4000	0.4150	0.4020	0.4170	0.3997	0.4147
hop0.25_zpad0.5	0.3950	0.4100	0.3970	0.4120	0.3980	0.4130	0.3967	0.4117

Appendix

Table 15: Sensitivity to STFT hyperparameters (hop ratio and zero-padding) on ETTh1 with iTransformer. The best results are highlighted.

Frequency-Domain Time Series Modeling

See all 22 papers

May 31, 2026

FAiT: Frequency-Aware Inverted Transformer for Multivariate Time Series Forecasting

While Transformer-based architectures have established themselves as a dominant paradigm in Multivariate Time Series Forecasting (MTSF), their core self-attention mechanism inherently functions as a low-pass filter, systematically smoothing out high-frequency signals vital for sharp local changes. Recent advancements have increasingly incorporated frequency-domain operations to address this bias, however, most existing designs rely on fixed spectral bases and apply sequence-wise (uniform) modulation, implicitly assuming a time-invariant frequency response. This overlooks a key property of real-world series that their spectral characteristics often evolve over time, making uniform modulation insufficient for capturing fine-grained temporal dynamics. To tackle these limitations, we propose FAiT, a Frequency-Aware inverted Transformer. Specifically, FAiT rectifies the spectral bias internally through Inverted Attention, which interprets the attention map as a learnable low-pass operator and constructs a dedicated complementary high-pass branch by inverting the attention matrix to recover attenuated transient signals. Furthermore, FAiT introduces Dynamic Temporal-Frequency Modulation (DTFM), which synthesizes instance-conditioned weights to adaptively re-calibrate the energy of spectral sub-bands, enabling fine-grained control over evolving multi-scale patterns. Extensive experiments on widely used benchmarks demonstrate that FAiT consistently outperforms state-of-the-art Transformer-based and frequency-enhanced baselines, while maintaining computational efficiency.

Peng He, Yao Liu, Yanglei Gan +3

University of Electronic Science and Technology of China · Chengdu, Sichuan, China

Multivariate Time Series Forecasting Frequency-Domain Feature Learning Transformer Attention Time Series Transformers Multiscale Time-Series Modeling

Ask PDF

May 31, 2026

FreqLite: A Lightweight Frequency-Decomposed Linear Model with Adaptive Reversible Normalization for Robust Long-Term Time-Series Forecasting

Long-term time-series forecasting needs models that are accurate yet efficient enough for commodity hardware. Lightweight linear forecasters are remarkably strong in this regime, yet they leave two openings: reversible instance normalization (RevIN) de-normalizes the entire horizon with a single lookback statistic, which is inaccurate under non-stationarity, and time-domain trend/seasonal decomposition relies on a fixed, non-adaptive filter. We present FreqLite, an ultra-lightweight, channel-independent frequency-decomposed linear forecaster: a learnable, lossless, partition-of-unity spectral filter splits the input into bands that are forecast by per-band linear heads and, unlike low-pass-truncation approaches, the high-frequency band is retained and modeled. FreqLite is the best lightweight model on the standard long-term forecasting benchmarks and, at long lookback (L=336), attains a lower average error than a PatchTST Transformer (0.3244 vs. 0.3587 MSE) while using 4x fewer parameters, 2.2x less memory, and 2.2x less time per epoch on a single 4 GB laptop GPU; although modest in magnitude, its improvements are statistically significant under paired Wilcoxon tests across all matched cells (p < 1e-5). We further introduce Adaptive Reversible Instance Normalization (A-RevIN), a regime-adaptive reversible normalization that strictly generalizes RevIN (recovered exactly when its gate is closed), engages under non-stationarity, and reduces to RevIN without harm on stationary data. We validate this on both a real strongly non-stationary dataset (ILI, up to ~5% MSE reduction) and a controlled synthetic drift sweep in which A-RevIN's benefit and its learned gate both rise monotonically with injected non-stationarity. Every component is independently ablatable (Linear and RLinear are special cases of FreqLite), and all results are reproducible on commodity hardware.

Mirza Samad Ahmed Baig, Syeda Anshrah Gillani

Fandaqah, Al Khobar, Saudi Arabia · Hamdard University, Karachi, Pakistan

Long-Term Time Series Forecasting Frequency-Domain Feature Learning Time Series Forecasting Non-Stationary Time Series Forecasting Frequency-Domain Time Series Modeling

Ask PDF

May 17, 2026

Towards Principled Test-Time Adaptation for Time Series Forecasting

Test-time adaptation (TTA) has recently emerged as a promising approach for improving time series forecasting (TSF) under distribution shift. Existing TSF-TTA methods differ in how they utilize revealed targets, yet the resulting adaptation protocols remain heterogeneous and lack a clearly unified formulation. To address this issue, we revisit TSF-TTA from the perspective of protocol cleanliness and propose an adaptation protocol based solely on matured ground truth, yielding a more principled setting for adaptation. Under this protocol, we further diagnose existing adapters in the frequency domain and find that their prediction corrections often exhibit limited and weakly structured spectral modifications. Motivated by this diagnosis, we propose Frequency-Aware Calibration (FAC), a lightweight calibration method that directly parameterizes prediction corrections in the frequency domain. Across diverse datasets, forecasting horizons, and source forecasters, FAC achieves competitive and consistent performance while requiring substantially fewer trainable parameters than the compared TSF-TTA adapters.

Haochun Wang, Ruichen Xu, Georgios Kementzidis +3

Stony Brook University

Test-Time Adaptation Time Series Forecasting

Ask PDF

Abstract

Frequency-Domain Time Series Modeling

Explore similar work

FAiT: Frequency-Aware Inverted Transformer for Multivariate Time Series Forecasting

FreqLite: A Lightweight Frequency-Decomposed Linear Model with Adaptive Reversible Normalization for Robust Long-Term Time-Series Forecasting

Towards Principled Test-Time Adaptation for Time Series Forecasting