TIFO: Time-Invariant Frequency Operator for Stationarity-Aware Representation Learning in Time Series

Xihao Piao; Zheng Chen; Lingwei Zhu; Yushun Dong; Yasuko Matsubara; Yasushi Sakurai

cs.LGFeb 19, 2026

TIFO: Time-Invariant Frequency Operator for Stationarity-Aware Representation Learning in Time Series

Authors: Xihao Piao, Zheng Chen, Lingwei Zhu, Yushun Dong, Yasuko Matsubara, Yasushi Sakurai

Organizations: SANKEN Osaka University Osaka, Japan · School of Information Science and Technology Great Bay University Dongguan, China · Department of Computer Science Florida State University

Abstract

Nonstationary time series forecasting suffers from the distribution shift issue due to the different distributions that produce the training and test data. Existing methods attempt to alleviate the dependence by, e.g., removing low-order moments from each individual sample. These solutions fail to capture the underlying time-evolving structure across samples and do not model the complex time structure. In this paper, we aim to address the distribution shift in the frequency space by considering all possible time structures. To this end, we propose a Time-Invariant Frequency Operator (TIFO), which learns stationarity-aware weights over the frequency spectrum across the entire dataset. The weight representation highlights stationary frequency components while suppressing non-stationary ones, thereby mitigating the distribution shift issue in time series. To justify our method, we show that the Fourier transform of time series data implicitly induces eigen-decomposition in the frequency space. TIFO is a plug-and-play approach that can be seamlessly integrated into various forecasting models. Experiments demonstrate our method achieves 18 top-1 and 6 top-2 results out of 28 forecasting settings. Notably, it yields 33.3% and 55.3% improvements in average MSE on the ETTm2 dataset. In addition, TIFO reduces computational costs by 60% -70% compared to baseline methods, demonstrating strong scalability across diverse forecasting models.

Figures & tables

Figure 1: Overview of TIFO . Before training, we first transfer all samples into the frequency domain and measure their cross-sample stationarity at the dataset level (steps 1 & 2). These features are then used to learn frequency weights that measure frequency stationarity (step 3). During training, each input sample is transformed into the frequency domain and then weighted by the learned stationarity weights. Finally, they are transformed to the time domain to serve as input to the forecasting models (steps 4 & 5). TIFO is optimized using the forecasting loss along with the backbone model.

Models	PatchTST ( Nie et al., 2023 )				iTransformer ( Liu et al., 2024b )
Methods	+ TIFO		Ori		+ TIFO		Ori
Metric	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTh1	0.438 $\pm$ 0.024	0.437 $\pm$ 0.035	0.480 $\pm$ 0.037	0.481 $\pm$ 0.031	0.445 $\pm$ 0.017	0.443 $\pm$ 0.026	0.511 $\pm$ 0.033	0.496 $\pm$ 0.036
ETTh2	0.379 $\pm$ 0.032	0.380 $\pm$ 0.038	0.604 $\pm$ 0.130	0.524 $\pm$ 0.027	0.376 $\pm$ 0.041	0.400 $\pm$ 0.057	0.813 $\pm$ 0.134	0.666 $\pm$ 0.072
ETTm1	0.390 $\pm$ 0.027	0.398 $\pm$ 0.025	0.419 $\pm$ 0.055	0.432 $\pm$ 0.047	0.396 $\pm$ 0.026	0.406 $\pm$ 0.056	0.447 $\pm$ 0.026	0.457 $\pm$ 0.061
ETTm2	0.280 $\pm$ 0.032	0.325 $\pm$ 0.031	0.420 $\pm$ 0.035	0.424 $\pm$ 0.044	0.283 $\pm$ 0.020	0.327 $\pm$ 0.026	0.633 $\pm$ 0.055	0.489 $\pm$ 0.041

Table 1: Multivariate forecasting results (average) with forecasting lengths

H\in\{96,192,336,720\}

for all datasets and fixed input sequence length

L=96

Models	MLP-based (DLinear ( Zeng et al., 2023 ) )										Transformer-based (iTransformer ( Liu et al., 2024b ) )
Methods	+ TIFO *		+ TIFO		+SAN		+ FAN		+ RevIN		+ TIFO *		+ TIFO		+SAN		+ FAN		+ RevIN
Metric	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTh1	0.413	0.424	0.407	0.419	0.421	0.427	0.469	0.473	0.460	0.456	0.455	0.449	0.445	0.443	0.466	0.455	0.504	0.488	0.463	0.452
ETTh2	0.339	0.384	0.337	0.384	0.342	0.387	0.459	0.469	0.561	0.518	0.378	0.408	0.376	0.400	0.392	0.413	0.543	0.515	0.385	0.412
ETTm1	0.341	0.372	0.357	0.375	0.344	0.376	0.367	0.392	0.413	0.407	0.389	0.398	0.396	0.406	0.401	0.406	0.424	0.428	0.406	0.410
ETTm2	0.255	0.316	0.256	0.313	0.260	0.318	0.278	0.339	0.350	0.413	0.285	0.334	0.283	0.327	0.287	0.336	0.337	0.385	0.294	0.337

Table 2: Multivariate forecasting results (average) with

H\in\{96,192,336,720\}

for all datasets and fixed input sequence length

L=96

. The best and second best results are highlighted. TIFO * represents the results where both TIFO and SAN are used in the backbones.

Figure 4: Frequency-domain analysis of Fourier basis Learning. From left to right: unprocessed spectra (Before), after FAN, and after applying TIFO . Each panel is a 3D box with time, frequency, and amplitude axes. We visualize four Fourier basis waves, $\zeta_{\omega_{1:4}}$ , to illustrate how each processing method alters the basis functions. In the frequency–amplitude plane, we plot three forecasting cases: the ground truth in blue and the forecasting results based on the processed input in red. The red diamonds mark key local peak frequencies.

Table 6

Table 5: Ablation study on frequency domain modeling on ETTh1 with iTransformer. (a) Impact of window functions and resolution under the DFT setting. (b) Comparison of reconstruction strategies: DFT vs. Short-Time Fourier Transform (STFT). The default settings with a rectangular window achieves the best performance.

Figure 8

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Illustration of z-score normalization in both time and frequency domains. (Top) Data generating distributions $p(x|t_{i})$ across different temporal structures $t_{i}$ are aligned after $z$ -score, sharing a common location and scale. (Bottom) Frequency-domain power spectra before and after $z$ -score. The frequency-domain distribution remains divergent after normalization.

Metric	Description	MSE	MAE
mu_sigma ( Ours )	Mean/Std (Unsupervised, Cross-sample stability)	0.3904	0.4071
entropy	Spectral Entropy (Intra-sample complexity)	0.3962	0.4106
corr	Correlation $\|r\|$ (Target-aware, MI approx.)	0.3948	0.4077

Appendix

Table 6: Comparison of different stationarity metrics on ETTh1 with iTransformer. mu_sigma denotes our default unsupervised metric; entropy focuses on intra-sample distribution; corr utilizes future labels to approximate Mutual Information.

Dataset	Source	Resolution	Channels	Time Range
Weather	Autoformer ( Wu et al., 2021 )	Every 10 minutes	21 (e.g., temperature, humidity)	2020
ETTh1	Informer ( Zhou et al., 2021 )	Hourly	7 states of a electrical transformer	2016-2017
ETTh2	Informer ( Zhou et al., 2021 )	Hourly	7 states of a electrical transformer	2017-2018
ETTm1	Informer ( Zhou et al., 2021 )	Every 15 minutes	7 states of a electrical transformer	2016-2017
ETTm2	Informer ( Zhou et al., 2021 )	Every 15 minutes	7 states of a electrical transformer	2017-2018
Electricity	UCI ML Repository	Hourly	321 clients’ consumption	2012-2014

Appendix

Table 7: Overview of Datasets

Models		DLinear ( Zeng et al., 2023 )								PatchTST ( Nie et al., 2023 )								iTransformer ( Liu et al., 2024b )
Methods		+ Ours*		+ Ours		+ SAN		+ RevIN		+ Ours*		+ Ours		+ SAN		+ RevIN		+ Ours*		+ Ours		+ SAN		+ RevIN
Metric		MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
Electricity	96	0.135	0.230	0.140	0.237	0.137	0.234	0.210	0.278	0.175	0.266	0.190	0.280	0.182	0.271	0.212	0.297	0.145	0.244	0.143	0.237	0.171	0.262	0.152	0.251
	192	0.149	0.245	0.155	0.249	0.151	0.247	0.210	0.304	0.183	0.273	0.195	0.286	0.186	0.276	0.213	0.300	0.169	0.266	0.159	0.252	0.180	0.270	0.165	0.255
	336	0.165	0.262	0.171	0.267	0.166	0.264	0.223	0.309	0.198	0.289	0.211	0.301	0.200	0.290	0.227	0.314	0.178	0.271	0.172	0.266	0.194	0.284	0.180	0.272
	720	0.198	0.291	0.208	0.298	0.201	0.295	0.257	0.349	0.233	0.317	0.253	0.334	0.237	0.322	0.268	0.344	0.210	0.311	0.205	0.295	0.237	0.319	0.227	0.312

Appendix

Table 8: Detailed results of comparing our proposal and other normalization methods. The best results are highlighted in bold . The second best are underlined .

Models		DLinear ( Zeng et al., 2023 )								iTransformer ( Liu et al., 2024b )
Methods		+ TIFO *		+ TIFO		+ FAN		+ RevIN		+ TIFO *		+ TIFO		+ FAN		+ RevIN
Metric		MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
Electricity	96	0.135	0.230	0.140	0.237	0.146	0.248	0.210	0.278	0.145	0.244	0.143	0.237	0.158	0.254	0.152	0.251
	192	0.149	0.245	0.155	0.249	0.163	0.264	0.210	0.304	0.169	0.266	0.159	0.252	0.170	0.263	0.165	0.255
	336	0.165	0.262	0.171	0.267	0.180	0.282	0.223	0.309	0.178	0.271	0.172	0.266	0.183	0.281	0.180	0.272
	720	0.198	0.291	0.208	0.298	0.216	0.316	0.257	0.349	0.210	0.311	0.205	0.295	0.208	0.306	0.227	0.312

Appendix

Table 9: Detailed results of comparing TIFO and FAN. The best results are highlighted in bold . The second best are underlined .

Figure 6: More results on train-test distance compactness: This figure shows another visualization of the JSD 2 amplitudes distribution distance between the train and test datasets on the electricity data (on a different channel). Each scatter point represents one frequency component. A smaller radius indicates a smaller distributional gap. Green and red colors represent the results before and after applying the learning method, respectively.

Figure 7: Frequency-domain analysis of Fourier basis Learning. ETTh1 dataset, Channel #1.

Figure 8: Frequency-domain analysis of Fourier basis Learning. ETTm1 dataset, Channel #2.

Figure 9: Frequency-domain analysis of Fourier basis Learning. Traffic dataset, Channel #237.

Figure 10: Frequency-domain analysis of Fourier basis Learning. ETTh1 dataset, Channel #6.

Figure 11: Frequency-domain analysis of Fourier basis Learning. Weather dataset, Channel #7.

Model	Electricity		ETTh1		ETTh2		ETTm1		ETTm2		Traffic		Weather
H	96	720	96	720	96	720	96	720	96	720	96	720	96	720
DLinear ( Zeng et al., 2023 )
+ TIFO	16.718	27.545	3.004	3.082	2.820	3.222	13.456	13.124	11.839	12.949	23.217	43.413	13.970	16.536
+ SAN	19.159	34.914	8.688	8.054	8.373	7.811	36.706	36.564	36.620	36.550	31.378	54.518	37.739	40.731
IMP(%)	12.8%	21.1%	65.5%	60.5%	66.6%	58.3%	63.1%	64.2%	67.8%	64.3%	26.0%	20.3%	63.0%	59.9%
PatchTST ( Nie et al., 2023 )

Appendix

Table 10: Running time comparison with forecasting lengths

H\in\{96,720\}

for all datasets and fixed input sequence length

L=96

. The best results are highlighted.

Setting	Run 0		Run 1		Run 2		Average
Setting	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
rect_K48	0.4250	0.4420	0.4020	0.4180	0.4050	0.4210	0.4107	0.4270
rect_K96	0.3902	0.4068	0.3939	0.4105	0.3974	0.4140	0.3938	0.4104
hann_K48	0.4130	0.4260	0.4100	0.4240	0.4120	0.4250	0.4117	0.4250
hann_K96	0.4110	0.4240	0.4090	0.4230	0.4130	0.4260	0.4110	0.4243

Appendix

Table 11: Ablation on window function and FFT resolution on ETTh1 with iTransformer. The best results are highlighted.

$\alpha$	H = 96		H = 192		H = 336		H = 720		Average
$\alpha$	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
0.00	0.4590	0.4740	0.5170	0.5100	0.5620	0.5330	0.5660	0.5520	0.5260	0.5172
0.25	0.4540	0.4690	0.5120	0.5050	0.5570	0.5280	0.5610	0.5470	0.5210	0.5122
0.50	0.4190	0.4340	0.4770	0.4700	0.5220	0.4930	0.5260	0.5120	0.4860	0.4773
0.75	0.3900	0.4050	0.4440	0.4380	0.4940	0.4650	0.4990	0.4840	0.4567	0.4480
1.00	0.3780	0.3930	0.4460	0.4390	0.4810	0.4520	0.4850	0.4710	0.4475	0.4388

Appendix

Table 12: Effect of scaling the learned TIFO weights at test time by

\alpha

on ETTh1 with iTransformer. The best average performance is highlighted.

Setting	Run 0		Run 1		Run 2		Average
Setting	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
Ori	0.3902	0.4068	0.3939	0.4105	0.3974	0.4140	0.3938	0.4104
hann_50_ola	0.5887	0.5148	0.5623	0.5053	0.5951	0.5152	0.5820	0.5118
sqrt_hann_50_wola	0.3999	0.4135	0.3985	0.4133	0.4032	0.4175	0.4005	0.4148
hann_75_ola	0.5872	0.5130	0.5627	0.5050	0.5944	0.5149	0.5814	0.5110

Appendix

Table 13: Comparison of reconstruction strategies (global DFT vs. OLA/WOLA) on ETTh1 with iTransformer. The best results are highlighted.

Setting	Run 0		Run 1		Run 2		Average
Setting	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
no_update	0.3902	0.4068	0.3939	0.4105	0.3974	0.4140	0.3938	0.4104
ema_0.9	0.3916	0.4101	0.3930	0.4105	0.3952	0.4136	0.3932	0.4114
ema_0.99	0.3903	0.4085	0.3938	0.4110	0.3966	0.4146	0.3936	0.4114

Appendix

Table 14: Effect of online EMA refresh of frequency masks during inference on ETTh1 with iTransformer. The best results are highlighted.

Setting	Run 0		Run 1		Run 2		Average
Setting	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
hop0.125_zpad0.0	0.3950	0.4100	0.3980	0.4130	0.3990	0.4140	0.3973	0.4123
hop0.125_zpad0.5	0.3920	0.4070	0.3940	0.4090	0.3950	0.4100	0.3937	0.4087
hop0.125_zpad1.0	0.3930	0.4080	0.3950	0.4100	0.3960	0.4110	0.3947	0.4097
hop0.25_zpad0.0	0.3970	0.4120	0.4000	0.4150	0.4020	0.4170	0.3997	0.4147
hop0.25_zpad0.5	0.3950	0.4100	0.3970	0.4120	0.3980	0.4130	0.3967	0.4117

Appendix

Table 15: Sensitivity to STFT hyperparameters (hop ratio and zero-padding) on ETTh1 with iTransformer. The best results are highlighted.

Frequency-Domain Time Series Modeling

See all 22 papers

FAiT: Frequency-Aware Inverted Transformer for Multivariate Time Series Forecasting
May 31, 2026Peng He, Yao Liu, Yanglei Gan +3Multivariate Time Series Forecasting Frequency-Domain Feature Learning
University of Electronic Science and Technology of China · Chengdu, Sichuan, China
FreqLite: A Lightweight Frequency-Decomposed Linear Model with Adaptive Reversible Normalization for Robust Long-Term Time-Series Forecasting
May 31, 2026Mirza Samad Ahmed Baig, Syeda Anshrah GillaniLong-Term Time Series Forecasting Frequency-Domain Feature Learning
Fandaqah, Al Khobar, Saudi Arabia · Hamdard University, Karachi, Pakistan
Towards Principled Test-Time Adaptation for Time Series Forecasting
May 17, 2026Haochun Wang, Ruichen Xu, Georgios Kementzidis +3Test-Time Adaptation Time Series Forecasting
Stony Brook University

May 31, 2026J/K move · Enter open · S save

FAiT: Frequency-Aware Inverted Transformer for Multivariate Time Series Forecasting

Peng He, Yao Liu, Yanglei Gan, Run Lin+2

University of Electronic Science and Technology of China · Chengdu, Sichuan, China

Multivariate Time Series Forecasting Frequency-Domain Feature Learning Transformer Attention Time Series Transformers Multiscale Time-Series Modeling

Open paper AskPDF arXiv

While Transformer-based architectures have established themselves as a dominant paradigm in Multivariate Time Series Forecasting (MTSF), their core self-attention mechanism inherently functions as a low-pass filter, systematically smoothing out high-frequency signals vital for sharp local changes. Recent advancements have increasingly incorporated frequency-domain operations to address this bias, however, most existing designs rely on fixed spectral bases and apply sequence-wise (uniform) modulation, implicitly assuming a time-invariant frequency response. This overlooks a key property of real-world series that their spectral characteristics often evolve over time, making uniform modulation insufficient for capturing fine-grained temporal dynamics. To tackle these limitations, we propose FAiT, a Frequency-Aware inverted Transformer. Specifically, FAiT rectifies the spectral bias internally through Inverted Attention, which interprets the attention map as a learnable low-pass operator and constructs a dedicated complementary high-pass branch by inverting the attention matrix to recover attenuated transient signals. Furthermore, FAiT introduces Dynamic Temporal-Frequency Modulation (DTFM), which synthesizes instance-conditioned weights to adaptively re-calibrate the energy of spectral sub-bands, enabling fine-grained control over evolving multi-scale patterns. Extensive experiments on widely used benchmarks demonstrate that FAiT consistently outperforms state-of-the-art Transformer-based and frequency-enhanced baselines, while maintaining computational efficiency.

Abstract

Frequency-Domain Time Series Modeling

Explore similar work

FAiT: Frequency-Aware Inverted Transformer for Multivariate Time Series Forecasting

FreqLite: A Lightweight Frequency-Decomposed Linear Model with Adaptive Reversible Normalization for Robust Long-Term Time-Series Forecasting

Towards Principled Test-Time Adaptation for Time Series Forecasting