Rethinking Time Series Tokenization from a Frequency Perspective

Zelong Tian; Ming Jin; Bo Du; Shirui Pan

cs.LGOct 20, 2026

Rethinking Time Series Tokenization from a Frequency Perspective

Authors: Zelong Tian, Ming Jin, Bo Du, Shirui Pan

Organizations: Griffith University

Abstract

Tokenization that partitions time series into subsequences has become a foundational paradigm in modern time series modeling. However, we find that current tokenization strategies inherently introduce spectral distortions, forcing the learned representations to diverge from the true underlying patterns such as trend, periodicity, and seasonality, thereby impairing model performance. Through theoretical analysis from a frequency perspective, we derive the boundary conditions under which such distortions occur. To overcome this distortion, we propose TimeFT, a nearly distortion-free frequency-based tokenizer for time series data. TimeFT obtains tokens through frequency-domain partitioning, frequency shifting, and Nyquist sampling. The method is parameter-free, incurs negligible computational complexity, and can serve as a drop-in replacement for existing tokenizers. Our extensive experiments on diverse tasks such as forecasting, classification, and anomaly detection demonstrate that TimeFT consistently and significantly improves performance.

Figures & tables

Figure 1: The key distinction between existing tokenization and TimeFT. Left: The processes of patch and phase tokenization in the time domain. Right: Spectral shifting moves the frequency spectrum, preventing aliasing by ensuring all components satisfy the Nyquist criterion.

Figure 2: The comparison of tokenization methods across the time domain, frequency domain and reconstruction versus origin. (a) Patch tokens lead to a completely different frequency spectrum and cause the loss of all information outside the rectangular window. (b) Phase tokens cause frequency components above the Nyquist frequency to completely alias into the low frequencies, thereby exhibiting erroneous patterns. (c) Our TimeFT effectively solves the above issues and achieves almost distortion - free tokenization. The further explanation of Figure 2 and the details of reconstruction methods are provided in the Appendix C .

Figure 3: The pipeline of TimeFT is implemented by (1) frequency domain decomposition, (2) spectral shifting and (3) information-preserving token extraction. Different colors and markers are used to denote the processes of operations. Based on the frequency threshold

B_{N}

derived from downsampling without information loss, TimeFT shifts all spectra of bandwidth

B_{N}

to the baseband, and then obtains almost distortion-free tokens by Nyquist sampling.

Models	TimeFT		EntroPE		TimeMosaic		PhaseFormer		TimeFilter		xPatch		PatchMLP		iTransformer		TimeMixer		PatchTST
	( Ours )		(2026)		(2026)		(2026)		(2025)		(2025)		(2025)		(2024)		(2024)		(2023)
Metric	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTm1	0.348	0.373	0.369	0.383	0.360	0.377	0.364	0.392	0.399	0.408	0.354	0.386	0.374	0.396	0.386	0.406	0.356	0.385	0.372	0.399
ETTm2	0.255	0.307	0.264	0.320	0.256	0.310	0.268	0.324	0.278	0.330	0.262	0.322	0.269	0.324	0.293	0.338	0.265	0.323	0.278	0.332
ETTh1	0.417	0.423	0.424	0.433	0.412	0.423	0.489	0.475	0.511	0.482	0.446	0.445	0.462	0.460	0.487	0.475	0.430	0.438	0.503	0.485
ETTh2	0.358	0.390	0.374	0.401	0.363	0.393	0.384	0.417	0.402	0.421	0.354	0.397	0.380	0.416	0.411	0.425	0.358	0.396	0.398	0.427

Table 1: For long-term forecasting task , we evaluate average MSE and MAE across prediction horizons

H\in\{96,192,336,720\}

. All baselines use a fixed input sequence length of 320. Best results are in bold and second best are underlined . The detailed performance is provided in Table 8 .

Models	TimeFT	EntroPE	TimeMosaic	PhaseFormer	PatchMLP	GPT4TS	PatchTST	TimesNet	MLP
	( Ours )	(2026)	(2026)	(2026)	(2025)	(2024)	(2023)	(2023)	-
Average	71.31	70.07	68.82	67.19	69.42	58.51	69.13	66.55	65.60

Table 2: For classification task , we evaluate accuracy (%) across all UEA datasets. Best results are in bold and second best are underlined . The detailed performance is provided in Table 9 .

Models	TimeFT	EntroPE	TimeMosaic	PhaseFormer	LightTS	TimesNet	PatchTST	DLinear	ETSformer	FEDformer
	( Ours )	(2026)	(2026)	(2026)	(2024)	(2023)	(2023)	(2023)	(2023)	(2022)
Average	86.12	85.38	82.50	83.01	84.23	85.24	82.79	82.46	82.87	84.97

Table 3: For anomaly detection task , we evaluate F1-score (%) on five benchmark datasets. Best results are in bold and second best are underlined . The detailed performance is provided in Table 11 .

Datasets	ETTm1	ETTm2	ETTh1	ETTh2	Weather	Electricity	Traffic
TimeFT (s)	1.492	1.376	0.352	0.362	3.799	28.50	50.27
Training (s)	299.5	177.5	80.55	127.8	833.7	2157	3629
Percentage	0.50%	0.78%	0.44%	0.28%	0.46%	1.32%	1.38%

Table 4: Comparison of TimeFT tokenization time and training time per epoch across different datasets. The percentage represents the proportion of TimeFT tokenization time relative to the training time per epoch.

Datasets	ETTm1		ETTm2		ETTh1		ETTh2		Weather		Electricity		Traffic
	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
PatchTST	0.321	0.361	0.181	0.270	0.403	0.423	0.324	0.374	0.155	0.204	0.142	0.245	0.403	0.285
+TimeFT	0.290	0.339	0.162	0.244	0.376	0.392	0.297	0.339	0.147	0.194	0.139	0.234	0.395	0.279
$\Delta\%$ Improvement	9.7%	6.1%	10.5%	9.6%	6.7%	7.3%	8.3%	9.4%	5.2%	4.9%	2.1%	4.5%	2.0%	2.1%
PhaseFormer	0.312	0.352	0.174	0.261	0.389	0.411	0.309	0.361	0.152	0.201	0.145	0.248	0.418	0.289
+TimeFT	0.293	0.341	0.161	0.249	0.379	0.397	0.299	0.343	0.149	0.193	0.143	0.242	0.406	0.278

Table 5: Plug-in performance on long-term forecasting task. Evaluation of TimeFT as a plug-in tokenization module on prediction horizon

H=96

and input length of 320. Experiments on more architectures are provided in Table 14 .

Datasets	ETTm1		ETTm2		ETTh1		ETTh2		Weather		Electricity		Traffic
	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
TimeFT	0.290	0.339	0.162	0.244	0.376	0.392	0.297	0.339	0.147	0.194	0.139	0.234	0.395	0.279
w/o Stage 1	0.328	0.364	0.175	0.261	0.412	0.421	0.315	0.358	0.162	0.211	0.151	0.255	0.408	0.288
w/o Stage 2	0.314	0.352	0.194	0.282	0.398	0.415	0.342	0.384	0.158	0.207	0.146	0.249	0.423	0.301
w/o Stage 3	0.375	0.412	0.228	0.315	0.482	0.485	0.396	0.432	0.189	0.242	0.178	0.294	0.475	0.342

Table 6: Ablation study on long-term forecasting task on prediction horizon

H=96

and input length of 320. The “w/o” indicates that the operations of the corresponding stage in Figure 3 are skipped in the pipeline, while the rest remains unchanged. Stage 1 is frequency domain decomposition, Stage 2 is spectral shifting and Stage 3 is information-preserving token extraction.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Illustration of spectral leakage and weak-signal masking caused by rectangular window truncation. (a) Rectangular window truncation of a time-domain signal. (b) Spectral characteristics of the Dirichlet kernel corresponding to the rectangular window. (c) Original spectrum containing adjacent strong and weak signals. (d) Masking of weak signals caused by sidelobe leakage after windowing. The decibel (dB) is a logarithmic unit, so a small difference can represent a large change.

Figure 7: Illustration of the picket - fence effect. (a) Picket-fence effect at linear magnitude. (b) Picket-fence effect at dB magnitude.

Figure 8: Illustration of spectral aliasing in the frequency domain. When the original spectrum is not confined to

(-\pi/S,\pi/S)

, high frequency components appear as spurious low frequency components.

Dataset	Dim	Dataset Size (Train, Val, Test)	Frequency	Domain Information
ETTm1	7	(34465, 11521, 11521)	15min	Energy Infrastructure
ETTm2	7	(34465, 11521, 11521)	15min	Energy Infrastructure
ETTh1	7	(8545, 2881, 2881)	Hourly	Energy Infrastructure
ETTh2	7	(8545, 2881, 2881)	Hourly	Energy Infrastructure
Weather	21	(36792, 5271, 10540)	10min	Meteorological Monitoring
Electricity	321	(18317, 2633, 5261)	Hourly	Electricity Consumption

Appendix

Table 7: Summary of datasets for long-term forecasting task. Dim denotes dimension, which is the variate number of each dataset. Dataset sizes represent the number of temporal observations in each partition (Training, Validation, Test). Frequency denotes the sampling interval of time points.

Models		TimeFT		EntroPE		TimeMosaic		PhaseFormer		TimeFilter		xPatch		PatchMLP		LightTS		iTransformer		TimeMixer		PatchTST		DLinear
Models		( Ours )		(2026)		(2026)		(2026)		(2025)		(2025)		(2025)		(2024)		(2024)		(2024)		(2023)		(2023)
Metric		MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTm1	96	0.290	0.339	0.316	0.349	0.304	0.344	0.312	0.352	0.355	0.373	0.293	0.347	0.305	0.357	0.313	0.363	0.327	0.371	0.307	0.352	0.321	0.361	0.299	0.344
	192	0.325	0.357	0.344	0.371	0.339	0.364	0.339	0.381	0.376	0.396	0.332	0.371	0.351	0.384	0.350	0.387	0.357	0.388	0.332	0.373	0.342	0.385	0.337	0.370
	336	0.360	0.381	0.386	0.394	0.375	0.385	0.375	0.398	0.401	0.413	0.364	0.395	0.387	0.403	0.399	0.418	0.393	0.412	0.362	0.390	0.386	0.410	0.369	0.385
	720	0.418	0.415	0.431	0.419	0.422	0.413	0.428	0.437	0.465	0.449	0.428	0.432	0.454	0.441	0.489	0.479	0.467	0.452	0.422	0.426	0.439	0.441	0.424	0.419

Appendix

Table 8: For long-term forecasting, we evaluate prediction horizons

H\in\{96,192,336,720\}

. All baselines use a fixed input sequence length of

320

. The term “Avg.” refers to the average metric over the four horizons. The best and second-best results are marked as best and second , respectively. The column “

1^{st}

Count” reports the number of times each method ranks first among all competitors.

Datasets	TimeFT	EntroPE	TimeMosaic	PhaseFormer	PatchMLP	GPT4TS	PatchTST	TimesNet	MLP
	( Ours )	(2026)	(2026)	(2026)	(2025)	(2024)	(2023)	(2023)	-
ArticularyWordRecognition	97.50	97.00	96.50	95.00	97.20	93.33	97.67	96.18	97.33
AtrialFibrillation	53.00	52.00	50.50	48.00	53.00	33.33	53.33	33.33	46.66
BasicMotions	98.00	98.00	97.00	95.50	98.00	92.50	92.50	100.00	85.00
Cricket	91.00	90.00	89.00	87.00	90.50	8.33	84.72	87.50	91.67
Epilepsy	85.00	84.00	82.50	80.00	84.50	85.51	65.94	78.13	60.14

Appendix

Table 9: We evaluate classification tasks on all UEA datasets. Best results are in bold and second best are underlined .

Dataset	Dim	Dataset Size (Train, Val, Test)	Length	Domain Information
SMD	38	(566724, 141681, 708420)	100	Server Machine
MSL	55	(44653, 11664, 73729)	100	Spacecraft
SMAP	25	(108146, 27037, 427617)	100	Spacecraft
SWaT	51	(396000, 99000, 449919)	100	Infrastructure
PSM	25	(105984, 26497, 87841)	100	Server Machine

Appendix

Table 10: Summary of datasets used for anomaly detection evaluation. Dim denotes the variate number of each dataset. Dataset sizes represent the number of temporal observations in each partition (Training, Validation, Test). Frequency denotes the sampling interval of time points.

Models	TimeFT	EntroPE	TimeMosaic	Phaseformer	LightTS	TimesNet	PatchTST	DLinear	ETSformer	FEDformer
	( Ours )	(2026)	(2026)	(2026)	(2024)	(2023)	(2023)	(2023)	(2023)	(2022)
SMD	85.52	84.75	84.25	84.85	82.53	84.61	84.62	77.10	83.13	85.08
MSL	82.95	81.95	78.42	79.15	78.95	81.84	78.70	84.88	85.03	78.57
SMAP	71.80	69.80	68.10	68.75	69.21	69.39	68.82	69.26	69.50	70.76
SWaT	93.65	93.10	85.90	86.12	93.33	93.02	85.72	87.52	84.91	93.19
PSM	96.68	97.28	95.85	96.20	97.15	97.34	96.08	93.55	91.76	97.23

Appendix

Table 11: For anomaly detection task, we evaluate F1-score (%) on five benchmark datasets. Best results are in bold and second best are underlined .

Noise ( $\sigma$ )	0		0.2		0.4		0.6		0.8		1.0
Metric	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
PatchTST	0.321	0.361	0.343	0.377	0.368	0.394	0.397	0.419	0.440	0.447	0.483	0.485
PhaseFormer	0.312	0.352	0.330	0.364	0.350	0.381	0.382	0.395	0.414	0.426	0.458	0.449
TimeFT (ours)	0.290	0.339	0.302	0.347	0.318	0.355	0.335	0.374	0.364	0.387	0.391	0.412

Appendix

Table 12: Long-term forecasting results on ETTm1 under Gaussian noise.

\sigma

is the standard deviation of the noise.

Noise ( $\sigma$ )	0.2	0.4	0.6	0.8	1.0
PatchTST	6.85%	14.64%	23.68%	37.07%	50.47%
PhaseFormer	5.77%	12.18%	22.44%	32.69%	46.79%
TimeFT (ours)	4.14%	9.66%	15.52%	25.52%	34.83%

Appendix

Table 13: Degradation Rate of the long-term forecasting task under Gaussian noise. Metric: MSE.

Dataset	ETTm1		ETTm2		ETTh1		ETTh2		Weather		Electricity		Traffic
Metric	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
PatchTST	0.321	0.361	0.181	0.270	0.403	0.423	0.324	0.374	0.155	0.204	0.142	0.245	0.403	0.285
+TimeFT	0.290	0.339	0.162	0.244	0.376	0.392	0.297	0.339	0.147	0.194	0.139	0.234	0.395	0.279
$\Delta$ % Improvement	9.7%	6.1%	10.5%	9.6%	6.7%	7.3%	8.3%	9.4%	5.2%	4.9%	2.1%	4.5%	2.0%	2.1%
PhaseFormer	0.312	0.352	0.174	0.261	0.389	0.411	0.309	0.361	0.152	0.201	0.145	0.248	0.418	0.289
+TimeFT	0.293	0.341	0.161	0.249	0.379	0.397	0.299	0.343	0.149	0.193	0.143	0.242	0.406	0.278

Appendix

Table 14: Plug-in experiments of the long-term forecasting task to show the general capability. All settings are the same as Table 5 .

Models	TimeFT (ours)		EntroPE		TimeMosaic		PhaseFormer
Metric	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTm1	0.348 $\pm$ 0.005	0.373 $\pm$ 0.004	0.369 $\pm$ 0.007	0.383 $\pm$ 0.006	0.360 $\pm$ 0.004	0.377 $\pm$ 0.008	0.364 $\pm$ 0.006	0.392 $\pm$ 0.005
ETTm2	0.255 $\pm$ 0.003	0.307 $\pm$ 0.004	0.264 $\pm$ 0.006	0.320 $\pm$ 0.005	0.256 $\pm$ 0.003	0.310 $\pm$ 0.007	0.268 $\pm$ 0.009	0.324 $\pm$ 0.006
ETTh1	0.417 $\pm$ 0.006	0.423 $\pm$ 0.005	0.424 $\pm$ 0.008	0.433 $\pm$ 0.004	0.412 $\pm$ 0.005	0.423 $\pm$ 0.007	0.489 $\pm$ 0.009	0.475 $\pm$ 0.004
ETTh2	0.358 $\pm$ 0.004	0.390 $\pm$ 0.007	0.374 $\pm$ 0.008	0.401 $\pm$ 0.005	0.363 $\pm$ 0.003	0.393 $\pm$ 0.006	0.384 $\pm$ 0.006	0.417 $\pm$ 0.005
Weather	0.232 $\pm$ 0.003	0.261 $\pm$ 0.005	0.237 $\pm$ 0.005	0.264 $\pm$ 0.007	0.228 $\pm$ 0.004	0.256 $\pm$ 0.008	0.233 $\pm$ 0.005	0.268 $\pm$ 0.009

Appendix

Table 15: Long-term forecasting results under different random seeds to show the effect of random seeds.

Layers	TimeFT Embedding Layer	PatchTST Embedding Layer	Total of Model
Params (MB)	0.699	0.031	50.337

Appendix

Table 16: Variation in Model Parameter Count.

Datasets	ETTm1	ETTm2	ETTh1	ETTh2	Weather	Electricity	Traffic
Patch	0.768	0.747	0.755	0.722	0.726	0.821	0.719
Phase	0.449	0.525	0.823	0.714	0.280	0.998	0.986
TimeFT	0.102	0.115	0.144	0.133	0.079	0.122	0.138
Reduction	77.3%	78.1%	80.9%	81.4%	71.8%	85.1%	80.8%

Appendix

Table 17: Reconstruction performance of different token types across different datasets. The percentage represents the MAE reduction of TimeFT tokens relative to the best one of others. Metric: MAE.

Figure 9: The visualization of tokenization methods across the time domain, frequency domain and reconstruction vs origin in ETTh1 dataset.

Figure 10: The visualization of tokenization methods across the time domain, frequency domain and reconstruction vs origin in ETTh2 dataset.

Figure 11: The visualization of tokenization methods across the time domain, frequency domain and reconstruction vs origin in ETTm1 dataset.

Figure 12: The visualization of tokenization methods across the time domain, frequency domain and reconstruction vs origin in ETTm2 dataset.

Figure 13: The visualization of tokenization methods across the time domain, frequency domain and reconstruction vs origin in Weather dataset.

Figure 14: The visualization of tokenization methods across the time domain, frequency domain and reconstruction vs origin in Electricity dataset.

Figure 15: The visualization of tokenization methods across the time domain, frequency domain and reconstruction vs origin in Traffic dataset.

Time Series Tokenization

See all 13 papers

TimeSqueeze: Dynamic Patching for Efficient Time Series Forecasting
Mar 11, 2026Sravan Kumar Ankireddy, Nikita Seleznev, Nam H. Nguyen +4Time Series Forecasting Time Series Transformers
University of Texas at Austin, Austin, TX, USA · Capital One, USA · University of Maryland, College Park, MD, USA
Beyond Uniform Tokens: Adaptive Compression for Time Series Language Models
Jun 11, 2026Jialin Gan, Xin Qiu, Guangzhe Chen +1LLM Inference Acceleration Multimodal Token Compression
Zhejiang University · Harbin Institute of Technology · Shandong University
Time Series as Language: A Universal Tokenizer for General-Purpose Time Series Foundation Models
May 31, 2026Yunhao Zhang, Ruiying Qi, Jiale Zheng +3Time Series Tokenization Time Series Generation
Shanghai Jiao Tong University · Huawei Noah’s Ark Lab

Mar 11, 2026J/K move · Enter open · S save

TimeSqueeze: Dynamic Patching for Efficient Time Series Forecasting

Sravan Kumar Ankireddy, Nikita Seleznev, Nam H. Nguyen, Yulun Wu+3

University of Texas at Austin, Austin, TX, USA · Capital One, USA · University of Maryland, College Park, MD, USA

Time Series Forecasting Time Series Transformers Time-Series Foundation Models

Open paper AskPDF arXiv

Transformer-based time series foundation models face a fundamental trade-off in choice of tokenization: point-wise embeddings preserve temporal fidelity but scale poorly with sequence length, whereas fixed-length patching improves efficiency by imposing uniform boundaries that may disrupt natural transitions and blur informative local dynamics. In order to address these limitations, we introduce TimeSqueeze, a dynamic patching mechanism that adaptively selects patch boundaries within each sequence based on local signal complexity. TimeSqueeze first applies a lightweight state-space encoder to extract full-resolution point-wise features, then performs content-aware segmentation by allocating short patches to information-dense regions and long patches to smooth or redundant segments. This variable-resolution compression preserves critical temporal structure while substantially reducing the token sequence presented to the Transformer backbone. Specifically for large-scale pretraining, TimeSqueeze attains up to 20x faster convergence and 8x higher data efficiency compared to equivalent point-token baselines. Experiments across long-horizon forecasting benchmarks show that TimeSqueeze consistently outperforms comparable architectures that use either point-wise tokenization or fixed-size patching.

Abstract

Time Series Tokenization

Explore similar work

TimeSqueeze: Dynamic Patching for Efficient Time Series Forecasting

Beyond Uniform Tokens: Adaptive Compression for Time Series Language Models

Time Series as Language: A Universal Tokenizer for General-Purpose Time Series Foundation Models