Harmonic Dataset Distillation for Time Series Forecasting

Seungha Hong; Sanghwan Jang; Wonbin Kweon; Suyeon Kim; Gyuseok Lee; Hwanjo Yu

cs.LGMar 4, 2026

Harmonic Dataset Distillation for Time Series Forecasting

Authors: Seungha Hong, Sanghwan Jang, Wonbin Kweon, Suyeon Kim, Gyuseok Lee, Hwanjo Yu

Organizations: Pohang University of Science and Technology (POSTECH), Republic of Korea · University of Illinois Urban-Champaign (UIUC), United States

Abstract

Time Series forecasting (TSF) in the modern era faces significant computational and storage cost challenges due to the massive scale of real-world data. Dataset Distillation (DD), a paradigm that synthesizes a small, compact dataset to achieve training performance comparable to that of the original dataset, has emerged as a promising solution. However, conventional DD methods are not tailored for time series and suffer from architectural overfitting and limited scalability. To address these issues, we propose Harmonic Dataset Distillation for Time Series Forecasting (HDT). HDT decomposes the time series into its sinusoidal basis through the FFT and aligns the core periodic structure by Harmonic Matching. Since this process operates in the frequency domain, all updates during distillation are applied globally without disrupting temporal dependencies of time series. Extensive experiments demonstrate that HDT achieves strong cross-architecture generalization and scalability, validating its practicality for large-scale, real-world applications.

Figures & tables

Figure 1: Illustrative examples of distilling an original dataset $\mathcal{X}$ into a synthetic dataset $\mathcal{S}$ with (a) Window-based and (b) Harmonic Dataset Distillation for TSF. In (a), windows randomly sampled from $\mathcal{X}$ are distilled into arbitrary positions within $\mathcal{S}$ . In (b), the sequence is decomposed into a sinusoidal basis, and selected harmonics ( $\mathcal{H}_{i}$ ) between $\mathcal{X}$ and $\mathcal{S}$ are aligned during distillation.

Figure 2: An overview of our method. Selected harmonics (red box) of synthetic data are updated through Harmonic Matching and Gradient Matching .

Backbone: DLinear
Dataset	ETTh1			ETTh2			ETTm1			ETTm2			Electricity			Traffic
	L	T	C	L	T	C	L	T	C	L	T	C	L	T	C	L	T	C
Random	0.945	0.757	0.664	1.860	0.406	0.359	0.919	0.732	0.666	1.504	0.256	0.234	0.400	0.327	0.351	1.112	0.938	0.911
DC	1.193	0.853	0.721	2.200	0.410	0.363	0.801	0.688	0.659	1.670	0.266	0.234	0.413	0.333	0.351	1.122	0.945	0.924
MTT	0.521	0.640	0.587	0.661	0.387	0.346	0.493	0.879	1.342	0.702	0.257	0.248	0.342	0.412	0.489	0.747	1.204	0.852
TESLA	0.576	0.535	0.530	0.724	0.381	0.348	0.443	0.475	0.438	0.428	0.231	0.220	0.388	0.413	0.394	1.087	0.956	0.953

Table 1: Overall dataset distillation performance in terms of MSE. The synthetic data size

M

is set to

384

. Bold indicates the best result, while underlined denotes the second-best result.

Figure 3: Scalability and cross-architecture generalization performance on the ETTh1 and Traffic datasets. The left two plots show scalability results for varying synthetic data sizes (

M

). The right two plots compare the performance between fixed- and cross-architecture settings, highlighting that HDT maintains a significantly smaller increase in MSE compared to others.

Method	ETTh1	ETTh2	ETTm1	ETTm2	Electricity	Traffic
Base	0.583	0.465	0.905	0.402	0.414	0.934
Base + Decomp.	0.545	0.420	0.814	0.325	0.376	0.902
HDT	0.420	0.334	0.386	0.206	0.226	0.760

Table 2: Ablation study on the components of HDT. Base denotes conventional gradient matching, and Decomp. refers to the use of the sinusoidal decomposition.

Backbone	Dataset	MTT	CondTSF	HDT
DLinear	ETTh1	$30.06$	$32.13$	$36.37$
DLinear	Electricity	$35.61$	$38.04$	$42.76$
iTransformer	ETTh1	$197.69$	$199.95$	$209.46$
iTransformer	Electricity	$253.90$	$262.10$	$264.90$

Table 3: Comparison of running times over 100 outer-loops in seconds. The cost increase of HDT is marginal.

Dataset	Full Training	Distilled	Speed-up
Electricity	1650.33	1.98	834x
Traffic	4266.45	2.32	1839x

Table 4: Training time comparison in seconds with the iTransformer backbone, demonstrating the significant efficiency gains achieved by training on the distilled dataset.

	Random	CondTSF	HDT	Full Data
MSE	358.51	197.95	46.63	44.25

Table 5: Distillation performance (MSE) on CA dataset with the DLinear backbone. HDT’s performance closely approaches that of full-data training.

Setting	MSE	Training Time
Zero-shot	1.972	-
Full Fine-tuning	1.383	3775.68s
Few-shot with HDT (end-to-end)	1.417	443.51s
Few-shot with HDT	1.417	47.23s
Comparison	+2.5%	80x (8.51x)

Table 6: Fine-tuning the Moirai-Large (311M) on ETTh1. The final row shows the MSE increase and training speed-up of our method relative to full fine-tuning.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	ETTh1	ETTh2	ETTm1	ETTm2	Electricity	Traffic	CA
Length	17,420	17,420	69,680	69,680	26,304	17,544	201,363
Features	7	7	7	7	321	862	8,600

Appendix

Table 7: Statistics of the datasets used for evaluation.

		$\lambda$
Dataset	$k$	$10^{-1}$	$10^{-2}$	$10^{-3}$	$10^{-4}$	$10^{-5}$
ETTh1	48	0.481	0.434	0.446	0.441	0.470
ETTh1	96	0.471	0.432	0.446	0.430	0.466
ETTm1	48	0.410	0.366	0.364	0.372	0.405
ETTm1	96	0.425	0.378	0.369	0.380	0.397
Electricity	48	0.256	0.209	0.212	0.215	0.235

Appendix

Table 8: Hyperparameter analysis results (MSE) on various datasets with the DLinear backbone.

Backbone: DLinear
Dataset	ETTh1			ETTh2			ETTm1			ETTm2			Electricity			Traffic
	L	T	C	L	T	C	L	T	C	L	T	C	L	T	C	L	T	C
Random	1.240	0.875	0.739	2.794	0.412	0.366	1.171	0.863	0.711	2.891	0.264	0.236	1.169	0.835	0.801	1.826	1.624	1.412
DC	1.221	0.896	0.740	3.083	0.414	0.366	1.198	0.857	0.720	2.598	0.272	0.238	1.176	0.855	0.801	1.832	1.596	1.382
MTT	1.146	2.489	3.429	3.038	1.971	1.841	1.098	2.582	3.844	2.919	0.333	0.572	0.973	0.933	0.923	1.488	1.479	1.460
TESLA	1.059	1.184	1.393	2.636	0.425	0.382	1.004	0.920	0.828	1.603	0.265	0.237	0.935	0.875	0.867	1.492	1.495	1.476

Appendix

Table 9: Overall distillation performance on the synthetic data size

M=192

in terms of MSE. Bold indicates the best result, while underlined denotes the second-best result.

Figure 4: Visual comparison of distilled datasets generated by different methods and backbones on ETTh1.

Figure 5: Visual comparison of distilled datasets generated by different methods and backbones on ETTm1.

Figure 6: Visual comparison of distilled datasets generated by different methods and backbones on ETTh2.

Figure 7: Visual comparison of distilled datasets generated by different methods and backbones on ETTm2.

Frequency-Domain Time Series Modeling

See all 22 papers

Effective Dataset Distillation for Spatio-Temporal Forecasting with Bi-dimensional Compression
Mar 11, 2026Taehyung Kwon, Yeonje Choi, Yeongho Kim +1Dataset Distillation Spatiotemporal Forecasting
Kim Jaechul Graduate School of AI, KAIST, Seoul, Republic of Korea
Olivia: Harmonizing Time Series Foundation Models with Power Spectral Density
May 17, 2026Jingru Fei, Kun Yi, Alex Xing Wang +3Zero-Shot Time Series Forecasting Frequency-Domain Feature Learning
Beijing Institute of Technology · North China Institute of Computing Technology · State Information Center +4
WinoTS: Wavelet-based Self-Distillation for Time Series Models
Sep 30, 2026Noam Major, Kathy Razmadze, Yoli ShavitSelf-Supervised Time Series Representation Learning Time Series Data Augmentation
Bar-Ilan University, Ramat-Gan, Israel

Mar 11, 2026J/K move · Enter open · S save

Effective Dataset Distillation for Spatio-Temporal Forecasting with Bi-dimensional Compression

Taehyung Kwon, Yeonje Choi, Yeongho Kim, Kijung Shin

Kim Jaechul Graduate School of AI, KAIST, Seoul, Republic of Korea

Dataset Distillation Spatiotemporal Forecasting

Open paper AskPDF arXiv

Spatio-temporal time series are widely used in real-world applications, including traffic prediction and weather forecasting. They are sequences of observations over extensive periods and multiple locations, naturally represented as multidimensional data. Forecasting is a central task in spatio-temporal analysis, and numerous deep learning methods have been developed to address it. However, as dataset sizes and model complexities continue to grow in practice, training deep learning models has become increasingly time- and resource-intensive. A promising solution to this challenge is dataset distillation, which synthesizes compact datasets that can effectively replace the original data for model training. Although successful in various domains, including time series analysis, existing dataset distillation methods compress only one dimension, making them less suitable for spatio-temporal datasets, where both spatial and temporal dimensions jointly contribute to the large data volume. To address this limitation, we propose STemDist, the first dataset distillation method specialized for spatio-temporal time series forecasting. A key idea of our solution is to compress both temporal and spatial dimensions in a balanced manner, reducing training time and memory. We further reduce the distillation cost by performing distillation at the cluster level rather than the individual location level, and we complement this coarse-grained approach with a subset-based granular distillation technique that enhances forecasting performance. On five real-world datasets, we show empirically that, compared to both general and time-series dataset distillation methods, datasets distilled by our STemDist method enable model training (1) faster (up to 6X) (2) more memory-efficient (up to 8X), and (3) more effective (with up to 12% lower prediction error).

Abstract

Frequency-Domain Time Series Modeling

Explore similar work

Effective Dataset Distillation for Spatio-Temporal Forecasting with Bi-dimensional Compression

Olivia: Harmonizing Time Series Foundation Models with Power Spectral Density

WinoTS: Wavelet-based Self-Distillation for Time Series Models