cs.LGMar 11, 2026

Effective Dataset Distillation for Spatio-Temporal Forecasting with Bi-dimensional Compression

Authors: Taehyung Kwon, Yeonje Choi, Yeongho Kim, Kijung Shin

Organizations: Kim Jaechul Graduate School of AI, KAIST, Seoul, Republic of Korea

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Spatio-temporal time series are widely used in real-world applications, including traffic prediction and weather forecasting. They are sequences of observations over extensive periods and multiple locations, naturally represented as multidimensional data. Forecasting is a central task in spatio-temporal analysis, and numerous deep learning methods have been developed to address it. However, as dataset sizes and model complexities continue to grow in practice, training deep learning models has become increasingly time- and resource-intensive. A promising solution to this challenge is dataset distillation, which synthesizes compact datasets that can effectively replace the original data for model training. Although successful in various domains, including time series analysis, existing dataset distillation methods compress only one dimension, making them less suitable for spatio-temporal datasets, where both spatial and temporal dimensions jointly contribute to the large data volume. To address this limitation, we propose STemDist, the first dataset distillation method specialized for spatio-temporal time series forecasting. A key idea of our solution is to compress both temporal and spatial dimensions in a balanced manner, reducing training time and memory. We further reduce the distillation cost by performing distillation at the cluster level rather than the individual location level, and we complement this coarse-grained approach with a subset-based granular distillation technique that enhances forecasting performance. On five real-world datasets, we show empirically that, compared to both general and time-series dataset distillation methods, datasets distilled by our STemDist method enable model training (1) faster (up to 6X) (2) more memory-efficient (up to 8X), and (3) more effective (with up to 12% lower prediction error).

Figures & tables

Fig. 1: Synthetic data compressed in both temporal and spatial dimensions in a balanced manner yields significantly lower training time and GPU memory usage compared to datasets of the same size compressed only along the temporal dimension. See Preliminary Experiment 1 for details.
SymbolDescription
Z∈RM×F×NZ\in\mathbb{R}^{M\times F\times N}Spatio-temporal time series
zt∈RF×Nz_{t}\in\mathbb{R}^{F\times N}tt -th observation in ZZ
MMTotal length of time series
FFNumber of (input) features
NNNumber of locations
Xt∈RLin×F×NX_{t}\in\mathbb{R}^{L_{in}\times F\times N}Input time series for time tt
TABLE I: Descriptions of frequently used notations
Fig. 2: An example illustrating location clustering, where eight locations in XT\mathcal{X}_{\mathcal{T}} are reduced into four locations (cluster centroids) in XC\mathcal{X}_{\mathcal{C}} . Entries with the same color correspond to the data for the same location. Refer to Example 1 for details.
Fig. 3: An illustrative example showing how the clustered spatio-temporal dataset (which is averaged within each cluster) are partitioned into two location subsets and distilled at the subset level. Entries with the same color correspond to the results for the same location. Refer to Example 2 for details.
 Data Dimensions# Total
 MMNNFFData Points
Real-world Datasets
GBA [ 2 ]1,9771,9772,3522,352114,649,904
GLA [ 2 ]1,9771,9773,8343,834117,579,818
ERA5 [ 67 ]2,1372,1376,5616,5616684,125,142
CAMS [ 68 ]2,5562,5567,0707,07066108,425,520
TABLE II: Dataset statistics.
TABLE III: Effectiveness of dataset-distillation methods. The synthetic dataset from each method is used to train MTGNN, equipped with the location encoders when necessary, for evaluation. The performances of our proposed method are highlighted in bold; the best performances in each setting are highlighted in yellow. O.O.M.: out of memory.
Fig. 4: Training on synthetic datasets distilled by STemDist is faster and more memory-efficient than training on those distilled by the baselines.
Graph wavenet [ 11 ]
 GBAGLAERA5CACAMS
MethodRelativeRelativeRelativeRelativeRelativeRelativeRelativeRelativeRelativeRelative
MAERMSEMAERMSEMAERMSEMAERMSEMAERMSE
K-Center [ 47 ]0.336 ±\pm 0.0270.427 ±\pm 0.0320.279 ±\pm 0.0210.367 ±\pm 0.0180.466 ±\pm 0.0170.510 ±\pm 0.0120.318 ±\pm 0.0320.396 ±\pm 0.0300.763 ±\pm 0.0550.796 ±\pm 0.028
DC [ 15 ]0.321 ±\pm 0.0360.395 ±\pm 0.0330.288 ±\pm 0.0310.312 ±\pm 0.0090.424 ±\pm 0.0070.470 ±\pm 0.0040.292 ±\pm 0.0430.369 ±\pm 0.0380.550 ±\pm 0.0060.688 ±\pm 0.004
CondTSF [ 23 ]0.346 ±\pm 0.0650.416 ±\pm 0.0660.302 ±\pm 0.0300.387 ±\pm 0.0280.464 ±\pm 0.0080.514 ±\pm 0.0070.322 ±\pm 0.0380.413 ±\pm 0.0310.531 ±\pm 0.0260.695 ±\pm 0.022
TABLE IV: Cross-model generalization of dataset-distillation methods. The synthetic dataset produced by each method is used to train various machine learning models for evaluation. The performances of our methods are highlighted in bold; and the best performances in each setting are highlighted in yellow.
Fig. 5: Scalability of STemDist. Its empirical distillation time grows (sub-)linearly with (a) the number of original time series, (b) the location count in the original dataset, and (c) the feature count in the original dataset.
Fig. 6: Comparison of distillation time per outer iteration across different dataset distillation methods.
Fig. 7: Ablation study. Distillation performance of STemDist and its variants over distillation time on GLA, ERA5, CAMS datasets
TABLE V: Effect of hyperparameters on the distillation performance of STemDist, measured by Relative RMSE under a total compression ratio of 1%.
More in this topic

Dataset Distillation

Dataset distillation compresses training data into a small synthetic or selected set that retains the information needed to train useful models.

See all 65 papers

Explore similar work

CardsList
Feb 22, 2026

Learning from Complexity: Exploring Dynamic Sample Pruning of Spatio-Temporal Training

Spatio-temporal forecasting is fundamental to intelligent systems in transportation, climate science, and urban planning. However, training deep learning models on the massive, often redundant, datasets from these domains presents a significant computational bottleneck. Existing solutions typically focus on optimizing model architectures or optimizers, while overlooking the inherent inefficiency of the training data itself. This conventional approach of iterating over the entire static dataset each epoch wastes considerable resources on easy-to-learn or repetitive samples. In this paper, we explore a novel training-efficiency techniques, namely learning from complexity with dynamic sample pruning, ST-Prune, for spatio-temporal forecasting. Through dynamic sample pruning, we aim to intelligently identify the most informative samples based on the model's real-time learning state, thereby accelerating convergence and improving training efficiency. Extensive experiments conducted on real-world spatio-temporal datasets show that ST-Prune significantly accelerates the training speed while maintaining or even improving the model performance, and it also has scalability and universality.

Wei Chen, Junle Chen, Yuqian Wu +2

HKUST(GZ) · HKUST

Data-Efficient LearningDataset PruningSpatiotemporal ForecastingNeural Network Training Dynamics
AskPDF
Mar 4, 2026

Harmonic Dataset Distillation for Time Series Forecasting

Time Series forecasting (TSF) in the modern era faces significant computational and storage cost challenges due to the massive scale of real-world data. Dataset Distillation (DD), a paradigm that synthesizes a small, compact dataset to achieve training performance comparable to that of the original dataset, has emerged as a promising solution. However, conventional DD methods are not tailored for time series and suffer from architectural overfitting and limited scalability. To address these issues, we propose Harmonic Dataset Distillation for Time Series Forecasting (HDT). HDT decomposes the time series into its sinusoidal basis through the FFT and aligns the core periodic structure by Harmonic Matching. Since this process operates in the frequency domain, all updates during distillation are applied globally without disrupting temporal dependencies of time series. Extensive experiments demonstrate that HDT achieves strong cross-architecture generalization and scalability, validating its practicality for large-scale, real-world applications.

Seungha Hong, Sanghwan Jang, Wonbin Kweon +3

Pohang University of Science and Technology (POSTECH), Republic of Korea · University of Illinois Urban-Champaign (UIUC), United States

Frequency-Domain Time Series ModelingDataset DistillationTime Series Forecasting
AskPDF
May 11, 2026

Dimensional Balance Improves Large Scale Spatiotemporal Prediction Performance

Accurate spatiotemporal pattern analysis is critical in fields such as urban traffic, meteorology, and public health monitoring. However, existing methods face performance bottlenecks, typically yielding only incremental gains and often exhibiting limited cross-domain transferability. We analyze this bottleneck through spatial and temporal entropy measures, which are used as diagnostic indicators of spatiotemporal complexity mismatch rather than as guarantees that entropy alignment alone yields better forecasting. Empirically, larger mismatch is often accompanied by higher prediction uncertainty, especially under a fixed model-capacity budget. Guided by this diagnostic, we propose a scalable, adaptive framework that harmonizes spatial and temporal feature representations. Spatial dimensionality is compressed via low-rank matrix embedding to preserve essential structure, while an extended temporal horizon captures long-range dependencies and mitigates cumulative errors arising from temporal heterogeneity. Extensive experiments on urban traffic, meteorological, and epidemic datasets demonstrate substantial accuracy gains and broad applicability across the evaluated domains, suggesting that the framework is promising for a wide range of spatiotemporal tasks beyond the current study. The code is available on GitHub at https://github.com/ST-Balance/ST-Balance.

Jing Chen, Shixiang Pan, Yujie Fan +3

School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, 310018, China · Key Laboratory of New Industrial Internet Control Technology, Hangzhou, 310018, China · College of Economics, China Jiliang University, Hangzhou, 314423, China

Long-Term Time Series ForecastingSpatiotemporal Forecasting
AskPDF