Learning from Complexity: Exploring Dynamic Sample Pruning of Spatio-Temporal Training

Wei Chen; Junle Chen; Yuqian Wu; Yuxuan Liang; Xiaofang Zhou

cs.LGFeb 22, 2026

Learning from Complexity: Exploring Dynamic Sample Pruning of Spatio-Temporal Training

Authors: Wei Chen, Junle Chen, Yuqian Wu, Yuxuan Liang, Xiaofang Zhou

Organizations: HKUST(GZ) · HKUST

Abstract

Spatio-temporal forecasting is fundamental to intelligent systems in transportation, climate science, and urban planning. However, training deep learning models on the massive, often redundant, datasets from these domains presents a significant computational bottleneck. Existing solutions typically focus on optimizing model architectures or optimizers, while overlooking the inherent inefficiency of the training data itself. This conventional approach of iterating over the entire static dataset each epoch wastes considerable resources on easy-to-learn or repetitive samples. In this paper, we explore a novel training-efficiency techniques, namely learning from complexity with dynamic sample pruning, ST-Prune, for spatio-temporal forecasting. Through dynamic sample pruning, we aim to intelligently identify the most informative samples based on the model's real-time learning state, thereby accelerating convergence and improving training efficiency. Extensive experiments conducted on real-world spatio-temporal datasets show that ST-Prune significantly accelerates the training speed while maintaining or even improving the model performance, and it also has scalability and universality.

Figures & tables

Panel of Figure 1 — Figure 1: The spatio-temporal data redundancy characteristics and statistical properties along the spatial and temporal dimensions, exemplified by the PeMS08 ( Song et al., 2020 ) dataset. For more statistical information on other datasets, please refer to Appendix A.1 .

Figure 2: Further analysis of spatio-temporal data insights. (a) Averaging Masking Effect: Low-error nodes dilute critical localized anomalies, necessitating a structural scoring mechanism beyond simple mean error. (b) Long-tail Stationarity Distribution: The dominance of stationary patterns motivates our stationarity-aware rescaling to prevent distribution shift and maintain representativeness.

Figure 3: The overall workflow of the ST-Prune for efficient data pruning during spatio-temporal training.

	Dataset	Pems08 ( MAPE (%) $\downarrow$ )				UrbanEV ( MAPE (%) $\downarrow$ )
	Remaining Ratio %	10	30	50	70	10	30	50	70
Static	Hard Random	13.93 ↓26.87%	14.80 ↓34.79%	12.07 ↓9.93%	12.06 ↓9.84%	35.89 ↓20.32%	31.90 ↓6.94%	31.52 ↓5.67%	30.69 ↓2.88%
	CD ( Agarwal et al., 2020 )	14.55 ↓32.51%	12.25 ↓11.57%	11.96 ↓8.93%	11.83 ↓7.74%	38.37 ↓28.63%	32.20 ↓7.95%	30.40 ↓1.91%	31.01 ↓3.96%
	Herding ( Welling, 2009 )	15.79 ↓43.81%	12.11 ↓10.29%	11.58 ↓5.46%	12.84 ↓16.94%	34.63 ↓16.09%	30.97 ↓3.82%	30.94 ↓3.72%	30.97 ↓3.82%
	K-Means ( Sener and Savarese, 2018 )	15.71 ↓43.08%	12.24 ↓11.48%	12.48 ↓13.66%	11.52 ↓4.92%	38.45 ↓28.90%	31.85 ↓6.77%	30.96 ↓3.79%	30.69 ↓2.88%
	Least Confidence ( Coleman et al., 2019 )	14.31 ↓30.33%	12.62 ↓14.94%	11.74 ↓6.92%	11.61 ↓5.74%	34.23 ↓14.75%	36.87 ↓23.60%	30.98 ↓3.86%	30.55 ↓2.41%

Table 1: Performance comparison to state-of-the-art dataset selection and pruning methods when remaining

\{10\%,30\%,50\%,70\%\}

of the full training set. All methods are trained using same ST backbone. (Additional MAE and RMSE results are provided in Appendix B )

Figure 4: The trade-off between per-epoch time and performance in UrbanEV . Specifically, we report the test performance when methods achieve per epoch times of {10%, 30%, 50%, 70%, 90%} of the full dataset training time. “Vanilla” denotes the full dataset training result.

Dataset

SD (

\#716

)

GBA (

\#2352

)

GLA (

\#3834

)

MAE

RMSE

MAPE (%)

TIME

MAE

RMSE

MAPE (%)

TIME

MAE

RMSE

MAPE (%)

TIME

Whole Dataset

24.13 ±1.14

38.03 ±1.32

15.77 ±0.87

2.07h

27.34 ±1.04

42.11 ±1.19

24.11 ±0.48

13.05h

26.26 ±0.84

41.12 ±1.47

16.06 ±0.63

30.53h

Soft Random (10%)

30.41

47.96

20.43

0.52h

33.05

48.71

39.30

2.97h

29.74

45.98

20.97

7.25h

ST-Prune (10%)

24.01

37.73

15.51

0.61h

27.32

41.94

24.05

4.92h

26.22

40.88

15.95

13.38h

ST-Prune (1%)

24.76

38.19

16.51

0.31h

28.16

42.77

23.64

2.84h

26.95

41.57

17.62

7.12h

Improvement

\uparrow 21.05\%

\uparrow 21.33\%

\uparrow 24.08\%

\times 6.68

\uparrow 17.33\%

\uparrow 13.90\%

\uparrow 38.80\%

\times 4.60

\uparrow 11.84\%

\uparrow 11.09\%

\uparrow 23.94\%

\times 4.29

Table 2: Comparison of ST-Prune with the fastest dynamic pruning method Soft Random on LargeST . TIME: Total wall-clock time.

Figure 5: Performance and efficiency trade-offs between w/o and w/ ST-Prune at different scales of the ST-foundation model OpenCity .

Figure 6: Performance variations across distinct architectures, optimizers, and tasks under data retention rates of

\{10\%,30\%,50\%,70\%\}

Figure 7: Ablation and parameter sensitivity analysis on UrbanEV dataset under training data retention rates of

\{10\%,30\%,50\%,70\%\}

Figure 8: Visualization of t-SNE fixed-corner clustering of the PEMS08 distribution with the full training set and a retention rate of 20%. The right panel shows the evolution of selected subsets through different pruning strategies at different training progresses (20% to 80%).

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Source	Dataset	Nodes	Time Range	Frames	Sampling Rate	Data Points
( Song et al., 2020 )	PEMS08	170	07/01/2016 – 08/31/2016	17,856	5 minutes	3.04 M
( Li et al., 2025a )	UrbanEV	275	09/01/2022 – 02/28/2023	4344	1 hour	1.19 M
LargeST ( Liu et al., 2024b )	GLA	3,834	01/01/2019 – 12/31/2019	35,040	15 minutes	134.3 M
	GBA	2,352	01/01/2019 – 12/31/2019	35,040	15 minutes	88.7 M
	SD	716	01/01/2019 – 12/31/2019	35,040	15 minutes	25.1 M

Appendix

Table A.1 : Summary of datasets used for our experiments. Degree: the average degree of each node. Data Points: multiplication of nodes and frames. M : million (

10^{6}

Panel of Figure 12 — Figure A.1 : The spatio-temporal redundancy characteristics and statistical properties of UrbanEV ( Li et al., 2025a ) .

Panel of Figure 13 — Figure A.2 : The spatio-temporal redundancy characteristics and statistical properties of SD ( Liu et al., 2024b ) .

Panel of Figure 14 — Figure A.3 : The spatio-temporal redundancy characteristics and statistical properties of GBA ( Liu et al., 2024b ) .

Panel of Figure 15 — Figure A.4 : The spatio-temporal redundancy characteristics and statistical properties of GLA ( Liu et al., 2024b ) .

Training epochs	100
Selection epochs	10
Batch size	256
Data Fraction	0.1 / 0.3 / 0.5 / 0.7
Optimizer	SGD
Learning rate	1e-3
Minimum Learning rate	1e-4

Appendix

Table A.2 : Hyperparameters setting.

	Dataset	Pems08 (MAE $\downarrow$ )				UrbanEV (MAE $\downarrow$ )
	Remaining Ratio %	10	30	50	70	10	30	50	70
Static	Hard Random	19.92 ↓16.90%	18.75 ↓10.04%	18.24 ↓7.04%	17.92 ↓5.16%	4.23 ↓18.82%	3.96 ↓11.24%	3.82 ↓7.30%	3.76 ↓5.62%
	CD ( Agarwal et al., 2020 )	19.99 ↓17.31%	18.50 ↓8.57%	18.12 ↓6.34%	17.86 ↓4.81%	4.34 ↓21.91%	3.95 ↓10.96%	3.81 ↓7.02%	3.79 ↓6.46%
	Herding ( Welling, 2009 )	20.14 ↓18.19%	18.51 ↓8.63%	18.16 ↓6.57%	17.84 ↓4.69%	4.37 ↓22.75%	3.92 ↓10.11%	3.88 ↓8.99%	3.74 ↓5.06%
	K-Means ( Sener and Savarese, 2018 )	20.21 ↓18.60%	18.59 ↓9.10%	18.28 ↓7.28%	17.81 ↓4.52%	4.33 ↓21.63%	4.00 ↓12.36%	3.81 ↓7.02%	3.77 ↓5.90%
	Least Confidence ( Coleman et al., 2019 )	20.61 ↓20.95%	19.48 ↓14.32%	18.14 ↓6.46%	18.11 ↓6.28%	4.22 ↓18.54%	4.47 ↓25.56%	3.82 ↓7.30%	3.71 ↓4.21%

Appendix

Table B.1 : Performance comparison to state-of-the-art dataset pruning methods when remaining

\{10\%,30\%,50\%,70\%\}

of the full set. All methods are trained using GWNet , and the reported metric MAE represent the average of five runs .

	Dataset	Pems08 (RMSE $\downarrow$ )				UrbanEV (RMSE $\downarrow$ )
	Remaining Ratio %	10	30	50	70	10	30	50	70
Static	Hard Random	31.29 ↓16.28%	29.24 ↓8.66%	28.72 ↓6.73%	28.29 ↓5.13%	8.24 ↓22.07%	7.73 ↓14.52%	7.35 ↓8.89%	7.24 ↓7.26%
	CD ( Agarwal et al., 2020 )	31.31 ↓16.35%	29.04 ↓7.92%	28.61 ↓6.32%	28.10 ↓4.42%	8.42 ↓24.74%	7.69 ↓13.93%	7.37 ↓9.19%	7.28 ↓7.85%
	Herding ( Welling, 2009 )	31.15 ↓15.76%	29.16 ↓8.36%	28.48 ↓5.83%	28.04 ↓4.20%	8.60 ↓27.41%	7.60 ↓12.59%	7.52 ↓11.41%	7.14 ↓5.78%
	K-Means ( Sener and Savarese, 2018 )	31.50 ↓17.06%	29.23 ↓8.62%	28.74 ↓6.80%	28.05 ↓4.24%	8.41 ↓24.59%	7.78 ↓15.26%	7.38 ↓9.33%	7.21 ↓6.81%
	Least Confidence ( Coleman et al., 2019 )	32.36 ↓20.25%	30.11 ↓11.89%	28.54 ↓6.06%	28.43 ↓5.65%	8.29 ↓22.81%	8.65 ↓28.15%	7.36 ↓9.04%	7.07 ↓4.74%

Appendix

Table B.2 : Performance comparison to state-of-the-art dataset pruning methods when remaining

\{10\%,30\%,50\%,70\%\}

of the full set. All methods are trained using GWNet , and the reported metric RMSE represent the average of five runs .

Data-Efficient Learning

Data-efficient learning develops methods that achieve accurate, robust models with fewer labeled examples, training samples, or interactions.

See all 239 papers

Effective Dataset Distillation for Spatio-Temporal Forecasting with Bi-dimensional Compression
Mar 11, 2026Taehyung Kwon, Yeonje Choi, Yeongho Kim +1Dataset Distillation Spatiotemporal Forecasting
Kim Jaechul Graduate School of AI, KAIST, Seoul, Republic of Korea
DYNA-PRUNER: Input-Adaptive Data-Model Co-Pruning for Efficient and Scalable Spatio-Temporal Media Prediction
Jun 13, 2026Fuyan Zhang, Yuqi Li, Qing Xu +2Efficient Neural Network Inference Spatiotemporal Forecasting
University of Edinburgh · The City College of New York, City University of New York · Institute of Computing Technology, Chinese Academy of Sciences +1
Efficient Prompt Learning for Traffic Forecasting
May 8, 2026Qianru Zhang, Xinyi Gao, Alexander Zhou +3OOD Generalization Prompt Tuning
School of Computing and Data Science The University of Hong Kong · School of Electrical Engineering and Computer Science The University of Queensland · Department of Computing The Hong Kong Polytechnic University

Mar 11, 2026J/K move · Enter open · S save

Effective Dataset Distillation for Spatio-Temporal Forecasting with Bi-dimensional Compression

Taehyung Kwon, Yeonje Choi, Yeongho Kim, Kijung Shin

Kim Jaechul Graduate School of AI, KAIST, Seoul, Republic of Korea

Dataset Distillation Spatiotemporal Forecasting

Open paper AskPDF arXiv

Spatio-temporal time series are widely used in real-world applications, including traffic prediction and weather forecasting. They are sequences of observations over extensive periods and multiple locations, naturally represented as multidimensional data. Forecasting is a central task in spatio-temporal analysis, and numerous deep learning methods have been developed to address it. However, as dataset sizes and model complexities continue to grow in practice, training deep learning models has become increasingly time- and resource-intensive. A promising solution to this challenge is dataset distillation, which synthesizes compact datasets that can effectively replace the original data for model training. Although successful in various domains, including time series analysis, existing dataset distillation methods compress only one dimension, making them less suitable for spatio-temporal datasets, where both spatial and temporal dimensions jointly contribute to the large data volume. To address this limitation, we propose STemDist, the first dataset distillation method specialized for spatio-temporal time series forecasting. A key idea of our solution is to compress both temporal and spatial dimensions in a balanced manner, reducing training time and memory. We further reduce the distillation cost by performing distillation at the cluster level rather than the individual location level, and we complement this coarse-grained approach with a subset-based granular distillation technique that enhances forecasting performance. On five real-world datasets, we show empirically that, compared to both general and time-series dataset distillation methods, datasets distilled by our STemDist method enable model training (1) faster (up to 6X) (2) more memory-efficient (up to 8X), and (3) more effective (with up to 12% lower prediction error).

Abstract

Data-Efficient Learning

Explore similar work

Effective Dataset Distillation for Spatio-Temporal Forecasting with Bi-dimensional Compression

DYNA-PRUNER: Input-Adaptive Data-Model Co-Pruning for Efficient and Scalable Spatio-Temporal Media Prediction

Efficient Prompt Learning for Traffic Forecasting