A key operation for massive data series collection analysis is similarity search. According to recent studies, SAX-based indexes offer state-of-the-art performance for similarity search tasks. However, their performance lags under high-frequency, weakly correlated, excessively noisy, or other dataset-specific properties. In this work, we propose Deep Embedding Approximation (DEA), a novel family of data series summarization techniques based on deep neural networks. Moreover, we describe SEAnet, a novel architecture especially designed for learning DEA, that introduces the Sum of Squares preservation property into the deep network design. We further enhance SEAnet with SEAtrans encoder. Finally, we propose novel sampling strategies, SEAsam and SEAsamE, that allow SEAnet to effectively train on massive datasets. Comprehensive experiments on 7 diverse synthetic and real datasets verify the advantages of DEA learned using SEAnet in providing high-quality data series summarizations and similarity search results.
Figures & tables
Fig. 1: Case studies where PAA and DFT work or fail to approximate and reconstruct series from RandWalk and Deep1B datasets. In both cases, DEA works to approximate and reconstruct series. All summarizations use the same memory budget.
Fig. 2: Replace PAA by DEA for SAX symbolization.
Fig. 3: Workflow of DEA-based approximate similarity search.
Fig. 4: The SEAnet architecture and the details of a dilated full-preactivation ResBlock.
Fig. 5: The SEAtrans encoder architecture.
Length
Before Scaling
×256/m
×1/m
m
Mean
Var
Mean
Var
Mean
Var
256
22.605
0.999
22.605
0.999
1.4128
0.0039
128
15.969
0.998
22.583
1.9961
1.4115
0.0078
96
13.820
0.997
22.569
2.6597
1.4105
0.0104
16
5.5692
0.984
22.277
15.743
1.3923
0.0615
8
3.8772
0.967
21.933
30.944
1.3708
0.1209
TABLE I: Mean and Variance of the distribution of pairwise distances between two ideal series. Scaling by 256/m is to preserve SoS by scaling DEA itself. Scaling by 1/m is the case where we scale both series and DEA in loss functions. (Note that for length 256 , scaling by 256/m does not change the original behavior.)
Fig. 6: SEAsam transformation and InvSAX [ 21 ] .
Fig. 7: SEAsam sampling workflow.
layer name
13-layer ( 96 )
15-layer ( 128 )
17-layer ( 256 )
ConvLayer
[3,96]
[3,128]
[3,256]
ConvBlock
[3,963,96]× 5
[3,1283,128]× 6
[3,2563,256]× 7
Linear
[9616]
[12816]
[25616]
parameters/M
0.585
1.234
5.712
MACs/B
0.053
0.152
1.412
TABLE II: SEAnet encoder architecture specifics for different series lengths. Building blocks ([kernel size, channel number] for the convolutional layer/block, [dimension] for the linear layer) are shown in brackets, multiplied by the numbers of blocks stacked.
Fig. 8: Improvements of employing SoS preservation in terms of 1st BSF tightness (positive values mean SoS is better); 100M datasets.
Fig. 9: Improvements of SEAsam over uniformly random sampling in terms of 1st BSF rightness (positive values mean SEAsam is better); 100M datasets.
Fig. 10: Leaf node coverage across different sampling sizes for uniformly random sampling (light gray) and SEAsam (dark red).
( a ) Averaged Distance Differences
( b ) Reconstruction RMS Error
Dataset
PAA
FDJNet
TimeNet
Incept
SEAnet-nD
SEAnet
PAA
TimeNet
Incept
SEAnet
RandWalk
1.3701
0.2794
0.4098
0.6285
0.2976
0.2194
0.3061
0.3354
0.3587
0.2604
F5
2.1152
0.1737
0.2103
0.2836
0.1692
0.1629
0.4214
0.2527
0.2708
0.2433
F10
5.0395
1.1943
1.9063
1.3958
1.1859
1.1672
0.6238
0.6799
0.5041
0.2635
SALD
3.2927
0.6247
0.6928
0.858
0.5748
0.6182
0.5586
0.5883
0.6831
0.5023
Deep1B
8.1095
0.9511
7.8478
0.9511
0.9083
0.9484
0.9207
1.0
0.6368
0.5418
TABLE III: (a) Averaged distance differences between pairs of series in the original and the embedded (PAA, DEA) spaces. (b) Root-Mean-Square error between original and reconstructed series. In both cases, best result (lower is better) is marked in bold, second best is underlined. (Results calculated using 10,000 series SEAsam sampled from datasets of 10-million series.)
Fig. 11: Distance scatters of 1,000 SEAsam sampled series pairs across different datasets for PAA (light gray) and SEAnet (dark red).
Fig. 12: Nearest neighbors’ coverage vs neighborhood size (higher is better).
Fig. 13: Approximate query answers quality: 1st BSF tightness vs dataset size (higher is better)
Fig. 14: Average distances among series of the visited leaf nodes vs dataset size (lower is better)
Fig. 15: Approximate query answers quality: 1st BSF tightness vs number of series visited; 100M series datasets (higher is better)
Fig. 16: SEAnet DEA vs. PAA on kNN classification accuracies of UCR18 dataset. Number of datasets where SEAnet wins-ties-losses compared with PAA are reported.
Fig. 17: Convergence procedures measured by compression error (averaged distance difference) LC across architectures on F5 dataset.
Fig. 18: Improvements of SEAsamE over SEAsam in terms of 1st BSF rightness (positive values mean SEAsamE is better); 100M datasets.
Fig. 19: Query answers quality upper-bounds: 1st BSF tightness vs number of series visited; 100M series datasets (higher is better).
Time series representation learning develops compact, transferable features that capture temporal patterns for forecasting, classification, retrieval, and anomaly detection.
We propose the Deep Distance Measurement Method (DDMM) to improve retrieval accuracy in unsupervised multivariate time series similarity retrieval. DDMM enables learning of minute differences within states in the entire time series and thereby recognition of minute differences between states, which are of interest to users in industrial plants. To achieve this, DDMM uses a learning algorithm that assigns a weight to each pair of an anchor and a positive sample, arbitrarily sampled from the entire time series, based on the Euclidean distance within the pair and learns the differences within the pairs weighted by the weights. This algorithm allows both learning minute differences within states and sampling pairs from the entire time series. Our empirical studies showed that DDMM significantly outperformed state-of-the-art time series representation learning methods on the Pulp-and-paper mill dataset and demonstrated the effectiveness of DDMM in industrial plants. Furthermore, we showed that accuracy can be further improved by linking DDMM with existing feature extraction methods through experiments with the combined model.
Susumu Naito, Kouta Nakata, Yasunori Taguchi
Corporate Laboratory, Toshiba Corporation, Kawasaki, Japan
Approximate Nearest Neighbor Search (ANNS) plays a pivotal role in modern deep learning pipelines. Recently, many ANNS systems have been proposed to either provide broad functionality or reach high performance. However, it is yet difficult to achieve both with minimal programming efforts. We propose ANNLib to address the gap. ANNLib is a library that provides a programming framework for achieving high performance and flexible functionality in ANNS systems, based on popular graph-based ANNS algorithms. We carefully decouple and independently optimize both the algorithm and the data structure components of an ANNS system. In addition, we integrate state-of-the-art algorithms and data structures into ANNLib as modules, along with our new designs. Users can choose combinations of components to implement sophisticated settings with high performance, such as filter search, fully dynamic updates, and historical queries on snapshots. Our experiments show that our new solution provides a simple interface for various applications and achieves comparable or even better performance than previous work, specifically for each application.
Zheqi Shen, Jingbo Su, Zijin Wan +2
UC Riverside Riverside CA, USA · W&M Williamsburg VA, USA
The rapid advancement of artificial intelligence (AI) has significantly accelerated research in time-series analysis, particularly in forecasting, classification, and generation tasks. Recent models, especially foundation models, benefit from time-series dataset similarity due to its significant role in source dataset selection for fine-tuning. However, many existing implementations for benchmarking time-series dataset similarity methods are fragmented and difficult to extend. To address this, we present a unified framework, the Time-Series Dataset Similarity Toolbox (TSDS-Toolbox). Our work enables (1) systematic and reproducible comparisons of time-series dataset similarity methods; (2) flexible extensibility for users to add customized datasets, similarity methods, and downstream time-series tasks; and (3) consistent evaluation of both dataset-level and series-level similarity methods through integrated time-series dataset reducers. The effectiveness of TSDS-Toolbox is validated through comprehensive experiments under diverse experimental settings. Our toolbox is publicly available.
Yen-Ku Liu, Hongjie Chen, Ryan A. Rossi +1
National Yang Ming Chiao Tung University Hsinchu, Taiwan · Dolby Laboratories Atlanta, Georgia, USA · Adobe Research San Jose, California, USA +1