The Multiverse of Time Series Machine Learning: an Archive for Multivariate Time Series Classification
Authors: Matthew Middlehurst, Aiden Rushbrooke, Ali Ismail-Fawaz, Maxime Devanne, Germain Forestier, Angus Dempster, Geoffrey I. Webb, Christopher Holder, Anthony Bagnall
Organizations: University of Bradford · University of East Anglia · Université de Haute-Alsace · Monash University · University of Southampton
Time series machine learning (TSML) is a growing research field that spans a wide range of tasks. The popularity of established tasks such as classification, clustering, and extrinsic regression has, in part, been driven by the availability of benchmark datasets. An archive of 30 multivariate time series classification datasets, introduced in 2018 and commonly known as the UEA archive, has since become an essential resource cited in hundreds of publications. We present a substantial expansion of this archive that more than quadruples its size, from 30 to 133 classification problems. We also release preprocessed versions of datasets containing missing values or unequal length series, bringing the total number of datasets to 147. Reflecting the growth of the archive and the broader community, we rebrand it as the Multiverse archive to capture its diversity of domains. The Multiverse archive includes datasets from multiple sources, consolidating other collections and standalone datasets into a single, unified repository. Recognising that running experiments across the full archive is computationally demanding, we recommend a subset of the full archive called Multiverse-core (MV-core) for initial exploration. To support researchers in using the new archive, we provide detailed guidance and a baseline evaluation of established and recent classification algorithms, establishing performance benchmarks for future research. We have created a dedicated repository for the Multiverse archive that provides a common aeon and scikit-learn compatible framework for reproducibility, an extensive record of published results, and an interactive interface to explore the results.
Figures & tables
Year
arXiv
Journal
Total
ML
Conv
RNN
Graph
Transf
Found
Deep
2019
12
0
3
0
2
0
0
1
0
0
2020
33
0
8
4
1
0
0
3
0
0
2021
72
28
12
6
3
2
0
0
1
0
2022
99
100
18
4
2
1
2
7
2
0
2023
138
173
15
3
3
0
1
6
1
1
2024
197
174
39
6
4
1
5
12
7
4
Table 1: Counts of papers roughly classified into one of the categories ML (machine learning), Conv (primarily CNN based for classification), Graph (learn structure on a graph representation), Transf (transformer based architecture using, e.g. attention or masking), foundation (model proposed as a backbone for multiple tasks which may be pretrained).
Year
No. papers
No. with code
No. datasets
Average
Algorithms
Average
2019
3
2
26–30
28.5
8–8
8.0
2020
8
2
6–30
15.1
2–16
6.6
2021
12
7
1–30
19.2
1–15
7.5
2022
18
15
4–30
19.0
1–14
7.9
2023
15
6
3–30
17.3
3–12
7.1
2024
39
19
3–30
21.4
3–19
10.3
Table 2: Estimates of the number of datasets and the number of algorithms used in papers using the UEA archive to support new MTSC algorithms.
Dataset
HC2
TimesNet
ConvTran
TapNet
LiteMVTime
ShapeNet
ArticularyWordRecognition
0.993
0.973
0.983
0.987
0.973
0.987
AtrialFibrillation
0.267
0.333
0.4
0.333
0.133
0.4
BasicMotions
1
0.973
1
1
1
1
Cricket
1
0.903
1
0.958
0.986
0.986
DuckDuckGeese
0.56
0.58
0.62
0.575
0.18
0.725
Epilepsy
1
0.877
0.985
0.971
0.993
0.987
Table 3: Published accuracy on the default test split of six classifiers proposed in the last five years on the 25 UEA datasets common to all papers. Classifiers are: ML algorithm HIVE-COTEv2 (HC2) Middlehurst et al. (2021) ; Foundation backbone TimesNet Wu et al. (2023) , Transformer approach ConvTran Foumani et al. (2024c) ; Convolutional approaches TapNet Zhang et al. (2020) and LiteMVTime Ismail-Fawaz et al. (2023b) ; and deep learning algorithm ShapeNet Li et al. (2021) . Best results in bold.
Table 4: Summary statistics for the 133 multivariate archive datasets, excluding missing value and unequal length variants. Provides statistics for the train set sizes (a), test set sizes (b), number of classes (c), number of time points (d) and number of dimensions (e) of the archive datasets. Also includes counts for series cleanliness before processing (f) and dataset type (g).
Dataset name
Train
Test
Channels
Length
Classes
Type
ArticularyWordRecognition
275
300
9
144
25
Motion
AtrialFibrillation
15
15
2
640
3
Biosignal
BasicMotions
40
40
6
100
4
Motion
CharacterTrajectories
1422
1436
3
60-182
20
Motion
Cricket
108
72
6
1197
12
Motion
DuckDuckGeese
50
50
1345
270
5
Audio
Table 5: The original 30 MTSC archive datasets. For datasets with unequal length series, both the minimum and maximum series lengths are shown.
Dataset name
Train
Test
Channels
Length
Classes
Type
Alzheimers
45
43
19
15000
3
Biosignal
Blink
500
450
4
510
2
Biosignal
ButtonPress
120
120
32
1000
2
Biosignal
EyesOpenShut
56
42
14
128
2
Biosignal
FeedbackButton
1700
1150
61
450
2
Biosignal
FeetHands
2835
1890
64
640
2
Biosignal
Table 6: The 20 new MTSC archive datasets from the EEG/MEG classification archive multivariate datasets, all of type Biosignal.
Dataset name
Train
Test
Channels
Length
Classes
Type
CrowdSourced
9,456
2,833
14
256
2
Biosignal
DREAMERA
140,638
29,608
14
256
2
Biosignal
DREAMERV
140,638
29,608
14
256
2
Biosignal
FordChallenge
29,003
7,254
30
40
2
Sensor
LenDB
983,483
261,459
3
540
2
Sensor
Opportunity
12,184
5,202
113
100
5
Motion
Table 7: The 18 new MTSC archive datasets from the Monash Monster archive multivariate datasets.
Figure 1: Given a skeleton sequence of F frames, where each frame contains J joints described by D -dimensional features, the sequence is represented as a multivariate time series of length F with J⋅D channels.
Repository
Subjects
Ex.
Joints
Dim.
Ch.
Notes
IRDS
29
9
25
3
75
Binary, one dataset per exercise
KERAAL
11
3
11
7
77
Binary and multiclass, 6 total
KIMORE
71
5
18
3
54
Regression reformulated as binary classification
KINECAL
–
4
25
3
75
Binary clinical risk prediction
SPHERE
6
1
15
3
45
Anomaly detection reformulated as binary classification
UCDHE
106
2
8
2
16
Binary and multiclass, 4 total
Table 8: Summary of the Rehab-Pile dataset repositories included in the Multiverse archive.
Dataset name
Train
Test
Channels
Length
Classes
Type
IRDS-EFL
223
26
75
60
2
Motion
IRDS-EFR
256
15
75
75
2
Motion
IRDS-SAL
213
56
75
82
2
Motion
IRDS-SAR
218
33
75
76
2
Motion
IRDS-SFE
237
15
75
95
2
Motion
IRDS-SFL
253
116
75
111
2
Motion
Table 9: The 39 new MTSC archive datasets from the classification portion of the Rehab-Pile archive, all of type Motion.
Dataset name
Train
Test
Channels
Length
Classes
Type
AppliancesEnergy_disc
95
42
24
144
2
Power
AustraliaRainfall_disc
112,186
48081
3
24
4
Sensor
BIDMC32HR_disc
5550
2399
2
4000
3
Biosignal
BIDMC32RR_disc
5471
2399
2
4000
3
Biosignal
BIDMC32SpO2_disc
5550
2399
2
4000
2
Biosignal
BeijingPM10Quality_disc
11,918
5048
9
24*
2
Sensor
Table 10: The 15 MTSC archive datasets from the discretised regression archive. For datasets with unequal length series, both the minimum and maximum series lengths are shown. HouseholdPowerConsumption is compressed to HPC for formatting.
Figure 2: The original target labels for the discretised regression problems. The targets for each case have been sorted and plotted in a scatter graph for each problem. The colour of each point represents its post-discretisation class, showing where class splits were made and the resulting class distribution.
Dataset name
Train
Test
Channels
Length
Classes
Type
AsphaltObstaclesCoordinates
390
391
3
111-736
4
Motion
AsphaltPavementTypeCoordinates
1055
1056
3
66-2371
3
Motion
AsphaltRegularityCoordinates
751
751
3
66-4201
2
Motion
AutomotiveRoadTrials
153
77
8
300
2
Sensor
BoneIntensitiesAgeGroup
600
445
8
270
3
Image
BoneProbAgeGroup
600
445
8
256
3
Image
Table 11: The 11 new standalone MTSC archive datasets. For datasets with unequal length series, both the minimum and maximum series lengths are shown.
Figure 3: An example hand X-ray, with the regions of interest around each of the phalanges of the middle finger (A). An example of a bone segmentation being converted into a univariate series (B). Images from the original PhD thesis Davis (2013) , all credit the author.
Figure 4: Photo examples of the device used to record data (A) and the type of road conditions present in the different street classes from the AsphaltPavementTypeCoordinates dataset (B). This figure is borrowed from Figure 1 and 3 of Souza (2018) which introduced the data. All credit to the original authors.
Figure 5: Illustration of the surface electromyography (sEMG) (A) and angle (B and C) sensors. This figure is borrowed from Figure 1 and 2 of the challenge GitHub ( https://github.com/Mvrjustid/EmoPainChallenge2020 ). All credit to the competition organisers.
Figure 6: An example of a counter movement jump split into separate stages: standing (a), preparation (b), take-off (c), flight (d), landing (e), and recovery (f). This figure is borrowed from Figure 1 of Hellmers et al. (2017) (unrelated to the source of this dataset). All credit to the original authors.
Figure 7: The classes used in TactileTextureRecognition, with visual examples of the robot arm and textures. This figure is borrowed from Figure 5 of Markert et al. (2021) . All credit to the original authors.
Dataset name
Train
Test
Channels
Length
Classes
Type
AsphaltObstaclesCoordinates_eq
390
391
3
298
4
Motion
AsphaltPavementTypeCoordinates_eq
1055
1056
3
400
3
Motion
AsphaltRegularityCoordinates_eq
751
751
3
384
2
Motion
BeijingPM10Quality_disc_nmv
11918
5048
9
24
2
Sensor
BeijingPM25Quality_disc_nmv
11918
5048
9
24
2
Sensor
BenzeneConcentration_disc_nmv
3349
5163
8
240
2
Sensor
Table 12: The 14 cleaned versions of datasets from the MTSC archive with unequal length series or missing values. This includes _eq datasets which are variants with equal length series and _nmv datasets which are variants with no missing values. HouseholdPowerConsumption is compressed to HPC for formatting.
Table 13: Summary statistics for the 66 Multiverse-core datasets. Provides statistics for the train set sizes (a), test set sizes (b), number of classes (c), number of time points (d), number of dimensions (e) of the archive datasets, and dataset type (f).
Figure 8: An accuracy rank critical difference diagram for multivariate-capable classifiers on 74 Multiverse datasets.
Figure 9: An MCM diagram for the top-performing multivariate-capable classifiers on 74 Multiverse datasets.
Figure 10: A scatter diagram comparing HC2 to CIF (A) and HC2 to ROCKET (B) on 74 Multiverse datasets.
Figure 11: An accuracy rank critical difference diagram for multivariate-capable classifiers on 42 Multiverse-core datasets.
Figure 12: An accuracy rank critical difference diagram for algorithms that completed all 66 Multiverse-core datasets.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset name
Train
Test
Channels
Length
Classes
Type
Alzheimers
45
43
19
15000
3
Biosignal
AppliancesEnergy_disc
95
42
24
144
2
Power
ArticularyWordRecognition
275
300
9
144
25
Motion
AsphaltObstaclesCoordinates
390
391
3
111-736
4
Motion
AsphaltObstaclesCoordinates_eq
390
391
3
298
4
Motion
AsphaltPavementTypeCoordinates
1055
1056
3
66-2371
3
Motion
Appendix
Table 14: Details for all 147 MTSC archive datasets. For datasets with unequal length series, both the minimum and maximum series lengths are shown. HouseholdPowerConsumption is compressed to HPC for formatting.
Table 15: The class values for all 133 archive datasets, excluding cleaned variants. Each class value is followed by the percentage of the dataset it represents (both train and test sets concatenated). The class names used in the .ts files for each problem are displayed. HouseholdPowerConsumption is compressed to HPC for formatting.
Dataset
Label split point(s)
Split description
AppliancesEnergy
Below average usage <15.81 Above average usage: ≥15.81
Midrange of target values
HPC1
Low consumption: <936 Medium consumption: ≥936,<2150 High consumption: ≥2150
Median of target values + and - standard deviation
HPC2
Low consumption: <121 Medium consumption: ≥121,<221 High consumption: ≥221
Median of target values + and - standard deviation
BenzeneConcentration
Below EU limit: <5 Above EU limit: ≥5
European Union air quality standards limit of 5 μg/m3 7 7 7 https://web.archive.org/web/20241118204135/https://environment.ec.europa.eu/topics/air/air-quality/eu-air-quality-standards_en
BeijingPM25Quality
Below 2013 median: <70 Above 2013 median: ≥70
2013 median of 74 Chinese cities from Table 1 of [ 34 ]
BeijingPM10Quality
Below 2013 median: <108 Above 2013 median: ≥108
2013 median of 74 Chinese cities from Table 1 of [ 34 ]
Appendix
Table 16: The original target label split points used to create class labels for the discretised regression problems. HouseholdPowerConsumption is compressed to HPC for formatting.
Arsenal
CIF
FP
HC2
MRHydra
QUANT
RDST
ROCKET
Alzheimers
0.442
0.419
0.349
0.488
0.395
0.419
0.419
AppliancesEnergy_disc
0.857
0.905
0.952
0.881
0.833
0.929
0.786
0.786
ArticularyWordRecognition
0.993
0.98
0.977
0.993
0.993
0.983
0.993
0.993
AsphaltObstaclesCoordinates_eq
0.854
0.824
0.839
0.857
0.859
0.818
0.867
0.854
AsphaltPavementTypeCoordinates_eq
0.86
0.917
0.942
0.889
0.912
0.933
0.896
0.864
AsphaltRegularityCoordinates_eq
0.936
0.977
0.991
0.971
0.988
0.988
0.983
0.939
Appendix
Table 17: Accuracy result for the top 8 classifiers from Section 6 for all 133 clean datasets. Experiments which produced no results are left blank.
1NN-DTW
Catch22
DrCIF
Dummy
H-IT
LITETime
RIST
STC
TDE
Alzheimers
0.279
0.488
0.442
0.419
0.326
0.372
0.442
0.372
0.372
AppliancesEnergy_disc
0.643
0.881
0.905
0.81
0.524
0.738
0.905
0.857
0.81
ArticularyWordRecognition
0.987
0.983
0.987
0.04
0.99
0.983
0.987
0.973
0.993
AsphaltObstaclesCoordinates_eq
0.578
0.808
0.821
0.284
0.854
0.847
0.821
0.821
0.818
AsphaltPavementTypeCoordinates_eq
0.583
0.923
0.934
0.386
0.963
0.964
0.937
0.855
0.839
AsphaltRegularityCoordinates_eq
0.703
0.976
0.987
0.507
0.995
0.995
0.987
0.951
0.947
Appendix
Table 18: Accuracy result for the remaining classifiers from Section 6 for all 133 clean datasets. Experiments which produced no results are left blank.
Recent years have witnessed the emergence of multivariate modeling using time series foundation models (TSFMs), which achieve advanced zero-shot generalization. Modern multivariate TSFMs are predominantly pretrained on multivariate synthetic data, which is easier to scale but may fail to capture the complex temporal dynamics and cross-variable relationships present in real-world time series. This raises a key question: Whether and to what extent the leading TSFMs trained with the real-world corpus perform better than those trained with synthetic data? To answer this, we establish the RMISC corpus, a considerably large-scale, high-quality, openly accessible, real-world, and multivariate time series archive that contains around 200 datasets and 142 billion time points across diverse domains. Furthermore, we pretrain four advanced TSFMs on univariate, synthetic multivariate, and real-world multivariate data and evaluate their zero-shot generalization capabilities on standard in-distribution and out-of-distribution benchmarks. Experimental results show that incorporating real-world multivariate data predominantly improves the generalization performance for both univariate and multivariate TSFMs. These results provide a deeper understanding of how real-world multivariate data contributes to the development of stronger TSFMs.
Qian Sun, Yong-Ming Tian, Jia-Wei Huang +2
State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing, China · School of Intelligent Science and Technology, Nanjing University, Suzhou, China · Nanjing University – Siemens Joint Research Center on Industrial AI, Suzhou, China +1
Time series classification (TSC) performance depends not only on architectural design but also on the diversity of input representations. In this work, we propose a scalable multi-scale convolutional framework that systematically integrates structured multi-representation inputs for univariate time series. We introduce two architectures: MRMS-Net, a hierarchical multi-scale convolutional network optimized for robustness and calibration, and LMRMS-Net, a lightweight variant designed for efficiency-aware deployment. In addition, we adapt LiteMV -- originally developed for multivariate inputs -- to operate on multi-representation univariate signals, enabling cross-representation interaction. We evaluate all models across 142 benchmark datasets under a unified experimental protocol. Critical Difference (CD) analysis confirms statistically significant performance differences among the top models. Results show that LiteMV achieves the highest mean accuracy, MRMS-Net provides superior probabilistic calibration (lowest NLL), and LMRMS-Net offers the best efficiency-accuracy tradeoff. Pareto analysis further demonstrates that multi-representation multi-scale modeling yields a flexible design space that can be tuned for accuracy-oriented, calibration-oriented, or resource-constrained settings. These findings establish scalable multi-representation multi-scale learning as a principled and practical direction for modern TSC. Reference implementation of MRMS-Net and LMRMS-Net is available at: https://github.com/alagoz/mrmsnet-tsc
Celal Alagöz, Mehmet Kurnaz, Farhan Aadil
Computer Engineering, Sivas Bilim ve Teknoloji Üniversitesi, Sivas, Türkiye
Time series classification (TSC) is dominated by a two-stage paradigm: train a feature encoder -- either from scratch on the target dataset or via pretraining on large corpora -- and then fit a task-specific classifier on top. While effective, this decoupling optimizes representation learning independently of the classification objective, requires per-dataset training, and prevents the model from exploiting label information during inference. We introduce TimEE, a 4.5M-parameter foundation model for end-to-end TSC via in-context learning. Given a labeled support set and a query time series, TimEE directly outputs a predicted class distribution in a single forward pass with no per-dataset training required. Following the prior-data fitted network (PFN) framework, TimEE is meta-trained exclusively on synthetic TSC tasks, where each task contains time series with distinct class identities arising from structured distributional shifts in the generative process. Despite seeing no real time series during pre-training, TimEE ranks first in ROC AUC (and third on accuracy) on the UCR benchmark among all compared methods, which include both foundation models and supervised deep learning baselines. To our knowledge, TimEE is the first purely synthetic-pretrained model to reach state-of-the-art performance on the UCR benchmark. These results establish end-to-end ICL with synthetic priors as a compelling, largely unexplored direction for TSC, with scaling, prior design, and richer generation mechanisms as natural avenues for improvement. Code is publicly available at http://github.com/automl/timee.
Jaris Küken, Shi Bin Hoo, Martin Mráz +2
University of Freiburg · Zuse School ELIZA Darmstadt · Prior Labs +1