TS2Vec is one of the strongest self-supervised methods for time series classification, introduced in 2022. It is designed to be trained anew on every dataset: self-supervised pre-training followed by linear probing. Since then, the field has shifted towards foundation models that reuse one encoder across datasets. Yet TS2Vec offers a compact convolutional architecture that operates directly on temporal samples and accepts variable-length inputs without resampling. We ask whether these properties can be retained when the encoder is pre-trained only once, and show that they can: TS2Vec can essentially be turned into a foundation model. Three changes suffice: pre-train on a synthetic corpus, feed the encoder a raw amplitude view and an instance-normalized shape view of the input through two small stems, and distill one specialist encoder per view into a single encoder. The result, TS2Vec-FM, is a 1.6M-parameter dilated convolutional network that, kept frozen, reaches 83.3% on UCR-128 and 75.3% on UEA-27 under linear probing, within 0.3 pp of MantisV2, the current state of the art, and above TS2Vec trained separately on each target dataset. What we find most interesting is how different TS2Vec-FM is from other foundation models in design: near-parity is reached without attention, without a patch tokenizer, and without intermediate-layer extraction, using an "old-school" convolutional encoder and a two-layer MLP stem.
Figures & tables
Figure 1: TS2Vec-FM architecture and training. (a) Raw and instance-normalized inputs pass through an MLP and a linear stem, respectively. Their concatenation feeds a shared convolutional trunk and temporal max pooling. (b) Two specialists are pretrained on synthetic data with the TS2Vec objective and then frozen. On the same data, each teacher supervises one 320-dimensional half of the student through timestamp-level and pooled cosine losses. Downstream, only a classifier on frozen student features is fitted.
Random forest
LogReg
Model
Type
Params
UCR
UEA
UCR
UEA
Catch22+
statistics
—
79.69
69.63
77.04
67.56
MOMENT
forecasting
161.4M
77.89
69.91
79.84
71.74
NuTime
classification
2.0M
77.32
71.99
79.05
74.91
TiRex
forecasting
16.5M
80.13
69.52
82.02
72.48
Chronos-2
forecasting
44.0M
80.02
69.67
82.44
72.43
Table 1: Zero-shot feature extraction accuracy (%), with parameters counted after layer pruning where applicable. The dashed rule separates the original per-dataset TS2Vec, which is not a foundation model: it trains one encoder on each dataset’s own training split (128 UCR and 27 UEA encoders of 0.6M parameters). ‡ With its original RBF-SVM probe it reaches 82.10 on UCR.
100K series
1M series
2M series
Encoder input
Training
UCR
UEA
UCR
UEA
UCR
UEA
raw only
contrastive
77.43 †
72.02 †
76.99
71.56
77.68
72.22
normalized only
contrastive
80.35 †
70.56 †
80.26
69.78
79.89
70.19
dual-input
contrastive
80.52 †
73.41 †
80.52 ‡
73.41 ‡
80.52 ‡
73.07 ‡
dual-input
distillation (ours)
81.08 †
73.90 †
81.62 †
73.48 †
81.76
73.60
Table 2: What each ingredient contributes (random forest, %) at 100K, 1M and 2M synthetic series. Instance normalization trades UEA for UCR at every scale, the specialists and the direct contrastive baseline do not improve with more data, and distillation consolidates the two specialists into the stronger single encoder at every scale. † mean over three pre-training seeds; ‡ with gradient clipping.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 2: Reference TS2Vec encoder. Ten width-64 residual blocks with dilation 2i are followed by a width-320 output block; one block is detailed in Figure 3 .
Figure 3: One TS2Vec residual block: two GELU–dilated-convolution pairs plus a skip connection (identity, or a 1×1 convolution when the width changes).
ours, per pre-training seed
Dataset
published
42
43
44
45
46
mean ± sd
PLAID
56.1
37.4
30.9
19.4
42.8
44.1
34.9 ± 10.1
Car
83.3
70.0
78.3
71.7
83.3
70.0
74.7 ± 5.9
DodgerLoopDay
56.2
47.5
48.8
48.8
50.0
52.5
49.5 ± 1.9
Herring
64.1
56.2
59.4
59.4
68.8
57.8
60.3 ± 4.9
FaceFour
93.2
86.4
84.1
90.9
89.8
89.8
88.2 ± 2.9
Appendix
Table 3: The eight datasets on which the seed-42 reproduction deviates from the published value by more than 5 pp, with a four-extra-seed diagnostic (per-dataset TS2Vec, RBF-SVM probe, %). The main per-dataset baseline uses seed 42.
Method
Probe
UCR acc.
W / T / L
of TS2Vec-FM
TS2Vec-FM
LogReg
83.32
—
TS2Vec/ds (reproduced)
LogReg
81.36
88 / 9 / 31
TS2Vec/ds (reproduced)
RBF-SVM
82.10
71 / 11 / 46
TS2Vec/ds (published)
RBF-SVM
82.87
62 / 10 / 56
Appendix
Table 4: TS2Vec-FM (one encoder, frozen, synthetic pre-training only) against TS2Vec trained per dataset (TS2Vec/ds) on UCR-128. W / T / L counts the datasets on which TS2Vec-FM wins, ties and loses against that row; p -values (two-sided Wilcoxon signed-rank) are given in the text.
Task
Benchmark
Metric
published
ours †
classification
UCR-125
accuracy
0.830
0.8257
UCR-125
avg. rank
1.82
1.876
UEA-29
accuracy
0.712
0.7055
UEA-29
avg. rank
2.40
2.776
UCR-125, B=4
accuracy
0.824
0.8267
UCR-125, B=16
accuracy
0.827
0.8250
Appendix
Table 5: Reproducing the original TS2Vec paper with unmodified released code. Accuracy and F1/precision/recall higher is better; MSE/MAE lower is better. † mean over three seeds (42, the paper’s own, plus 43 and 44; 1–3 for anomaly detection, as in the released scripts). Status uses a 5% relative tolerance.
Figure 4: Colours are shared across panels; accuracy is UCR-128 with the random-forest probe. (a) At 100K, the distilled student stabilizes after the first few epochs and remains above the same trunk trained contrastively; bands span the three pre-training seeds. (b) At 2M (one seed) the student peaks at epoch 7 and then declines, while the gradient-clipped contrastive control stays within 0.56 pp of its epoch-1 accuracy across ten epochs. (c) Probing each dilated residual block of the two 100K specialists: accuracy increases with depth and the final block is best for both.
Input handling
UCR RF
UCR LogReg
UEA RF
UEA LogReg
native lengths
81.76
83.32
73.60
75.30
resampled to 512
81.57
83.10
72.70
74.74
Appendix
Table 6: Native lengths versus resampling to 512 steps, for the released 2M model (epoch 7, %).
amplitude view
shape view
Batch
10−3
2⋅10−3
4⋅10−3
10−3
2⋅10−3
4⋅10−3
8
77.31
–
–
79.80
–
–
128
77.53
77.36
–
80.49
80.33
–
256
77.73
77.82
–
80.01
80.34
–
512
77.52
77.40
76.70
79.84
80.25
80.26
Appendix
Table 7: Teacher pre-training frontier at 100K (UCR random forest, best of the trained epochs, pre-training seed 42, %). Dashes are configurations we did not run.
Dataset
Catch22+
MOMENT
NuTime
TiRex
Chronos-2
TiViT-H
TiConvNext
MantisV2
TS2Vec / ds
TS2Vec-FM
ACSF1
82.3
82.0
75.0
85.0
84.7
85.0
86.7
82.3
87.7
86.7
Adiac
73.4
78.9
73.6
78.5
81.1
70.6
65.5
84.8
77.0
80.6
AllGestureWiimoteX
59.6
61.4
65.0
65.0
60.4
60.3
66.8
68.4
70.5
69.1
AllGestureWiimoteY
63.2
66.2
63.0
70.5
64.2
65.1
68.0
69.1
72.4
74.7
AllGestureWiimoteZ
54.6
57.7
62.2
60.6
62.0
61.3
60.7
68.8
68.0
69.7
ArrowHead
74.1
79.4
77.3
78.9
77.9
76.4
77.1
80.6
76.6
81.1
Appendix
Table 8: Per-dataset accuracy (%) on UCR-128 with the random-forest probe (mean over predictor seeds 0/1/2). TS2Vec / ds is the original TS2Vec trained on each dataset (the reference row of Table 1 ); TS2Vec-FM is our single frozen encoder. Best per row in bold.
Dataset
Catch22+
MOMENT
NuTime
TiRex
Chronos-2
TiViT-H
TiConvNext
MantisV2
TS2Vec / ds
TS2Vec-FM
ACSF1
70.0
75.0
70.0
75.0
86.0
83.0
83.0
76.0
85.0
85.0
Adiac
66.5
77.7
80.1
78.3
82.9
71.9
70.6
83.6
77.5
80.3
AllGestureWiimoteX
54.9
66.1
60.7
70.1
68.4
67.0
69.4
73.0
72.0
74.7
AllGestureWiimoteY
52.3
70.3
65.3
73.1
70.4
73.7
72.3
74.7
75.0
77.1
AllGestureWiimoteZ
48.3
60.6
54.1
63.4
66.0
67.7
65.9
68.1
65.6
72.3
ArrowHead
71.4
77.7
72.6
78.3
85.1
82.3
82.3
85.1
78.9
82.9
Appendix
Table 9: Per-dataset accuracy (%) on UCR-128 with the logistic-regression probe. Columns as in Table 8 .
Dataset
Catch22+
MOMENT
NuTime
TiRex
Chronos-2
TiViT-H
TiConvNext
MantisV2
TS2Vec/ds
TS2Vec-FM
ArticularyWordRecognition
96.0
98.4
99.1
99.0
99.3
98.3
98.0
99.2
96.0
99.1
BasicMotions
100.0
97.5
100.0
96.7
100.0
98.3
100.0
100.0
99.2
100.0
CharacterTrajectories
98.1
97.0
96.7
96.0
96.3
93.7
95.2
97.6
98.7
98.6
Cricket
94.4
97.2
99.1
99.5
98.6
100.0
99.5
98.6
97.7
98.6
DuckDuckGeese
51.3
44.7
42.7
42.0
41.3
43.3
44.0
54.0
53.3
50.7
EigenWorms
87.5
76.1
73.5
78.6
80.2
88.8
93.1
81.4
82.4
88.0
Appendix
Table 10: Per-dataset accuracy (%) on UEA-27 with the random-forest probe (mean over predictor seeds 0/1/2). TS2Vec/ds is the original TS2Vec trained on each dataset (the reference row of Table 1 ); TS2Vec-FM is our single frozen encoder. Best per row in bold.
Dataset
Catch22+
MOMENT
NuTime
TiRex
Chronos-2
TiViT-H
TiConvNext
MantisV2
TS2Vec/ds
TS2Vec-FM
ArticularyWordRecognition
93.3
98.7
99.3
99.0
99.3
98.3
97.7
99.3
99.0
99.3
BasicMotions
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
CharacterTrajectories
98.4
98.4
98.5
96.9
97.4
96.6
97.1
98.7
99.4
99.0
Cricket
90.3
98.6
100.0
100.0
100.0
100.0
100.0
98.6
98.6
100.0
DuckDuckGeese
42.0
48.0
50.0
38.0
34.0
48.0
56.0
50.0
56.0
56.0
EigenWorms
76.3
83.2
87.0
84.0
84.0
93.1
96.2
88.5
80.9
90.8
Appendix
Table 11: Per-dataset accuracy (%) on UEA-27 with the logistic-regression probe. Columns as in Table 10 .
Self-supervised foundation models have achieved remarkable success across domains, including time series. However, the potential of non-contrastive methods, a paradigm that has driven significant advances in computer vision, remains underexplored for time series. In this work, we adapt DINOv2-style self-distillation to pretrain a time series foundation model, building on the Mantis tokenizer and transformer encoder architecture as our backbone. Through a student-teacher framework, our method Utica learns representations that capture both temporal invariance via augmented crops and fine-grained local structure via patch masking. Our approach achieves state-of-the-art classification performance on both UCR and UEA benchmarks. These results suggest that non-contrastive methods are a promising and complementary pretraining strategy for time series foundation models.
Yessin Moakher, Youssef Attia El Hili, Vasilii Feofanov
Ecole Polytechnique, France · Huawei Noah’s Ark Lab
While foundation models have revolutionized various domains, their application to time series classification remains rather under-explored, with existing literature predominantly focused on forecasting. To bridge this gap, we introduce Mantis, a transformer-based foundation model pre-trained exclusively on synthetic data via self-supervised contrastive learning. We demonstrate that effective tokenization is critical to unlocking the full potential of transformers, proposing a novel token generator unit. Furthermore, we introduce an enhanced test-time methodology that bridges the performance gap between Mantis and strong specialized approaches by leveraging intermediate-layer representations, self-ensembling, and cross-model embedding fusion. Extensive experiments demonstrate that Mantis establishes a new state-of-the-art, outperforming existing foundation models across four diverse dataset collections covering various application domains.
Vasilii Feofanov, Songkang Wen, Shifeng Xie +10
1. Huawei Noah’s Ark Lab · 3. Paris Cit´e University · 4. Helmholtz Munich +3
Developing foundation models for time series classification is of high practical relevance, as such models can serve as universal feature extractors for diverse downstream tasks. Although early models such as Mantis have shown the promise of this approach, a substantial performance gap remained between frozen and fine-tuned encoders. In this work, we introduce methods that significantly strengthen zero-shot feature extraction for time series. First, we introduce Mantis+, a variant of Mantis pre-trained entirely on synthetic time series. Second, through controlled ablation studies, we refine the architecture and obtain MantisV2, an improved and more lightweight encoder. Third, we propose an enhanced test-time methodology that leverages intermediate-layer representations and refines output-token aggregation. In addition, we show that performance can be further improved via self-ensembling and cross-model embedding fusion. Extensive experiments on UCR, UEA, Human Activity Recognition (HAR) benchmarks, and EEG datasets show that MantisV2 and Mantis+ consistently outperform prior time series foundation models, achieving state-of-the-art zero-shot performance.