cs.LGApr 9, 2026

ADAPTive Input Training for Many-to-One Pre-Training on Time-Series Classification

Authors: Paul Quinlan, Qingguo Li, Xiaodan Zhu

Organizations: Electrical and Computer Engineering, Queen’s University · Ingenuity Labs Research Institute, Queen’s University · Mechanical and Materials Engineering, Queen’s University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Recent work on time-series models has leveraged self-supervised training to learn meaningful features and patterns in order to improve performance on downstream tasks and generalize to unseen modalities. While these pretraining methods have shown great promise in one-to-many scenarios, where a model is pre-trained on one dataset and fine-tuned on a downstream dataset, they have struggled to generalize to new datasets when more datasets are added during pre-training. This is a fundamental challenge in building foundation models for time-series data, as it limits the ability to develop models that can learn from a large variety of diverse datasets available. To address this challenge, we present a new pre-training paradigm for time-series data called ADAPT, which can efficiently align the physical properties of data in the time-series domain, enabling mixed-batch pre-training despite the extreme discrepancies in the input sizes and channel dimensions of pre-training data. We trained on 162 time-series classification datasets and set new state-of-the-art performance for classification benchmarks. We successfully train a model within the time-series domain on a wide range of datasets simultaneously, which is a major building block for building generalist foundation models in time-series domains.

Figures & tables

Figure 1. An overview of the adapt process and time-frequency masking algorithm. Each time-series is adaptive-pooled (in the [A] boxes) to a universal representation space and enable mixed-batch training among different modality types. At training time we add noise to the joint representation space in both the time and frequency space.
Table 2
Dataset: EpilepsyDataset: FD-B
Model Acc Prec Rec F1 TS-SD 89.5 5.2 80.2 22.4 76.5 14.9 77.7 18.6 TS2vec 94.0 0.4 90.6 1.2 90.4 1.2 90.5 0.7 CLOCS 95.1 0.3 93.0 0.7 91.3 1.7 92.1 0.7 Mixing-up 80.2 0.0 40.1 0.0 50.0 0.0 44.5 0.0 TS-TCC 92.5 1.0 94.5 0.5 81.8 2.6 86.3 2.2 SimCLR 90.7 3.4 92.2 1.7 78.6 10.7 81.8 10.0 TF-C 94.9 2.5 94.6 1.1 89.1 2.2 91.5 5.3 ADAPT(EEG) 88.6 3.5 82.2 4.2 91.1 1.6 84.8 3.9 ADAPT 93.6 2.9 90.1 4.9 91.7 0.9 88.5 4.6Model Acc Prec Rec F1 TS-SD 55.7 2.1 57.1 5.4 60.5 2.7 57.0 3.3 TS2vec 47.9 1.1 43.4 0.9 48.4 2.0 43.9 1.1 CLOCS 49.3 3.1 48.2 3.2 58.7 3.9 47.5 4.9 Mixing-up 67.9 2.5 71.5 3.4 76.1 2.0 72.7 2.3 TS-TCC 55.0 2.2 52.8 2.9 64.0 1.8 54.2 3.4 SimCLR 49.2 4.4 54.5 10.2 47.6 8.9 42.2 11.4 TF-C 69.4 2.3 75.6 3.5 72.0 2.6 74.9 2.7 ADAPT(EEG) 97.3 2.8 98.2 1.8 98.0 2.0 98.0 2.0 ADAPT 91.2 4.2 91.0 4.1 91.8 4.2 88.8 5.7
Dataset: GestureDataset: EMG
Model Acc Prec Rec F1 TS-SD 69.2 4.4 67.0 4.7 68.7 4.9 66.6 4.4 TS2vec 69.2 3.3 65.5 3.6 68.5 3.5 65.7 3.9 CLOCS 44.3 5.2 42.4 7.9 44.3 5.2 40.1 6.0 Mixing-up 69.3 2.3 67.2 2.3 69.3 2.3 65.0 3.1 TS-TCC 71.9 3.5 71.4 3.5 71.7 3.7 69.8 3.6 SimCLR 48.0 5.9 59.5 16.2 54.1 19.5 49.6 18.7 TF-C 76.4 2.0 77.3 3.6 74.3 2.7 75.7 3.1 ADAPT(EEG) 72.5 1.2 70.8 0.8 72.5 1.2 70.7 0.7 ADAPT 77.0 2.5 74.9 3.7 77.0 2.5 75.1 2.9Model Acc Prec Rec F1 TS-SD 46.1 0.0 15.5 0.0 33.3 0.0 21.1 0.0 TS2vec 78.5 3.2 80.4 7.5 67.9 4.0 67.7 5.0 CLOCS 69.9 3.2 53.1 7.5 53.5 2.9 51.4 4.1 Mixing-up 30.2 5.3 11.0 1.3 25.8 4.6 15.4 2.0 TS-TCC 78.9 1.9 58.5 9.7 63.1 9.9 59.0 9.5 SimCLR 61.5 5.8 53.6 17.2 49.9 12.1 47.1 14.9 TF-C 81.7 2.9 72.7 3.5 81.6 2.9 76.8 3.1 ADAPT(EEG) 96.6 4.5 94.4 6.1 97.3 3.6 95.0 6.2 ADAPT 98.5 1.2 96.7 2.7 98.8 1.0 97.6 2.0
Table 3. Overall classification accuracy of ADAPT compared to other state-of-the-art pre-trained models for time-series classification. Baseline implementations are from [ 41 ] and are pretrained on the SleepEEG dataset. We pre-train two models, ADAPT(EEG) which is only pre-trained on the SleepEEG datasets, and ADAPT which is trained on all of the pretraining datasets in Section 5.2. The subscripts are standard deviations.
Figure 2. Model accuracy on UCR and UEA datasets compared to basic dataset properties. The “Length” and “Channel Dimension” corresponds to the dimensionality of the data or the total length of the input and number of channels at each time period. “Number of Classes” corresponds to the number of classes for classification in the downstream dataset, and “Total Dimensional Size” refers to length of the input multiplied by the number of channel dimensions. Overall, we can see that the performance is not strongly linked to the physical properties of the datasets. The number of classes of a dataset could represent an increase in dataset complexity and the model has a weak negative correlation as expected.
Figure 3. T-SNE visualisation of the time and frequency inputs before and after adaptive pooling on the Gesture dataset. We note that there is no discernible drop in the quality of inputs after they are transformed. The relative groupings and distributions between the 8 different classes remain largely intact.
More in this topic

Time Series Representation Learning

Time series representation learning develops compact, transferable features that capture temporal patterns for forecasting, classification, retrieval, and anomaly detection.

See all 94 papers

Explore similar work

CardsList
Mar 2, 2026

UTICA: Multi-Objective Self-Distllation Foundation Model Pretraining for Time Series Classification

Self-supervised foundation models have achieved remarkable success across domains, including time series. However, the potential of non-contrastive methods, a paradigm that has driven significant advances in computer vision, remains underexplored for time series. In this work, we adapt DINOv2-style self-distillation to pretrain a time series foundation model, building on the Mantis tokenizer and transformer encoder architecture as our backbone. Through a student-teacher framework, our method Utica learns representations that capture both temporal invariance via augmented crops and fine-grained local structure via patch masking. Our approach achieves state-of-the-art classification performance on both UCR and UEA benchmarks. These results suggest that non-contrastive methods are a promising and complementary pretraining strategy for time series foundation models.

Yessin Moakher, Youssef Attia El Hili, Vasilii Feofanov

Ecole Polytechnique, France · Huawei Noah’s Ark Lab

Self-Supervised Time Series Representation LearningTime-Series Foundation ModelsTime Series ClassificationSelf-Supervised LearningSelf-Distillation
AskPDF
Feb 21, 2025

Mantis: Lightweight Foundation Model for Time Series Classification

While foundation models have revolutionized various domains, their application to time series classification remains rather under-explored, with existing literature predominantly focused on forecasting. To bridge this gap, we introduce Mantis, a transformer-based foundation model pre-trained exclusively on synthetic data via self-supervised contrastive learning. We demonstrate that effective tokenization is critical to unlocking the full potential of transformers, proposing a novel token generator unit. Furthermore, we introduce an enhanced test-time methodology that bridges the performance gap between Mantis and strong specialized approaches by leveraging intermediate-layer representations, self-ensembling, and cross-model embedding fusion. Extensive experiments demonstrate that Mantis establishes a new state-of-the-art, outperforming existing foundation models across four diverse dataset collections covering various application domains.

Vasilii Feofanov, Songkang Wen, Shifeng Xie +10

1. Huawei Noah’s Ark Lab · 3. Paris Cit´e University · 4. Helmholtz Munich +3

Time Series ClassificationSelf-Supervised Time Series Representation LearningTime Series TokenizationTime-Series Foundation Models
AskPDF
Jul 22, 2026

Post-Training in Time Series Foundation Models: A Unifying Framework

Time series foundation models (TSFMs) have emerged as general-purpose models for time series analysis, but pretraining alone is often insufficient for reliable downstream deployment. Bridging this gap requires further intervention to handle domain shift, task heterogeneity, limited supervision, and computational constraints, which motivates post-training as a broad class of methods to adapt, augment, compose, calibrate, or specialize pretrained TSFMs for downstream tasks. In this work, we analyze TSFM post-training methods based on their locus of intervention in the prediction pipeline, yielding five categories: parameter adaptation, context augmentation, model composition, output processing and uncertainty control, and compression and specialization. Within each category, we study main representative methods and discuss their current limitations. We further identify future directions toward controlled adaptation, reliable context construction, uncertainty-aware model composition, calibrated output processing, and deployment-aware specialization. Overall, by providing a unifying framework for the emerging TSFM post-training landscape, this work aims to support future research to navigate the design space between a pretrained TSFM and its reliable downstream deployment.

Shifeng Xie, Ambroise Odonnat, Zehao Xiao +7

Noah’s Ark Lab · Université Paris Cité · Inria +2

Model CompressionTime-Series Foundation Models
AskPDF