Recurrent Domain Incremental Adaptation of Time-Series Foundation Models
Organizations: Nanyang Technological University, Singapore · Institute of Advanced Intelligence and Computing (IAIC), A*STAR, Singapore · Department of Computer Science, Khalifa University, UAE
Abstract
Pretrained Time-Series Foundation Models (TSFMs) provide strong representations for downstream classification tasks. Yet adapting a TSFM under changing operating conditions remains challenging. A model must improve on each newly observed condition without sacrificing conditions it has already learned, often from only a small labeled calibration set and without knowing the operating condition at inference time. We find that this difficulty has an important geometric structure. Under an operating-condition shift, the embedding space of a TSFM does not move uniformly: some local regions change substantially, while others remain nearly unchanged. Consequently, a global adaptation can unnecessarily move embeddings that were already correctly classified. To address this, we propose the novel PACT (Piecewise Adaptation via Chart Transport). PACT partitions the frozen source embedding space into local charts and learns lightweight residual transports for the regions affected by each new condition. An explicit identity path is always retained, and a learned gate softly selects among the identity and accumulated transports for each input. The pretrained TSFM encoder, classifier head, and previously learned transports remain fixed. With only 5% labeled calibration data per arriving condition and no condition identity at test time, PACT maintains an average macro-F1 above 0.95 across a four-condition bearing-fault stream. Under homogeneous tasks and label spaces, similar retention-adaptation behavior is observed across public benchmarks. Code is available at https://github.com/Sachithx/PACT.
Figures & tables
| FD | WISDM | HHAR | UCIHAR | Sleep-EDF | ||||||||||||
| Method | AMF 1 | RF | PRF | AMF 1 | RF | PRF | AMF 1 | RF | PRF | AMF 1 | RF | PRF | AMF 1 | RF | PRF | |
| No-DA | Source-only | 0.7084 | 0.0000 | 0.0000 | 0.5263 | 0.0000 | 0.0000 | 0.6768 | 0.0000 | 0.0000 | 0.5024 | 0.0000 | 0.0000 | 0.5313 | 0.0000 | 0.0000 |
| Few-shot | NCM prototypes | 0.7336 | 0.0009 | 0.0001 | 0.6761 | 0.1245 | 0.0492 | 0.7145 | 0.0026 | 0.0007 | 0.5436 | 0.1099 | 0.0551 | 0.5043 | 0.0203 | 0.0099 |
| TTA/SFDA | CoTTA | 0.7309 | 0.2625 | 0.1262 | 0.7793 | 0.1853 | 0.0703 | 0.8013 | 0.1123 | 0.0291 | 0.6704 | 0.2534 | 0.1041 | 0.5532 | 0.1413 | 0.0696 |
| EATA | 0.7546 | 0.2240 | 0.1021 | 0.7882 | 0.1739 | 0.0666 | 0.8063 | 0.1043 | 0.0267 | 0.6904 | 0.2265 | 0.0913 | 0.5693 | 0.1212 | 0.0580 | |
| MAPU† | 0.5797 | 0.1555 | 0.0490 | 0.4081 | 0.0926 | 0.0541 | 0.5017 | 0.1707 | 0.0910 | 0.2555 | 0.1593 | 0.1184 | 0.3296 | 0.1527 | 0.1012 | |
| Chart-count sweep | Equal-capacity placement | |||||
| Dataset | Source -means | Random projection | ||||
| FD | 0.9386 | 0.9484 | 0.9575 | 8 | 0.9575 | 0.8936 |
| WISDM | 0.8717 | 0.8774 | 0.8420 | 4 | 0.8774 | 0.8078 |
| HHAR | 0.8318 | 0.8369 | 0.8090 | 4 | 0.8367 | 0.7839 |
| UCI-HAR | 0.7743 | 0.7862 | 0.7725 | 4 | 0.7864 | 0.7553 |
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | Stream | Domain sequence |
| FD | 1 | |
| 2 | ||
| 3 | ||
| 4 | ||
| WISDM | 1 | |
| 2 |
| Dataset | Stream SD | Seeds | PACT | Comparator |
| FD | 0.0106 | 3 | ||
| WISDM | 0.0197 | 2 | ||
| HHAR | 0.0214 | 3 | ||
| UCI-HAR | 0.0144 | 2 | ||
| Sleep-EDF | 0.0316 | 3 |
| Full-history capacity | Full history | 512 embeddings | ||||
| Dataset | Embeddings | DER++ slots | PACT | Replay | PACT | Replay |
| FD | 2,104 | 315 | 0.9575 | 0.9096 | 0.9556 | 0.9118 |
| WISDM | 978–1,280 | 1,916–2,507 | 0.8774 | 0.8629 | 0.8206 | 0.8461 |
| HHAR | 1,630–1,708 | 3,193–3,346 | 0.8367 | 0.8648 | 0.8252 | 0.8622 |
| UCI-HAR | 1,026–1,066 | 679–705 | 0.7864 | 0.8169 | 0.7513 | 0.8078 |
| Sleep-EDF | 1,927–2,524 | 492–644 | 0.6129 | 0.6357 | 0.6003 | 0.6255 |
| Dataset | Source -means | Random projection | Streams won | |
| FD | 8 | 0.9575 | 0.8936 | 4/4 |
| WISDM | 4 | 0.8774 | 0.8078 | 4/4 |
| HHAR | 4 | 0.8367 | 0.7839 | 4/4 |
| UCI-HAR | 4 | 0.7864 | 0.7553 | 4/4 |
| UCI-HAR | 8 | 0.7725 | 0.7023 | 4/4 |
| Dataset | |||
| FD | 0.0018 | 0.0021 | 0.0022 |
| WISDM | 0.0317 | 0.0292 | 0.0458 |
| HHAR | 0.0193 | 0.0171 | 0.0142 |
| UCI-HAR | 0.0549 | 0.0436 | 0.0462 |
| Dataset | PACT ( ) | PASS adaptation | Pooled FeCAM |
| FD | 0.9507 | 0.8392 | 0.9195 |
| WISDM | 0.8576 | 0.8084 | 0.8297 |
| UCI-HAR | 0.7941 | 0.7629 | 0.7939 |
| Target update | |||||||||
| Dataset | Initial | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
| FD | .9876 | – | – | – | – | – | |||
| WISDM | .9345 | ||||||||
| HHAR | .9673 | ||||||||
| UCI-HAR | .9357 | ||||||||
| Sleep-EDF | .7591 | ||||||||
| Dataset | PACT | Embedding replay | Head fine-tuning |
| FD | |||
| WISDM | |||
| HHAR | |||
| UCI-HAR | |||
| Sleep-EDF |
| Diagnostic | FD | HHAR | UCI-HAR |
| Source examples with | 99.9 | 97.5 | 77.9 |
| Originally correct source examples certified | 99.8 | 97.5 | 71.1 |
| Certified predictions preserved | 100.0 | 100.0 | 100.0 |
| Originally correct source predictions preserved | 99.9 | 99.7 | 92.7 |
| Target chart–condition cells masked | 33.3 | 19.5 | 10.2 |
| Temperature ( ) | Minimum support ( ) | |||||
| Dataset | 0.25 | 0.5 | 1.0 | 1 | 5 | 10 |
| FD | 0.9589 | 0.9591 | 0.9599 | 0.9585 | 0.9591 | 0.9517 |
| UCI-HAR | 0.7834 | 0.7911 | 0.8048 | 0.7860 | 0.7911 | 0.7834 |
| Calibration setting | Metric | Streams | FD | UCI-HAR |
| Class-stratified | AMF 1 | 4 | 0.9575 | 0.7864 |
| 85% from one class | AMF 1 | 4 | 0.8667 | 0.7726 |
| Class 0 omitted | Class 0 recall | 4 | 0.995 | 0.154 |
| Class 1 omitted | Class 1 recall | 1 | 0.138 | 0.016 |
Domain-Incremental Learning
Domain-incremental learning studies how models adapt to changing input distributions while retaining performance on previously encountered domains.
See all 61 papers