Organizations: Institute of Advanced Technology University of Science and Technology of China Hefei, China · Department of Automation University of Science and Technology of China Hefei, China
Irregular multivariate time series forecasting (IMTSF) requires capturing temporal dynamics and cross-variable dependencies from sparse, asynchronous observations. Existing methods often rely on learned temporal encoders, coupling historical representation construction with model optimisation and repeatedly processing the same observations. We propose CTLite, a lightweight forecasting framework that decouples historical summarisation from learned prediction. Specifically, a parameter-free continuous-time moment encoder summarises local levels and trends at multiple temporal scales while suppressing poorly supported slope estimates. A compact readout combines damped local extrapolation with a low-rank cross-variable correction that accounts for observation freshness, target availability, and forecast horizon. Since the historical summaries are independent of trainable parameters and forecast queries, they can be reused across optimisation steps and repeated predictions for an unchanged history without altering predictions or parameter gradients. Experiments on four real-world benchmarks show that CTLite achieves the best or second-best mean MSE and MAE among the evaluated methods with only 88-2,537 trainable parameters. Under matched GPU workloads at batch size 32, CTLite reduces inference latency by 18.6-87.3% compared with the latest models, including the cost of historical encoding. Reusing summaries further reduces total runtime by approximately 52% over 16 forecast calls on the same history compared with recomputing the encoder for every call, including the initial encoding cost. The code is at: https://github.com/SSMa913/NICLab-CTLite.
Figures & tables
Fig. 1: Compact prediction and historical reuse. (a) Test MSE normalised by last-value forecasting on the same test records. (b) Time for 16 forecast calls on one unchanged batch, normalised by recomputing the history for every call. Reuse includes one initial encoding. Bars show five-seed means and sample Standard Deviation (SD).
Fig. 2: Overall CTLite framework. F and L denote fixed and learned computations, respectively.
Dataset
HumanActivity
USHCN
PhysioNet †
MIMIC
Metric
MSE ↓
MAE ↓
MSE ↓
MAE ↓
MSE ↓
MAE ↓
MSE ↓
MAE ↓
PrimeNet
4.2507 ± 0.0041
1.7018 ± 0.0011
0.4930 ± 0.0015
0.4954 ± 0.0018
0.8128 ± 0.00004
0.6977 ± 0.00006
0.9073 ± 0.0001
0.6614 ± 0.0001
NeuralFlows
0.1722 ± 0.0090
0.3150 ± 0.0094
0.2087 ± 0.0258
0.3157 ± 0.0187
0.3910 ± 0.00303
0.4360 ± 0.00201
0.6085 ± 0.0101
0.5306 ± 0.0066
CRU
0.1387 ± 0.0073
0.2607 ± 0.0092
0.2168 ± 0.0162
0.3180 ± 0.0248
0.5917 ± 0.00340
0.5604 ± 0.00199
0.5895 ± 0.0092
0.5151 ± 0.0048
mTAN
0.0993 ± 0.0026
0.2219 ± 0.0047
0.5561 ± 0.2020
0.5015 ± 0.0968
0.3711 ± 0.00459
0.4231 ± 0.00356
0.9408 ± 0.1126
0.6755 ± 0.0459
SeFT
1.3786 ± 0.0024
0.9762 ± 0.0007
0.3345 ± 0.0022
0.4083 ± 0.0084
0.7912 ± 0.00223
0.6860 ± 0.00079
0.9230 ± 0.0015
0.6628 ± 0.0008
TABLE I: Forecasting performance on four IMTS datasets (mean ± standard deviation). Bold and underline mark the best and second-best means. † denotes repaired PhysioNet.
Dataset
N
Records (Samples)
Observed targets
Train
Val
Test
Train
Val
Test
HumanActivity
12
949
193
218
25,872
5,205
5,910
USHCN
5
901
101
112
2,964
334
368
PhysioNet
36
9,703
1,079
1,199
217,124
24,061
27,008
MIMIC
96
17,212
1,913
2,125
187,419
20,874
23,015
TABLE II: Split sizes and observed target counts.
Dataset
Model
Params ↓
Train (ms) ↓
Infer (ms) ↓
Throughput ↑
Train MiB ↓
Infer MiB ↓
HumanActivity
APN
28,789
8.905 ± 0.034
3.530 ± 0.021
9.065 ± 0.055
463.83 ± 1.00
229.90 ± 0.41
GraFITi
249,409
8.887 ± 0.053
4.205 ± 0.007
7.610 ± 0.012
280.21 ± 1.08
62.92 ± 0.31
CTLite
365
2.232 ± 0.017
0.665 ± 0.008
48.119 ± 0.579
18.46 ± 0.04
10.24 ± 0.03
USHCN
APN
1,970
3.440 ± 0.024
1.062 ± 0.006
30.119 ± 0.175
176.30 ± 2.71
69.54 ± 1.04
GraFITi
495,617
8.516 ± 0.114
3.940 ± 0.055
8.122 ± 0.113
339.13 ± 8.80
137.81 ± 3.01
CTLite
88
2.205 ± 0.016
0.654 ± 0.003
48.926 ± 0.250
17.70 ± 0.01
9.55 ± 0.01
TABLE III: Matched efficiency at batch size 32 (mean ± standard deviation). Throughput is in thousands of records/s, and memory is peak allocated MiB. Bold is best, underline second best.
Dataset
Fixed rates Λ
r
H
Learning rate
Batch size
Weight decay
Params
HumanActivity
{0,1,4,16,64}
1
8
0.01
16
0.0001
365
USHCN
{0,8}
1
2
0.0003
4
0.01
88
PhysioNet
{0,2,8,32}
4
4
0.001
64
0.0003
889
MIMIC
{0,2,8,32,128}
4
8
0.001
32
0.0001
2,537
TABLE IV: Final model and training settings. All tasks use constant learning rates and raw Adam weights.
Dataset
Seed
Val MSE
Val MAE
Test MSE
Test MAE
HumanActivity
2024
0.045863
0.118920
0.042599
0.117128
2025
0.046127
0.119902
0.042174
0.116331
2026
0.047570
0.120034
0.044177
0.117122
2027
0.046258
0.118295
0.041547
0.116031
2028
0.045992
0.119011
0.042301
0.116980
USHCN
2024
0.353462
0.308676
0.166031
0.272905
TABLE V: Validation and test errors of the complete model by seed.
Fig. 4: Full per-request CPU inference without cached histories. Bars show five-seed mean median latency, and sample SD. The log axis accommodates model differences. Input tensors are CPU resident; disk IO and process startup are excluded.
Fig. 5: Batch scaling. Full inference above and full training steps below. Error bars show sample SD over five seeds.
Dataset
Encode once
Cached inference
Cached train step
HumanActivity
0.370 ± 0.004
0.336 ± 0.004
1.817 ± 0.024
USHCN
0.371 ± 0.003
0.331 ± 0.004
1.786 ± 0.006
PhysioNet
0.376 ± 0.008
0.341 ± 0.009
1.801 ± 0.038
MIMIC
0.373 ± 0.005
0.331 ± 0.003
1.806 ± 0.031
TABLE VI: Historical encoding and computation with a stored summary (ms).
Dataset
HumanActivity
USHCN
PhysioNet
MIMIC
Variant
MSE ↓
MAE ↓
MSE ↓
MAE ↓
MSE ↓
MAE ↓
MSE ↓
MAE ↓
Full model
0.04256 ± 0.00098
0.11672 ± 0.00051
0.16482 ± 0.00084
0.27068 ± 0.00241
0.28762 ± 0.00017
0.35265 ± 0.00096
0.40476 ± 0.00083
0.38438 ± 0.00087
No cross residual
0.04259 ± 0.00080
0.11672 ± 0.00131
0.16475 ± 0.00093
0.26739 ± 0.00097
0.28982 ± 0.00008
0.35309 ± 0.00091
0.42632 ± 0.00062
0.39337 ± 0.00052
Index/no slope
0.04287 ± 0.00060
0.11730 ± 0.00114
0.16000 ± 0.00064
0.26774 ± 0.00265
0.28840 ± 0.00015
0.35295 ± 0.00114
0.40187 ± 0.00135
0.38406 ± 0.00183
TABLE VII: Ablation studies. Bold marks the lowest error in each column.
Irregular Multivariate Time Series (IMTS) are characterized by uneven intervals between consecutive timestamps, which carry sampling pattern information valuable and informative for learning temporal and variable dependencies. In addition, IMTS often exhibit diverse dependencies across multiple time scales. However, many existing multi-scale IMTS methods use resampling to obtain the coarse series, which can alter the original timestamps and disrupt the sampling pattern information. To address the challenge, we propose ReIMTS, a Recursive multi-scale modeling approach for Irregular Multivariate Time Series forecasting. Instead of resampling, ReIMTS keeps timestamps unchanged and recursively splits each sample into subsamples with progressively shorter time periods. Based on the original sampling timestamps in these long-to-short subsamples, an irregularity-aware representation fusion mechanism is proposed to capture global-to-local dependencies for accurate forecasting. Extensive experiments demonstrate an average performance improvement of 27.1% in the forecasting task across different models and real-world datasets. Our code is available at https://github.com/Ladbaby/PyOmniTS.
Boyuan Li, Zhen Liu, Yicheng Luo +1
School of Computer Science and Engineering, South China University of Technology
Irregular Multivariate Time Series (IMTS) are common in practice, yet their irregular sampling complicates effective modeling. Existing approaches typically either (i) design specialized architectures that limit the reuse of proven Multivariate Time Series (MTS) models, or (ii) map IMTS onto regular temporal grids through interpolation, which may distort temporal dynamics by introducing artificial values. To address these limitations, we propose a new input-embedding-based approach. We identify that the key bottleneck lies not in the backbone architecture, but in conventional embedding layers that assume uniform sampling. In this work, we introduce QuITE (Query-Based Irregular Time Series Embedding), a simple yet effective plug-and-play embedding module for IMTS. QuITE employs learnable query tokens to aggregate irregular observations through a single self-attention layer, directly producing backbone-compatible latent representations without artificial value generation or architectural modification. Extensive experiments on real-world benchmarks show that QuITE consistently improves MTS models, yielding average relative gains of up to 54.7% in forecasting and 15.8% in classification across diverse datasets and backbone architectures. Code is available at: https://github.com/Meaningfull9502/QuITE.
Irregularly sampled multivariate time series arise in many real-world domains such as healthcare, climate science, and large-scale monitoring systems. Characterized by asynchronous observations and heterogeneity across variables, such time series pose significant challenges for forecasting. In this work, we introduce Cross-Variable Temporal Attention (CVTA), a short-term, continuous-time forecasting framework that learns prediction functions from its compact, interpretable state summarizing recent temporal dynamics across variables. CVTA uses Variable-Conditioned Embeddings to project heterogeneous variables into variable-specific latent spaces, and applies temporal attention to capture cross-variable dependencies from this state. Furthermore, we demonstrate that CVTA can be integrated as an auxiliary module into existing state-of-the-art prediction models through a dynamic meta-decision model. Extensive experiments on multiple real-world benchmarks show that CVTA achieves strong short-term predictive accuracy, supports efficient online inference, improves existing prediction models when used as an auxiliary module, and learns attention patterns that align well with domain knowledge.
Ankitkumar Joshi, Milos Hauskrecht
Department of Computer Science, University of Pittsburgh, Pittsburgh, PA 15260