RTSM: Adapting Recursive Models to Multivariate Time-Series Classification

Georg Tenzing; Anna Tegon; Luca Benini; Thorir Mar Ingolfsson

cs.LGOct 16, 2026

RTSM: Adapting Recursive Models to Multivariate Time-Series Classification

Authors: Georg Tenzing, Anna Tegon, Luca Benini, Thorir Mar Ingolfsson

Organizations: Integrated Systems Laboratory, ETH Zürich, Switzerland

Abstract

Accurate multivariate time-series classification requires extracting temporal features and integrating information across a sequence. Weight-tied refinement offers a way to repeatedly process these features without allocating new parameters at each stage. Applying this principle to time series requires choosing both a temporal representation and a latent-state organization suited to classification. We introduce the Recursive Time-Series Model (RTSM), which combines a learned convolutional temporal tokenizer, single-state weight-tied refinement, and a pooled classification readout. Across all 30 UEA datasets, RTSM achieves 72.81% mean accuracy under validation-based configuration and checkpoint selection, the highest observed mean among the 17 evaluated deep-learning models and 1.60 percentage points above MambaSL. Ablations on eight datasets favor the proposed tokenizer over a simple convolutional embedding and the single-state model over the tested two-state variants. The complete classifier reaches 86.10% mean accuracy versus 84.79% for a separately trained tokenizer-only classifier. These findings identify temporal encoding and state organization as key design choices for adapting recursive architectures to time series. The resulting classifier combines competitive accuracy with a shared refinement module whose parameter count is independent of computational depth.

Figures & tables

Figure 1: RTSM encodes a time series into input tokens

X

(green) and repeatedly updates one latent state

y

(purple). (a) The tokenizer extracts temporal features before forming patches. Temporal order enters through the tokenizer; refinement uses no positional encoding. (b) Each supervision step maps

y^{(s-1)}

y^{(s)}

through

H=3

applications of the shared module

R_{\theta}

, adding

X

before every cycle. The initial state broadcasts a fixed random vector

y_{\mathrm{init}}\in\mathbb{R}^{d}

across tokens. Training retains gradients through the final

G=1

cycle. (c) Each step receives a classification loss, a halt loss, and an optimizer update; the carried state is detached before the next step. Training uses adaptive halting with exploration, up to 16 steps. Evaluation uses a fixed 16 steps (48 refinement cycles).

Component	TRM	RTSM
Input	Discrete token embeddings	Convolutional temporal tokens from real-valued sequences
State	Prediction state $y$ and auxiliary state $z$	One token-level latent state $y$ refined using the input tokens
Readout	Per-token class logits	Pooled sequence-level class and correctness logits
Training	Deep supervision; gradients through the final cycle; correctness-based halting	Same recipe, with correctness targets for sequence classification

Table 1: From TRM to RTSM : the main adaptations and retained training mechanisms. The pooled readout follows MambaSL ( Jung & Kim, 2026 ) ; Appendix C.3 gives the full component comparison.

Figure 2: Mean classification accuracy (%) across all 30 UEA datasets for the 17 evaluated deep-learning models under validation-based configuration and checkpoint selection. Each dataset receives equal weight. Full per-dataset results and pairwise Wilcoxon tests appear in Appendix B .

	State structure
Tokenizer	None	Two-state shared $f$	Two-state separate $f_{H}/f_{L}$	Single state
Conv1D	73.72	78.96	79.69	–
RTSM	84.79	84.98	84.29	86.10

Table 2: Tokenizer and state structure: mean accuracy (%) over eight datasets and three seeds. “None” applies the prediction head directly to tokenizer outputs. The two-state variants use shared or separate update networks within the TRM training recipe (Appendix B.3 ); a dash marks an unevaluated combination.

Figure 3: RTSM versus the tokenizer-only configuration on all eight ablation datasets. Positive differences favor RTSM . Open circles show the accuracy difference for each seed (0, 1, or 2); diamonds and numerical labels show their mean. Small vertical offsets separate the seed markers. Datasets are ordered by their mean difference. Full per-seed accuracies appear in Appendix B.3 .

Within-step depth $H$					Training supervision
1	2	3	4	5	One step	Adaptive, up to 16	Fixed 16
85.77	85.60	86.10	85.61	85.78	85.98	86.10	85.05

Table 3: Recursive depth and training supervision: mean accuracy (%) over the same eight datasets and three seeds. Left: vary the number of recursive cycles

H

within each supervision step, using adaptive supervision capped at 16 steps. Right: vary the supervision schedule at

H=3

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

	MambaSL Protocol		Validation Protocol
Setting	DL	RTSM	DL	RTSM
Train epochs	100	100	100	100
Seed	2021	2021	2021	2021
Batch size	16 †	dataset-specific	16	dataset-specific
Learning rate	$10^{-3}$	$10^{-4}$	$10^{-3}$	$10^{-4}$
LR scheduler	none	constant	none	constant

Appendix

Table 4: Basic training settings under the MambaSL and validation protocols.

Dataset	Abbr.	Train Cases	Test Cases	Length	Channels	Classes
EthanolConcentration	EC	261	263	1751	3	4
FaceDetection	FD	5890	3524	62	144	2
Handwriting	HW	150	850	152	3	26
Heartbeat	HB	204	205	405	61	2
JapaneseVowels	JV	270	370	7–29	12	9
PEMS-SF	PEMS	267	173	144	963	7

Appendix

Table 5: Overview of the 30 UEA datasets.

Figure 4: Mean classification accuracy (%) under the MambaSL protocol, which uses official test-set performance for configuration and checkpoint selection. The comparison includes RTSM, 16 other deep-learning models, and five non-deep-learning baselines. Solid bars show averages across all 30 UEA datasets; hatched bars show the 10-dataset subset. Each dataset receives equal weight within its respective average.

Table 6: Classification accuracy (%) of models on the 30 datasets of the UEA archive under the MambaSL protocol. The best and the second-best are highlighted in bold and underline, respectively.

Table 7: (Continued.) Classification accuracy (%) of models on the 30 datasets of the UEA archive under the MambaSL protocol. The best and the second-best are highlighted in bold and underline, respectively.

	Linear/MLP			Convolutional/spectral/multiscale
	DLinear ( 2023 )	LightTS ( 2022 )	MTS- Mixers ( 2025 )	TimesNet ( 2023 )	Modern TCN ( 2024 )	TSLANet ( 2024 )	Time Mixer++ ( 2025b )
EC	25.480	28.140	22.050	28.520	23.950	30.420	24.330
FD	67.820	64.560	65.580	66.030	64.100	65.490	64.390
HW	21.530	19.530	14.940	27.760	19.650	56.350	26.000
HB	73.170	73.170	73.660	77.070	68.780	77.070	75.610
JV	95.680	92.970	92.970	97.840	96.220	96.490	97.030

Appendix

Table 8: Classification accuracy (%) of deep-learning models on the 30 UEA datasets under the validation protocol. Best and second-best results are shown in bold and underlined, respectively.

Table 9: (Continued.) Classification accuracy (%) of deep-learning models on the 30 datasets of the UEA archive under the validation protocol. The best and the second-best are highlighted in bold and underline, respectively.

Main-text ablations
Configuration	Avg. Acc. (%)
Conv_tokenizer_only	$73.72\pm 0.71$
Conv_3H6L1G_steps16_shared	$78.96\pm 1.03$
Conv_3H6L1G_steps16_separate	$79.69\pm 0.62$
tokenizer_only	$84.79\pm 0.41$
3H6L1G_steps16_shared	$84.98\pm 1.04$

Appendix

Table 10: Classification accuracy (%) across the recursive architecture and training ablations on eight UEA datasets (CT, DDG, HB, JV, LSST, NATO, PD, and SAD) under validation-based selection. Each seed’s score is the equally weighted mean across these datasets. Entries report the mean

\pm

sample standard deviation of the three scores for seeds 0, 1, and 2. Bold marks the reference configuration.

Dataset	Seed 0	Seed 1	Seed 2	Mean $\pm$ Std.	Seed 0	Seed 1	Seed 2	Mean $\pm$ Std.	$\Delta$
	3H1G_steps16				tokenizer_only
CharacterTrajectories	99.234	99.095	98.886	$\mathbf{99.072\pm 0.175}$	95.891	96.031	96.379	$96.100\pm 0.251$	+2.971
PenDigits	98.571	98.742	99.085	$\mathbf{98.799\pm 0.262}$	98.485	98.113	98.228	$98.275\pm 0.190$	+0.524
SpokenArabicDigits	98.408	98.681	98.590	$\mathbf{98.560\pm 0.139}$	96.362	96.771	97.317	$96.817\pm 0.479$	+1.743
JapaneseVowels	97.297	97.838	96.757	$\mathbf{97.297\pm 0.541}$	96.757	97.027	96.757	$96.847\pm 0.156$	+0.450
NATOPS	91.111	89.444	91.111	$90.555\pm 0.962$	89.444	92.778	93.333	$\mathbf{91.852\pm 2.103}$	-1.296

Appendix

Table 11: Comparison of the proposed 3H1G_steps16 configuration with the tokenizer_only baseline across the eight ablation datasets. Results report classification accuracy (%) for each seed and the mean

\pm

sample standard deviation across three seeds. In the Overall row, datasets are averaged equally within each seed before computing the mean and standard deviation.

\Delta

denotes the difference in mean accuracy between the proposed configuration and the tokenizer-only baseline.

Dataset	Seed 0	Seed 1	Seed 2	Mean $\pm$ Std.	Seed 0	Seed 1	Seed 2	Mean $\pm$ Std.	$\Delta$
	3H1G_steps16				3H6L1G_steps16_shared
CharacterTrajectories	99.234	99.095	98.886	$\mathbf{99.072\pm 0.175}$	99.304	98.189	98.886	$98.793\pm 0.563$	+0.279
PenDigits	98.571	98.742	99.085	$\mathbf{98.799\pm 0.262}$	98.856	98.571	98.370	$98.599\pm 0.244$	+0.200
SpokenArabicDigits	98.408	98.681	98.590	$\mathbf{98.560\pm 0.139}$	97.954	98.090	98.363	$98.136\pm 0.208$	+0.424
JapaneseVowels	97.297	97.838	96.757	$\mathbf{97.297\pm 0.541}$	98.108	96.486	96.486	$97.027\pm 0.936$	+0.271
NATOPS	91.111	89.444	91.111	$\mathbf{90.555\pm 0.962}$	82.778	77.778	95.556	$85.371\pm 9.168$	+5.185

Appendix

Table 12: Comparison of the proposed 3H1G_steps16 configuration with the 3H6L1G_steps16_shared two-state variant across the eight ablation datasets. Results report classification accuracy (%) for each seed and the mean

\pm

sample standard deviation across three seeds. In the Overall row, datasets are averaged equally within each seed before computing the mean and standard deviation.

\Delta

denotes the difference in mean accuracy between the proposed configuration and the shared-network two-state variant.

Dataset	Seed 0	Seed 1	Seed 2	Mean $\pm$ Std.	Seed 0	Seed 1	Seed 2	Mean $\pm$ Std.	$\Delta$
	3H6L1G_steps16_shared				tokenizer_only
CharacterTrajectories	99.304	98.189	98.886	$\mathbf{98.793\pm 0.563}$	95.891	96.031	96.379	$96.100\pm 0.251$	+2.693
PenDigits	98.856	98.571	98.370	$\mathbf{98.599\pm 0.244}$	98.485	98.113	98.228	$98.275\pm 0.190$	+0.324
SpokenArabicDigits	97.954	98.090	98.363	$\mathbf{98.136\pm 0.208}$	96.362	96.771	97.317	$96.817\pm 0.479$	+1.319
JapaneseVowels	98.108	96.486	96.486	$\mathbf{97.027\pm 0.936}$	96.757	97.027	96.757	$96.847\pm 0.156$	+0.180
NATOPS	82.778	77.778	95.556	$85.371\pm 9.168$	89.444	92.778	93.333	$\mathbf{91.852\pm 2.103}$	-6.481

Appendix

Table 13: Comparison of the 3H6L1G_steps16_shared two-state variant with the tokenizer_only baseline across the eight ablation datasets. Results report classification accuracy (%) for each seed and the mean

\pm

sample standard deviation across three seeds. In the Overall row, datasets are averaged equally within each seed before computing the mean and standard deviation.

\Delta

denotes the difference in mean accuracy between the shared-network two-state variant and the tokenizer-only baseline.

Panel of Figure 5 — Figure 5: Complete breadth–depth grid for RTSM . Macro-averaged oracle pass@ $B$ accuracy and RMS latent-state residual for stochastic RI+NI trajectories across 28 UEA datasets.

$D\backslash B$	1	2	4	8	16	32	64	128	256	512
1	72.03	73.01	74.25	74.82	75.48	75.91	76.34	76.70	77.14	77.71
2	72.48	72.86	73.14	73.48	73.79	73.91	74.35	74.69	75.11	75.23
4	72.82	73.01	73.18	73.55	73.65	73.73	73.81	73.86	73.96	74.03
8	72.57	72.71	72.85	72.95	73.02	73.07	73.12	73.15	73.23	73.26
16	72.54	72.65	72.80	72.93	72.97	73.00	73.06	73.32	73.38	73.41
32	72.78	72.90	73.01	73.12	73.17	73.22	73.27	73.29	73.35	73.37

Appendix

Table 14: Macro-averaged oracle pass@

B

accuracy (%) across the full depth–breadth grid, over 28 UEA datasets.

Panel of Figure 6 — Figure 6: Deterministic recursive dynamics of RTSM . Macro-averaged classification accuracy, cross-entropy loss, and RMS latent-state residual across 64 recursion steps on 28 UEA datasets.

Step	Accuracy (%)	CE loss	RMS residual
1	72.42	2.5983	1.41722
2	72.53	2.8313	0.19541
4	72.15	2.9168	0.03303
8	72.56	2.9377	0.01080
16	72.61	2.9470	0.00716
32	72.83	2.9476	0.00684

Appendix

Table 15: Deterministic trajectory metrics at selected recursion steps, macro-averaged over 28 datasets.

	Det.	Oracle	Headroom	Best-Q	Majority	Min-res.
RTSM	72.84	73.36	+0.52	72.66	72.76	72.74

Appendix

Table 16: Macro-averaged accuracy (%) and headroom (pp) at the maximum evaluated budget,

D=64

and

B=512

. Oracle pass@

B

uses ground-truth labels and is an upper bound; Best-Q, majority, and minimum-residual selection are label-free.

Statistic	Headroom (pp)
Mean	+0.52
Median	0.00
Q25	0.00
Q75	+0.37
IQR	0.37
Minimum	0.00

Appendix

Table 17: Distribution of per-dataset headroom (oracle pass@

B

minus deterministic accuracy) at

D=64

B=512

, over 28 UEA datasets.

Dataset	Det.	Oracle	Headroom	Best-Q	Majority	Min-res.
PS	24.16	29.56	+5.40	23.08	23.89	23.53
EC	29.66	34.98	+5.32	25.86	28.52	28.52
FD	67.22	68.22	+0.99	66.97	66.91	67.00
EP	94.93	95.65	+0.72	95.65	94.93	94.93
NATO	95.56	96.11	+0.56	95.56	95.56	95.56
LSST	65.37	65.90	+0.53	65.41	65.45	65.41

Appendix

Table 18: Per-dataset results at

D=64

B=512

, sorted by headroom. All values are classification accuracy (%) except headroom, reported in percentage points.

Component	HRM ( Wang et al., 2025a )	TRM ( Jolicoeur-Martineau, 2025 )	RTSM (ours)
State initialization	frozen random vectors	frozen random vectors	frozen random vector ( $y$ only)
Update networks	separate $f_{L}$ , $f_{H}$	one shared network $f$	one shared network $R_{\theta}$
Gradient schedule	one-step gradient: final $f_{L}$ and $f_{H}$ updates only	full backpropagation through the final cycle ( $G=1$ )	full backpropagation through the final cycle ( $G=1$ )
Deep supervision	up to $S_{\max}=16$ steps, states detached between steps	up to $S_{\max}=16$ steps, states detached between steps	up to $S_{\max}=16$ steps, states detached between steps
Halting, training	Q-learning halt/continue head with exploration	correctness-based halt score $q$ with BCE and exploration	same as TRM, using sequence-level correctness
Halting, evaluation	fixed depth $S_{\mathrm{eval}}$	fixed depth $S_{\mathrm{eval}}$	fixed depth $S_{\mathrm{eval}}$

Appendix

Table 19: Component-level comparison of HRM, TRM, and RTSM using the framework of Section 2.2 . The comparison follows the released HRM and TRM implementations. HRM provides architectural context; the HRM-style ablation in Section 4.2 changes state structure and network sharing within the TRM training recipe.

H/L/G

denotes cycles, inner auxiliary-state updates, and gradient-tracked cycles.

Time Series Classification

Time series classification assigns labels to sequential data, using temporal patterns to support decisions in health, sensing, science, and industry.

See all 174 papers

Reshape and Recur: Improving SSMs with Input Reshaping and Depth Recurrence
May 15, 2026Mónika Farsang, Ramin Hasani, Daniela Rus +1Parameter Sharing State Space Models
1TU Wien · MIT CSAIL · Liquid AI
MambaSL: Exploring Single-Layer Mamba for Time Series Classification
Apr 16, 2026Yoo-Min Jung, Leekyung KimMamba Selective SSMs
Department of Industrial Engineering, Seoul National University
From Retrieval to Recognition:Knowledge-Enhanced Post-training of Time Series Foundation Models for Classification
Oct 19, 2026Zhen Liu, Yucheng Wang, Yingpeng Du +5Time Series Classification Time-Series Foundation Models
Nanyang Technological University, Singapore · Institute of Advanced Intelligence and Computing (IAIC), A*STAR, Singapore · AI Center of Excellence for Manufacturing (AIMfg), A*STAR, Singapore +2

May 15, 2026J/K move · Enter open · S save

Reshape and Recur: Improving SSMs with Input Reshaping and Depth Recurrence

Mónika Farsang, Ramin Hasani, Daniela Rus, Radu Grosu

1TU Wien · MIT CSAIL · Liquid AI

Parameter Sharing State Space Models

Open paper AskPDF arXiv

State Space Models (SSMs) are increasingly deployed in the Edge because they offer, at comparable performance, a smaller memory/training/inference footprint, compared to Large Language Models (LLMs). These three advantages are a direct consequence of the time recurrence inherent in the SSMs architecture. Here, we further improve this recurrent architecture by positively answering two previously underexplored, orthogonal questions: (1) Can we reduce SSMs memory-footprint without any performance penalty, by also employing depth recurrence? (2) Can we increase SSMs performance by using a fixed and consistent time-granularity across all tasks? The first question is somewhat unexpected, given that SSMs are already recurrent. However, the orthogonal depth recurrence further decreases SSMs memory footprint. We show that a looped SSM with

k

parameters adaptively iterated

M

times, achieves a performance comparable to a standard SSM with

k \cdot L

independent parameters, where

M \leq L

. The second question is also unexpected given the time-recurrent nature of the SSMs architecture. However, it makes perfect sense for the time-parallel training of SSMs on the entire input sequence. We show that concatenating time steps for lower-dimensional sequence elements, or flattening and re-chunking the joint feature-time dimension for high-dimensional ones, can improve the baseline by enhancing the way information is presented to the model. Our results for both extensions lead to consistent benefits across four representative SSM architectures: LRU, S5, LinOSS, LrcSSM.

Abstract

Time Series Classification

Explore similar work

Reshape and Recur: Improving SSMs with Input Reshaping and Depth Recurrence

MambaSL: Exploring Single-Layer Mamba for Time Series Classification

From Retrieval to Recognition:Knowledge-Enhanced Post-training of Time Series Foundation Models for Classification