Riccati State Space Models: Non-iterative Parallelization for Nonlinear Sequence Modeling

Mónika Farsang; Ramin Hasani; Daniela Rus; Radu Grosu

cs.LGSep 28, 2026

Riccati State Space Models: Non-iterative Parallelization for Nonlinear Sequence Modeling

Authors: Mónika Farsang, Ramin Hasani, Daniela Rus, Radu Grosu

Organizations: TU Wien · MIT CSAIL · Liquid AI

Abstract

State space models (SSMs) achieve efficient sequence processing because their affine state updates are closed under composition and can therefore be evaluated with an associative parallel scan. Nonlinear recurrent models can provide richer, state-dependent dynamics, but generally lose this compositional structure: parallel evaluation then requires iterative methods that repeatedly linearize and scan the recurrence. We ask, what state-dependent nonlinear dynamics can be designed to remain exactly composable? We answer by introducing RiccatiSSM, a nonlinear SSM, in which each state dimension follows an input-conditioned Riccati differential equation. Its quadratic state dependence makes the local Jacobian explicitly state-dependent, while its exact per-step flow under piecewise-constant inputs is a Möbius transformation. Since Möbius maps are closed under composition and compose through $2\times 2$ matrix multiplication, the complete nonlinear state trajectory can be evaluated exactly with a single associative parallel scan, without iterative linearization. We further derive a constrained parameterization that ensures bounded, contractive dynamics, and avoids poles in the fractional-linear state update. Across long-sequence classification, regression, and forecasting tasks, RiccatiSSM achieves competitive predictive performance while reducing runtime by $22{-}33\%$ compared to the nonlinear LrcSSM under matched architectures. These results demonstrate that state-dependent nonlinear dynamics can retain exact composability and be evaluated efficiently within a single parallel scan.

Figures & tables

	DEER/ELK-based models	RiccatiSSM
Dynamics	any $f(x_{t-1},u_{t})$	Riccati: $\alpha+\beta x+\gamma x^{2}$
Scan element	affine ( $(J_{t},c_{t})$ , $J_{t}$ the state Jacobian)	fractional-linear, $M_{t}\in\mathbb{R}^{2\times 2}$
Composition	approximate (per iteration)	exact
Sweeps per layer	until convergence ( $K$ , data-dependent)	$1$ , fixed
Depth per layer	$\mathcal{O}(K\log T)$	$\mathcal{O}(\log T)$
Work per layer	$\mathcal{O}(K\,TD)$	$\mathcal{O}(TD)$

Table 1: RiccatiSSM compared with DEER/ELK-style ( Lim et al., 2024 ; Gonzalez et al., 2024 ) parallelization of general nonlinear recurrences. Both evaluate state-dependent dynamics with associative scans, but DEER/ELK approximate the general nonlinear update by Newton iteration updates until convergence, whereas RiccatiSSM restricts the dynamics to a family that composes exactly in a single step.

T

is the sequence length,

D

the number of channels, and

K

the number of sweeps.

Figure 1: RiccatiSSM evaluates a nonlinear recurrence by a detour through a two-dimensional linear one. Bottom (grey, dashed): the states

x_{t}

of one channel, whose Möbius updates are nonlinear in the state. Composing such maps directly requires clearing nested fractions, so the bottom row offers no cheap merge to scan over. Top (blue, solid): the same states as homogeneous pairs

(p_{t},q_{t})

, where each step is the matrix

M_{t}

. The initial state is lifted once,

x_{0}\mapsto(x_{0},1)

, and states are recovered by

x_{t}=p_{t}/q_{t}

. Matrix multiplication is associative, so the top row collapses into prefix products

P_{t}=M_{t}\cdots M_{1}

under one associative scan of depth

\mathcal{O}(\log T)

, with no fixed-point iterations; all

D

channels scan in parallel in the same round.

	Heart	SCP1	SCP2	Ethanol	Motor	Worms
Sequence length	405	896	1,152	1,751	3,000	17,984
Input size	61	6	7	2	63	6
#Classes	2	2	2	4	2	5
Transformer ‡	70.5 $\pm$ 0.1	84.3 $\pm$ 6.3	49.1 $\pm$ 2.5	40.5 $\pm$ 6.3	50.5 $\pm$ 3.0	OOM
RFormer ‡	72.5 $\pm$ 0.1	81.2 $\pm$ 2.8	52.3 $\pm$ 3.7	34.7 $\pm$ 4.1	55.8 $\pm$ 6.6	90.3 $\pm$ 0.1
NRDE †	73.9 $\pm$ 2.6	76.7 $\pm$ 5.6	48.1 $\pm$ 11.4	31.4 $\pm$ 4.5	54.0 $\pm$ 7.8	77.2 $\pm$ 7.1

Table 2: Test accuracy comparison of different models. The performance of the models marked by † is reported from Rusch and Rus (2025) , those with ‡ from Moreno-Pino et al. (2024) , and

\diamond

from Farsang and Grosu (2025) . The same hyperparameter tuning protocol and dataset splitting over the same 5 seeds were used. Bold highlights the top 3 models.

Model	MSE $\times 10^{-2}$ ( $\downarrow$ )
NRDE † ( Morrill et al., 2021 )	9.90 $\pm$ 0.97
NCDE † ( Kidger et al., 2020 )	13.54 $\pm$ 0.69
Log-NCDE † ( Walker et al., 2024 )	9.56 $\pm$ 0.59
LRU † ( Orvieto et al., 2023 )	12.17 $\pm$ 0.49
S5 † ( Smith et al., 2023 )	12.63 $\pm$ 1.25
LinOSS-IMEX † ( Rusch and Rus, 2025 )	7.50 $\pm$ 0.46

Table 3: Mean squared error (MSE ×

10^{-2}

) for different models on the PPG-DaLiA dataset. The performance of the models marked by † is reported from Rusch and Rus (2025) and

\diamond

from Farsang and Grosu (2025) . All results are averaged over 5 seeds.

Model	Mean Absolute Error ( $\downarrow$ )
Informer † ( Zhou et al., 2021 )	$0.731$
LogTrans † ( Li et al., 2019 )	$0.773$
Reformer † ( Kitaev et al., 2020 )	$1.575$
LSTMa † ( Bahdanau et al., 2014 )	$1.109$
LSTnet † ( Lai et al., 2018 )	$0.757$
S4 † ( Gu et al., 2022 )	$0.5783$

Table 4: Mean absolute error on the weather dataset predicting 720 future time steps based on 720 past time steps. The performance of the models marked by † is reported from Rusch and Rus (2025) .

Figure 2: Time per step (seconds) for RiccatiSSM and LrcSSM across three UEA time-series classification datasets using the same architecture on an NVIDIA A100 GPU. RiccatiSSM evaluates each layer with a single parallel scan, whereas LrcSSM requires approximately three quasi-DEER iterations. The runtime gap is largest for the longer EigenWorms sequences.

Model	Input-dependent coefficients	MSE ( $\downarrow$ )
RiccatiSSM (ours)	$\alpha(u),\beta(u),\gamma(u),\varepsilon(u)$	$\mathbf{7.15\pm 1.01}$
$\varepsilon$ ablation	$\alpha(u),\beta(u),\gamma(u)$	$8.45\pm 0.38$
Reduced input dependence	$\alpha(u),\beta,\gamma,\varepsilon$	$8.71\pm 1.10$
RiccatiSSM (LRC-tied)	LRC Taylor-matched	$8.47\pm 0.47$

Table 5: Ablation of the RiccatiSSM parameterization on PPG-DaLiA. Test MSE (

\times 10^{-2}

, lower is better), reported as mean

\pm

standard deviation over five seeds.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Factor isolated	Comparison	Relative RMSE
Truncation effect (Exact integration)	RiccatiSSM-ZOH vs. LrcSSM-RK4	$1.73\times 10^{-5}$
Truncation effect (Euler integration)	RiccatiSSM-Euler vs. LrcSSM-Euler	$1.73\times 10^{-5}$
Integrator choice LrcSSM	LrcSSM-Euler vs. LrcSSM-RK4	$7.16\times 10^{-3}$

Appendix

Table 6: Relative RMSE over

T=512

steps, receiving white-noise input.

Figure 3: Hidden-state trajectories of 5 neurons under three models with matched Taylor coefficients driven by white-noise input. LrcSSM-RK4 (black, 64 RK4 substeps) serves as ground truth for the full dynamics; LrcSSM-Euler (blue dashed) is the model’s single Euler step as in [ Farsang and Grosu, 2025 ] ; RiccatiSSM-ZOH (red dotted) solves the quadratic (Riccati) truncation exactly via zero-order hold. The inset shows the first 30 timesteps (dashed box). Close match between RiccatiSSM-ZOH and both LrcSSM variants indicates that the dominant discretisation error comes from the Euler integrator rather than the quadratic truncation of the vector field.

Figure 4: Reachable constraint-coefficient space under free vs. LRC-tied Riccati parameterizations, sampled over 500k random initializations. (Left) Distribution of normalized stability coefficient $\alpha/\alpha_{max}$ : the free variant saturates at $\pm 1$ , while the LRC-tied variant concentrates near zero due to double-squashing through the circuit nonlinearities. (Center) Distribution of curvature coefficient $\gamma$ : LRC-tied outputs are similarly compressed relative to the free range. (Right) Joint ( $\gamma,\alpha/\alpha_{max}$ ) reachable region; the LRC-tied parameterization occupies a strictly smaller subspace, reflecting the structural constraints imposed by the underlying neural circuit dynamics.

	lr	hidden dim.	state-space dim.	number of layers
Heart	$10^{-3}$	64	64	4
SCP1	$10^{-4}$	64	16	6
SCP2	$10^{-3}$	16	16	2
Ethanol	$10^{-3}$	64	16	6
Motor	$10^{-3}$	16	256	4
Worms	$10^{-3}$	16	16	4

Appendix

Table 7: Optimized hyperparameters used for the classification, PPG-DaLiA and Weather datasets.

	NRDE	NCDE	Log-NCDE	LRU	S5	Mamba	S6	LinOSS-IMEX	LinOSS-IM	LrcSSM	RiccatiSSM
Heart	9539	1177	826	8	11	34	4	4	7	23	21
SCP1	1014	973	635	9	17	7	3	42	38	12	33
SCP2	1404	1251	583	9	9	32	7	55	22	15	12
Ethanol	2256	2217	2056	16	9	255	4	48	8	15	26
Motor	7616	3778	730	51	16	35	34	128	11	31	27
Worms	5386	24595	1956	94	31	122	68	37	90	33	23

Appendix

Table 8: Runtime in seconds for the models for 1000 training steps, using the best configuration for each. Values for the models are taken from Rusch and Rus [2025] and Farsang and Grosu [2025] .

Explore similar work

Jun 29, 2026cs.LG

MuonSSM: Orthogonalizing State Space Models for Sequence Modeling

State space models (SSMs) have emerged as efficient linear-time alternatives to attention for long-sequence modeling. However, existing SSMs often suffer from instability and memory degradation over extended horizons due to poorly conditioned first-order updates and unbalanced update geometry. We introduce MuonSSM, a general framework that stabilizes SSM training by explicitly conditioning the geometry of memory updates rather than the recurrent transition matrix. MuonSSM augments SSMs with a momentum-based pathway and a lightweight Newton Schulz transformation on low-rank input injections, yielding bounded and spectrally conditioned updates while preserving parallel scan complexity. Theory shows that MuonSSM improves gradient propagation, mitigates spectral amplification, and enriches memory representations over long horizons. Extensive experiments across language, vision, and time-series benchmarks show consistent gains in accuracy, robustness, and long-context performance when integrated into diverse SSM backbones. These results establish geometric conditioning of updates as a principled pathway to stable, scalable sequence modeling.

Thai-Khanh Nguyen, Ngoc-Bich-Uyen Vo, Thieu N. Vo +2

Faculty of Information Technology, Dainam University, Hanoi University of Science and Technology, Hanoi 10000, Vietnam · Faculty of Artificial Intelligence, Posts and Telecommunications Institute of Technology, Hanoi 10000, Vietnam · Department of Computer Science, University of Bath, Bath BA2 7AY, UK +1

State Space Models Sequence Modeling Momentum

Ask PDF

May 15, 2026cs.LG

Reshape and Recur: Improving SSMs with Input Reshaping and Depth Recurrence

State Space Models (SSMs) are increasingly deployed in the Edge because they offer, at comparable performance, a smaller memory/training/inference footprint, compared to Large Language Models (LLMs). These three advantages are a direct consequence of the time recurrence inherent in the SSMs architecture. Here, we further improve this recurrent architecture by positively answering two previously underexplored, orthogonal questions: (1) Can we reduce SSMs memory-footprint without any performance penalty, by also employing depth recurrence? (2) Can we increase SSMs performance by using a fixed and consistent time-granularity across all tasks? The first question is somewhat unexpected, given that SSMs are already recurrent. However, the orthogonal depth recurrence further decreases SSMs memory footprint. We show that a looped SSM with

k

parameters adaptively iterated

M

times, achieves a performance comparable to a standard SSM with

k \cdot L

independent parameters, where

M \leq L

. The second question is also unexpected given the time-recurrent nature of the SSMs architecture. However, it makes perfect sense for the time-parallel training of SSMs on the entire input sequence. We show that concatenating time steps for lower-dimensional sequence elements, or flattening and re-chunking the joint feature-time dimension for high-dimensional ones, can improve the baseline by enhancing the way information is presented to the model. Our results for both extensions lead to consistent benefits across four representative SSM architectures: LRU, S5, LinOSS, LrcSSM.

Mónika Farsang, Ramin Hasani, Daniela Rus +1

1TU Wien · MIT CSAIL · Liquid AI

Time-Series Classification State Space Models Intrinsic Dimensionality

Ask PDF

May 16, 2025cs.LG

Regularity and Stability Properties of Selective SSMs with Discontinuous Gating

Selective State-Space Models (SSMs) such as Mamba have become central to long-sequence modeling. Still, their stability is poorly understood: their state-space coefficients are modulated online by a token-dependent gating signal, making the recurrence neither linear time-invariant nor classically nonlinear. We study continuous-time selective SSMs through passivity, dissipativity, and Input-to-State Stability (ISS), explicitly separating the selection signal

x(\cdot)

from the driving input

u(\cdot)

. We obtain four results: exponential forgetting under strict dissipativity; a canonical

\mathrm{AUC}_{\mathrm{loc}}

quadratic storage for the frozen-selection subsystem that accommodates discontinuous gating; a parametric LMI together with universal kernel constraints and "irreversible forgetting" under universal quadratic storage; and sufficient conditions for global ISS uniformly over admissible selection schedules. We then bridge to practice by deriving a sampled block LMI for the Mamba selective-scan core, which is used as a differentiable training-time regularizer. Across seven standard time-series datasets and four prediction horizons, the regularizer reduces sampled Mamba-core LMI violations by roughly

92\%

28/28

pairs at a clean-MSE cost of less than

0.018\%

. It improves internal Mamba passivity and state-norm diagnostics under injected perturbations. Our results turn classical control-theoretic tools into verifiable structural and training criteria for selective SSMs, while honestly scoping which guarantees transfer to a deep selective-scan architecture.

Nikola Zubić, Davide Scaramuzza

Robotics and Perception Group University of Zurich

State Space Models Discontinuous Gating Regularity Passive Unsupervised Detection

Ask PDF