Riccati State Space Models: Non-iterative Parallelization for Nonlinear Sequence Modeling

Mónika Farsang; Ramin Hasani; Daniela Rus; Radu Grosu

cs.LGSep 28, 2026

Riccati State Space Models: Non-iterative Parallelization for Nonlinear Sequence Modeling

Authors: Mónika Farsang, Ramin Hasani, Daniela Rus, Radu Grosu

Organizations: TU Wien · MIT CSAIL · Liquid AI

Abstract

State space models (SSMs) achieve efficient sequence processing because their affine state updates are closed under composition and can therefore be evaluated with an associative parallel scan. Nonlinear recurrent models can provide richer, state-dependent dynamics, but generally lose this compositional structure: parallel evaluation then requires iterative methods that repeatedly linearize and scan the recurrence. We ask, what state-dependent nonlinear dynamics can be designed to remain exactly composable? We answer by introducing RiccatiSSM, a nonlinear SSM, in which each state dimension follows an input-conditioned Riccati differential equation. Its quadratic state dependence makes the local Jacobian explicitly state-dependent, while its exact per-step flow under piecewise-constant inputs is a Möbius transformation. Since Möbius maps are closed under composition and compose through $2\times 2$ matrix multiplication, the complete nonlinear state trajectory can be evaluated exactly with a single associative parallel scan, without iterative linearization. We further derive a constrained parameterization that ensures bounded, contractive dynamics, and avoids poles in the fractional-linear state update. Across long-sequence classification, regression, and forecasting tasks, RiccatiSSM achieves competitive predictive performance while reducing runtime by $22{-}33\%$ compared to the nonlinear LrcSSM under matched architectures. These results demonstrate that state-dependent nonlinear dynamics can retain exact composability and be evaluated efficiently within a single parallel scan.

Figures & tables

	DEER/ELK-based models	RiccatiSSM
Dynamics	any $f(x_{t-1},u_{t})$	Riccati: $\alpha+\beta x+\gamma x^{2}$
Scan element	affine ( $(J_{t},c_{t})$ , $J_{t}$ the state Jacobian)	fractional-linear, $M_{t}\in\mathbb{R}^{2\times 2}$
Composition	approximate (per iteration)	exact
Sweeps per layer	until convergence ( $K$ , data-dependent)	$1$ , fixed
Depth per layer	$\mathcal{O}(K\log T)$	$\mathcal{O}(\log T)$
Work per layer	$\mathcal{O}(K\,TD)$	$\mathcal{O}(TD)$

Table 1: RiccatiSSM compared with DEER/ELK-style ( Lim et al., 2024 ; Gonzalez et al., 2024 ) parallelization of general nonlinear recurrences. Both evaluate state-dependent dynamics with associative scans, but DEER/ELK approximate the general nonlinear update by Newton iteration updates until convergence, whereas RiccatiSSM restricts the dynamics to a family that composes exactly in a single step.

T

is the sequence length,

D

the number of channels, and

K

the number of sweeps.

Figure 1: RiccatiSSM evaluates a nonlinear recurrence by a detour through a two-dimensional linear one. Bottom (grey, dashed): the states

x_{t}

of one channel, whose Möbius updates are nonlinear in the state. Composing such maps directly requires clearing nested fractions, so the bottom row offers no cheap merge to scan over. Top (blue, solid): the same states as homogeneous pairs

(p_{t},q_{t})

, where each step is the matrix

M_{t}

. The initial state is lifted once,

x_{0}\mapsto(x_{0},1)

, and states are recovered by

x_{t}=p_{t}/q_{t}

. Matrix multiplication is associative, so the top row collapses into prefix products

P_{t}=M_{t}\cdots M_{1}

under one associative scan of depth

\mathcal{O}(\log T)

, with no fixed-point iterations; all

D

channels scan in parallel in the same round.

	Heart	SCP1	SCP2	Ethanol	Motor	Worms
Sequence length	405	896	1,152	1,751	3,000	17,984
Input size	61	6	7	2	63	6
#Classes	2	2	2	4	2	5
Transformer ‡	70.5 $\pm$ 0.1	84.3 $\pm$ 6.3	49.1 $\pm$ 2.5	40.5 $\pm$ 6.3	50.5 $\pm$ 3.0	OOM
RFormer ‡	72.5 $\pm$ 0.1	81.2 $\pm$ 2.8	52.3 $\pm$ 3.7	34.7 $\pm$ 4.1	55.8 $\pm$ 6.6	90.3 $\pm$ 0.1
NRDE †	73.9 $\pm$ 2.6	76.7 $\pm$ 5.6	48.1 $\pm$ 11.4	31.4 $\pm$ 4.5	54.0 $\pm$ 7.8	77.2 $\pm$ 7.1

Table 2: Test accuracy comparison of different models. The performance of the models marked by † is reported from Rusch and Rus (2025) , those with ‡ from Moreno-Pino et al. (2024) , and

\diamond

from Farsang and Grosu (2025) . The same hyperparameter tuning protocol and dataset splitting over the same 5 seeds were used. Bold highlights the top 3 models.

Model	MSE $\times 10^{-2}$ ( $\downarrow$ )
NRDE † ( Morrill et al., 2021 )	9.90 $\pm$ 0.97
NCDE † ( Kidger et al., 2020 )	13.54 $\pm$ 0.69
Log-NCDE † ( Walker et al., 2024 )	9.56 $\pm$ 0.59
LRU † ( Orvieto et al., 2023 )	12.17 $\pm$ 0.49
S5 † ( Smith et al., 2023 )	12.63 $\pm$ 1.25
LinOSS-IMEX † ( Rusch and Rus, 2025 )	7.50 $\pm$ 0.46

Table 3: Mean squared error (MSE ×

10^{-2}

) for different models on the PPG-DaLiA dataset. The performance of the models marked by † is reported from Rusch and Rus (2025) and

\diamond

from Farsang and Grosu (2025) . All results are averaged over 5 seeds.

Model	Mean Absolute Error ( $\downarrow$ )
Informer † ( Zhou et al., 2021 )	$0.731$
LogTrans † ( Li et al., 2019 )	$0.773$
Reformer † ( Kitaev et al., 2020 )	$1.575$
LSTMa † ( Bahdanau et al., 2014 )	$1.109$
LSTnet † ( Lai et al., 2018 )	$0.757$
S4 † ( Gu et al., 2022 )	$0.5783$

Table 4: Mean absolute error on the weather dataset predicting 720 future time steps based on 720 past time steps. The performance of the models marked by † is reported from Rusch and Rus (2025) .

Figure 2: Time per step (seconds) for RiccatiSSM and LrcSSM across three UEA time-series classification datasets using the same architecture on an NVIDIA A100 GPU. RiccatiSSM evaluates each layer with a single parallel scan, whereas LrcSSM requires approximately three quasi-DEER iterations. The runtime gap is largest for the longer EigenWorms sequences.

Model	Input-dependent coefficients	MSE ( $\downarrow$ )
RiccatiSSM (ours)	$\alpha(u),\beta(u),\gamma(u),\varepsilon(u)$	$\mathbf{7.15\pm 1.01}$
$\varepsilon$ ablation	$\alpha(u),\beta(u),\gamma(u)$	$8.45\pm 0.38$
Reduced input dependence	$\alpha(u),\beta,\gamma,\varepsilon$	$8.71\pm 1.10$
RiccatiSSM (LRC-tied)	LRC Taylor-matched	$8.47\pm 0.47$

Table 5: Ablation of the RiccatiSSM parameterization on PPG-DaLiA. Test MSE (

\times 10^{-2}

, lower is better), reported as mean

\pm

standard deviation over five seeds.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Factor isolated	Comparison	Relative RMSE
Truncation effect (Exact integration)	RiccatiSSM-ZOH vs. LrcSSM-RK4	$1.73\times 10^{-5}$
Truncation effect (Euler integration)	RiccatiSSM-Euler vs. LrcSSM-Euler	$1.73\times 10^{-5}$
Integrator choice LrcSSM	LrcSSM-Euler vs. LrcSSM-RK4	$7.16\times 10^{-3}$

Appendix

Table 6: Relative RMSE over

T=512

steps, receiving white-noise input.

Figure 3: Hidden-state trajectories of 5 neurons under three models with matched Taylor coefficients driven by white-noise input. LrcSSM-RK4 (black, 64 RK4 substeps) serves as ground truth for the full dynamics; LrcSSM-Euler (blue dashed) is the model’s single Euler step as in [ Farsang and Grosu, 2025 ] ; RiccatiSSM-ZOH (red dotted) solves the quadratic (Riccati) truncation exactly via zero-order hold. The inset shows the first 30 timesteps (dashed box). Close match between RiccatiSSM-ZOH and both LrcSSM variants indicates that the dominant discretisation error comes from the Euler integrator rather than the quadratic truncation of the vector field.

Figure 4: Reachable constraint-coefficient space under free vs. LRC-tied Riccati parameterizations, sampled over 500k random initializations. (Left) Distribution of normalized stability coefficient $\alpha/\alpha_{max}$ : the free variant saturates at $\pm 1$ , while the LRC-tied variant concentrates near zero due to double-squashing through the circuit nonlinearities. (Center) Distribution of curvature coefficient $\gamma$ : LRC-tied outputs are similarly compressed relative to the free range. (Right) Joint ( $\gamma,\alpha/\alpha_{max}$ ) reachable region; the LRC-tied parameterization occupies a strictly smaller subspace, reflecting the structural constraints imposed by the underlying neural circuit dynamics.

	lr	hidden dim.	state-space dim.	number of layers
Heart	$10^{-3}$	64	64	4
SCP1	$10^{-4}$	64	16	6
SCP2	$10^{-3}$	16	16	2
Ethanol	$10^{-3}$	64	16	6
Motor	$10^{-3}$	16	256	4
Worms	$10^{-3}$	16	16	4

Appendix

Table 7: Optimized hyperparameters used for the classification, PPG-DaLiA and Weather datasets.

	NRDE	NCDE	Log-NCDE	LRU	S5	Mamba	S6	LinOSS-IMEX	LinOSS-IM	LrcSSM	RiccatiSSM
Heart	9539	1177	826	8	11	34	4	4	7	23	21
SCP1	1014	973	635	9	17	7	3	42	38	12	33
SCP2	1404	1251	583	9	9	32	7	55	22	15	12
Ethanol	2256	2217	2056	16	9	255	4	48	8	15	26
Motor	7616	3778	730	51	16	35	34	128	11	31	27
Worms	5386	24595	1956	94	31	122	68	37	90	33	23

Appendix

Table 8: Runtime in seconds for the models for 1000 training steps, using the best configuration for each. Values for the models are taken from Rusch and Rus [2025] and Farsang and Grosu [2025] .

Explore similar work

MuonSSM: Orthogonalizing State Space Models for Sequence Modeling
Jun 29, 2026Thai-Khanh Nguyen, Ngoc-Bich-Uyen Vo, Thieu N. Vo +2State Space Models Sequence Modeling
Faculty of Information Technology, Dainam University, Hanoi University of Science and Technology, Hanoi 10000, Vietnam · Faculty of Artificial Intelligence, Posts and Telecommunications Institute of Technology, Hanoi 10000, Vietnam · Department of Computer Science, University of Bath, Bath BA2 7AY, UK +1
Reshape and Recur: Improving SSMs with Input Reshaping and Depth Recurrence
May 15, 2026Mónika Farsang, Ramin Hasani, Daniela Rus +1Time-Series Classification State Space Models
1TU Wien · MIT CSAIL · Liquid AI
Regularity and Stability Properties of Selective SSMs with Discontinuous Gating
May 16, 2025Nikola Zubić, Davide ScaramuzzaState Space Models Discontinuous Gating
Robotics and Perception Group University of Zurich

Jun 29, 2026, cs.LGJ/K move · Enter open · S save

MuonSSM: Orthogonalizing State Space Models for Sequence Modeling

Thai-Khanh Nguyen, Ngoc-Bich-Uyen Vo, Thieu N. Vo, Tan M. Nguyen+1

Faculty of Information Technology, Dainam University, Hanoi University of Science and Technology, Hanoi 10000, Vietnam · Faculty of Artificial Intelligence, Posts and Telecommunications Institute of Technology, Hanoi 10000, Vietnam · Department of Computer Science, University of Bath, Bath BA2 7AY, UK+1

State Space Models Sequence Modeling Momentum

Open paper AskPDF arXiv

State space models (SSMs) have emerged as efficient linear-time alternatives to attention for long-sequence modeling. However, existing SSMs often suffer from instability and memory degradation over extended horizons due to poorly conditioned first-order updates and unbalanced update geometry. We introduce MuonSSM, a general framework that stabilizes SSM training by explicitly conditioning the geometry of memory updates rather than the recurrent transition matrix. MuonSSM augments SSMs with a momentum-based pathway and a lightweight Newton Schulz transformation on low-rank input injections, yielding bounded and spectrally conditioned updates while preserving parallel scan complexity. Theory shows that MuonSSM improves gradient propagation, mitigates spectral amplification, and enriches memory representations over long horizons. Extensive experiments across language, vision, and time-series benchmarks show consistent gains in accuracy, robustness, and long-context performance when integrated into diverse SSM backbones. These results establish geometric conditioning of updates as a principled pathway to stable, scalable sequence modeling.

Abstract

Explore similar work

MuonSSM: Orthogonalizing State Space Models for Sequence Modeling

Reshape and Recur: Improving SSMs with Input Reshaping and Depth Recurrence

Regularity and Stability Properties of Selective SSMs with Discontinuous Gating