Naga: Vedic Encoding for Deep State Space Models

Melanie Schaller; Nick Janssen; Bodo Rosenhahn

cs.LGNov 17, 2025

Naga: Vedic Encoding for Deep State Space Models

Authors: Melanie Schaller, Nick Janssen, Bodo Rosenhahn

Abstract

This paper presents Naga, a deep State Space Model (SSM) encoding approach inspired by structural concepts from Vedic mathematics. The proposed method introduces a bidirectional representation for time series by jointly processing forward and time-reversed input sequences. These representations are then combined through an element-wise (Hadamard) interaction, resulting in a Vedic-inspired encoding that enhances the model's ability to capture temporal dependencies across distant time steps. We evaluate Naga on multiple long-term time series forecasting (LTSF) benchmarks, including ETTh1, ETTh2, ETTm1, ETTm2, Weather, Traffic, and ILI. The experimental results show that Naga outperforms 28 current state of the art models and demonstrates improved efficiency compared to existing deep SSM-based approaches. The findings suggest that incorporating structured, Vedic-inspired decomposition can provide an interpretable and computationally efficient alternative for long-range sequence modeling.

Figures & tables

Figure 1: Comparison between conventional multiplication in Mamba2 [ 3 ] ; (left) and the proposed Vedic element-wise decomposition in Naga (right). While Mamba2 [ 3 ] performs standard matrix multiplications for state updates, Naga introduces bidirectional input reversal and element-wise operations inspired by Vedic mathematics. This decomposition enhances gradient flow and enables the model to capture complex temporal dependencies more efficiently.

Criterion	Naga (Ours)	S-Mamba [ 23 ]	LTSMamba	Mamba-2 [ 2 ]
Cell Structure	Mamba2-like block applied at feature level using Vedic decomposition.	Mamba Block with parametrized SSM-kernels, bidirectional.	Mamba2-Cell with SSD matrices, TDB, ICB.	SSD-based Mamba2Cell with semi-separable attention kernel.
Selectivity	✓ Adaptive, token-wise feature selection via element-wise decomposition.	✓ Dynamic kernel selection, token-wise adaptive.	± Gated forget mechanism + adaptive channel re-ranking.	✓ Grouped-value attention with selective SSD-state spaces.
Bidirectionality	✓ Bidirectional Vedic encoding.	✓ Bidirectional Mamba Blocks.	✓ Bidirectional TDB and ICB structures with attention.	± Theoretically bidirectional (configurable).
Inter-Channel Awareness	± Moderate – features processed individually, easing interaction learning.	✗ Weak (standard multivariate input).	✓ Strong via ICB and adaptive re-ranking.	± Medium (grouped-value attention).
Output Head	Linear head over last Mamba2 time steps.	Feedforward reconstructor layer.	TDB/ICB fused with feedforward and ranking outputs.	LayerNorm followed by linear readout.
Efficiency	✓ Lightweight – element-wise, parallelizable multiplication.	✓ Lightweight, $\mathcal{O}(n)$ , dependent on SSM efficiency.	✗ Heavier due to TDB+ICB complexity.	✓ Highly efficient due to optimized SSD structure.

Table 1: Comparison of Naga with S-Mamba, LTSMamba, and Mamba-2 across architectural and computational characteristics.

Figure 2: Overview of Naga method.

Dataset	Pred. length	Epochs	Runtime [s]	MSE (Naga)
ETTh1	96	16	131.02	0.124
ETTh2	720	73	213.31	0.139
ETTm1	720	35	2145.10	0.117
ETTm2	96	23	275.82	0.119
Weather	96	45	812.64	0.089
Traffic	96	38	1150.77	0.006

Table 2: Benchmarking results of Naga across LTSF datasets. Reported values correspond to training runtime (in seconds) until early stopping, number of epochs, and forecasting accuracy in terms of Mean Squared Error (MSE). For all experiments we use a fixed look-back window of

d=96

(ETTh1, ETTm2, Weather, Traffic) and

d=720

(ETTh2, ETTm1), following standard LTSF practice.

Figure 3: Memory usage versus mean squared error (MSE) values for various long-term forecasting models across multiple benchmark datasets. The prediction horizon ( Pred_length ) is indicated in each title. Naga is highlighted with a star symbol. (As we wanted to show the error values against memory usage, we avoided to use tabular results although both MSE as well as MAE were calculated due to a lack of space).

Sequence Modeling

Sequence modeling studies how to represent, predict, classify, and generate ordered data such as text, time series, events, and biological sequences.

See all 250 papers

Reshape and Recur: Improving SSMs with Input Reshaping and Depth Recurrence
May 15, 2026Mónika Farsang, Ramin Hasani, Daniela Rus +1Parameter Sharing State Space Models
1TU Wien · MIT CSAIL · Liquid AI
LogSig-SSM: Time-Series Modelling with Multi-Scale Log-Signature Compression for State-Space Models
Oct 4, 2026Felix Oury, Nicolas Calvo Peiro, Reiko J. TanakaIrregular Time-Series Modeling Multivariate Time Series Forecasting
Department of Computing Imperial College London · Department of Bioengineering Imperial College London
Model- and Layer-Level Closed-Loop Forecasting with SSM-Based Spiking Neural Networks
Oct 20, 2026Erik Stolt, Sanja Karilanova, Ayca OzcelikkaleTime Series Forecasting Spiking Neural Networks
Department of Electrical Engineering, Uppsala University, Uppsala, Sweden

May 15, 2026J/K move · Enter open · S save

Reshape and Recur: Improving SSMs with Input Reshaping and Depth Recurrence

Mónika Farsang, Ramin Hasani, Daniela Rus, Radu Grosu

1TU Wien · MIT CSAIL · Liquid AI

Parameter Sharing State Space Models

Open paper AskPDF arXiv

State Space Models (SSMs) are increasingly deployed in the Edge because they offer, at comparable performance, a smaller memory/training/inference footprint, compared to Large Language Models (LLMs). These three advantages are a direct consequence of the time recurrence inherent in the SSMs architecture. Here, we further improve this recurrent architecture by positively answering two previously underexplored, orthogonal questions: (1) Can we reduce SSMs memory-footprint without any performance penalty, by also employing depth recurrence? (2) Can we increase SSMs performance by using a fixed and consistent time-granularity across all tasks? The first question is somewhat unexpected, given that SSMs are already recurrent. However, the orthogonal depth recurrence further decreases SSMs memory footprint. We show that a looped SSM with

k

parameters adaptively iterated

M

times, achieves a performance comparable to a standard SSM with

k \cdot L

independent parameters, where

M \leq L

. The second question is also unexpected given the time-recurrent nature of the SSMs architecture. However, it makes perfect sense for the time-parallel training of SSMs on the entire input sequence. We show that concatenating time steps for lower-dimensional sequence elements, or flattening and re-chunking the joint feature-time dimension for high-dimensional ones, can improve the baseline by enhancing the way information is presented to the model. Our results for both extensions lead to consistent benefits across four representative SSM architectures: LRU, S5, LinOSS, LrcSSM.

Abstract

Sequence Modeling

Explore similar work

Reshape and Recur: Improving SSMs with Input Reshaping and Depth Recurrence

LogSig-SSM: Time-Series Modelling with Multi-Scale Log-Signature Compression for State-Space Models

Model- and Layer-Level Closed-Loop Forecasting with SSM-Based Spiking Neural Networks