cs.LGOct 29, 2026

TensorSSM: Memorizing Spatial Position in Dynamic State Space for Tensor Core Generation

Authors: Sheng Zhang, Xinyi Wang, Maolin Wang, Xiangyu Zhao

Organizations: City University of Hong Kong, Tat Chee Avenue 83, Hong Kong, 999077, China · Columbia University, 116th St & Broadway, New York, 10027, NY, USA

Abstract

High-dimensional tensors are ubiquitous in real-world applications but lead to massive storage overhead. Functional tensor decomposition addresses this by employing neural networks to map element coordinates to tensor cores, significantly reducing the parameters for data compression. However, existing functional methods typically flatten high-order tensors into 1D sequences. This folding operation causes highly overlapping local coordinate sequences and struggles to capture complex cross-mode spatial dependencies, severely limiting their expressiveness on diverse datasets. To tackle these challenges, we propose TensorSSM, a novel Implicit Neural Representation (INR) for functional decomposition method that memorizes spatial positions within a dynamic State Space Model (SSM). Specifically, TensorSSM first decomposes mode-wise coordinates into sequences spit by K-ary rule, encoding spatial tokens from coarse to fine, which effectively captures spatial variations. Furthermore, we extend the SSM's state space to explicitly map cross-mode spatial dependencies, allowing inter-mode interactions to enhance the updates of each mode. By integrating cross-mode information from previous hidden states into the current state transitions, TensorSSM effectively expands tensor representation along two dimensions: the sequence dimension for coarse-to-fine spatial tokens, and the state dimension for cross-mode memorization. This design effectively integrates spatial information into the state space and the tensor core, substantially advancing the state adaptability and enhancing the expressiveness of the data compression. Extensive experiments demonstrate that TensorSSM achieves significantly high reconstruction accuracy and storage efficiency, exhibiting robust expressiveness across diverse tensor datasets without relying on strict spatial assumptions.

Figures & tables

Explore similar work

CardsList
  1. Fast Tensorization of Neural Networks via Slice-wise Feature Distillation

    May 19, 2026Safa Hamreras, Sukhbinder Singh, Román OrúsTensor NetworksTensor Decomposition

  2. A New Tensor Network: Tubal Tensor Train and Its Applications

    Mar 11, 2026Salman Ahmadi-Asl, Valentin Leplat, Anh-Huy Phan +1Tensor DecompositionTT Decomposition

  3. Scaling State-Space Models on Multiple GPUs with Tensor Parallelism

    Feb 24, 2026Anurag Dutt, Nimit Shah, Hazem Masarani +1LLM Inference AccelerationTensor Parallelism