eess.SYOct 16, 2025

A Deep State-Space Model Compression Method using Upper Bound on Output Error

Authors: Hiroki Sakamoto, Kazuhiro Sato

Organizations: Department of Mathematical Informatics, Graduate School of Information Science and Technology, The University of Tokyo, Tokyo 113-8656, Japan

Abstract

We study deep state-space models (Deep SSMs) that contain linear quadratic-output (LQO) systems as internal blocks and present a compression method with a provable output error guarantee. We first derive an upper bound on the output error between two Deep SSMs and show that the bound can be expressed in terms of the h2h^2-error norms between the layerwise LQO systems. In particular, we show that reducing the h2h^2 approximation errors of the LQO systems placed in shallow layers is effective in reducing the derived upper bound on the output error. Next, we formulate an optimization problem for the derived upper bound and develop a gradient-based MOR method. In the numerical experiments, using the IMDb task from the LRA benchmark, we demonstrate the effectiveness of the proposed upper-bound-based compression method. In particular, we show that the number of trainable parameters can be reduced by approximately 60% without retraining while maintaining the performance of the original model.

Figures & tables

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. On the Expressive Power and Limitations of Multi-Layer SSMs

    Apr 16, 2026Nikola Zubić, Qian Li, Yuyi Wang +1CoT ReasoningState Space Models

  2. Latent Matters: Learning Deep State-Space Models

    Feb 26, 2026Alexej Klushyn, Richard Kurle, Maximilian Soelch +2State Space ModelsLatent Dynamics Modeling

  3. Predicting LLM Compression Degradation from Spectral Statistics

    Apr 20, 2026Mingxue XuLLM CompressionLow-Rank Matrix Decomposition