cs.SDOct 15, 2026

Frame-Wise Streaming for Efficient Learned Speech Compression

Authors: Yuhan Liu, Jun Xu, Fengxi Zhang, Zhengxue Cheng, Li Song

Organizations: School of Information Science and Electronic Engineering, Shanghai Jiao Tong University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Speech compression is increasingly important in scenarios such as real-time and satellite communication. Neural codecs have shown superior Rate-Distortion (RD) performance over traditional signal-processing codecs. Models with long-range dependency encoders/decoders and entropy models outperform convolution-based structures, but they still lack streaming support since they rely on sequence-level correlations. In this paper, we propose StreamLSC, a frame-level streaming learned speech compression model built on three fundamental components of streaming networks, i.e. Boundary-Cached Causal Convolution, Intra-Frame Layer Normalization, and RNN-mode RWKV. Our model incorporates a streamable entropy quantizer module, ensuring efficient context model while preserving causal constraints. Experiments show that the streaming model achieves performance comparable to its offline counterpart, attaining state-of-the-art rate-distortion results, significant gains in subjective evaluations, -51.37% (ViSQOL) and -53.99% (PESQ) BD-RATE savings on average, and improvements of 0.2154 and 0.5156 in BD-ViSQOL and BD-PESQ.

Figures & tables

Fig. 1: Subjective evaluation results using MUSHRA. The center point of each circle represents the average value, and the size reflects variability of listener ratings. Our model achieves superior perceptual quality at ultra-low bitrates.
Fig. 2: Illustration of model architectures. (a) Previous non-streamable method. The whole audio sequence is processed at a time. (b) StreamLSC, frame-level streaming processing mode. (c) Streamable model components: FS-CRM block with explicit state propagation in the BCC and RWKV layers. Red blocks denote causal convolutions or convolution-based modules, while green blocks denote inherently streamable components.
Fig. 3: Objective evaluation results: ViSQOL and PESQ scores of the proposed model compared with baselines. The proposed StreamLSC achieves frame-level streamability while maintaining RD performance consistent with the non-streaming version, significantly outperforming the baseline solution with a performance gain of 0.2154 BD-ViSQOL and 0.5156 BD-PESQ.
Frame length80 ms160 ms320 ms640 ms
ViSQOL4.82574.82834.83154.8319
TABLE I: Effect of frame length on quality (ViSQOL).
Training StrategyBitrate (bps)ViSQOLPESQ
Full GAN Training3529.004.45003.1800
Two-stage GAN Training3377.294.53983.1780
TABLE II: Comparison of training strategies on test_clean and test_others datasets.
ModelRTF
StreamLSC (Ours)5.39×10−25.39\times 10^{-2}
FunCodec5.45×10−35.45\times 10^{-3}
EnCodec4.695×10−34.695\times 10^{-3}
DAC6.367×10−36.367\times 10^{-3}
BigCodec1.504×10−21.504\times 10^{-2}
TABLE III: Model parameters and real-time factor (RTF) on RTX 4090.
More in this topic

Neural Audio Codecs

Neural audio codecs learn compact representations for compressing and reconstructing speech, music, and other sounds.

See all 87 papers

Explore similar work

CardsList

  1. Benchmarking Neural Speech Compression from a Rate-Distortion Perspective

    Jun 10, 2026Jun Xu, Zhengxue Cheng, Fengxi Zhang +3Neural Audio CodecsSpeech Processing

    School of Information Science and Electronic Engineering, Shanghai Jiao Tong University, Shanghai 200240, China

  2. ZipCodec: Ultra-Low-Frame-Rate Streaming Speech Coding

    Sep 10, 2026Luca Della Libera, Cem Subakan, Mirco RavanelliAudio Token CompressionNeural Audio Codecs

    Concordia University · Mila-Quebec AI Institute · Universit´e Laval

  3. Q-SPT: Learnable Query-Based Compression for Low-Frame-Rate Speech Tokenization

    Oct 1, 2026Jeeyoung Yun, Seohwan Yun, Sungwoong KimAudio Token CompressionNeural Audio Codecs

    Korea University, Seoul, Republic of Korea