cs.SDJun 15, 2026

Probing Low Frame Rate Degradation in Neural Audio Codecs

Authors: Alex GichambaMoise Busogi

Organizations: Carnegie Mellon University Africa, Rwanda

Abstract

Low frame rates in neural audio codecs are attractive for autoregressive speech synthesis, where the generation cost scales linearly with the sequence length. Recent work has demonstrated that codecs can operate at 12.5 Hz and below, but the mechanisms underlying low frame rate degradation remain insufficiently understood. We investigate these mechanisms through a controlled frame rate ablation. We reproduce a quality cliff at 6.25 Hz reported in previous works and evaluate candidate explanations: phonemic collisions and codebook saturation, neither of which shows evidence of a fundamental barrier. The cliff is instead caused by suboptimal training configuration: fixed clip duration during training yields too few tokens at low frame rates, starving the decoder of inter-token context. Once corrected, WER degrades smoothly with phonemic load down to 3.1 Hz and 1.6 Hz, suggesting the inference-time efficiency gains of low frame rate codecs are more accessible than previously assumed.

Explore similar work

CardsList
  1. ZipCodec: Ultra-Low-Frame-Rate Streaming Speech Coding

    Sep 12, 2026Luca Della Libera, Cem Subakan, Mirco RavanelliNeural Audio CodecsVideo Coding

  2. LACE: Layer-Wise Compression for Dynamic Frame Rate Codecs

    Sep 15, 2026Thanapat Trachu, Samuele Cornell, William Chen +1Neural Audio CodecsFrame Rate