The Equalizer: Introducing Shape-Gain Decomposition in Neural Audio Codecs

Samir Sadok; Laurent Girin; Xavier Alameda-Pineda

cs.SDFeb 17, 2026

The Equalizer: Introducing Shape-Gain Decomposition in Neural Audio Codecs

Authors: Samir Sadok, Laurent Girin, Xavier Alameda-Pineda

Organizations: Inria at Univ. Grenoble Alpes, CNRS, LJK, France · Univ. Grenoble Alpes, Grenoble-INP, GIPSA-lab, France

Abstract

Neural audio codecs (NACs) typically encode the short-term energy (gain) and normalized structure (shape) of speech/audio signals jointly within the same latent space. As a result, they are poorly robust to a global variation of the input signal level in the sense that such a variation has a strong influence on the embedding vectors at the output of the encoder and their quantization. This methodology is inherently inefficient, leading to codebook redundancy and suboptimal bitrate-distortion performance. To address these limitations, we propose to introduce shape-gain decomposition, widely used in classical speech and audio coding, into the NAC framework. The principle of the proposed Equalizer methodology, easily applicable to any NAC, is to decompose the input signal---before the NAC encoder---into gain and normalized shape vector on a short-term basis. The shape vector is processed by the NAC, while the gain is quantized with scalar quantization and transmitted separately. The output (decoded) signal is reconstructed from the normalized output of the NAC and the quantized gain. Our experiments conducted on speech signals with four different prominent codecs show that this general methodology enables a substantial gain in bitrate-distortion performance, as well as a massive reduction in quantizer complexity.

Figures & tables

Fig. 1 : Effect of global input gain variation on BigCodec, SpeechTokenizer and DAC latent representations. Top: Average embedding norm (normalized to $0$ dB). Middle: Average cosine similarity with the $0$ dB reference embedding. Bottom: Discrete code stability after quantization of the embedding vectors. The vertical dashed line marks $0$ dB, the reference gain.

Fig. 2 : Architecture of the proposed Equalizer NAC pipeline. Block 1: The input signal $\mathbf{s}$ is decomposed into a temporal gain envelope and an equalized waveform $\bar{\mathbf{s}}$ using short-term analysis, normalization, and OLA synthesis. Block 2: The resulting successive shape vectors are processed by a NAC applying vector quantization on the corresponding embedding vectors. Block 3: In parallel, the gain is quantized with scalar quantization (typically $\mu$ -law). Block 4: The decoded equalized output waveform and the quantized gain are used to generate the final output waveform using again short-term analysis-synthesis with OLA.

Codec	Emb. vec.	Num. of RVQ	Codebook	Bitrate
	rate ( $f$ in Hz)	stages ( $N_{q}$ )	size ( $C$ )	( $r$ in kbps)
EnCodec	50	8	128–1024	2.8–4.0
EnCodec*	50	2, 4, 6	1024	1.0–3.0
DAC	50	6, 8, 10, 12	1024	3.0–6.0
BigCodec	80	1	1024–8192	0.8–1.04
WavTokenizer	40	1	4096	0.48

TABLE I : Configuration of the NACs as used in our experiments.

Panel of Figure 4 — Fig. 3 : Performance of EnCodec-Eq (a) and BigCodec-Eq (b) (solid lines) and the corresponding baseline codecs (dashed lines) as a function of the input gain $\alpha$ , for four different codebook sizes $C$ . Scores are averaged over approximately $5.4$ h of test signals from the Librispeech-test dataset. For both codecs, the Equalizer version remains nearly invariant to changes in $\alpha$ , whereas the baseline models degrade as $\alpha$ deviates from $0$ dB. Furthermore, the Equalizer versions consistently outperform their corresponding baselines even at $0$ dB.

Codec	Equalizer	PESQ $\uparrow$	STOI $\uparrow$	MCD $\downarrow$	PESQ $\uparrow$	STOI $\uparrow$	MCD $\downarrow$	PESQ $\uparrow$	STOI $\uparrow$	MCD $\downarrow$	PESQ $\uparrow$	STOI $\uparrow$	MCD $\downarrow$	PESQ $\uparrow$	STOI $\uparrow$	MCD $\downarrow$
		$-12$ dB			$-4$ dB			$0$ dB			$+4$ dB			$+12$ dB
WavTok. (0.48 kbps)	✗	1.19	0.782	7.24	1.28	0.812	6.93	1.31	0.818	6.85	1.31	0.819	6.78	1.26	0.808	6.62
WavTok. (0.48 kbps)	✓	1.58	0.860	5.80	1.58	0.860	5.85	1.58	0.860	5.87	1.58	0.860	5.86	1.58	0.860	5.59
BigCodec (1.04 kbps)	✗	2.08	0.904	4.55	2.22	0.913	4.24	2.20	0.912	4.20	2.15	0.909	4.23	1.35	0.776	5.31
BigCodec (1.04 kbps)	✓	2.46	0.930	3.69	2.46	0.930	3.71	2.46	0.930	3.71	2.46	0.930	3.71	2.46	0.930	3.70
EnCodec* (3 kbps)	✗	2.17	0.898	4.78	2.31	0.910	4.54	2.33	0.911	4.48	2.32	0.908	4.56	1.86	0.878	4.87

TABLE II : PESQ, STOI, and MCD scores obtained in our experiments for the four codecs, with and without combination with The Equalizer and for five selected values of the global input gain. All codecs were retrained from scratch on the LibriSpeech-100 dataset. The Equalizer version (✓) demonstrates near-perfect robustness to input gain variations as opposed to the corresponding baseline (✗).

Fig. 4 : Performance of the proposed Equalizer methodology and of the baselines, as a function of the (total) bitrate, and for the four considered codecs (two versions for EnCodec). Each score is averaged over $\approx 5.4$ h of test signals from the Librispeech-test dataset and over an input gain range of $\pm 12$ dB. The Equalizer (solid lines) consistently outperforms the corresponding baseline (dashed lines).

Fig. 5 : Results of the pairwise preference test.

Neural Audio Codecs

Neural audio codecs learn compact representations for compressing and reconstructing speech, music, and other sounds.

See all 87 papers

GS-Codec: A Gaussian-Splatting Bottleneck for Neural Audio Coding
Oct 3, 2026Ron Aluf, Alon Canfi, Eliya NachmaniNeural Audio Codecs
School of Electrical and Computer Engineering Ben-Gurion University of the Negev
Language-Statistical Analysis of Neural Audio Codec Tokens Across Architectures, Corpora, and Noise Conditions
Aug 31, 2026Joonyong Park, Shinnosuke Takamichi, David M. Chan +3Neural Audio Codecs Residual VQ
Grad. School of Information Science and Technology, The University of Tokyo, Tokyo, Japan · Faculty of Science and Technology, Keio University, Tokyo, Japan · Berkeley Artificial Intelligence Research Lab (BAIR), University of California, Berkeley, CA, USA
Towards Interpretable Framework for Neural Audio Codecs via Sparse Autoencoders: A Case Study on Accent Information
Mar 18, 2026Shih-Heng Wang, Tiantian Feng, Aditya Kommineni +4Neural Audio Codecs Sparse Autoencoders
University of Southern California, USA

Oct 3, 2026J/K move · Enter open · S save

GS-Codec: A Gaussian-Splatting Bottleneck for Neural Audio Coding

Ron Aluf, Alon Canfi, Eliya Nachmani

School of Electrical and Computer Engineering Ben-Gurion University of the Negev

Neural Audio Codecs

Open paper AskPDF arXiv

Neural audio codecs compress waveforms into compact discrete tokens that underpin speech language models, real-time communication, and large-scale audio storage. Almost every dominant design, including residual vector quantization, finite scalar quantization, and single-codebook variants, follows the VQ-VAE template by partitioning the encoder latent through a learned codebook or a fixed scalar grid. We ask whether this partition is necessary. We introduce GS-Codec, a neural speech codec whose bottleneck is a parametric signal decomposition rather than a quantizer. We adapt Gaussian splatting from 3D scene reconstruction to one-dimensional latents. An inner optimization loop fits each encoder segment as a weighted sum of 1D Gaussian primitives. The decoder then reconstructs the waveform from the rendered sum. To avoid the cost of this iterative inner loop at inference time, we additionally train a lightweight GS Predictor Net that regresses the primitive parameters in a single forward pass. The encoder and decoder are trained end-to-end through the inner loop, with no quantizer anywhere in the training pipeline: the bottleneck is the decomposition itself, and scalar quantization is applied only post-training to the fitted parameters. Rather than relying on discrete codebook stages for bitrate control, our representation exposes a fine-grained rate-quality tradeoff: a single trained checkpoint supports post-training bitrate control by varying the number of primitives and the per-parameter bit depth, with no retraining required. GS-Codec matches or exceeds well-established open-source codecs such as EnCodec and DAC on speaker similarity (SIM), intelligibility (STOI), and perceptual quality (UTMOS) at comparable bitrates, while achieving comparable semantic performance (WER). Code and audio samples are available at https://ronaluf.github.io/gs-codec/

Abstract

Neural Audio Codecs

Explore similar work

GS-Codec: A Gaussian-Splatting Bottleneck for Neural Audio Coding

Language-Statistical Analysis of Neural Audio Codec Tokens Across Architectures, Corpora, and Noise Conditions

Towards Interpretable Framework for Neural Audio Codecs via Sparse Autoencoders: A Case Study on Accent Information