Overview

Neural audio codecs compress sound by mapping waveforms into compact latent representations, often using residual vector quantization, then reconstructing audio with a learned decoder. Current approaches balance perceptual fidelity with bitrate, intelligibility, speaker identity, and attributes such as emotion or pitch, using perceptual losses, self-supervised features, and separate semantic and acoustic streams. Researchers also design codec tokens for downstream generation with language models, while developing streaming codecs, variable frame rates, and hybrid discrete-continuous representations. A central challenge is that lower rates and more predictable tokens can lose acoustic detail, and quality metrics do not always reveal whether information needed for speech or other tasks survives compression.

Latest papers

91 papers9 papers in the last four weeks, down 40% on the four weeks before. 0.1% of all new papers.

Oct 29, 2025

PitchFlower: A flow-based neural audio codec with pitch controllability

We present PitchFlower, a flow-based neural audio codec with explicit pitch controllability. Our approach promotes pitch disentanglement through a simple perturbation: during training, F0 contours are flattened and randomly shifted at the input, while the true F0 is provided as conditioning to regenerate the original audio. A vector-quantization bottleneck prevents pitch recovery, and a flow-based decoder generates high quality audio. Experiments show that PitchFlower achieves accurate pitch control at the level of DSP baselines but at much higher audio quality, and performs on par with state-of-the-art neural approaches. Notably, despite using WORLD-transformed audio for training, our method filters out the vocoder's inherent artifacts, revealing a strong resilience of deep generative modeling to input degradation. This finding suggests that our framework provides a simple and extensible path that could be extended to other speech attributes.