Overview

Neural audio codecs compress sound by mapping waveforms into compact latent representations, often using residual vector quantization, then reconstructing audio with a learned decoder. Current approaches balance perceptual fidelity with bitrate, intelligibility, speaker identity, and attributes such as emotion or pitch, using perceptual losses, self-supervised features, and separate semantic and acoustic streams. Researchers also design codec tokens for downstream generation with language models, while developing streaming codecs, variable frame rates, and hybrid discrete-continuous representations. A central challenge is that lower rates and more predictable tokens can lose acoustic detail, and quality metrics do not always reveal whether information needed for speech or other tasks survives compression.

Latest papers

CardsList

91 papers9 papers in the last four weeks, down 40% on the four weeks before. 0.1% of all new papers.

  1. Is Semantics Enough for Speech Mean Opinion Score Prediction?

    Sep 3, 2026Tianyu Lan, Yufei Shi, Yang Ai +3Human Preference EvaluationNeural Audio Codecs

  2. BiMTokenizer: Preserving Semantic-Acoustic Balance in Low-Bitrate Speech Tokenization via Bidirectional State-Space Modeling

    Sep 1, 2026Xin Zhang, Lin Li, Chuanbo Liu +2Neural Audio CodecsVector Quantization

  3. Language-Statistical Analysis of Neural Audio Codec Tokens Across Architectures, Corpora, and Noise Conditions

    Aug 31, 2026Joonyong Park, Shinnosuke Takamichi, David M. Chan +3Neural Audio CodecsResidual VQ

  4. Geometric Iterative Retrieval for Neural Audio Codec Resynthesis

    Aug 19, 2026Leo Schmidt-Traub, Frédéric Berdoz, Luca A. Lanzendörfer +1Neural Audio CodecsResidual VQ

  5. ReLMCodec: Designing Predictable Speech Tokens from Pre-Quantization Phoneme Structure

    Aug 8, 2026Zixiang Wan, Xusheng Yang, Zheng Wang +1Neural Audio CodecsAudio Tokenization

  6. LILAC: An Idempotent Neural Speech Codec

    Aug 6, 2026June Young Yi, Dongwook Lee, Jiheum Yeom +1Neural Audio Codecs

  7. On the Geometry of Music Bandwidth Extension in Latent Spaces of Audio Codecs

    Aug 4, 2026Hendrik Vincent Koops, Hao Hao Tan, Elio QuintonNeural Audio CodecsNeural Representation Geometry

  8. MeloCodec: Harnessing Melodic Priors for High-Fidelity Singing Voice Representation

    Aug 4, 2026Yizhong Geng, Wenxin Fu, Kecan Mao +7Audio Representation LearningNeural Audio Codecs

  9. Stable Autoregressive Speech Generation with Low-Frame-Rate High-Dimensional Continuous Tokens

    Jul 31, 2026Yi Luo, Rongzhi Gu, Jixun YaoFlow MatchingAudio Representation Learning

  10. Dissecting Sensitivity to Training Language in Self-Supervised Speech Learning Using Neural Audio Codec Tokens

    Jul 28, 2026Daigo Takizawa, Tomohiko Nakamura, Samuele Cornell +3Audio Representation LearningNeural Audio Codecs

  11. Investigating Codec-Internal Latent Audio Watermarking for Neural Codec Robustness

    Jul 23, 2026Zi Hu, Houmin Sun, Linxi Li +4Audio WatermarkingNeural Audio Codecs

  12. HARP: Harmonic-Aware Residual Partitioning for Neural Audio Codecs

    Jul 18, 2026Qiaoyu Yang, Lixing He, Binyue Deng +1Audio Representation LearningNeural Audio Codecs

  13. Can Tokens Compete? Token Representations against Supervised CNN Backbones for BirdCLEF+ 2026

    Jul 16, 2026Anthony Miyaguchi, Murilo Gustineli, Adrian CheungAudio Representation LearningNeural Audio Codecs

  14. Streaming Neural Speech Codecs through Time-Invariant Representations

    Jul 6, 2026Kélian Estève, Salima Mhdaffar, Mickael Rouvier +2Audio Representation LearningNeural Audio Codecs

  15. Neural Audio Codec with Adjustable Token Temporal Resolution Using Sampling-Frequency-Independent Convolutional Layers

    Jul 2, 2026Tomohiko Nakamura, Wataru Nakata, Kanami Imamura +1Neural Audio CodecsAudio Tokenization

  16. Beyond Cross-Reconstruction: Probing-Based Disentanglement Evaluation for Acoustic Teleportation Codecs

    Jun 30, 2026Philipp Grundhuber, Emanuël A. P. HabetsDisentangled Representation LearningNeural Audio Codecs

  17. DTM-Codec: Dynamic Token Masking for VFR Speech Coding with Efficient Boundary Selection

    Jun 28, 2026Hoyeol Sohn, Juhan NamNeural Audio CodecsNeural Network Compression

  18. HybridCodec: Modeling Discrete and Continuous Representations for Efficient Speech Language Models

    Jun 26, 2026Artem Ploujnikov, Francesco Verdini, Samir Sadok +1Neural Audio CodecsMultimodal Token Compression

  19. Elastic Time: Dynamic Frame Rate Bottlenecks for Neural Audio Coding

    Jun 25, 2026Dimitrios Bralios, Paris Smaragdis, Minje KimNeural Audio CodecsAutoencoders

  20. ProsoCodec: Prosody-Oriented Speech Codec for Voice Conversion

    Jun 20, 2026Jeongsoo Choi, Ji-Hoon Kim, Shujie Hu +1Audio Representation LearningNeural Audio Codecs

  21. Towards Detecting Neural Audio Codec Synthesized Heart Sounds

    Jun 19, 2026Girish, Orchid Chetia Phukan, Mohd Mujtaba Akhtar +3Neural Audio CodecsAudio Deepfake Detection

  22. NAC: Neural Action Codec for Vision-Language-Action Models

    Jun 19, 2026Ahad Jawaid, Yu XiangNeural Audio CodecsResidual VQ

  23. Probing Low Frame Rate Degradation in Neural Audio Codecs

    Jun 15, 2026Alex Gichamba, Moise BusogiNeural Audio Codecs

  24. Bridging the SEA Gap: An Initial Benchmark for Neural Audio Codec-Synthesized Speech Deepfakes in South-East Asian Languages

    Jun 14, 2026Orchid Chetia Phukan, Girish, Mohd Mujtaba Akhtar +1Neural Audio CodecsAudio Deepfake Detection

  25. From Tokens to Faces: Investigating Discrete Speech Representations for 3D Facial Animation

    Jun 11, 2026Pedro Correa, Olivier Perrotin, Samir Sadok +2Audio Representation LearningNeural Audio Codecs

  26. Self-Guidance: Enhancing Neural Codecs via Decoder Manifold Alignment

    Jun 11, 2026Xiang Li, Yixuan Zhou, Jingran Xie +2Neural Audio CodecsVector-Quantized Variational Autoencoder