Overview

Neural audio codecs compress sound by mapping waveforms into compact latent representations, often using residual vector quantization, then reconstructing audio with a learned decoder. Current approaches balance perceptual fidelity with bitrate, intelligibility, speaker identity, and attributes such as emotion or pitch, using perceptual losses, self-supervised features, and separate semantic and acoustic streams. Researchers also design codec tokens for downstream generation with language models, while developing streaming codecs, variable frame rates, and hybrid discrete-continuous representations. A central challenge is that lower rates and more predictable tokens can lose acoustic detail, and quality metrics do not always reveal whether information needed for speech or other tasks survives compression.

Latest papers

CardsList

91 papers9 papers in the last four weeks, down 40% on the four weeks before. 0.1% of all new papers.

  1. ZONOS3-Mini Technical Report

    Nov 12, 2026Gabriel Clark, Sofian Mejjoute, George Close +1TTS SynthesisZero-Shot TTS

  2. AudioCodex: Streaming Autoencoders for Live Audio

    Nov 9, 2026Marco Pasini, Javier Nistal, Cyran Aouameur +2Neural Audio CodecsAutoencoders

  3. Lightweight Neural Prediction for MDCT-Based Perceptual Audio Coding

    Nov 6, 2026Niccolo Abate, Gabriel Soule, Marina BosiNeural Audio CodecsPredictive Coding

  4. A Tutorial on Generative Models for Music Audio

    Oct 28, 2026Mateusz Modrzejewski, Joanna Kurkowska, Dorien HerremansMusic GenerationAudio Diffusion Models

  5. Probing the phon* representations of the Mimi codebooks

    Oct 26, 2026Artem Saloev, Nicolas Ballier, Erin PacquetetNeural Audio CodecsDiscrete Speech Representation Learning

  6. RubatoCodec: Budgeted Adaptive Speech Tokenization with Streaming Decoding

    Oct 21, 2026Chihiro Arata, Shohei Taniguchi, Gouki Minegishi +1Neural Audio CodecsAudio Token Compression

  7. Frame-Wise Streaming for Efficient Learned Speech Compression

    Oct 15, 2026Yuhan Liu, Jun Xu, Fengxi Zhang +2Neural Audio CodecsSpeech Processing

  8. Probing Neural Audio Codecs with Speaker Traits and Speech Characteristics

    Oct 14, 2026Jihwan Lee, Tiantian Feng, Minu Kim +6Neural Audio CodecsSpeech Processing

  9. FOAdapter: A Plug-and-Play Spatial Tokenizer for Low-Bitrate FOA Speech Coding

    Oct 12, 2026Kirak Kim, Yoonjeong Park, Juhan Nam +2Spatial AudioNeural Audio Codecs

  10. BridgeCodec: Mamba Enhanced Neural Audio Codec with Schrödinger Bridge at Low Bitrate

    Oct 11, 2026Zijian Lin, Jing Yang, Jinghao Luo +3Neural Audio Codecs

  11. HE-NAC: High-Efficiency Neural Audio Coding via Hierarchical and Residual Multi-Band Quantization

    Oct 11, 2026Darius Pétermann, Wootaek Lim, Seungkwon Beack +1Neural Audio CodecsResidual VQ

  12. SemiCodec: Bridging Speech Tokenization and High-fidelity Reconstruction with Semi-discrete Representations

    Oct 10, 2026Kecan Mao, Nan Xu, Zhaoyang Meng +4Neural Audio CodecsAudio Representation Learning

  13. Feature Encoding in VAE-based Audio Decoders: Effects of Input, Depth and Distribution

    Oct 6, 2026Louis McCallum, Mick GriersonVariational AutoencodersAudio Representation Learning

  14. Exposing and Mitigating Neural Codec Vulnerabilities in Audio Deepfake Detection

    Oct 5, 2026Abdullah, Awais Khan, Khalid Mahmood MalikNeural Audio CodecsAudio Deepfake Detection

  15. AudioGAR: Bridging Reconstruction and Generation in Latent Audio Generative Models

    Oct 5, 2026Xianghong Fang, Geeyang Tay, Wentao Ma +2Neural Audio CodecsAudio Diffusion Models

  16. GS-Codec: A Gaussian-Splatting Bottleneck for Neural Audio Coding

    Oct 3, 2026Ron Aluf, Alon Canfi, Eliya NachmaniNeural Audio Codecs

  17. Q-SPT: Learnable Query-Based Compression for Low-Frame-Rate Speech Tokenization

    Oct 1, 2026Jeeyoung Yun, Seohwan Yun, Sungwoong KimAudio Token CompressionNeural Audio Codecs

  18. Neural Audio Codec for Robust Audio Deepfake Detection

    Sep 30, 2026Jungwoo Kim, Joonyong Park, Junyoung Koh +1Neural Audio CodecsAudio Deepfake Detection

  19. Multi-Rate Bandwidth Extension by Token Completion in Neural Audio Codecs

    Sep 29, 2026Benoît Ginies, Olivier Fercoq, Gaël RichardAudio Representation LearningNeural Audio Codecs

  20. Interpreting and Evaluating Dynamic-Rate Speech Codec Boundaries

    Sep 29, 2026Han Wang, Jiaqi Li, Yingda Shen +2Neural Audio CodecsAudio Tokenization

  21. Narrowband Voice Communication Using Streaming Neural Compression

    Sep 21, 2026Dahong Luo, Anannya Trehan, Aritrik Ghosh +1Efficient Neural Network InferenceNeural Audio Codecs

  22. LACE: Layer-Wise Compression for Dynamic Frame Rate Codecs

    Sep 15, 2026Thanapat Trachu, Samuele Cornell, William Chen +1Audio Token CompressionNeural Audio Codecs

  23. Tracing the Origins: Legacy Codec Identification in Neural Audio Transcoding

    Sep 14, 2026Wonje Heo, Shinee Youn, Yooshin Kim +2Neural Audio CodecsDigital Forensics

  24. TokenMapper: A Step Toward Interoperable Speech Token Translation

    Sep 11, 2026Tal Kozakov, Tal Rosenwein, Eliya NachmaniNeural Audio CodecsAudio Tokenization

  25. How Robust Are Neural Audio Codecs for African Speech? A Multi-Task Benchmark and the Limits of Perceptual Quality

    Sep 10, 2026Chibuzor Okocha, Christan Earl GrantNeural Audio CodecsSpeech Processing

  26. ZipCodec: Ultra-Low-Frame-Rate Streaming Speech Coding

    Sep 10, 2026Luca Della Libera, Cem Subakan, Mirco RavanelliAudio Token CompressionNeural Audio Codecs

  27. PACodec: A Low-bitrate Neural Speech Codec with Parallel Additive Vector Quantization

    Sep 3, 2026Fei Liu, Yang Ai, Xiao-Hang Jiang +1Neural Audio CodecsSpeech Processing