Audio Token Compression

Follow this topic Ask about this topic

Momentum

6 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 22

All topics

Q-SPT: Learnable Query-Based Compression for Low-Frame-Rate Speech Tokenization
Oct 1, 2026Jeeyoung Yun, Seohwan Yun, Sungwoong KimAudio Token Compression Neural Audio Codecs
Korea University, Seoul, Republic of Korea
Audio Token Attention Is Predictable Before the Language Model Runs
Sep 30, 2026Kyoungjun Park, Yunzhe Li, Lili QiuAudio Token Compression Audio Understanding
The University of Texas at Austin
OmniRoute: Mapping Temporal Semantic Evidence to Audio-Visual Token Budgets for Efficient Omnimodal Large Language Models
Sep 29, 2026Yuchen Deng, Zidang Cai, Feidiao Yang +4Efficient Multimodal Inference Audio Token Compression
Shenzhen International Graduate School, Tsinghua University, China · Pengcheng Laboratory, China
Joint Analysis of Latent Dimensionality and Frame Rate in Continuous Audio Encoders
Sep 24, 2026Kyudan Jung, Sehyun Lee, Son-ha Jo +2Audio Token Compression Audio Representation Learning
KAIST AI · Seoul National University · NAVER Cloud
LACE: Layer-Wise Compression for Dynamic Frame Rate Codecs
Sep 15, 2026Thanapat Trachu, Samuele Cornell, William Chen +1Audio Token Compression Neural Audio Codecs
Language Technologies Institute, Carnegie Mellon University, Pittsburgh, USA
ZipCodec: Ultra-Low-Frame-Rate Streaming Speech Coding
Sep 12, 2026Luca Della Libera, Cem Subakan, Mirco RavanelliAudio Token Compression Neural Audio Codecs
Concordia University · Mila-Quebec AI Institute · Universit´e Laval
Compressing Streaming Neural Audio Encoders via Latent-Space Distillation
Sep 3, 2026Prasanth Yadla, Mohammad Samragh, Dongseong Hwang +7Audio Token Compression Audio Representation Learning
Apple
Stride-k Subsampling: Train-Free Audio Token Reduction for Whisper
Aug 31, 2026Chanhee Cho, Junhyuk Choi, Bugeun KimAudio Token Compression Automatic Speech Recognition
Department of Artificial Intelligence Chung-Ang University, Seoul, Republic of Korea
Deferred Audio Pruning with Local Audio-Visual Dynamics for Omni-LLMs
Aug 9, 2026Kyeongyoon Lee, Hongyeob Kim, Youngeun Kim +1Efficient Multimodal Inference Audio Token Compression
Sungkyunkwan University · AWS AI Labs
VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inference
Aug 9, 2026Wenxu Jia, Dongjie Fu, Xize Cheng +6Audio Token Compression LLM Inference Acceleration
1Zhejiang University · 2Meituan
KVAE: Family of Tokenizers for Multimodal Generative Models
Aug 6, 2026Andrey Shutkin, Denis Parkhomenko, Ivan Kirillov +11Variational Autoencoders Audio Token Compression
Kandinsky Lab
OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Models
Aug 4, 2026Wanshun Su, Yang Shi, Feihu Liu +10Efficient Multimodal Inference Audio Token Compression
Northwestern Polytechnical University · Peking University · Alibaba Group +1
Allocation Before Ranking: Decoupled Token Compression for OmniLLMs
Aug 3, 2026Zhenghui Guo, Yilin Yang, Yuanbin Man +5Audio Token Compression Unified Multimodal Models
Department of Computer Science, University of Houston · Department of Computer Science and Engineering, The University of Texas at Arlington
OmniDelta: Skill-Driven Budget Allocation for Token Compression in OmniLLMs
Jul 28, 2026Haoyang Huang, Wenjie Huang, Tianqi Xu +14Efficient Multimodal Inference Audio Token Compression
Zhejiang University · Qwen Application, Alibaba · Carnegie Mellon University +2
OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models
Jul 25, 2026Jinsen Su, Yongdong Luo, Yuexiao Ma +3Efficient Multimodal Inference Audio Token Compression
Media Analytics and Computing Lab, Xiamen University, Xiamen, China · Institute of Artificial Intelligence, Xiamen University, Xiamen, China · School of Informatics, Xiamen University, Xiamen, China +1
TLDR: Compressing Audio Tokens for Efficient Autoregressive Text-to-Speech
Jun 8, 2026Yejin Lee, Junwon Moon, Hyoeun Kim +3Efficient Transformer Inference Audio Token Compression
Sungkyunkwan University · University of Seoul
CleanCodec: Efficient and Robust Speech Tokenization via Perceptually Guided Encoding
Jun 3, 2026Eugene Kwek, Feng Liu, Rui Zhang +1Audio Token Compression Audio Representation Learning
Pennsylvania State University · Drexel University
LoSATok: Low-dimensional Semantic-Acoustic Tokenizer for Cross-Domain Audio Understanding and Generation
May 27, 2026Zhisheng Zhang, Xiang Li, Yixuan Zhou +3Audio Token Compression Audio Representation Learning
1Shenzhen International Graduate School, Tsinghua University, China · *Work conducted while interning at ModelBest. · 2ModelBest Inc., China
Locality Matters for Training-Free Audio Token Compression in Audio-Language Models
May 24, 2026Jiale Luo, Xiaoyu Liang, Haoji HuToken Merging Audio Token Compression
1Zhejiang University
OmniRefine: Alignment-Aware Cooperative Compression for Efficient Omnimodal Large Language Models
May 12, 2026Yuchen Deng, Zidang Cai, Hai-Tao Zheng +3Audio-Visual Understanding Efficient Multimodal Inference
Tsinghua Shenzhen International Graduate School, Tsinghua University · Pengcheng Laboratory
HeadRouter: Dynamic Head-Weight Routing for Task-Adaptive Audio Token Pruning in Large Audio Language Models
Apr 26, 2026Peize He, Yaodi Luo, Xiaoqian Liu +7Audio Token Compression Self-Attention
1EPIC Lab, Shanghai Jiao Tong University · 2DAIL Tech · 3Northeastern University +2
Do Modern Video-LLMs Need to Listen? A Benchmark Audit and Scalable Remedy
Sep 22, 2025Geewook Kim, Minjoon SeoVLM Evaluation Audio-Visual Understanding
NAVER Cloud AI, South Korea · KAIST AI, South Korea

Oct 1, 2026, cs.SDJ/K move · Enter open · S save

Q-SPT: Learnable Query-Based Compression for Low-Frame-Rate Speech Tokenization

Jeeyoung Yun, Seohwan Yun, Sungwoong Kim

Korea University, Seoul, Republic of Korea

Audio Token Compression Neural Audio Codecs Audio Tokenization Speech Language Models

Open paper AskPDF arXiv

Neural speech codecs increasingly serve as tokenizers for speech language models (SLMs). Lowering the frame rate reduces the computational and memory costs of SLMs, but makes it difficult to preserve both linguistic information and acoustic detail. Existing approaches rely on rule-based compression: average pooling can discard linguistic information, whereas similarity-based merging uses a fixed threshold on adjacent-frame similarity and applies the resulting boundaries to the acoustic stream. We propose Q-SPT, a low-frame-rate dual-stream speech tokenizer with separate, context-aware, learnable query-based compressors specialized for semantic and acoustic representations. In particular, queries at a fixed rate independently attend to the semantic and acoustic streams as separate key-value sources, enabling stream-specific, context-aware aggregation through two separately learned compressors. In addition, an autoregressive text loss explicitly supervises the semantic compressor to preserve linguistic information. Experimental results show that Q-SPT achieves the best reconstruction among the evaluated codecs at the same frame rate. In downstream SLMs, it yields the best speech recognition accuracy and text-to-speech perceptual quality with competitive intelligibility.

Momentum

Related topics

Latest papers 22