Multimodal Token Compression

Overview

Multimodal token compression reduces the number or size of image, video, audio, and other input representations so models can process longer inputs with less computation and memory. Common approaches prune or merge redundant tokens, pool them into learned bottlenecks, allocate budgets using queries or modality-specific signals, and compress key-value caches. These methods are evaluated on tasks such as visual question answering, video understanding, audio comprehension, and multimodal retrieval, often by comparing accuracy with latency, FLOPs, or memory use at different compression levels. The central challenge is deciding what information can safely be removed: useful evidence varies by task, modality, and model layer, and aggressive compression can discard fine details or weaken joint understanding and generation.

Latest papers

CardsList

101 papers5 papers in the last four weeks, down 50% on the four weeks before. 0.0% of all new papers.

  1. Transfer Once, Ask Locally: Compact Audio-Visual Memory for On-Device Question Answering

    Nov 17, 2026Qitong Wang, Jihui Jin, Divya Kothandaraman +2Audio-Visual QAMultimodal QA

  2. VoluCompose: Constructing Compact Volumetric Visual Representations for Medical Image Understanding

    Nov 5, 2026Zanting Ye, Yunrui Lin, Shengyuan Liu +4Visual Token CompressionMultimodal Token Compression

  3. R2^2-Omni: Reducing Structural and Precision Redundancy in Omni-LLMs for Long-Form Generation

    Nov 2, 2026Bolong Feng, Pengfei Zhou, Zhenglin Wan +2Multimodal Large Language ModelsMultimodal Token Compression

  4. Calibrating Attention-Based Token Pruning for Efficient Multi-Image Synthesis in UMMs

    Oct 29, 2026Zimeng Wu, Qijian Tian, Penghui Du +3Efficient Multimodal InferenceMultimodal Token Compression

  5. Rethinking Sequence Compression with Test-time Regression

    Oct 22, 2026Sihan Yang, Jie Huang, Hanxuan Li +4LLM Context CompressionMultimodal Token Compression

  6. PACE: Pareto-Adaptive Compression for Efficient Native MLLMs

    Oct 15, 2026Lei Tian, Chunzheng Zhu, Kai Wu +2Multimodal Token CompressionVisual Token Compression

  7. Bottlenecked semantic compression by representation reconstruction

    Oct 12, 2026Xiaoya Tang, Shilong Liu, Daniel Smolyak +3Visual Token CompressionEfficient ViTs

  8. LapMerge: Differentiable Exact-Budget Token Merging in Linear Time

    Oct 9, 2026Magdalena Trędowicz, Aleksandra Nowak, Tomasz Steifer +3Token MergingVisual Token Merging

  9. Instruction-Conditioned Electromagnetic Spectrum Understanding via Budget-Adaptive Signal Tokenization

    Oct 8, 2026Lei Zhai, Zhihao Chang, Shuyuan Yang +1Multimodal Token CompressionMultimodal Instruction Tuning

  10. DIPrune: Task-Aware Token Pruning with Dual Importance for Efficient Multimodal Language Models

    Oct 6, 2026Shuo Yang, Changbai Li, Linlin Yang +6Multimodal Large Language ModelsToken Pruning

  11. TRIM-ReID: Duplication-Aware Token Reduction and Modality-Aligned Interaction for Multi-Modal Object Re-Identification

    Oct 3, 2026Wanke Xia, Ruiding Zhu, Xingguo Xu +6Multimodal FusionMultimodal Token Compression

  12. ResComEmb: Effective and Efficient Multimodal Embedding via Residual Homogeneity Compression

    Sep 29, 2026Zijing Cai, Yuzhe Wang, Jingxian Zhu +2Multimodal IRMultimodal Embedding

  13. OmniRoute: Mapping Temporal Semantic Evidence to Audio-Visual Token Budgets for Efficient Omnimodal Large Language Models

    Sep 29, 2026Yuchen Deng, Zidang Cai, Feidiao Yang +4Efficient Multimodal InferenceAudio Token Compression

  14. From Token Importance to Conditional Removability: Rethinking Visual Token Pruning in Multimodal Large Language Models

    Sep 22, 2026Shengli He, Yongchao Liang, Roumeng He +5Visual Token PruningMultimodal Token Compression

  15. SLICEChat: Progressive In-Encoder Token Pruning for Whole-Slide Pathology Language Models

    Sep 21, 2026Ali Kerem Bozkurt, Baris Cem Bakay, Ibrahim Kulac +3Efficient Multimodal InferenceMultimodal Large Language Models

  16. Layer-Wise Gate-Controlled Prompt Truncation in a Multimodal Chest X-Ray Classifier

    Sep 6, 2026Jingtao Lei, Hongji Li, Dexiang ShuPrompt TuningMultimodal Classification

  17. SinkPruner: Sink-Free Visual Token Pruning for Multimodal Large Language Models

    Sep 1, 2026Shiyu Li, Zi-Yuan Hu, Shijia Huang +3Efficient VLM InferenceMultimodal Large Language Models

  18. Visual Token Coding for Video Multimodal Large Language Models

    Aug 28, 2026Chenxin Fang, Tao Chen, JunChao You +3Multimodal Token CompressionVideo Token Compression

  19. VIG: Visual Information Gain as a Reward Signal for Multimodal Chain-of-Thought Compression

    Aug 22, 2026Wen Luo, Xiaohan Yi, Xiaotao Huang +1Multimodal CoT ReasoningMultimodal Token Compression

  20. SEER: Long-Context Reasoning via Selective Visual-Text Compression

    Aug 16, 2026Jiawei Xu, Zhilin Zhai, Jinrui Fang +6Long-Context Language Model InferenceVLM Reasoning

  21. Reading is not Reasoning: Bridging the Agentic Policy Gap in Vision-Text Compression

    Aug 9, 2026Cheng Fan, Junyi Zhou, Tingzhang Luo +5Cross-Modal Knowledge DistillationMultimodal Token Compression

  22. Deferred Audio Pruning with Local Audio-Visual Dynamics for Omni-LLMs

    Aug 9, 2026Kyeongyoon Lee, Hongyeob Kim, Youngeun Kim +1Efficient Multimodal InferenceAudio Token Compression

  23. VLZip: Unified Visual and Textual Compression for Interleaved Long-Context Modeling

    Aug 9, 2026Yuqi Zhang, Cheng Chen, Yuyu Guo +6Vision-Language ModelsEfficient VLM Inference

  24. KVAE: Family of Tokenizers for Multimodal Generative Models

    Aug 6, 2026Andrey Shutkin, Denis Parkhomenko, Ivan Kirillov +11Variational AutoencodersAudio Token Compression