Multimodal Token Compression

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, up 33% on the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 70

All topics

DIPrune: Task-Aware Token Pruning with Dual Importance for Efficient Multimodal Language Models
Oct 6, 2026Shuo Yang, Changbai Li, Linlin Yang +6Multimodal Large Language Models Token Pruning
Beihang University · Communication University of China · National University of Singapore
ResComEmb: Effective and Efficient Multimodal Embedding via Residual Homogeneity Compression
Sep 29, 2026Zijing Cai, Yuzhe Wang, Jingxian Zhu +2Multimodal IR Multimodal Embedding
University of Science and Technology of China · Hefei University of Technology · National University of Singapore
OmniRoute: Mapping Temporal Semantic Evidence to Audio-Visual Token Budgets for Efficient Omnimodal Large Language Models
Sep 29, 2026Yuchen Deng, Zidang Cai, Feidiao Yang +4Efficient Multimodal Inference Audio Token Compression
Shenzhen International Graduate School, Tsinghua University, China · Pengcheng Laboratory, China
SLICEChat: Progressive In-Encoder Token Pruning for Whole-Slide Pathology Language Models
Sep 21, 2026Ali Kerem Bozkurt, Baris Cem Bakay, Ibrahim Kulac +3Efficient Multimodal Inference Multimodal Large Language Models
Kocı University, Department of Computer Engineering · Kocı University, Department of Pathology · Hacettepe University, Department of Artificial Intelligence and Data Engineering
EMMI: Edge Multi-Modal Intelligence for Communication-Efficient MLLM Inference via Fused Representation Compression
Sep 11, 2026Motahare Mounesan, Irfan KhanEfficient Multimodal Inference Edge Computing
Texas A&M University
SinkPruner: Sink-Free Visual Token Pruning for Multimodal Large Language Models
Sep 1, 2026Shiyu Li, Zi-Yuan Hu, Shijia Huang +3Efficient VLM Inference Multimodal Large Language Models
1The Chinese University of Hong Kong · 2Weitu AI · 3Peking University
Accelerating Unified Multimodal Models with Core-Expansion Routing and Unified Computation Scheduling
Aug 29, 2026Wengyi Zhan, Chenqian Yan, Songwei Liu +2Efficient Multimodal Inference Unified Multimodal Models
1Xiamen University · 2ByteDance
Post-Hoc Sparse Coding of Latent Communication Between Vision-Language Model Agents
Aug 10, 2026Di Wu, Xiaohui ZhuSparse Autoencoders Multimodal Token Compression
Xi’an Jiaotong-Liverpool University
Reading is not Reasoning: Bridging the Agentic Policy Gap in Vision-Text Compression
Aug 9, 2026Cheng Fan, Junyi Zhou, Tingzhang Luo +5Cross-Modal Knowledge Distillation Multimodal Token Compression
City University of Hong Kong · Huawei Technologies Ltd.
Deferred Audio Pruning with Local Audio-Visual Dynamics for Omni-LLMs
Aug 9, 2026Kyeongyoon Lee, Hongyeob Kim, Youngeun Kim +1Efficient Multimodal Inference Audio Token Compression
Sungkyunkwan University · AWS AI Labs
VLZip: Unified Visual and Textual Compression for Interleaved Long-Context Modeling
Aug 9, 2026Yuqi Zhang, Cheng Chen, Yuyu Guo +6Vision-Language Models Efficient VLM Inference
Shanghai Key Lab of Intell. Info. Processing, School of CS, Fudan University, China · Shanghai Innovation Institute, China · Ant Group, China
KVAE: Family of Tokenizers for Multimodal Generative Models
Aug 6, 2026Andrey Shutkin, Denis Parkhomenko, Ivan Kirillov +11Variational Autoencoders Audio Token Compression
Kandinsky Lab
CARVE: Cross-Slice Anisotropic Reallocation of Visual Evidence for Efficient 3D Medical Volume Understanding
Aug 5, 2026Zhenyu Yi, Qiang Hu, Zhenhao Li +33D Medical Imaging Medical VQA
School of Biomedical Engineering, Shanghai Jiao Tong University · 2Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology
EvtGraph: Event-Adaptive Compression for Sparse Temporal Graph Learning in Multimodal Time Series
Aug 5, 2026Ziqian Wang, Tingxiong Xiao, Yuxiao Cheng +1Irregular Time-Series Modeling Efficient Multimodal Inference
Department of Automation Tsinghua University Beijing, China
OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Models
Aug 4, 2026Wanshun Su, Yang Shi, Feihu Liu +10Efficient Multimodal Inference Audio Token Compression
Northwestern Polytechnical University · Peking University · Alibaba Group +1
Same Semantics, Different Paths: Self-Improving Alignment for Vision-Text Compression
Aug 3, 2026Tianyu Liang, Xiangxi Zheng, Yilin Wang +1VLM Adaptation Multimodal Token Compression
Southeast University Nanjing, China · Nanjing University Nanjing, China · Zhejiang University Hangzhou, China +1
Allocation Before Ranking: Decoupled Token Compression for OmniLLMs
Aug 3, 2026Zhenghui Guo, Yilin Yang, Yuanbin Man +5Audio Token Compression Unified Multimodal Models
Department of Computer Science, University of Houston · Department of Computer Science and Engineering, The University of Texas at Arlington
3DZip: Spatial-Aware Feature Diversity-Guided Token Compression for 3D Question Answering
Aug 2, 2026Changwoo Baek, Kyeongbo KongMultimodal Token Compression 3D VQA
Department of Electrical and Electronics Engineering, Pusan National University, Busan, Republic of Korea
RAGOCR: Optical Compression of Retrieval-Augmented Text via Visual Representation
Aug 1, 2026Jiayang Yu, Jialun Zhong, Lei ZouMultimodal RAG Retrieval-Augmented Generation
Wangxuan Institute of Computer Technology, Peking University Beijing, China
CodeShrink: Adaptive Visual Compression for Efficient Multimodal Code Understanding
Jul 31, 2026Wenxin Tang, Jingyu Xiao, Zhenyu Liu +6Efficient Multimodal Inference Multimodal Large Language Models
Tsinghua University · The Chinese University of Hong Kong · Northwestern Polytechnical University +4
MedARC: Training-Free Adaptive Redundancy Compression of Visual Tokens for 3D Medical Vision-Language Models
Jul 29, 2026Yitao Zhu, Mengjun Liu, Yingji Fu +2Efficient VLM Inference Medical VLMs
Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Hong Kong · School of Integrated Circuits and Electronics, Beijing Institute of Technology, China · 3Mental Health Research Center, The Hong Kong Polytechnic University, Hong Kong +1
OmniDelta: Skill-Driven Budget Allocation for Token Compression in OmniLLMs
Jul 28, 2026Haoyang Huang, Wenjie Huang, Tianqi Xu +14Efficient Multimodal Inference Audio Token Compression
Zhejiang University · Qwen Application, Alibaba · Carnegie Mellon University +2
Omni-Prune: Query-Aware Unified Token Pruning for Efficient Omnimodal Large Language Models
Jul 26, 2026Yiming Zhong, Chang Nie, Caifeng ShanEfficient Multimodal Inference Multimodal Large Language Models
1Nanjing University
OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models
Jul 25, 2026Jinsen Su, Yongdong Luo, Yuexiao Ma +3Efficient Multimodal Inference Audio Token Compression
Media Analytics and Computing Lab, Xiamen University, Xiamen, China · Institute of Artificial Intelligence, Xiamen University, Xiamen, China · School of Informatics, Xiamen University, Xiamen, China +1
Autoregressive EHR Foundation Models with Multimodal Inputs
Jul 24, 2026Yuxuan Liu, Joshua Placidi, Jinpei Han +3Clinical Outcome Prediction Multimodal Token Compression
Dept. of Computing, Imperial College London, London, United Kingdom · UKRI Centres in AI for Health, United Kingdom · School of Public Health, Imperial College London, London, United Kingdom +2
Out of Sight, Still in Mind: Token Compression for Omni-LLMs
Jul 23, 2026Suho Yoo, Youngjoon Jang, Hyebin Cho +1Efficient Multimodal Inference Multimodal Large Language Models
KAIST · VGG, University of Oxford
Beyond Independent Optimization: Compression, MoE Routing, and Quantization Interactions in Multimodal Edge Intelligence
Jul 23, 2026Jay Gor, Karm Dave, Akshita Abrol +3Efficient Multimodal Inference VLM Quantization
Nirma University · Singapore Institute of Technology, Singapore · Department of CE-AI and Big Data, Marwadi University
VisCo: Leveraging Large Language Models as Intrinsic Encoders for Visual Token Compression
Jul 14, 2026Yupeng Zheng, Kai Zou, Bin Liu +1Efficient VLM Inference Large Vision-Language Models
Anhui Province Key Laboratory of Digital Security, University of Science and Technology of China Hefei, China
OmniFocus: Query-Guided Modality-Balanced Token Compression for Omni-Modal Large Language Models
Jul 3, 2026Shijie Cao, Qingyu Zhang, Boxi Yu +6Efficient Multimodal Inference Multimodal Large Language Models
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences · University of Limerick +1
Combating Textual Noise and Redundancy: Entropy-Aware Dense Visual Token Pruning
Jul 2, 2026Xuehui Wang, Xuankun Yang, Wei ShenEfficient VLM Inference Multimodal Token Compression
Shanghai Jiao Tong University, Shanghai, China
ERA: Entropy-Guided Visual Token Pruning with Rectified Attention for Efficient MLLMs
Jun 30, 2026Yuhao Wang, Mu Qiao, Haiwen Diao +5Efficient VLM Inference Multimodal Large Language Models
School of Future Technology, Dalian University of Technology, Dalian 116024, China · S-Lab, Nanyang Technological University, Singapore 639798 · OPPO Research Institute, Shenzhen 518000, China +1
AVTok: 1D Unified Tokenization for Holistic Audio-Video Generation
Jun 29, 2026Kien T. Pham, I Chieh Chen, Qifeng Chen +1Audio-Video Generation Audio-Visual Alignment
The Hong Kong University of Science and Technology
HybridCodec: Modeling Discrete and Continuous Representations for Efficient Speech Language Models
Jun 26, 2026Artem Ploujnikov, Francesco Verdini, Samir Sadok +1Neural Audio Codecs Multimodal Token Compression
Mila, Quebec AI Institute, Canada · Concordia University, Canada · Sapienza University of Rome, Italy +1
TOPS: First-Principles Visual Token Pruning via Constructing Token Optimal Preservation Sets for Efficient MLLM Inference
Jun 25, 2026Tinghao Wang, Yichen Guo, Rui Huang +11Efficient VLM Inference Multimodal Large Language Models
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Nanyang Technological University · University of Electronic Science and Technology of China +1
AVOC: Enhancing Hour-Level Audio-Video Understanding in Omni-Modal LLMs via Retrieval-Inspired Token Compression
Jun 23, 2026Yijing Chen, Wenhui Tan, Xiaoyi Yu +7Audio-Visual Understanding Multimodal Large Language Models
Gaoling School of Artificial Intelligence, Renmin University of China · Huawei Inc.
Spectral Evolution-Guided Token Pruning in Multimodal Large Language Models
Jun 23, 2026Bin Chen, Yuxiang Cai, Yadan Luo +3Efficient Multimodal Inference Multimodal Large Language Models
School of Software Technology, Zhejiang University, Ningbo, China · Zhejiang Key Laboratory of Digital-Intelligence Service Technology, China · The University of Queensland, St Lucia, QLD, Australia +2
Stepwise Token Selection for Efficient Multimodal Large Language Models
Jun 14, 2026Landi He, Shawn Young, Lijian XuEfficient Multimodal Inference Multimodal Large Language Models
Shenzhen University of Advanced Technology
Beyond Uniform Tokens: Adaptive Compression for Time Series Language Models
Jun 11, 2026Jialin Gan, Xin Qiu, Guangzhe Chen +1LLM Inference Acceleration Multimodal Token Compression
Zhejiang University · Harbin Institute of Technology · Shandong University
Task-Aware Structured Memory for Dynamic Multi-modal In-Context Learning
Jun 10, 2026Zhirui Chen, Ziwei Chen, Ling ShaoMemory-Augmented Language Models Multimodal ICL
UCAS-Terminus AI Lab, University of Chinese Academy of Sciences, China · ByteDance Inc.
One Token per Multimodal Evidence: Latent Memory for Resource-Constrained QA
Jun 9, 2026Zhi Zheng, Ziqiao Meng, Hao Luan +2Multimodal RAG Multimodal Memory
School of Computing, National University of Singapore
Audio-Visual Exchange-Aware Token Pruning for Efficient Audio-Visual Captioning
Jun 9, 2026Zihan Meng, Dexiang Hong, Weidong Chen +3Audio-Visual Understanding Multimodal Token Compression
University of Science and Technology of China
Planning-aligned Token Compression for Long-Context Autonomous Driving
Jun 5, 2026Zhixuan Liang, Yuxiao Chen, Yurong You +12End-to-End Autonomous Driving Vector-Quantized Variational Autoencoder
NVIDIA Research · School of Computing and Data Science, The University of Hong Kong
Differentiable Efficient Operator Search
Jun 3, 2026Xiaohuan Pei, Jiyuan Zhang, Yuanfan Guo +4Efficient Multimodal Inference Multimodal Token Compression
1The University of Sydney · 2ByteDance · 3Shanghai Jiao Tong University +1
InfoMerge: Information-aware Token Compression for Efficient Video Large Language Models
Jun 1, 2026Xinxin Liu, Shiwei Gan, Xiao Liu +3Efficient VLM Inference Video-Language Models
State Key Laboratory of Novel Software Technology, Nanjing University Nanjing 210023, China
On the Limits of Token Reduction for Efficient Unified Vision Language Training
May 31, 2026Siyi Chen, Weiming Zhuang, Jingtao Li +1Vision-Language Models Unified Multimodal Models
University of Michigan · Sony AI
Wavelet as Tokenizer: Preliminary Results on a Shared Wavelet Token Schema for Natural Signals
May 30, 2026Shenghao DingWavelet Methods Multimodal Token Compression
Yet Another AI
ETC: Extreme Token Compression via Task-aware Visual Information Distillation in VLMs
May 30, 2026Yiling Gao, Hongchen Wei, Zhenzhong ChenEfficient VLM Inference Large Vision-Language Models
School of Remote Sensing and Information Engineering, Wuhan University
OmniMem: Perturbation-aware Memory Compression for Streaming Audio-Visual LLMs
May 26, 2026Guangzhi Sun, Yixuan Li, Yudong Yang +1Streaming Video Understanding Multimodal Large Language Models
ByteDance · Department of Engineering, University of Cambridge · Tsinghua University
O-MARC: Omni Memory-Augmented Compression Distillation for Efficient Video Understanding
May 26, 2026Peiran Wu, Yunze Liu, Chi-Hao Wu +2Audio-Visual Understanding Efficient Multimodal Inference
University of Bristol · Memories.ai Research · University of Central Florida
Locality Matters for Training-Free Audio Token Compression in Audio-Language Models
May 24, 2026Jiale Luo, Xiaoyu Liang, Haoji HuToken Merging Audio Token Compression
1Zhejiang University
ST-SimDiff: Balancing Spatiotemporal Similarity and Difference for Efficient Video Understanding with MLLMs
May 21, 2026Bingjun Luo, Tony Wang, Chaoqi Chen +1Multimodal Token Compression Video Token Compression
Tsinghua University · Shenzhen University · Xidian University
Enhancing Visual Token Representations for Video Large Language Models via Training-Free Spatial-Temporal Pooling and Gridding
May 21, 2026Bingjun Luo, Tony Wang, Hanqi Chen +1Video-Language Models Multimodal Token Compression
1Tsinghua University · 2Shenzhen University · 3Xidian University
Stage-adaptive Token Selection for Efficient Omni-modal LLMs
May 19, 2026Zijie Xin, Jie Yang, Ruixiang Zhao +4Efficient Multimodal Inference Multimodal Large Language Models
Renmin University of China · WeChat Vision, Tencent Inc.
OmniSelect: Dynamic Modality-Aware Token Compression for Efficient Omni-modal Large Language Models
May 18, 2026Morunliu Yang, Ruotao Xu, Le Li +6Efficient Multimodal Inference Multimodal Large Language Models
1Soochow University · Peking University
OmniDrop: Layer-wise Token Pruning for Omni-modal LLMs via Query-Guidance
May 14, 2026Yeo Jeong Park, Hyemi Jang, Minseo Choi +3Efficient Multimodal Inference Multimodal Large Language Models
Samsung Research
OmniRefine: Alignment-Aware Cooperative Compression for Efficient Omnimodal Large Language Models
May 12, 2026Yuchen Deng, Zidang Cai, Hai-Tao Zheng +3Audio-Visual Understanding Efficient Multimodal Inference
Tsinghua Shenzhen International Graduate School, Tsinghua University · Pengcheng Laboratory
Keep What Audio Cannot Say: Context-Preserving Token Pruning for Omni-LLMs
May 12, 2026Chaeyoung Jung, Kyeongha Rho, Joon Son ChungAudio-Visual Understanding Efficient Multimodal Inference
Korea Advanced Institute of Science and Technology (KAIST)
How Many Visual Tokens Do Multimodal Language Models Need? Scaling Visual Token Pruning with F^3A
May 9, 2026YiJie Huang, Yiqun Zhang, Zhuoyue Jia +7Efficient VLM Inference Large Vision-Language Models
School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China · Shanghai Artificial Intelligence Laboratory · School of Computer and Communication Engineering, Northeastern University, Qinhuangdao 066004, China
HY-Himmel Technical Report: Hierarchical Interleaved Multi-stream Motion Encoding for Long Video Understanding
May 4, 2026Haopeng Jin, Hongzhu Yi, Wenlong Zhao +6Video Understanding Video-Language Models
Tencent · University of Chinese Academy of Sciences · Beijing Forestry University
HeadRouter: Dynamic Head-Weight Routing for Task-Adaptive Audio Token Pruning in Large Audio Language Models
Apr 26, 2026Peize He, Yaodi Luo, Xiaoqian Liu +7Audio Token Compression Self-Attention
1EPIC Lab, Shanghai Jiao Tong University · 2DAIL Tech · 3Northeastern University +2

Oct 6, 2026, cs.CVJ/K move · Enter open · S save

DIPrune: Task-Aware Token Pruning with Dual Importance for Efficient Multimodal Language Models

Shuo Yang, Changbai Li, Linlin Yang, Huobin Tan+5

Beihang University · Communication University of China · National University of Singapore

Multimodal Large Language Models Token Pruning Multimodal Token Compression Visual Token Pruning

Open paper AskPDF arXiv

Recent training-free pruning approaches for Multimodal Large Language Models (MLLMs) effectively cut computational overhead by exploiting visual redundancy or text-vision attention. However, they frequently suffer from semantic degradation due to their task-agnostic design or unreliable attention estimates. Based on our empirical analysis, we have found that this issue arises because salient tokens in shallow layers persistently suppress emerging semantic ones through numerical inertia, leading to premature discarding of signals crucial for deep reasoning. To address the aforementioned issue, from the task-oriented aspects, we first reformulate training-free pruning as a minimization of the distortion in the final task loss and derive a tractable, token-wise upper bound to serve as a surrogate objective. Specifically, this formulation inherently reveals a previously neglected inter-layer term that accounts for gradients across layers. Accordingly, for the implementation, we propose DIPrune, a rank-based framework that employs a dual importance scoring mechanism to jointly optimize intra-layer static feature saliency and inter-layer dynamic semantic evolution. Extensive experiments on LLaVA and Qwen-VL demonstrate that DIPrune consistently achieves state-of-the-art results.

Momentum

Related topics

Latest papers 70