Visual Token Compression

Follow this topic Ask about this topic

Vision encoders represent images and video as patch or frame tokens, and long token sequences can make multimodal models costly to run, especially during prefill and memory-intensive video reasoning. Methods include training-free pruning and merging, learned resamplers, quantization and codecs, and adaptive token budgets guided by text queries, spatial structure, or temporal redundancy. They are evaluated on tasks such as visual question answering, video understanding, OCR, segmentation, retrieval, generation, and robot control. The central tension is that fewer tokens do not always mean faster end-to-end inference, and aggressive compression can discard details needed for a particular question or task.

Latest papers

271 papers46 papers in the last four weeks, up 229% on the four weeks before. 0.3% of all new papers.

ASAP: Attention-Shift-Aware Pruning for Efficient LVLM Inference
Mar 15, 2026Surendra Pathak, Bo HanEfficient VLM Inference Visual Token Pruning
George Mason University, Fairfax, VA, USA
DASH: Dynamic Audio-Driven Semantic Chunking for Efficient Omnimodal Token Compression
Mar 15, 2026Bingzhou Li, Tao HuangAudio-Visual Understanding Efficient Multimodal Inference
Shanghai Jiao Tong University, Shanghai, China · Tongji University, Shanghai, China
EVATok: Adaptive Length Video Tokenization for Efficient Visual Autoregressive Generation
Mar 12, 2026Tianwei Xiong, Jun Hao Liew, Zilong Huang +3Video Token Compression Visual Token Compression
The University of Hong Kong · ByteDance Seed
ForensicZip: More Tokens are Better but Not Necessary in Forensic Vision-Language Models
Mar 12, 2026Yingxin Lai, Zitong Yu, Jun Wang +3Visual Token Pruning Visual Token Compression
Great Bay University · Shenzhen University · Harbin Institute of Technology +1
UniCompress: Token Compression for Unified Vision-Language Understanding and Generation
Mar 11, 2026Ziyao Wang, Chen Chen, Jingtao Li +4Multimodal Token Compression Visual Token Compression
Sony AI · University of Maryland, College Park
Frequency-Modulated Visual Restoration for Matryoshka Large Multimodal Models
Mar 11, 2026Qingtao Pan, Zhihao Dou, Shuo LiEfficient VLM Inference Multimodal Large Language Models
Case Western Reserve University
Language-Guided Token Compression with Reinforcement Learning in Large Vision-Language Models
Mar 11, 2026Sihan Cao, Jianwei Zhang, Pengcheng Zheng +7Efficient VLM Inference Visual Token Pruning
School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China · School of Robotics and Advanced Manufacture, Harbin Institute of Technology, Shenzhen, China · College of Information and Control Engineering, Xi’an University of Architecture and Technology, Xi’an, China
DepthCache: Depth-Guided Training-Free Visual Token Merging for Vision-Language-Action Model Inference
Mar 11, 2026Yuquan Li, Lianjie Ma, Han Ding +1Efficient VLA Model Inference Visual Token Compression
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China. · School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, China.
Prune Redundancy, Preserve Essence: Vision Token Compression in VLMs via Synergistic Importance-Diversity
Mar 10, 2026Zhengyao Fang, Pengyuan Lyu, Chengquan Zhang +3Visual Token Pruning Visual Token Compression
Harbin Institute of Technology, Shenzhen · Independent Researcher · Peng Cheng Laboratory
Granulon: Awakening Pixel-Level Visual Encoders with Adaptive Multi-Granularity Semantics for MLLM
Mar 9, 2026Junyuan Mao, Qiankun Li, Linghao Meng +5Large Vision-Language Models Fine-Grained Visual Perception
National University of Singapore · Nanyang Technological University
StructSAM: Structure- and Spectrum-Preserving Token Merging for Segment Anything Models
Mar 7, 2026Duy M. H. Nguyen, Tuan A. Tran, Duong Nguyen +17Token Merging Image Segmentation
University of Stuttgart · DFKI · Max Planck Research School +10
DC-DiT: Adaptive Compute and Elastic Inference for Visual Generation via Dynamic Chunking
Mar 6, 2026Akash Haridas, Utkarsh Saxena, Parsa Ashrafi Fashi +3Adaptive Tokenization Diffusion Transformer
Advanced Micro Devices Inc. (AMD)
Keeping the Evidence Chain: Semantic Evidence Allocation for Training-Free Token Pruning in Video Temporal Grounding
Mar 5, 2026Jiaqi Li, Shuntian Zheng, Yixian Shen +4Temporal Video Grounding Vision-Language Grounding
University of Warwick, Coventry CV4 7AL, United Kingdom · University of Amsterdam, Amsterdam 1012 WX, Netherlands · Amazon AGI, The United States of America
TC-SSA: Token Compression via Semantic Slot Aggregation for Gigapixel Pathology Reasoning
Mar 1, 2026Zhuo Chen, Shawn Young, Lijian XuVisual Token Compression Whole-Slide Image Analysis
Shenzhen University of Advanced Technology Shenzhen, Guangdong, China · University of Nottingham NingBo China NingBo, ZheJiang, China FoSE, Department of Mathematics with Applied Mathematics
Random Wins All: Rethinking Grouping Strategies for Vision Tokens
Feb 28, 2026Qihang Fan, Yuang Ai, Huaibo Huang +1Vision Transformer Efficient ViTs
MAIS & NLPR, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
TokenCom: Vision-Language Model for Multimodal and Multitask Token Communications
Feb 28, 2026Feibo Jiang, Siwei Tu, Li Dong +5Joint Source-Channel Coding Vision-Language Models
Hunan Provincial Key Laboratory of Intelligent Computing and Language Information Processing, Hunan Normal University, Changsha, China. · School of Information Science and Engineering, Hunan Normal University, Changsha, China. · Changsha Social Laboratory of Artificial Intelligence, Hunan University of Technology and Business, Changsha, China. +5
Stateful Token Reduction for Long-Video Hybrid VLMs
Feb 27, 2026Jindong Jiang, Amala Sanjay Deshmukh, Kateryna Chumachenko +7Hybrid Mamba-Transformer Architectures Efficient VLM Inference
NVIDIA
Spatio-Temporal Token Pruning for Efficient High-Resolution GUI Agents
Feb 26, 2026Zhou Xu, Bowen Zhou, Qi Wang +2GUI Agents Visual Token Pruning
Tsinghua University, Shenzhen, China · Xidian University, Xi’an, China · The Chinese University of Hong Kong, Hong Kong, China
Tokenizing Semantic Segmentation with Run Length Encoding
Feb 25, 2026Abhineet Singh, Justin Rozeboom, Nilanjan RaySemantic Segmentation Image Tokenization
Department of Computing Science, University of Alberta, Edmonton, Canada
ApET: Approximation-Error Guided Token Compression for Efficient VLMs
Feb 23, 2026Qiankun Ma, Ziyao Zhang, Haofei Wang +3Visual Token Compression Efficient VLM Inference
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences · Peng Cheng Laboratory · University of Chinese Academy of Sciences +2
Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Framework
Feb 23, 2026Yibo Yan, Mingdong Ou, Yi Cao +5Visual Document Retrieval Multi-Vector Retrieval
Hong Kong University of Science and Technology (Guangzhou) · Alibaba Cloud Computing · Hong Kong University of Science and Technology
DUET-VLM: Dual stage Unified Efficient Token reduction for VLM Training and Inference
Feb 21, 2026Aditya Kumar Singh, Hitesh Kandala, Pratik Prabhanjan Brahma +2Efficient VLM Inference Multimodal Token Compression
Advanced Micro Devices, Inc. (AMD)
Going Down Memory Lane: Scaling Tokens for Video Stream Understanding with Dynamic KV-Cache Memory
Feb 20, 2026Vatsal Agarwal, Saksham Suri, Matthew Gwilliam +2Streaming Video Understanding Memory-Augmented VLMs
University of Maryland, College Park · TikTok
AFFMAE: Scalable Vision Pre-Training for High-Resolution Microscopy Segmentation on Desktop Hardware
Feb 18, 2026David Smerkous, Zian Wang, Behzad NajafianMasked Autoencoders Self-Supervised Pre-Training
Department of Laboratory Medicine & Pathology, University of Washington, Seattle WA, USA · Electrical Engineering & Computer Science, Oregon State University, Corvallis, OR, USA · Paul G. Allen School of Computer Science & Engineering, University of Washington, Seattle WA, USA
LGQ: Learnable Geometric Quantization for Image Tokenization
Feb 17, 2026Idil Bilge Altun, Mert Onur Cakiroglu, Elham Buxton +2Image Tokenization Vector Quantization
Luddy School of Informatics, Computing, and Engineering, Indiana University Bloomington, Bloomington, IN, USA · Department of Computer Science, University of Illinois Springfield, Springfield, IL, USA · College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar
CausalEmbed: Auto-Regressive Multi-Vector Generation in Latent Space for Visual Document Embedding
Jan 29, 2026Jiahao Huo, Yu Huang, Yibo Yan +7Multimodal Embedding Multi-Vector Retrieval
The Hong Kong University of Science and Technology (Guangzhou) · Alibaba Cloud Computing · The Hong Kong University of Science and Technology +1
SSDD: Single-Step Diffusion Decoder for Efficient Image Tokenization
Oct 6, 2025Théophane Vallaeys, Jakob Verbeek, Matthieu CordImage Tokenization Image Reconstruction
Meta Fundamental AI Research · Sorbonne University
EfficientUICoder: A Bidirectional Token Compression Framework for Efficient MLLM-Based UI Code Generation
Sep 15, 2025Jingyu Xiao, Zhongyi Zhang, Yuxuan Wan +3Efficient Multimodal Inference Code Generation
The Chinese University of Hong Kong, Hong Kong, China · Huazhong University of Science and Technology, Wuhan, China · Singapore Management University, Singapore, Singapore +1
DART: Differentiable Dynamic Adaptive Region Tokenizer for Vision Foundation Models
Jun 12, 2025Shicheng Yin, Kaixuan Yin, Yang Liu +2Vision Foundation Models Efficient ViTs
Sun Yat-sen University, China
ToDRE: Effective Visual Token Pruning via Token Diversity and Task Relevance
May 24, 2025Duo Li, Zuhao Yang, Xiaoqin Zhang +2Efficient VLM Inference Large Vision-Language Models
CCDS, NTU, Singapore · CCST, ZJUT, China · Terminus AI Lab, UCAS, China

Mar 15, 2026J/K move · Enter open · S save

ASAP: Attention-Shift-Aware Pruning for Efficient LVLM Inference

Surendra Pathak, Bo Han

George Mason University, Fairfax, VA, USA

Efficient VLM Inference Visual Token Pruning LVLM Pruning Visual Token Compression

Open paper AskPDF arXiv

While Large Vision-Language Models (LVLMs) demonstrate exceptional multi-modal capabilities, the quadratic computational cost of processing high-resolution visual tokens remains a critical bottleneck. Though recent token reduction strategies attempt to accelerate inference, such methods inadequately exploit attention values and fail to address token redundancy. More critically, they overlook the ``attention shift'' phenomenon inherent in LVLMs, which skews token attention scores. In this work, we propose ASAP, a novel training-free, KV-Cache-compatible pruning recipe that comprehensively addresses these limitations. First, we mitigate the attention shift by utilizing a dynamic bidirectional soft attention mask, ensuring the selection of genuinely informative tokens rather than naive attention-based selection. Second, we posit that high semantic redundancy within the token set degrades performance. We therefore introduce a weighted soft merging component that merges semantically similar tokens, preserving only the most feature-dense visual patches for subsequent layers. ASAP achieves virtually lossless compression of visual context, retaining 99.02% of the original LLaVA-NeXT-7B performance while aggressively slashing computational FLOPs by ~80%.