Visual Token Compression

Follow this topic Ask about this topic

Momentum

20 papers in the last four weeks, up 67% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 184

All topics

AdaMerge: Salience-Aware Adaptive Token Merging for Training-Free Acceleration of Vision Transformers
May 26, 2026Semi Lee, Hyejin Go, Hyesong ChoiToken Merging Self-Attention
Electronic Engineering · Soongsil University · Seoul, South Korea
Channel-wise Vector Quantization
May 25, 2026Wei Song, Tianhang Wang, Yitong Chen +5Image Tokenization Visual Tokenization
Shanghai Innovation Institute · Westlake University · Zhejiang University +2
VEN-VL: A Visual Ensemble MoE Framework for Effective and Efficient Multi-Modal Understanding
May 25, 2026Yinghao Wu, Zhuoyan Luo, Yiyao Yu +3Efficient Multimodal Inference Mixture-of-Experts Models
Tsinghua University
Efficient One-Step Diffusion Restoration Model with Compact Token Compression and Linear Attention
May 22, 2026Bingtian Qiao, Yue Shi, Yingjie Zhou +3Diffusion Models for Image Restoration Diffusion Model Inference Acceleration
Shanghai Jiao Tong University · Fuzhou University · Shanghai AI Laboratory
ASAP: Attention Sink Anchored Pruning
May 21, 2026Jaehyuk Lee, Hanyoung Kim, Yanggee Kim +1Self-Attention Efficient ViTs
Department of Mathematics Korea University Seoul, Republic of Korea · Korea University Seoul, Republic of Korea
MuKV: Multi-Grained KV Cache Compression for Long Streaming Video Question-Answering
May 21, 2026Junbin Xiao, Jiajun Chen, Tianxiang Sun +2KV Caching Video QA
University of Science and Technology of China · National University of Singapore
Enhancing Visual Token Representations for Video Large Language Models via Training-Free Spatial-Temporal Pooling and Gridding
May 21, 2026Bingjun Luo, Tony Wang, Hanqi Chen +1Video-Language Models Multimodal Token Compression
1Tsinghua University · 2Shenzhen University · 3Xidian University
ORBIS: Output-Guided Token Reduction with Distribution-Aware Matching for Video Diffusion Acceleration
May 21, 2026Hangyeol Lee, Joo-Young KimToken Merging Video Diffusion Models
KAIST · Daejeon, South Korea
Head-Aware Key-Value Compression for Efficient Autoregressive Image Generation
May 20, 2026Guotao Liang, Baoquan Zhang, Zhiyuan Wen +1KV Caching Autoregressive Image Generation
Harbin Institute of Technology, Shenzhen · Peng Cheng Laboratory
DynaTok: Temporally Adaptive and Positional Bias-Aware Token Compression for Video-LLMs
May 19, 2026Minyoung Park, Taehun Kong, Sangjun AhnVideo-Language Models Video Token Compression
LG Electronics, Seoul, South Korea
AQuaUI: Visual Token Reduction for GUI Agents with Adaptive Quadtrees
May 19, 2026Yuankai Li, Tinghui Zhu, Ha Min Son +3Efficient Multimodal Inference GUI Agents
UC Davis
Vision Foundation Models as Generalist Tokenizers for Image Generation
May 18, 2026Anlin Zheng, Qi Han, Xin Wen +5Vision Foundation Models Image Tokenization
University of Hong Kong, Pokfulam, Hong Kong · StepFun
TinySAM 2: Extreme Memory Compression for Efficient Track Anything Model
May 18, 2026Zhaoyuan Ding, Yijing Yang, Han Shu +1Video Object Segmentation Memory-Efficient Inference
Huawei Noahs Ark Lab
A More Word-like Image Tokenization for MLLMs
May 18, 2026Hyun Lee, Hyemin Jeong, Yejin Kim +4Image Tokenization Efficient VLM Inference
Seoul National University · Ewha Womans University
An Efficient Streaming Video Understanding Framework with Agentic Control
May 18, 2026Jinming Liu, Jianguo Huang, Zhaoyang Jia +7Streaming Video Understanding Adaptive Model Routing
1Shanghai Jiao Tong University · 2Eastern Institute of Technology, Ningbo, China · 3Microsoft Research Asia
LiteFrame: Efficient Vision Encoders Unlock Frame Scaling in Video LLMs
May 17, 2026Jihwan Kim, Nikhil Parthasarathy, Danfeng Qin +5Efficient VLM Inference Video-Language Models
1Google DeepMind · 2Seoul National University
LRCP: Low-Rank Compressibility Guided Visual Token Pruning for Efficient LVLMs
May 15, 2026Hongyu Lu, Feng Zhang, Wenwei Jin +5Efficient VLM Inference Large Vision-Language Models
1Xiaohongshu · 2Harbin Institute of Technology · 3Fudan University
Representative Attention For Vision Transformers
May 14, 2026Yuntong Li, Hainuo Wang, Hengxing Liu +2Vision Transformer Efficient ViTs
College of Intelligence and Computing, Tianjin University, Tianjin 300350, China
A Picture is Worth a Thousand Words? An Empirical Study of Aggregation Strategies for Visual Financial Document Retrieval
May 14, 2026Ho Hung Lim, Yi YangMulti-Vector Retrieval Information Retrieval
The Hong Kong University of Science and Technology
InsightTok: Improving Text and Face Fidelity in Discrete Tokenization for Autoregressive Image Generation
May 14, 2026Yang Yue, Fangyun Wei, Tianyu He +10Image Tokenization Image Generation
Tsinghua University · Microsoft Research
AdaFocus: Adaptive Relevance-Diversity Sampling with Zero-Cache Look-back for Efficient Long Video Understanding
May 13, 2026Xiao Yang, Yingzhe Ma, Haoxuan Yu +2Efficient VLM Inference Long-Video Understanding
University of Electronic Science and Technology of China · China
G $^2$ TR: Generation-Guided Visual Token Reduction for Separate-Encoder Unified Multimodal Models
May 12, 2026Junxian Li, Kai Liu, Zizhong Ding +4Efficient Multimodal Inference Unified Multimodal Models
Shanghai Jiao Tong University · Huawei Technologies Ltd
See What Matters: Differentiable Grid Sample Pruning for Generalizable Vision-Language-Action Model
May 12, 2026Yixu Feng, Zinan Zhao, Yanxiang Ma +4Vision-Language-Action Models VLMs for Robotics
The University of Sydney · City University of Hong Kong · StellarEdge Robotics
OTT-Vid: Optimal Transport Temporal Token Compression for Video Large Language Models
May 12, 2026Minseok Kang, Minhyeok Lee, Jungho Lee +6Efficient VLM Inference Video-Language Models
Yonsei University · LG Electronics · KIST
DORA: Dynamic Online Reinforcement Agent for Token Merging in Vision Transformers
May 12, 2026Kaixuan He, Song Chen, Yi KangEfficient ViTs Visual Token Merging
University of Science and Technology of China, Hefei, China · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center
ReVision: Scaling Computer-Use Agents via Temporal Visual Redundancy Reduction
May 11, 2026Amirhossein Abaskohi, Yuhang He, Peter West +3Computer-Use Agents Long-Horizon Agent Tasks
University of British Columbia · Microsoft Research
Mutual Enhancement Between Global Tokens and Patch Tokens: From Theory to Practice
May 11, 2026Xiusheng Huang, Xin Jiang, Jun Zhao +2Learned Image Compression Image Tokenization
The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences · Beijing Academy of Artificial Intelligence
Network-Efficient World Model Token Streaming
May 11, 2026Shatadal Mishra, Ahmadreza Moradipari, Nejib AmmarIntelligent Transportation Systems World Models
InfoTech Labs, Toyota Motor North America R&D, Mountain View, CA, USA
Fre-Res: Frequency-Residual Video Token Compression for Efficient Video MLLMs
May 10, 2026Yigui Feng, Qinglin Wang, Yang Liu +1Efficient VLM Inference Visual Tokenization
The College of Computer Science, National University of Defense Technology, Changsha, Hunan, China · College of Computer Science, National University of Defense Technology, Deya Road 109, Changsha, Hunan 410073, China · The Shien-Ming Wu School of Intelligent Engineering, South China University of Technology, Guangzhou, Guangdong, China +1
LLaVA-UHD v4: What Makes Efficient Visual Encoding in MLLMs?
May 9, 2026Kechen Fang, Yihua Qin, Chongyi Wang +3Efficient VLM Inference Multimodal Large Language Models
Tsinghua University · ModelBest

May 26, 2026, cs.CVJ/K move · Enter open · S save

AdaMerge: Salience-Aware Adaptive Token Merging for Training-Free Acceleration of Vision Transformers

Semi Lee, Hyejin Go, Hyesong Choi

Electronic Engineering · Soongsil University · Seoul, South Korea

Token Merging Self-Attention Vision Transformer Efficient ViTs Visual Token Merging Visual Token Compression

Open paper AskPDF arXiv

The quadratic cost of self-attention in Vision Transformers (ViTs) constitutes a fundamental bottleneck for practical deployment, motivating a vibrant line of research on token reduction. Among existing approaches, token merging (ToMe) has emerged as an elegant training-free solution; yet its design rests on an unspoken premise of token equality, which contravenes the well-documented non-uniformity of self-attention and leads to information loss in high-salience tokens under aggressive compression. We address this limitation with AdaMerge, a token-merging framework based on two complementary mechanisms. First, salience-weighted similarity leverages column-wise feature-affinity centrality as a token-importance proxy and incorporates the resulting salience scores into the bipartite matching score, ensuring that pivotal tokens contribute more strongly to the merged representation. Second, adaptive merging intensity uses pre-computed layer-wise similarity statistics to dynamically modulate the per-layer reduction count in accordance with input-specific redundancy. On ImageNet-1k with ViT-B/16, AdaMerge consistently outperforms ToMe, PiToMe, and DSM across all FLOPs-matched regimes. The accuracy gap widens monotonically with compression: at the 13.4G FLOPs operating point, AdaMerge sustains a Top-1 degradation of only -1.06%, compared to -1.45% for PiToMe and -4.62% for DSM. To our knowledge, AdaMerge is the first to combine salience-weighted similarity and adaptive per-layer reduction into a single training-free token merging framework, advancing the accuracy-FLOPs Pareto frontier of ViT acceleration.

Momentum

Related topics

Latest papers 184