Visual Token Compression

Follow this topic Ask about this topic

Momentum

20 papers in the last four weeks, up 67% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 184

All topics

Spatial Matryoshka Training for Multi-Granularity Visual Document Retrieval
Aug 30, 2026Trishan Singha Roy, Arkadeep Acharya, Vishwajeet Kumar +2Hierarchical Representation Learning Matryoshka Representation Learning
IIT Delhi · IBM
PACE: A Unified Condense-and-Extract Paradigm for Fast VLM Inference
Aug 27, 2026Junjie Liu, Shengyuan Ye, Xu ChenEfficient VLM Inference Visual Token Pruning
Sun Yat-sen University · Power Dispatch Control Center, Guangdong Power Grid Co., Ltd. Guangzhou, China · Shenzhen Loop Area Institute, Shenzhen, China
Token-Oriented Semantic Communication with Pretrained Vision Transformers
Aug 26, 2026Jiwoong Im, Minwoo Kim, Jaeho Lee +2Learned Image Compression Semantic Communication
Department of Electrical Engineering, Pohang University of Science and Technology (POSTECH), Pohang 37673, South Korea
Just Noticeable Difference Modeling for Token Compression in Vision-Language-Action Models
Aug 21, 2026Zhuoyuan Li, Rui Zhao, Jin Wang +5Vision-Language-Action Models Visual Token Pruning
Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong · School of Computer Science and Engineering, Nanyang Technological University, Singapore 639798 · Bytedance Inc. +2
MergeOver: Post-Training Token Merging for Recursive Vision Transformers
Aug 13, 2026Junseo Kim, Uraz Odyurt, Amirreza YousefzadehVision Transformer Efficient ViTs
Computer Architecture for Embedded Systems, University of Twente Enschede, The Netherlands
When Vision Becomes Text: Visual Token Pruning via Cross-Modal Residual Guidance in VLMs
Aug 11, 2026Congyang Ou, Ruike Song, Yang Zhou +3Efficient VLM Inference Large Vision-Language Models
Northwestern Polytechnical University · Chongqing University · MiLM Plus, Xiaomi Inc.
Can Coding Agents Solve Repository-Level Issues with Rendered Code? An Exploratory Study of Visual Representations
Aug 10, 2026Weijie Liang, Yuanfeng Song, Xing Chen +3Automated Program Repair AI Coding Agents
The Hong Kong University of Science and Technology, Hong Kong, China · ByteDance, China
Not All Visual Tokens Are Equally Safe to Remove:Consequence-Sensitive Visual Token Compression
Aug 10, 2026Jingbo Wen, Liang He, Mingyu Cao +4Efficient VLM Inference Visual Token Compression
1The University of Sydney · 2Tongji University · University of Surrey +3
Visual Token Codec: Unleashing Spatial Redundancy for ViT Feature Coding
Aug 9, 2026Donghui Feng, Fengxi Zhang, Changsheng Gao +6Visual Token Compression
AnchorFold: A Focus-Then-Fold Framework via Recursive Attention Propagation for Efficient Multi-Vector Visual Document Retrieval
Aug 9, 2026Haoyu Zuo, Yibo Yan, Xin Zou +4Multi-Vector Retrieval Visual Document Retrieval
Hong Kong University of Science and Technology (Guangzhou) · Alibaba Cloud Computing · Hong Kong University of Science and Technology
Resolution Meets Reduction: Efficient Visual Context for 3D Radiology Report Generation
Aug 9, 2026Jonathan Suprijadi, Raphael Stock, Moritz Langenberg +10Radiology Report Generation Radiology VLMs
Division of Medical Image Computing, German Cancer Research Center (DKFZ), Heidelberg, Germany
KVAE: Family of Tokenizers for Multimodal Generative Models
Aug 6, 2026Andrey Shutkin, Denis Parkhomenko, Ivan Kirillov +11Variational Autoencoders Audio Token Compression
Kandinsky Lab
Persistent Object Narratives for Token-Efficient Video Language Models
Aug 5, 2026Junzhe Chen, Siyuan Meng, Xiaojie GuoVisual Tokenization Video-Language Models
Tianjin University
HiSC: Hierarchical Spatial Clustering Token Compression for Efficient 3D Scene Understanding
Aug 5, 2026Jiuhe Qu, Yingping Liang, Ying Fu3D Spatial Reasoning Visual Token Pruning
Beijing Institute of Technology Beijing, China
CARVE: Cross-Slice Anisotropic Reallocation of Visual Evidence for Efficient 3D Medical Volume Understanding
Aug 5, 2026Zhenyu Yi, Qiang Hu, Zhenhao Li +33D Medical Imaging Medical VQA
School of Biomedical Engineering, Shanghai Jiao Tong University · 2Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology
RUTA: Principled Visual Token Allocation via Rate-Utility Optimization
Aug 4, 2026Jian Zou, Xiaoyu Xu, Zhihua Wang +3Efficient VLM Inference Visual Token Pruning
City University of Hong Kong · Google Inc.
OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Models
Aug 4, 2026Wanshun Su, Yang Shi, Feihu Liu +10Efficient Multimodal Inference Audio Token Compression
Northwestern Polytechnical University · Peking University · Alibaba Group +1
When Do Fewer Visual Tokens Accelerate Multimodal Inference? A Break-Even Study Across Decision Locations and Hardware
Aug 4, 2026Hao Dou, Ruiwen TianEfficient VLM Inference Visual Token Pruning
Harbin Institute of Technology, Harbin, China
Messages, Not Tokens: Grounded Coresets for Faithful VLM Compression
Aug 3, 2026Long Qian, Jiaqi Wei, Bingke Zhu +2Efficient VLM Inference Coreset Selection
Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Engineering, Cardiff University, Cardiff, United Kingdom · School of Future Technology, University of Chinese Academy of Sciences, Beijing, China +1
Same Semantics, Different Paths: Self-Improving Alignment for Vision-Text Compression
Aug 3, 2026Tianyu Liang, Xiangxi Zheng, Yilin Wang +1VLM Adaptation Multimodal Token Compression
Southeast University Nanjing, China · Nanjing University Nanjing, China · Zhejiang University Hangzhou, China +1
Decoupling semantics from vision: A framework for faithful visual-text compression evaluation
Aug 3, 2026Yonghan Gao, Zehong Chen, Lijian Xu +3Multimodal Model Evaluation Visual Token Compression
Shenzhen University of Advanced Technology · Shenzhen Technology University
CRAFT: Compression via Recursive Adaptive Fusion of Video Tokens for Vision-Language Models
Aug 3, 2026Yu Chen, Xiaohong Li, Xiaole Wang +3Efficient VLM Inference Large Vision-Language Models
1Evermind · 2Shanda Group · 3Peking University
Rethinking Video Token Compression with a Global Codebook: Learning Once, Compressing Everywhere
Aug 2, 2026Jiayang He, Tianling Xu, Diancheng Kang +4Efficient VLM Inference Video-Language Models
Southern University of Science and Technology · Imperial College London · Jilin University
RAGOCR: Optical Compression of Retrieval-Augmented Text via Visual Representation
Aug 1, 2026Jiayang Yu, Jialun Zhong, Lei ZouMultimodal RAG Retrieval-Augmented Generation
Wangxuan Institute of Computer Technology, Peking University Beijing, China
ORCA: ORgan-Centroid Aggregation for Training-Free 3D CT Visual Token Compression
Jul 31, 2026Renjie Liang, Zijian Xu, Jinqian Pan +6Medical VLMs Visual Token Merging
University of Florida, Gainesville, FL, USA · University of Southern California, Los Angeles, CA, USA · National University of Singapore, Singapore
ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understanding
Jul 30, 2026Mingkang Dong, Muxin Pu, Jie Li +8Memory-Augmented VLMs Streaming Video Understanding
Universiti Malaya · Monash University · SJTU +5
Calibrate Before Reason: Robust Visual Token Reduction against Semantic Drift in VLMs
Jul 30, 2026Jiasheng Li, Zhong Ji, Yan Zhang +1VLM Robustness Efficient VLM Inference
Tianjin University, Tianjin, China · Tsinghua University, Beijing, China
MedARC: Training-Free Adaptive Redundancy Compression of Visual Tokens for 3D Medical Vision-Language Models
Jul 29, 2026Yitao Zhu, Mengjun Liu, Yingji Fu +2Efficient VLM Inference Medical VLMs
Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Hong Kong · School of Integrated Circuits and Electronics, Beijing Institute of Technology, China · 3Mental Health Research Center, The Hong Kong Polytechnic University, Hong Kong +1
MAViE: A Multi-scale Adaptive Vision Encoder for Fine-grained Visual Perception and Efficient Multimodal Reasoning
Jul 27, 2026Shaofei LeiEfficient VLM Inference Multi-Scale Feature Fusion
GOTS: Greedy Orthogonal Token Selection for High-Resolution Vision-Language Models
Jul 27, 2026Jun Ling, Tao Huang, Junzhuo Liu +2Efficient VLM Inference Large Vision-Language Models
School of Computer Science and Engineering, University of Electronic Science and Technology of China

Aug 30, 2026, cs.AIJ/K move · Enter open · S save

Spatial Matryoshka Training for Multi-Granularity Visual Document Retrieval

Trishan Singha Roy, Arkadeep Acharya, Vishwajeet Kumar, Jaydeep Sen+1

IIT Delhi · IBM

Hierarchical Representation Learning Matryoshka Representation Learning Multi-Vector Retrieval Visual Document Retrieval Visual Token Compression

Open paper AskPDF arXiv

Multi-modal late-interaction retrievers achieve strong retrieval on visually rich documents by representing each page as per patch embeddings and matching at the token level. However, this approach incurs high storage costs. Existing compression methods typically fix a single compression level at indexing time, limiting flexibility. We present ColSNAP (Spatial Nested Average Pooling)1, a training method that generates a nested hierarchy of compression levels directly from a backbone's patch grid. By spatially pooling patch embeddings into pro- gressively coarser tiers and training all tiers simultaneously, a single model learns to support retrieval at multiple compression levels without architectural changes. Crucially, a single encoding pass yields every tier, enabling the accuracy-storage trade-off to be configured at indexing time to match avail- able storage budgets, rather than being fixed during training. We demonstrate that models trained using ColSNAP maintain near full-resolution retrieval performance under substantial compression and that ColSNAP transfers effectively across multiple late-interaction backbones, and achieves most of its improvements via a lightweight adaptation stage applied to a pre-trained retriever.

Momentum

Related topics

Latest papers 184