Visual Token Compression

Ask about this topic

Overview

Vision encoders represent images and video as patch or frame tokens, and long token sequences can make multimodal models costly to run, especially during prefill and memory-intensive video reasoning. Methods include training-free pruning and merging, learned resamplers, quantization and codecs, and adaptive token budgets guided by text queries, spatial structure, or temporal redundancy. They are evaluated on tasks such as visual question answering, video understanding, OCR, segmentation, retrieval, generation, and robot control. The central tension is that fewer tokens do not always mean faster end-to-end inference, and aggressive compression can discard details needed for a particular question or task.

Latest papers

CardsList

271 papers46 papers in the last four weeks, up 229% on the four weeks before. 0.3% of all new papers.

  1. ConvRoute-ViT: Convolutional Token Routing for Efficient Vision Transformers

    Oct 28, 2026Manish Das, Shiv Ram Dubey, Satish Kumar SinghVision TransformerEfficient ViTs

    IIIT Allahabad, Prayagraj, India

  2. WithinVAE: Compact Visual Autoencoding with Self-Distilled Semantics

    Oct 27, 2026Yuzhou Ji, Jiepeng Wang, Tianyu Jin +4Variational AutoencodersVisual Token Compression

    Shanghai Jiao Tong University · Institute of Artificial Intelligence, China Telecom

  3. TP-JSCC: Efficient Token Pruning and Joint Source-Channel Coding for LVLMs-based 3D-VQA in Cloud-Edge Collaboration

    Oct 26, 2026Zhengxin Li, Chongzhen Tian, Hui Yuan +2Visual Token Compression3D VQA

    School of Control Science and Engineering, Shandong University, Jinan 250061, China · Key Laboratory of Machine Intelligence and System Control, Ministry of Education, Jinan 250061, China · Shandong Key Laboratory of Intelligent Communication and Sensing-Computing Integration, Jinan 250100, China +2

  4. VTC-AUDIT: An Interventional Reliability Audit for Visual-Token Compression under Paired Image Shifts

    Oct 25, 2026Isaac, LiVisual Token CompressionVLM Evaluation

    Rucheng · University of Pittsburgh

  5. Rescuing Your Pruning: Rethinking Visual Token Pruning and Post-Training Recovery in MLLMs

    Oct 24, 2026Peiran Yin, Hao Wu, Zihao Yang +3Visual Token PruningEfficient VLM Inference

    EIT-NLP Lab, Eastern Institute of Technology, Ningbo · Shanghai Jiao Tong University · Uppsala University

  6. Patch-Derived or Register Tokens? Comparing Variable-Length Image Tokenizers

    Oct 24, 2026Philipp Schenkel, Tobias Schürmann, Omer Sahin TasImage TokenizationVisual Token Compression

    FZI Research Center for Information Technology · Karlsruhe Institute of Technology

  7. Consommé: Conditioned Token Merging for Split Inference

    Oct 23, 2026Man Ngo, Jan S. RellermeyerSplit InferenceEfficient ViTs

    Leibniz University Hannover Hannover, Germany

  8. Rethinking Sequence Compression with Test-time Regression

    Oct 22, 2026Sihan Yang, Jie Huang, Hanxuan Li +4LLM Context CompressionMultimodal Token Compression

    The Chinese University of Hong Kong · Independent Researcher · Token Foundry, Alibaba Group

  9. GazeLens for Efficient Egocentric Video Question Answering with Simple Gaze-Driven Adaptive Token Reduction

    Oct 20, 2026Trong-Thang Pham, Anh Nguyen, Hien Van Nguyen +1Egocentric Video QAEfficient VLM Inference

  10. ReTrace: Visual Backtracking for Efficient Multimodal Large Language Models

    Oct 18, 2026Guangzong Si, Dong Wang, Yifan Yu +3Efficient VLM InferenceVisual Token Compression

    University of Science and Technology of China · Eastern Institute of Technology, Ningbo · Tsinghua University +2

  11. Query-adaptive Recurrent Tokenization

    Oct 16, 2026Bavesh Balaji, Pranav Gopinath, Pouya Sadeghi +1Visual TokenizationEfficient VLM Inference

    Critical ML Lab, University of Waterloo

  12. Thinking from Above: Efficient Spatial Reasoning with Bird's-Eye-View Visual Tokens

    Oct 15, 2026Faizan Siddiqui, Nirav Savaliya, Sahith Reddy Chada +1Visual Spatial ReasoningBird's-Eye-View Representation

    Honda Research Institute USA

  13. PACE: Pareto-Adaptive Compression for Efficient Native MLLMs

    Oct 15, 2026Lei Tian, Chunzheng Zhu, Kai Wu +2Multimodal Token CompressionVisual Token Compression

    ByteDance · Hunan University · Institute of Automation, Chinese Academy of Sciences

  14. FastVLA: Task-Conditioned and Temporally Adaptive Visual Token Compression for Efficient Vision-Language-Action Models

    Oct 15, 2026Jiuyi Xu, Chengyuan Qian, Zhengzhong Tu +2Efficient VLA Model InferenceVisual Token Compression

    Colorado School of Mines · Texas A&M University · Microsoft AI

  15. Visual Tokens on Demand: Visual Budgeting for Scalable and Efficient Multi-Image Reasoning in Multimodal LLMs

    Oct 13, 2026Aimon Rahman, Joymallya Chakraborty, Vidit Bansal +1Multi-Image ReasoningEfficient Multimodal Inference

    Amazon

  16. Just Respect the Semantics: Image-Token Compression via Pre-LLM Semantic Variable-Resolution Tokenization and In-LLM Relevance Pruning

    Oct 13, 2026Tianhao Fu, Austin Wang, Charles Chen +5Visual Token CompressionEfficient VLM Inference

    University of Toronto, Toronto, ON, Canada · Vector Institute, Toronto, ON, Canada · McGill University, Montreal, QC, Canada +2

  17. Scaling Vision Transformers Without Training: Prune-and-Merge Token Compression

    Oct 13, 2026Vatsal, Soumil Malhotra, Priyanshu Chahal +1Vision TransformerEfficient ViTs

    Department of Mathematics, Indian Institute of Technology Roorkee

  18. TwinPrune: Density-Aware Two-Phase Token Pruning for Vision-Language Models

    Oct 13, 2026Kai Liu, Anqi Li, Junxian Li +4Efficient VLM InferenceVisual Token Pruning

    Shanghai Jiao Tong University · Huawei Technologies Ltd

  19. Bottlenecked semantic compression by representation reconstruction

    Oct 12, 2026Xiaoya Tang, Shilong Liu, Daniel Smolyak +3Visual Token CompressionEfficient ViTs

    Scientific Computing and Imaging Institute University of Utah · Princeton AI Lab Princeton University · National Institutes of Health

  20. WIRE: Weight Space Implicit Representation Encoding of Frozen Video LLMs

    Oct 12, 2026Ameen Ali, Tamim Zoabi, Lior Wolf +1Efficient VLM InferenceVideo-Language Models

    Tel Aviv University · INSIGHT Lab, Ben-Gurion University of the Negev, Israel

  21. ConDenseDrive: Knowledge-Dense, Token-Sparse Visual Encoder for End-to-End Autonomous Driving

    Oct 12, 2026Shengzhen Li, Yongkang Li, Hao Gao +5End-to-End Autonomous DrivingEfficient ViTs

    Huazhong University of Science and Technology · Yinwang Intelligent Technology Co. Ltd.

  22. When 1+1 Backfires: Rethinking Relevance-Diversity Balancing for Sample-Aware Visual Token Reduction

    Oct 12, 2026Yihang Huang, Peisong Wen, Chenhao Zhou +2Efficient VLM InferenceVisual Token Pruning

    State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences School of Computer Science and Technology, University of Chinese Academy of Sciences Beijing, China · School of Computer Science and Technology, University of Chinese Academy of Sciences Beijing, China · State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences Beijing Academy of Artificial Intelligence Beijing, China

  23. Semantically Steerable Image Tokenization

    Oct 11, 2026Danilo de Goede, Mohammad Mahdi Derakhshani, Yuki M. Asano +1Image TokenizationVisual Token Compression

    QUVA Lab, University of Amsterdam · VIS Lab, University of Amsterdam · Fundamental AI Lab, University of Technology Nuremberg

  24. LapMerge: Differentiable Exact-Budget Token Merging in Linear Time

    Oct 9, 2026Magdalena Trędowicz, Aleksandra Nowak, Tomasz Steifer +3Token MergingVisual Token Merging

    Jagiellonian University in Kraków · Institute of Fundamental Technological Research, Polish Academy of Sciences · Centre for Credible AI, Warsaw University of Technology

  25. MixTok: Adaptive Visual Tokenization with Global 1D Tokens and Sparse 2D Refinement

    Oct 9, 2026Chenge Li, Yefei He, Xinyu Liu +3Visual TokenizationVisual Token Compression

    Zhejiang University · The Hong Kong University of Science and Technology · University of Adelaide

  26. GeoGR: Spatial-Semantic Token Compression and Group Routing for Efficient 3D VLMs

    Oct 9, 2026Yongxin Zheng, Hao Wu, Fang Li +8Layer Skipping in Language Model Decoding3D VLMs

    EIT-NLP Lab, Eastern Institute of Technology, Ningbo · Shanghai Jiao Tong University · The Hong Kong Polytechnic University +1

  27. Look Back, Think Ahead: Visual Memory on Demand for Efficient Multimodal Reasoning

    Oct 8, 2026Yicheng Xue, Han Wu, Jufeng Yang +4VLM ReasoningEfficient Multimodal Inference

    City University of Hong Kong · Zhejiang University · Peking University +2

  28. V-CoLA: Vision Token Compression with Linear Attention

    Oct 8, 2026Hao Jiang, Yiru Mao, Tianpeng Bu +9Visual Token CompressionEfficient VLM Inference

    Alibaba Cloud Computing, Alibaba Group

  29. QuadTok: Quadtree Visual Tokenizer for Autoregressive Image Generation

    Oct 7, 2026Yucheng Mao, Zeyuan Chen, Xiaojun Shan +4Visual TokenizationAutoregressive Image Generation

    University of California, San Diego

  30. Understanding and Mitigating Token-Pruning-Induced Vulnerabilities in VLMs

    Oct 7, 2026Shuailong Wang, Xinyu Lyu, Shengming Yuan +3Visual Token PruningVLM Robustness

    University of Electronic Science and Technology of China, Chengdu, China · Southwestern University of Finance and Economics, Chengdu, China · Tongji University, Shanghai, China +1