cs.CVNov 19, 2026

VidToC: Rethinking Video Token Compression for Multimodal Large Language Models

Authors: Zihan Xiao, Boshen Xu, Pei Fu, Jianzhong Ju, Zhenbo Luo, Jian Luan, Qin Jin

Organizations: AIM3 Lab, Renmin University of China · MiLM Plus, Xiaomi Inc · MiLM Plus, Xiaomi Inc.

Abstract

Extending multimodal large language models (MLLMs) from images to videos faces substantial memory and latency challenges due to the massive number of visual tokens. These challenges have driven the development of numerous visual token compression methods and more efficient LLM architectures. However, existing compression methods are often evaluated with different training setups, tasks, and models, making fair comparison difficult. Meanwhile, the growing use of hybrid Mamba-Transformer architectures raises questions about whether compression strategies transfer across backbones. We present VidToC, a controlled evaluation framework for representative compression algorithms, LLM backbones, and time encoding strategies, covering training-free and fine-tuned settings on three complementary video understanding tasks. Our results reveal task-dependent compression behavior and characterize the effects of compression placement, time encoding, and backbone choice. These findings provide empirical guidance for designing and selecting video token compression strategies across tasks and training settings.

Figures & tables

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Visual Token Coding for Video Multimodal Large Language Models

    Aug 28, 2026Chenxin Fang, Tao Chen, JunChao You +3Multimodal Token CompressionVideo Token Compression

  2. Video Codec-Guided Training-Free Token Compression for Video Large Language Models

    Oct 9, 2026Xihua Sheng, Ruijie Liu, Chang Wen ChenVideo Token CompressionVideo-Language Models

  3. KiToke: Distribution-Aware Token Compression for Video Large Language Models

    Apr 3, 2026Haifeng Huang, Biyin Xu, Yang LiVideo Token CompressionVisual Token Compression