cs.CVApr 21, 2025

Quicksviewer: An LMM with Early Video Compression using Annealed Gumbel-Softmax

Authors: Ji Qi, Yuan Yao, Yushi Bai, Bin Xu, Juanzi Li, Zhiyuan Liu, Tat-Seng Chua

Organizations: Tsinghua University · National Univesity of Singapore

Abstract

Large Multimodal Models (LMMs) uniformly perceive video frames, creating computational inefficiency for videos with inherently varying temporal redundancy. This paper present Quicksviewer, an LMM with new perceiving paradigm that partitions a nonuniform redundant video into varying cubes using Gumbel Softmax, followed by a unified resampling for each cube to achieve efficient video understanding. We further propose a Gumbel-noise annealing strategy that enables stable end-to-end LMM training with the discrete cubing network. This simple and intuitive approach dynamically compress video online based on the momentum of frame differences, significantly reducing spatiotemporal complexity (overall 45×\times compression rate), while enabling efficient training with large receptive field. We train the model from a language backbone through three progressive stages, each incorporating lengthy videos on average of 420s/1fps thanks to the perceiving efficiency. With only 0.8M total video samples, our model outperforms contemporary LMMs with the same backbone and comparable parameters amount. An experiment in A/B pretraining with same backbone and training data shows that, w/ cubing achieves lower training loss and higher benchmark accuracy (+2.1) than the fixed partitioning baseline, strictly validating the benefits. Moreover, the front-loaded compression in encoding substantially reduces latency in the computationally expensive prefill and decoding, making it suited to mainstream EPD disaggregated inference deployment. Code: https://quicksviewer.github.io.

Figures & tables

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Small Vision-Language Models are Smart Compressors for Long Video Understanding

    Apr 9, 2026Junjie Fei, Jun Chen, Zechun Liu +13Long-Video UnderstandingVideo Token Compression

  2. FlashGaze: Training-Free Multi-Scale Patch Pruning For Efficient Video Understanding

    Oct 3, 2026Ziye Zhu, Yanghao Zhou, Lixing Tan +3Efficient VLM InferenceVisual Token Pruning

  3. EarlyTom: Early Token Compression Completes Fast Video Understanding

    May 28, 2026Hesong Wang, Xin Jin, Lu Lu +4Video UnderstandingEfficient VLM Inference