Large Multimodal Models (LMMs) uniformly perceive video frames, creating computational inefficiency for videos with inherently varying temporal redundancy. This paper present Quicksviewer, an LMM with new perceiving paradigm that partitions a nonuniform redundant video into varying cubes using Gumbel Softmax, followed by a unified resampling for each cube to achieve efficient video understanding. We further propose a Gumbel-noise annealing strategy that enables stable end-to-end LMM training with the discrete cubing network. This simple and intuitive approach dynamically compress video online based on the momentum of frame differences, significantly reducing spatiotemporal complexity (overall 45× compression rate), while enabling efficient training with large receptive field. We train the model from a language backbone through three progressive stages, each incorporating lengthy videos on average of 420s/1fps thanks to the perceiving efficiency. With only 0.8M total video samples, our model outperforms contemporary LMMs with the same backbone and comparable parameters amount. An experiment in A/B pretraining with same backbone and training data shows that, w/ cubing achieves lower training loss and higher benchmark accuracy (+2.1) than the fixed partitioning baseline, strictly validating the benefits. Moreover, the front-loaded compression in encoding substantially reduces latency in the computationally expensive prefill and decoding, making it suited to mainstream EPD disaggregated inference deployment. Code: https://quicksviewer.github.io.
Figures & tables
Figure 2: Left: Three-stage training. Middle and right: Controlled A/B pretraining with the same backbone (Qwen2.5 + SigLIP) and training data (150K captions). Cubing yields lower training loss and improves benchmark accuracy (+2.1) over fixed partitioning at the same input token budget.
Model
Visual tokens / original frame
Compression
Max train frames
Eval frames
D600 (%)
Video-MME w/o subs (%)
Trained compression models
VideoLLaMA2 7B
72
8x
16
16
97.33
47.9
PLLaVA 7B
144
4x
16
16
97.33
-
Kangaroo 8B
128
8x
160
64
89.33
56.0
TimeMarker 8B
36*
16x*
128
120*
80.00*
57.3
LongVU 7B
≤ 144
Adaptive
-
1 FP
-
-
Table 1: Comparison of Quicksviewer with representative video LMMs and compression methods. Metrics are compiled from official implementations or reported results; see Appendix C for definitions and details. Quicksviewer achieves 45 × compression and 62.3% accuracy on Video-MME.
Figure 3: Left: The network architecture of Quicksviewer, that performs unified understanding of videos and images through visual tokens from cascaded modules. Right: The cubing network, that partitions an online video into nonuniform cubes based on Gumbel Softmax.
Figure 4: (a) Domain-specific results on VideoMME. (b) Inference time comparison with LLaVA-Next-Video. (c) Distribution of cube lengths on VideoMME.
Figure 5: The "Visual Lag" phenomenon occurring during the model’s cube-based segmental comprehension, where current cubes incorporate terminal frames from preceding event scenes to enable retrospective understanding.
Figure 6: (a) Left: Gumbel noise progressively anneals to 0.001 following the decaying learning rate with cosine scheduler. (b) Right: Compared to non-annealed training (cyan curve), adding Gumbel noise annealing (purple curve) yields more stable and superior loss convergence.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Stage-1
Stage-2
Stage-3
Vision
Resolution
384×384
384×384
384×384
FPS,#Frames
1, Max 420
1, Max 420
1, Max 420
Data
Image-Text #samples
LCS, OBELICS
LLaVAOV-SingleImage
LLaVAOV-MultiImages
558K, 20K
2.99M
100K
Video-Text #samples
FineVideo, ANetCaptions
FineVideo, ShareGPT4Video, ANetCaptions
Sec. 3. 4
87K, 8K
118K
599K
Appendix
Table 4: Detailed configuration for each training stage.
Modality
Task
# Samples
Dataset
Image-Text
Interleaved Pairs
20K
OBELICS
Single-Image Captioning
558K
LCS
Video-Text
Captioning
95K
FineVideoCaptions, ANetCaptions
Appendix
Table 5: Training data statistics for the alignment stage.
Modality
Task
# Samples
Dataset
Text
Instruction
93K
Evo-Instruct
Image-Text
Interleaved Pairs
20K
OBELICS
Single-Image Captioning
50K
LCS
Single-Image Tasks
2.8M
LLaVAOneVision
Video-Text
Captioning
5K
FineVideoCaptions, AnetCaptions
VQA
75K
FineVideoQAs
Appendix
Table 6: Training data statistics for the pre-training stage.
Modality
Task
# Samples
Dataset
Text
Instruction
9K
Evo-Instruct
Image-Text
Single-Image Tasks
40K
LLaVA-OneVision-SingleImage
Multi-Images Tasks
100K
LLaVA-OneVision-MultiImages
Video-Text
Captioning
52K
TextVR, MovieChat, YouCook2
Dense Captioning
4K
ShareGPT4Video
Classification
1K
Kinetics-710
Appendix
Table 7: Training data statistics for the supervised fine-tuning stage.
Model
Visual tokens / original frame
Compression
Max train frames
Eval frames
D600 (%)
Video-MME w/o subs (%)
Trained compression models
LLaMA-VID 7B
2
128x
-
1 FPS
conditional 0
-
VideoLLaMA2 7B
72
8x
16
16
97.33
47.9
VideoLLaMA3 7B
Dynamic
Dynamic
180
≤ 180
≥ 70.00
66.2
PLLaVA 7B
144
4x
16
16
97.33
-
Chat-UniVi 7B
7 at F=64
36.57x
64
64
89.33
-
Appendix
Table 8: Comparison of representative efficient video LMMs publicly available by June 30, 2025. Video-MME results use the no-subtitle setting. BIMBA follows the v1 128-frame setup (training cap assumed). Video-XL’s count of 144 tokens/frame excludes additional visual summarization tokens (VSTs). - denotes unavailable or unresolved values, including unverified subtitle protocols; N/A denotes not applicable. D600=100[1−min(600,Feval)/600] is the sampling-budget omission for 600 input frames, not an architectural limit or an observed adaptive frame-drop rate. Capacity0 means the reported frame cap is at least 600; “+ adaptive” indicates additional content-dependent pruning. Formatting tokens are excluded. Asterisks mark conditional settings: TimeSuite’s ratio uses the raw spatial patch grid; TimeMarker’s values use a 600-second video; MiniCPM-V’s ratio assumes a single square 448-pixel view; ReTaKe’s capacity does not account for adaptive frame selection.
Figure 7: Qualitative analysis showns that Quicksviewer effectively understands lengthy documentary and sports videos, as well as informative single and multiple images.