Multimodal Large Language Models (MLLMs) have demonstrated strong performance in video understanding, yet efficiently processing long, high-resolution videos remains challenging. Such videos often contain substantial spatiotemporal redundancy, and processing redundant visual tokens can incur avoidable computational overhead. Many existing methods prune visual tokens during or after vision transformer (ViT) encoding, leaving much of the encoding cost unaddressed. Some approaches prune patches before encoding but rely on learned auxiliary networks for patch selection, incurring additional training and inference overhead. To address these limitations, we propose FlashGaze, a training-free method that reduces spatiotemporal redundancy before ViT encoding without introducing auxiliary networks. FlashGaze uses pixel-space differences as a proxy for information loss and employs Quadtree Dynamic Programming to jointly optimize patch dropping, merging, and keeping under a fixed budget. Experiments on two MLLM backbones across multiple benchmarks demonstrate substantial efficiency gains while largely preserving accuracy. On Qwen3-VL-8B, FlashGaze retains 98% of the full-input baseline accuracy on LongVideoBench while achieving up to 5.4x and 17x speedups in ViT encoding and MLLM prefill, respectively, and reducing peak GPU memory usage by a factor of 1.8. These efficiency gains enable the model to process videos with more frames and higher resolutions on the same GPU hardware, unlocking video understanding at scales previously out of reach.
Figures & tables
Figure 1: We propose FlashGaze, a training-free method that enables MLLMs to process 1.5K-frame and 4K-resolution videos. Existing methods prune visual tokens before or inside the MLLM, leaving substantial redundant computation in the ViT. In contrast, FlashGaze prunes patches directly at the video input stage, retaining 98% of the full-input baseline accuracy while speeding up the ViT and prefill stages by up to 5.4 × and 17 × , respectively. We also visualize the pruning results, showing FlashGaze selectively retains informative visual patches.
Figure 2: Visualization of pruned videos. FlashGaze mainly retains patches in regions that change over time in (a) and (b). In the continuously changing driving scene in (c), FlashGaze uses coarse-scale patches for low-information areas such as the sky, while preserving fine-scale patches for detail-rich regions such as traffic signs.
Figure 3: Processing pipeline of FlashGaze. FlashGaze performs patch pruning on video frames sequentially, based on visual memory and current frames (left). During patch pruning, FlashGaze estimates the costs of Drop and Merge using current frame Ft and memory Mt (middle). Given the costs and budget, FlashGaze determines the action assignment via Quadtree DP (right).
Method
LongVideoBench
VideoMMMU
HLVid
Perc.
Relat.
Overall
TTFT(s)
Perc.
Comp.
Adapt.
Overall
TTFT(s)
Overall
TTFT(s)
Qwen3-VL-8B
+ToMe
73.2
60.4
66.4
24.3
78.0
69.0
48.3
65.1
23.7
71.6
25.7
+FastV
72.8
60.4
66.2
28.7
79.6
67.0
50.7
65.8
28.2
70.9
30.1
+VisionZip
72.6
60.3
66.0
25.4
79.0
68.3
49.0
65.4
24.6
71.2
27.5
+FlashVID
72.0
59.7
65.4
26.5
77.3
64.7
49.3
63.8
25.8
72.3
28.9
Table 1: Performance comparison with other methods. FlashGaze achieves the highest accuracy and lowest TTFT across high-resolution and long-video benchmarks. * FastV is omitted for NVILA-8B-Video due to pre-pruning context overflow.
general video
long video
high-res & long
Models
Max #F
Max Res
VideoMME (w/o sub)
NExT-QA (mc)
VideoMMMU (overall)
LongVideoBench (overall)
MLVU (m-avg)
HLVid (test)
LLaVA-OV-8B ( Li et al., 2025 )
32
384
58.2
79.4
-
56.5
64.7
41.1
LLaVA-Video-7B ( Zhang et al., 2024 )
64
384
63.3
83.2
36.1
58.2
70.8
-
VideoLLaMA3-7B ( Zhang et al., 2025 )
180
384
66.2
84.5
47.0
59.8
73.0
38.8
InternVL3.5-8B ( Wang et al., 2025 )
64
448
66.0
-
-
62.1
70.2
39.9
VideoChat-Flash ( Li et al., 2026 )
10000
448
65.3
-
41.7
64.7
74.7
46.6
Table 2: Scalability across various benchmarks. FlashGaze scales Qwen3-VL-8B to 1.5K-frame and 4K-resolution videos, unlocking longer context for better video understanding.
Table 3: Efficiency under different retained ratios. FlashGaze achieves the lowest TTFT and peak GPU memory across all settings while largely preserving accuracy.
Figure 4: Latency across input scales and retained ratios. FlashGaze can choose different retained ratios to balance accuracy and efficiency. At a 20% retained ratio, FlashGaze still maintains strong accuracy. At this ratio, with 256 frames and 1792 resolution, it achieves 7.3 × ViT, 19.7 × prefill, and 15 × TTFT speedups.
Method
Type
HLVid
L-VidBench
Acc.
Acc.
Patch embedding
Feature-space
69.3
65.8
SigLIP
Semantic
65.1
64.2
LPIPS
Perception
69.0
66.0
Pixel (ours)
Pixel
71.7
66.3
Table 4: Ablation on cost estimation and optimization methods.
Figure 5: Analysis of how FlashGaze enables strong video understanding. Visual representations remain highly similar between FlashGaze and the vanilla setting (left). FlashGaze enables MLLMs to allocate more attention to dynamic tokens and achieves better accuracy, suggesting that increased attention to dynamic tokens may contribute to the accuracy improvement (right).
Video Large Language Models (VLLMs) excel in video understanding, but their excessive visual tokens pose a significant computational challenge for real-world applications. Current methods aim to enhance inference efficiency by visual token pruning. However, they do not consider the dynamic characteristics and temporal dependencies of video frames, as they perceive video understanding as a multi-frame task. To address these challenges, we propose MMG-Vid, a novel training-free visual token pruning framework that removes redundancy by Maximizing Marginal Gains at both segment-level and token-level. Specifically, we first divide the video into segments based on frame similarity, and then dynamically allocate the token budget for each segment to maximize the marginal gain of each segment. Subsequently, we propose a temporal-guided DPC algorithm that jointly models inter-frame uniqueness and intra-frame diversity, thereby maximizing the marginal gain of each token. By combining both stages, MMG-Vid can maximize the utilization of the limited token budget, significantly improving efficiency while maintaining strong performance. Extensive experiments demonstrate that MMG-Vid can maintain over 99.5% of the original performance, while effectively reducing 75% visual tokens and accelerating the prefilling stage by 3.9x on LLaVA-OneVision-7B. Code is available at https://github.com/ylnxxts/MMG-Vid.
Junpeng Ma, Qizhe Zhang, Ming Lu +4
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Fudan University · Taobao & Tmall Group of Alibaba
Multi-modal large language models (MLLMs) have advanced general-purpose video understanding but struggle with long, high-resolution videos -- they process every pixel equally in their vision transformers (ViTs) or LLMs despite significant spatiotemporal redundancy. We introduce AutoGaze, a lightweight module that removes redundant patches before processed by a ViT or an MLLM. Trained with next-token prediction and reinforcement learning, AutoGaze autoregressively selects a minimal set of multi-scale patches that can reconstruct the video within a user-specified error threshold, eliminating redundancy while preserving information. Empirically, AutoGaze reduces visual tokens by 4x-100x and accelerates ViTs and MLLMs by up to 19x, enabling scaling MLLMs to 1K-frame 4K-resolution videos and achieving superior results on video benchmarks (e.g., 67.0% on VideoMME). Furthermore, we introduce HLVid: the first high-resolution, long-form video QA benchmark with 5-minute 4K-resolution videos, where an MLLM scaled with AutoGaze improves over the baseline by 10.1% and outperforms the previous best MLLM by 4.5%. Project page: https://autogaze.github.io/.
Long-form video understanding remains challenging for Video Large Language Models (VideoLLMs), as the dense frame sampling introduces massive visual tokens while sparse sampling risks missing critical temporal evidence and leading to LLM hallucination. Existing training-free token reduction methods either treat videos equally as static images or rely on segment-level merging heuristics, which weaken fine-grained spatiotemporal modeling and introduce additional overhead. In this paper, we propose EchoPrune, a lightweight and training-free token pruning method that improves temporal resolution under a fixed LLM-side visual token budget. Our core idea is to interpret redundant video tokens as temporal echoes: if a token is well reconstructed from the previous frame, it is merely a temporally redundant echo; otherwise, it may capture new events, motion, or query-relevant visual evidence. Based on this insight, EchoPrune scores visual tokens by (i) query-guided crossmodal relevance and (ii) temporal reconstruction error, measured by correspondence matching and echo matching across consecutive frames. The selected tokens preserve task-relevant cues and temporal novelty while suppressing predictable redundancy, allowing VideoLLMs to observe more frames without increasing the decoding budget. Extensive experiments on LLaVA-OV, Qwen2.5VL, and Qwen3VL across six video understanding benchmarks show that EchoPrune enables VideoLLMs to process up to 20x frames under the same token budget, yielding improved performance (+8.6%) and inference speedup (5.6x for prefilling) on Qwen2.5VL-7B.
Jiameng Li, Minye Wu, Jiezhang Cao +2
KU Leuven · Shanghai Jiaotong University · Weill Cornell Medicine