Video large language models (VLLMs) rely on long visual token sequences to understand complex temporal content, which makes visual token compression important for efficient inference. Existing training-free compression methods mainly reduce the number of tokens sent to the language model, but the compression process itself can still require many visual feature comparisons, especially for intra-frame token protection and inter-frame temporal token merging. This paper proposes a video codec-guided training-free token compression method for VLLMs that uses coding information already available in compressed video as a low-cost search prior. The method aligns reference-frame dependencies, motion vectors, and residuals with the visual token grid. Residual information forms compact candidate pools for token protection, while motion information is propagated along reference-frame chains to construct localized candidates for temporal token merging between sparsely sampled frames. Local visual feature similarity comparison is then used within these candidates to make the final compression decisions, without modifying the base VLLM. Experiments on three representative VLLMs and four video-understanding benchmarks show that the proposed method achieves performance comparable to the state-of-the-art training-free baseline FlashVID, and in some settings surpasses it, while substantially reducing the feature comparisons required by token protection and temporal matching. These results demonstrate that codec information can effectively improve the accuracy-complexity tradeoff of video token compression.
Figures & tables
Fig. 1: Illustration of video codec information extraction and codec-guided local token matching. (a) Video codec information extraction. (b) Global visual token matching and codec-guided local token matching.
Fig. 2: Comparison of the performance and feature-comparison costs of our method and the state-of-the-art baseline FlashVID [ 13 ] .
Fig. 3: Overview of our proposed video codec-guided training-free token compression framework for video large language models.
Fig. 4: Illustration of the procedure for aligning block-level coding information to the visual token grid.
Fig. 5: Illustration of the residual-guided token protection method.
Fig. 6: Illustration of motion-guided temporal merging through reference-chain propagation.
Fig. 7: Subjective comparison results between FlashVID and our method.
ρ
Selection Reduction (%)
Temporal Reduction (%)
Video-MME
25%
79.8
64.7
59.0 (58.1)
20%
87.2
64.7
58.3 (58.0)
15%
93.0
64.7
57.0 (57.5)
10%
96.6
64.7
56.7 (57.4)
TABLE III: Reduction of feature comparisons relative to FlashVID on Video-MME with LLaVA-OneVision. The Video-MME column reports our score, with the FlashVID score shown in parentheses.
Method
Short
Medium
Long
Overall
Motion-Guided Only
71.4
56.5
48.9
59.0
Residual-Guided Only
70.3
55.8
49.0
58.4
Combined
70.4
56.8
49.8
59.0
TABLE IV: Effectiveness of motion and residual guidance on Video-MME with LLaVA-OneVision ( ρ=25% ).
α
Short
Medium
Long
Overall
0
70.9
55.9
48.9
58.6
0.1
70.9
55.4
49.3
58.6
0.2
70.4
56.6
49.8
59.0
TABLE V: Effectiveness of temporal residual increase on Video-MME with LLaVA-OneVision ( ρ=25% ).
r
Short
Medium
Long
Overall
1
70.3
56.3
49.7
58.8
2
70.4
56.8
49.8
59.0
3
70.3
55.6
49.9
58.6
4
70.9
56.8
49.8
59.1
TABLE VI: Effectiveness of the local search radius on Video-MME with LLaVA-OneVision ( ρ=25% ).
Fig. 8: Effectiveness of temporal residual increase and local search radius. The top part shows the protected tokens under different temporal residual increase weights. Adjusting α changes the protected tokens while keeping the protection budget fixed. The bottom part shows the candidate regions for local matching under different search radii r .