ForensicZip: More Tokens are Better but Not Necessary in Forensic Vision-Language Models

Yingxin Lai; Zitong Yu; Jun Wang; Linlin Shen; Yong Xu; Xiaochun Cao

cs.CVMar 12, 2026

ForensicZip: More Tokens are Better but Not Necessary in Forensic Vision-Language Models

Authors: Yingxin Lai, Zitong Yu, Jun Wang, Linlin Shen, Yong Xu, Xiaochun Cao

Organizations: Great Bay University · Shenzhen University · Harbin Institute of Technology · School of Cyber Science and Technology, Sun Yat-sen University

Abstract

Multimodal Large Language Models (MLLMs) enable interpretable multimedia forensics by generating textual rationales for forgery detection. However, processing dense visual sequences incurs high computational costs, particularly for high-resolution images and videos. Visual token pruning is a practical acceleration strategy, yet existing methods are largely semantic-driven, retaining salient objects while discarding background regions where manipulation traces such as high-frequency anomalies and temporal jitters often reside. To address this issue, we introduce ForensicZip, a training-free framework that reformulates token compression from a forgery-driven perspective. ForensicZip models temporal token evolution as a Birth-Death Optimal Transport problem with a slack dummy node, quantifying physical discontinuities indicating transient generative artifacts. The forensic scoring further integrates transport-based novelty with high-frequency priors to separate forensic evidence from semantic content under large-ratio compression. Experiments on deepfake and AIGC benchmarks show that at 10% token retention, ForensicZip achieves $2.97\times$ speedup and over 90% FLOPs reduction while maintaining state-of-the-art detection performance.

Figures & tables

Figure 1: Quantitative analysis of forensic-semantic misalignment. (a) Recall of retained tokens against ground-truth forgery regions under varying compression ratios. Semantic-driven criteria show high sensitivity to compression ratios, while our transport-based novelty score maintains stable coverage of localized anomalies. Shaded regions denote

\pm 1

standard deviation across evaluation samples. (b) Distribution of temporal Optimal Transport (OT) costs. The distinct high-cost tail in forged videos indicates physical discontinuities (birth-death events), supporting the use of our augmented OT formulation.

Figure 2: Empirical motivation for ForensicZip. (a) Inverse Correlation: Density plot of semantic saliency (cross-modal attention) vs. forensic evidence (IoU with forgery masks) on FakeClue/LOKI. (b) Temporal Discontinuity: KDE of inter-frame matching costs.

Figure 3: Overall framework of ForensicZip. ForensicZip measures transport novelty across adjacent video frames and selectively preserves vision tokens with high physical inconsistency (i.e., transient artifacts), thereby achieving plug-and-play forensic MLLM inference acceleration.

Method	Backbone: FakeVLM				Backbone: FakeShield				Avg. FLOPs(T)	Avg. Lat.(ms)	GPU Mem.(GB)	Speedup
Method	FakeClue	LOKI	DD-VQA	DMimage	PhotoShop	DeepFake	AIGC	DFFD	Avg. FLOPs(T)	Avg. Lat.(ms)	GPU Mem.(GB)	Speedup
Upper Bound (100% Tokens)
Vanilla	98.60	84.30	93.20	94.00	94.43	98.00	93.00	98.00	3.102	4520.2	26.7	1.00 $\times$
Retain 50% Tokens ( $\downarrow$ 50% visual tokens)
FastV (ECCV’24)	95.23	78.15	89.52	88.24	90.15	93.42	88.54	93.22	2.150	3228.6	26.6	1.40 $\times$
SparseVLM (ICML’25)	96.14	79.53	90.45	89.66	91.24	94.15	89.23	94.44	2.155	3180.4	26.5	1.42 $\times$

Table 1: Quantitative Performance of ForensicZip on FakeVLM and FakeShield Backbones. “Vanilla” denotes the full-token upper bound. Avg. Lat. (ms) is end-to-end per-sample inference latency, GPU Mem. (GB) is peak GPU memory, and Speedup is w.r.t. Vanilla; efficiency metrics are averaged over all datasets in this table. ( Red text indicates performance collapse.)

Methods	LLM Generation $\downarrow$	Model Generation $\downarrow$	Total $\downarrow$	GPU Peak $\downarrow$	Throughput $\uparrow$	Performance $\uparrow$
Methods	Latency (s)	Latency (s)	Latency (min:sec)	Memory (GB)	(samples/s)	Acc (%)
FakeVLM-7B (Vanilla)	1472.5	1478.6	67:59	26.7	1.23	98.60
Retention Ratio = 25%
FastV [ECCV’24]	885.2 ( $\downarrow$ 39.9%)	892.4 ( $\downarrow$ 39.6%)	41:30 ( $\downarrow$ 38.9%)	26.5 ( $\downarrow$ 0.7%)	2.00 ( $\uparrow$ 1.63 $\times$ )	82.34 ( $\downarrow$ 16.2)
SparseVLM [ICML’25]	870.5 ( $\downarrow$ 40.9%)	878.2 ( $\downarrow$ 40.6%)	40:55 ( $\downarrow$ 39.8%)	26.6 ( $\downarrow$ 0.4%)	2.03 ( $\uparrow$ 1.65 $\times$ )	85.63 ( $\downarrow$ 12.9)
LLaVA-Scissor [NeurIPS’25]	852.1 ( $\downarrow$ 42.1%)	860.5 ( $\downarrow$ 41.8%)	40:10 ( $\downarrow$ 40.9%)	26.5 ( $\downarrow$ 0.7%)	2.07 ( $\uparrow$ 1.68 $\times$ )	91.45 ( $\downarrow$ 7.1)

Table 2: Efficiency comparisons on the FakeClue dataset. LLM Generation Latency: time for LLM-only response generation; Model Generation Latency: time for the full model to generate response; Total Latency: total time to complete the FakeClue benchmark; GPU Peak Memory: maximum VRAM usage during inference; and Throughput: number of samples processed per second. Comparisons are conducted at 25% token retention .

Backbone	Method	#Tokens	LLM Gen. Lat. (s) $\downarrow$	Total Lat. (min:sec) $\downarrow$	Peak Mem. (GB) $\downarrow$	Throughput (samples/s) $\uparrow$	FLOPs (T) $\downarrow$	Acc. $\uparrow$
LLaVA-1.5-7B	Vanilla	576	618.0	21:43 (1.00 $\times$ )	18.5	0.64	3.8	85.9
	+ SparseVLM (ICML’25)	64	512.4	18:24 (1.18 $\times$ )	20.2	0.76	1.3	75.1
	+ VisionTrim (ICLR’26)	64	448.5	16:10 (1.34 $\times$ )	19.1	0.86	1.0	84.5
	+ ForensicZip (Ours)	64	412.0	14:52 (1.46 $\times$ )	16.8	0.94	0.8	86.1
LLaVA-NeXT-7B	Vanilla	2880	1008.4	38:04 (1.00 $\times$ )	46.2	0.35	12.6	86.3
	+ SparseVLM (ICML’25)	320	846.2	31:43 (1.20 $\times$ )	48.5	0.42	2.5	78.5

Table 3: Comprehensive efficiency and performance results on the POPE [ Li-hallucination-2023 ] benchmark. Comparison of generation latency, end-to-end time, peak GPU memory, throughput, and accuracy on a single NVIDIA A100 GPU.

Table 7

Figure 4: Ablation Study. Impact of (a) joint weights $\lambda/\eta$ , (b) regularization $\varepsilon_{\mathrm{ot}}$ , (c) penalty $c$ , and (d) iterations on FakeClue accuracy.

Visual Token Pruning

Visual token pruning reduces the image and video patches processed by transformers to lower inference cost while preserving task-relevant information.

See all 276 papers

ForensicsTok: Forensics-Guided Tokenized Modeling for Image Tampering Localization
Jun 23, 2026Lei Xu, Haowei Wang, Shen Chen +3Large Vision-Language Models Image Forgery Localization
1Shenzhen University · 2Tencent Youtu Lab
Rescuing Your Pruning: Rethinking Visual Token Pruning and Post-Training Recovery in MLLMs
Oct 24, 2026Peiran Yin, Hao Wu, Zihao Yang +3Visual Token Pruning Efficient VLM Inference
EIT-NLP Lab, Eastern Institute of Technology, Ningbo · Shanghai Jiao Tong University · Uppsala University
FORGE: Forensic Reasoning with Grounded Evidence
Mar 20, 2025Rohit Kundu, Shan Jia, Vishal Mohanty +2VLM Adaptation Explainable Deepfake Detection
University of California, Riverside · Google LLC

Jun 23, 2026J/K move · Enter open · S save

ForensicsTok: Forensics-Guided Tokenized Modeling for Image Tampering Localization

Lei Xu, Haowei Wang, Shen Chen, Taiping Yao+2

1Shenzhen University · 2Tencent Youtu Lab

Large Vision-Language Models Image Forgery Localization

Open paper AskPDF arXiv

Multi-modal Large Language Models (MLLMs) offer powerful reasoning for forensic tasks, yet existing approaches utilizing exogenous segmentation decoders often suffer from suboptimal localization. The reliance on stitched pipelines introduces information bottlenecks during backpropagation, which dilutes spatial signals and is limited by semantic priors of the segmentor. To address these limitations, we propose ForensicsTok, which reformulates image manipulation localization as an autoregressive sequence generation task. ForensicsTok directly generates spatially grounded token sequences, enabling precise mask prediction without intermediary supervision. Specifically, we introduce a Token Splatting Decoder (TSD) to map tokens to binary masks via codebook-aware code smoothing, which mitigates sharp gradients from deterministic detokenizers. Furthermore, to capture diverse tampering clues, we propose a Hierarchical Expert Fusion (HEF) module that injects multi-scale features from a forensic expert model. This unified architecture effectively compensates for the lack of forensic priors in standard MLLMs. Extensive experiments on six benchmarks show that ForensicsTok substantially improves over existing MLLM-based baselines and slightly improves over strong forensic expert baselines, while exhibiting stronger robustness to perturbations.

Abstract

Visual Token Pruning

Explore similar work

ForensicsTok: Forensics-Guided Tokenized Modeling for Image Tampering Localization

Rescuing Your Pruning: Rethinking Visual Token Pruning and Post-Training Recovery in MLLMs

FORGE: Forensic Reasoning with Grounded Evidence