Look Before You Judge: Training-Free Region Mining for Grounded and Explainable Deepfake Detection

Chia-Ling Chen; Yu-Ting Ta; Jian-Yu Jiang-Lin; Tai-Ming Huang; Ling Lo; Po-Ching Chen; Yan-Tsung Wang; Pei-Heng Li

cs.CVSep 28, 2026

Look Before You Judge: Training-Free Region Mining for Grounded and Explainable Deepfake Detection

Authors: Chia-Ling Chen, Yu-Ting Ta, Jian-Yu Jiang-Lin, Tai-Ming Huang, Ling Lo, Po-Ching Chen, Yan-Tsung Wang, Pei-Heng Li, +3 more

Organizations: National Taiwan University · National Tsing Hua University · National Yang Ming Chiao Tung University

Abstract

Multimodal large language models (MLLMs) can explain deepfake verdicts in natural language, but such explanations are not necessarily visually grounded in the visual evidence underlying the prediction. A model may describe plausible artifacts inferred from language priors rather than from image evidence. Existing grounding methods improve visual reliance through decoding or attention interventions, but they generally strengthen grounding over the entire image, making them ill-suited for forensic artifacts that are subtle, spatially localized, and image-dependent. We propose Look Before You Judge, a training-free framework that formulates explainable deepfake detection as a sequential evidence acquisition process. Instead of directly predicting image authenticity from holistic visual reasoning, our framework first identifies image-specific candidate evidence regions by contrasting the MLLM's decoder-to-visual attention between an original image and its Gaussian-blurred counterpart. The identified regions are then inspected individually, and the resulting local evidence is integrated with the global image context before reaching a final verdict. The framework operates without manipulation masks, external forensic models, or parameter updates, making it directly applicable to off-the-shelf MLLMs. Across five open-source MLLMs on TriDF and MMTD-Set, our framework improves detection accuracy by up to 12.8%, reduces CHAIR by up to 33.4% and hallucination rate by up to 21.3%, and outperforms representative training-free decoding and attention methods.

Figures & tables

Figure 1: Look Before You Judge. A general MLLM produces an ungrounded explanation and an incorrect authentic verdict. Our framework instead inspects localized candidate regions and aggregates the resulting evidence to reach the correct manipulated verdict.

Figure 2: Overview of Look Before You Judge . Stage 1 derives a blur-sensitive visual prior by contrasting output-to-visual attention between the input image and its blurred counterpart. Stage 2 constructs candidate forensic regions from high-response tokens. Stage 3 performs region-wise inspection with attention steering and aggregates the filtered evidence to produce the final prediction and explanation. The entire framework is training-free and requires only frozen MLLMs.

Method	TriDF					MMTD-Set
	ACC $\uparrow$	Cover $\uparrow$	CHAIR $\downarrow$	Hal $\downarrow$	$F^{0.5}$ $\uparrow$	DeepFake		AIGC-Editing
	ACC $\uparrow$	Cover $\uparrow$	CHAIR $\downarrow$	Hal $\downarrow$	$F^{0.5}$ $\uparrow$	ACC $\uparrow$	F1 $\uparrow$	ACC $\uparrow$	F1 $\uparrow$
Open-source models
InternVL-3.5-8B	0.4176	0.0270	0.9745	1.0000	0.0296	0.5235	0.4936	0.5200	0.4419
InternVL-3.5-8B + Ours	0.5458	0.2239	0.6407	0.7875	0.2564	0.5591	0.6278	0.5260	0.5917
InternVL-3.5-14B	0.4213	0.0395	0.9763	0.9980	0.0217	0.5300	0.5488	0.5105	0.4872

Table 1: Overall Quantitative Comparison. Results across five open-source MLLMs on TriDF and MMTD-Set, with commercial MLLMs reported as performance references.

Method	TriDF
Method	ACC $\uparrow$	Cover $\uparrow$	CHAIR $\downarrow$	Hal $\downarrow$	$F^{0.5}$ $\uparrow$
Vanilla	0.4176	0.0270	0.9745	1.0000	0.0296
+ VCD	0.4241	0.0372	0.9777	1.0000	0.0212
+ AttnReal	0.4206	0.0719	0.9750	0.9993	0.0246
+ Ours	0.5458	0.2239	0.6407	0.7875	0.2564

Table 2: Comparison with Training-Free Grounding. Results on TriDF using InternVL-3.5-8B.

Method	TriDF
Method	ACC $\uparrow$	Cover $\uparrow$	CHAIR $\downarrow$	Hal $\downarrow$	$F^{0.5}$ $\uparrow$
Vanilla	0.4176	0.0270	0.9745	1.0000	0.0296
Prompt-only	0.5113	0.1369	0.8545	0.9927	0.1340
w/o blur contrast	0.4867	0.1920	0.8600	0.9568	0.1132
w/o component steering	0.4884	0.1890	0.8574	0.9548	0.1182
Ours	0.5458	0.2239	0.6407	0.7875	0.2564

Table 3: Ablation Study. Effects of evidence-before-judgment prompting, blur-contrastive region mining, and component-wise steering on TriDF.

Figure 3: Qualitative Spatial Grounding. Candidate regions discovered by Stages 1 and 2 align with annotated tampered areas without localization supervision.

Figure 4: From Global Error to Local Evidence. Top: Vanilla inference overlooks the annotated blending and texture artifacts and incorrectly predicts the image as authentic. Bottom: Our framework discovers three candidate regions, inspects them independently, and aggregates the retained local evidence under a global view to produce the correct manipulated verdict.

Method	TriDF
Method	ACC $\uparrow$	Cover $\uparrow$	CHAIR $\downarrow$	Hal $\downarrow$	$F^{0.5}$ $\uparrow$
Vanilla	0.4176	0.0270	0.9745	1.0000	0.0296
Prompt-only	0.5113	0.1369	0.8545	0.9927	0.1340
Self-Proposed Region Inspection	0.5076	0.0919	0.8870	0.9495	0.0836
Ours	0.5458	0.2239	0.6407	0.7875	0.2564

Table S1: Comparison with Self-Proposed Region Inspection. Results on TriDF using InternVL-3.5-8B.

Method	TriDF
Method	ACC $\uparrow$	Cover $\uparrow$	CHAIR $\downarrow$	Hal $\downarrow$	$F^{0.5}$ $\uparrow$
Vanilla	0.4176	0.0270	0.9745	1.0000	0.0296
+ Facial Regions	0.5176	0.2096	0.6727	0.8014	0.2343
+ Ours	0.5458	0.2239	0.6407	0.7875	0.2564

Table S2: Comparison with Alternative Region Selection Method. Results on TriDF using InternVL-3.5-8B.

Figure S1: Qualitative result on a manipulated image generation example. Vanilla inference overlooks the localized artifacts and predicts the image as authentic. Our framework identifies blur and texture/color inconsistencies in the selected regions and produces the correct manipulated verdict.

Figure S2: Qualitative result on an authentic face. Vanilla inference hallucinates facial artifacts and produces a false positive. Our framework assigns only weak, uncertain, or negative evidence to the inspected regions and recovers the correct authentic verdict.

Figure S3: Qualitative comparison when both methods correctly predict manipulation. Vanilla inference provides generic and weakly supported claims, whereas our framework grounds its explanation in blending and texture irregularities around the hairline and forehead.

Figure S4: Representative failure case on a manipulated outdoor scene. Our framework identifies a relevant background-blur cue, but the cue is also compatible with natural focus variation and lacks sufficient complementary evidence. Both methods therefore predict the image as authentic.

Method	Inference Efficiency
Method	Avg. Invoc. $\downarrow$	Time (s/img) $\downarrow$	Rel. Latency $\downarrow$	Peak GPU Mem. (GB) $\downarrow$
Vanilla	1.000	13.480	1.000 $\times$	16.129
Ours	6.970	19.951	1.480 $\times$	16.624

Table S3: Inference efficiency comparison. inference efficiency on a balanced 100-image subset of TriDF. Both methods use the same InternVL-3.5-8B backbone under identical single-GPU settings. A model invocation denotes either an autoregressive generation or an explicit attention forward evaluation.

Explore similar work

FORGE: Forensic Reasoning with Grounded Evidence
Mar 20, 2025Rohit Kundu, Shan Jia, Vishal Mohanty +2Deepfake Detection Digital Forensics
University of California, Riverside · Google LLC
VIGIL: Part-Grounded Structured Reasoning for Generalizable Deepfake Detection
Mar 23, 2026Xinghan Li, Junhao Xu, Jingjing ChenDeepfake Detection Digital Forensics
College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China.
DF-CBM: Region-Aware Concept Bottleneck Models for Deepfake Detection
Sep 28, 2026Georgios Tsoumplekas, Vazgken Vanian, Alexandros Doumanoglou +4Deepfake Detection Unsupervised Detection
Department of Networks and Digital Media, Kingston University London, UK · Centre for Research and Technology Hellas (CERTH), Thessaloniki, Greece · Department of Computer Science, Kingston University London, UK

Mar 20, 2025, cs.CVJ/K move · Enter open · S save

FORGE: Forensic Reasoning with Grounded Evidence

Rohit Kundu, Shan Jia, Vishal Mohanty, Athula Balachandran+1

University of California, Riverside · Google LLC

Deepfake Detection Digital Forensics Multimodal Contrastive Learning Forge Evidence-Grounded Questions

Open paper AskPDF arXiv

Forensic deepfake analysis demands more than binary classification: investigators need region-grounded natural language explanations they can verify against the image. Multimodal large language models (MLLMs) are a natural fit, but pretrained MLLMs fail systematically, producing globally coherent text that misses the small localized cues defining manipulations. We argue this is an inductive bias problem rather than a capacity issue: the image-text contrastive objective training MLLM visual encoders optimizes for whole-image semantic summaries, not patch-level forensic detail. The same mismatch explains why prior deepfake reasoning methods target either face manipulation or fully AI-generated content, never both. We propose FORGE, which addresses the mismatch by routing a second visual stream into the language model from a Vision-Only Model (VOM) trained on dense patch prediction rather than image-text alignment. The MLLM's native encoder and the VOM operate on a shared patch grid, which lets us interleave their tokens with preserved spatial correspondence; we show this beats naive concatenation. A two-stage adapter training protocol (generic image-caption alignment, then joint task-specific optimization) prevents the localized stream from overfitting to training-domain manipulations. Across face-manipulated and fully synthetic content, FORGE produces region-referential explanations answering fine-grained attribute queries ("Does the eyes/nose/mouth look real or fake?") and substantially outperforms in-domain baselines on cross-domain evaluations; region-specific evaluation and human studies confirm explanation faithfulness.

Abstract

Explore similar work

FORGE: Forensic Reasoning with Grounded Evidence

VIGIL: Part-Grounded Structured Reasoning for Generalizable Deepfake Detection

DF-CBM: Region-Aware Concept Bottleneck Models for Deepfake Detection