PATE-Forensics: Perception-as-Tool for Explainable Deepfake Forensics with General-Purpose MLLMs

Yaqi Li; Jielun Peng; Yabin Wang; Jincheng Liu; Xiaopeng Hong

cs.CVAug 19, 2026

PATE-Forensics: Perception-as-Tool for Explainable Deepfake Forensics with General-Purpose MLLMs

Authors: Yaqi Li, Jielun Peng, Yabin Wang, Jincheng Liu, Xiaopeng Hong

Organizations: Harbin Institute of Technology

Abstract

Existing explainable deepfake forensic methods typically rely on task-adapted MLLM to jointly address detection, localization, and explanation. Inspired by agent-style tool use, we instead introduce a Perception-as-Tool paradigm and instantiate it as PATE-Forensics, which architecturally decouples detection and localization from explanation generation while coupling detection and localization as tightly as possible within a forensic perception tool. The DINOv3-based tool couples a multi-granularity detection module that integrates global, patch-level, and segment-level evidence with a cue-guided localization module by spatializing the patch-level and segment-level evidence into forgery score maps that guide dense mask prediction. The original image and forensic perception outputs produced by the tool form structured forensic context for a general-purpose MLLM, which is guided by prompt constraints to generate explanations without task-specific fine-tuning. On DDL-X Track 3, PATE-Forensics achieves the best official score of 0.89, outperforming the second-ranked team by 0.19 points. Our code is available at https://github.com/yqli00000/PATE-Forensics.

Figures & tables

Figure 1: Comparison with existing method jointly addressing detection, localization, and explanation. Existing methods rely on task-adapted MLLMs, either using MLLM outputs to guide dedicated detection and localization modules or directly producing all three task outputs. In contrast, PATE-Forensics keeps a general-purpose MLLM frozen and externalizes detection and localization as a forensic perception tool, whose outputs, together with the original image, form the structured forensic context for explanation generation.

Figure 2: Overview of PATE-Forensics. Following the Perception-as-Tool paradigm, a forensic perception tool performs detection and localization through two coupled modules. The global, patch, and segment branches provide multi-granularity evidence for fake/real detection, while the cue-guided localization module fuses detection-stage patch and segment score maps with multi-layer DINOv3 features to predict a dense forgery mask. The original image and forensic perception outputs form structured forensic context for a general-purpose MLLM. Guided by prompt constraints, the MLLM generates the final explanation without task-specific fine-tuning.

Rank	Team	Score	ACC	IoU	BERT	Rub.
1	HIT_VIRLAB	0.8940	0.1995	0.3079	0.0943	0.2923
2	CBSR-WTD	0.7025	0.1776	0.2495	0.0813	0.1941
3	Track Killer	0.6857	0.1562	0.2363	0.0851	0.2081
4	hellohello	0.6181	0.1601	0.2200	0.0769	0.1610
5	XQ999	0.5770	0.1601	0.1985	0.0585	0.1599

Table 1: Final results compared with other teams on DDL-X Track 3. CBSR-WTD denotes CBSR_what_the_dog_doing.

Figure 3: Qualitative examples of MLLM-based explanation. For the real image (top), no manipulated region is detected. For the fake image (bottom), the predicted bounding-box coordinates localize the manipulated regions. The explanations are abridged for readability.

	Detection Evidence			Localization Input			Metric
Configuration	$z_{cls}$	$p_{agg}$	$s_{agg}$	$F_{1}$ – $F_{4}$	$S_{p}$	$S_{s}$	ACC	IoU
Baseline	✓	✗	✗	✓	✗	✗	0.9914	0.7283
$+$ Patch-level evidence	✓	✓	✗	✓	✗	✗	0.9925	0.7375
$+$ Segment-level evidence	✓	✓	✓	✓	✗	✗	0.9939	0.7410
$+$ Coarse forgery cues	✓	✓	✓	✓	✓	✓	0.9961	0.7910

Table 2: Progressive component-wise ablation of PATE-Forensics. Components are added cumulatively from top to bottom. The baseline uses

z_{cls}

for detection and multi-layer features

F_{1}

–

F_{4}

for localization. The final row denotes the full PATE-Forensics model.

Explore similar work

Look Before You Judge: Training-Free Region Mining for Grounded and Explainable Deepfake Detection
Sep 28, 2026Chia-Ling Chen, Yu-Ting Ta, Jian-Yu Jiang-Lin +8Explainable Deepfake Detection Deepfake Detection
National Taiwan University · National Tsing Hua University · National Yang Ming Chiao Tung University
MSU Team at the Explainable Deepfake Detection Challenge 2026: Grounded Artifact Evidence for Deepfake Detection
Oct 7, 2026Artem Filippov, Aleksandr Gushchin, Kirill Koltsov +2Deepfake Detection Explainable Deepfake Detection
Lomonosov Moscow State University Moscow, Russia · MSU Institute for Artificial Intelligence Moscow, Russia
FORGE: Forensic Reasoning with Grounded Evidence
Mar 20, 2025Rohit Kundu, Shan Jia, Vishal Mohanty +2VLM Adaptation Explainable Deepfake Detection
University of California, Riverside · Google LLC

Sep 28, 2026J/K move · Enter open · S save

Look Before You Judge: Training-Free Region Mining for Grounded and Explainable Deepfake Detection

Chia-Ling Chen, Yu-Ting Ta, Jian-Yu Jiang-Lin, Tai-Ming Huang+7

National Taiwan University · National Tsing Hua University · National Yang Ming Chiao Tung University

Explainable Deepfake Detection Deepfake Detection Visual Grounding VLM Hallucination Mitigation

Open paper AskPDF arXiv

Multimodal large language models (MLLMs) can explain deepfake verdicts in natural language, but such explanations are not necessarily visually grounded in the visual evidence underlying the prediction. A model may describe plausible artifacts inferred from language priors rather than from image evidence. Existing grounding methods improve visual reliance through decoding or attention interventions, but they generally strengthen grounding over the entire image, making them ill-suited for forensic artifacts that are subtle, spatially localized, and image-dependent. We propose Look Before You Judge, a training-free framework that formulates explainable deepfake detection as a sequential evidence acquisition process. Instead of directly predicting image authenticity from holistic visual reasoning, our framework first identifies image-specific candidate evidence regions by contrasting the MLLM's decoder-to-visual attention between an original image and its Gaussian-blurred counterpart. The identified regions are then inspected individually, and the resulting local evidence is integrated with the global image context before reaching a final verdict. The framework operates without manipulation masks, external forensic models, or parameter updates, making it directly applicable to off-the-shelf MLLMs. Across five open-source MLLMs on TriDF and MMTD-Set, our framework improves detection accuracy by up to 12.8%, reduces CHAIR by up to 33.4% and hallucination rate by up to 21.3%, and outperforms representative training-free decoding and attention methods.

Abstract

Explore similar work

Look Before You Judge: Training-Free Region Mining for Grounded and Explainable Deepfake Detection

MSU Team at the Explainable Deepfake Detection Challenge 2026: Grounded Artifact Evidence for Deepfake Detection

FORGE: Forensic Reasoning with Grounded Evidence