Automated detection has become the essential safeguard against realistic synthetic media, but deploying it at scale requires training on sensitive data from multiple organizations, putting privacy and the transparency demanded by forensic review in direct conflict. Federated learning resolves the privacy side by sharing only model updates rather than raw data, but whether this comes at the cost of trustworthy gradient-based explanations, which analysts rely on to decide whether or not to trust a flagged clip, has not been studied. We present the systematic study of Grad-CAM explanation faithfulness under federated deepfake detection, evaluating an EfficientNet-B4 detector on FaceForensics++ across five federated clients under varying degrees of data heterogeneity. We find that the relationship between federation and explanation quality is non-monotonic: at near-IID (Independent and Identically Distributed Data) and moderate heterogeneity, federated models match or exceed the faithfulness of a centralized baseline, but at extreme heterogeneity, explanations break down entirely, becoming spatially incoherent and, on average, causally misleading. This degradation is not a general cost of federated training but a threshold effect tied to client heterogeneity, and we offer practical guidance for identifying when privacy-preserving deepfake detection can be deployed safely and when its explanations can no longer be trusted. Code and data partitions are available for reproducibility.
Figures & tables
Research Pillar
Established
Missing
Federated Learning [ 4 , 5 , 6 , 7 , 8 ]
FedAvg, FedProx, convergence analysis, weight divergence under non-IID data, Dirichlet α as the standard heterogeneity benchmark
Effect of gradient divergence on explanation faithfulness remains unexplored
Deepfake Detection [ 9 , 10 , 11 , 12 ]
High-performance detectors with Grad-CAM, SHAP, adversarial robustness, and centralized explainability
Federated implementation and evaluation of explanation reliability
XAI Faithfulness [ 2 , 3 , 13 , 14 ]
Grad-CAM, AOPC, sanity checks, and saliency reliability provide objective explanation evaluation
No application of these metrics to federated deepfake detection under statistical heterogeneity
Federated XAI [ 15 , 16 ]
Explainability integrated into FL for credit assessment and intrusion detection
High-dimensional visual forensics, quantitative comparison between centralized and federated explanations, and heterogeneity-aware faithfulness evaluation
TABLE I: Summary of the Converging Literatures
Fig. 1: Mean AOPC for each configuration (centralized, α=1.0,0.5,0.3,0.1 ), with error bars showing the standard deviation.
Fig. 2: Qualitative Grad-CAM comparison. The centralized baseline (center) yields coherent focal activations, whereas the federated model at extreme heterogeneity ( α=0.1 , right) produces diffuse, spatially incoherent heatmaps.
Fig. 3: Explanation IoU across communication rounds for α∈{0.1,0.3,0.5,1.0} .
Synthetic facial imagery has made the distinction between authentic and generated content increasingly important for digital forensics and online trust. This study examines an explainable deep-learning workflow for image-level deepfake detection using two contrasting classification strategies: a compact convolutional model trained from scratch and an ImageNet-pretrained MobileNetV2 model adapted through two-stage transfer learning. Experiments were conducted with the balanced 140k Real and Fake Faces dataset, comprising 140,000 facial images. Under the recorded validation protocol, the augmented from-scratch model reached 84.15% accuracy (loss 0.3625), compared with 95.90% accuracy (loss 0.1074) for the fine-tuned pretrained model. Gradient-weighted class activation mapping (Grad-CAM) was used to visualize image regions associated with predictions, and a Flask-based prototype was developed for side-by-side inspection. These observations support the practical value of pretrained visual representations in the studied setting while underscoring the need to interpret validation performance alongside dataset provenance, explanation limitations, and evaluation scope. The reported results are validation results, not held-out test estimates.
Explainable deepfake detection extends binary classification by requiring models to not only predict authenticity but also provide interpretable justifications. This expanded scope is critical in practice, where users like forensic analysts need insight into the rationale behind the detection. Despite advancements, current approaches suffer from two critical deficiencies: (1)vulnerability to image quality degradation: detection accuracy plummets on low-quality samples, while naive augmentation strategies may induce feature drift and impair performance as diversity expands. (2) factually flawed explanations: explanation models may omit manipulation evidence or hallucinate irrelevant details, undermining interpretability. To address it, we propose a framework with two innovations. For robust deepfake detection, we introduce Feature-robust Augmentation, which comprises diversified degradation-aware augmentation strategies, and a supervised contrastive learning pattern paired with a mean-teacher architecture that stabilizes features against augmentations through consistency constraints. For explanation, we devise an evidence-grounded preference optimization process that guides model to prioritize genuine manipulation traces by learning from chosen-rejected explanation pairs, where rejected samples are constructed via evidence omission or irrelevant information injection. The proposed approach wins the first place in ACM Multimedia 2026 Explainable Deepfake Detection Challenge.The code is available at https://github.com/oceanflowlab/EDD.git.
Zhu Xu, Jiaqi Tang, Pokai Chen +2
Wangxuan Institute of Computer Technology, Peking University Beijing, China
Existing explainable deepfake forensic methods typically rely on task-adapted MLLM to jointly address detection, localization, and explanation. Inspired by agent-style tool use, we instead introduce a Perception-as-Tool paradigm and instantiate it as PATE-Forensics, which architecturally decouples detection and localization from explanation generation while coupling detection and localization as tightly as possible within a forensic perception tool. The DINOv3-based tool couples a multi-granularity detection module that integrates global, patch-level, and segment-level evidence with a cue-guided localization module by spatializing the patch-level and segment-level evidence into forgery score maps that guide dense mask prediction. The original image and forensic perception outputs produced by the tool form structured forensic context for a general-purpose MLLM, which is guided by prompt constraints to generate explanations without task-specific fine-tuning. On DDL-X Track 3, PATE-Forensics achieves the best official score of 0.89, outperforming the second-ranked team by 0.19 points. Our code is available at https://github.com/yqli00000/PATE-Forensics.