cs.CVNov 11, 2026

Honest Fusion Evaluation for Cross-Domain Deepfake Detection: Leakage Audits and Tail Robustness

Authors: Luke Collins, Chang-Tsun Li, Duc Thanh Nguyen

Organizations: School of Information Technology, Deakin University, Waurn Ponds, Australia

Abstract

Cross-domain video deepfake detection increasingly uses frozen foundation-model experts, combined by calibrated score averaging, a per-video router picking one expert, or a learned head over concatenated features. Expressive routing and feature fusion are often reported to beat plain averaging. Across a 22-dataset frozen-expert testbed, we find those gains are largely evaluation artifacts. A corner theorem locates the gap routing aims to close: because AUC depends only on score ordering, the best any score-space router can do is an oracle reading the test label to pick each video's expert, a ceiling no realisable, label-blind router reaches. The theorem does not order real routers against a calibrated global average; the case against routing is empirical, and no router we test beats that average. Feature fusion reads raw features, falls outside the theorem, and is audited empirically. Reported gains trace to leakage: holding out one dataset leaks siblings sharing identities and reals (over-crediting routing), while holding out one manipulation keeps that dataset's reals in training (over-crediting feature fusion). A 2×22\times 2 over fold and evaluation granularity pins the +2.6+2.6pp feature-fusion "win" to the fold alone: 12/12 significant under leaky folds vs. 0/12 under honest family-held-out folds, where both gains vanish. Fusion still pays off in the tail of hard videos and families: as experts are added, videos every expert misclassifies drop from 15.2% to 2.3%, and global calibrated fusion (InvB) beats the best single expert on worst-family AUC, selective risk (AURC), and low-false-positive recall, stably across protocols. Practitioners should calibrate and combine globally, hold out whole dataset families, and measure tail performance. For score-space routing under ranking metrics, honest evaluation and expert diversity matter more than combiner complexity.

Figures & tables

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. When Are Predictions Enough? An Evaluation Protocol for Frozen Expert Composition

    Nov 9, 2026Alessandro Pereira, Keith J Ransom, Lewis MitchellEnsemble LearningStacked Ensemble Learning

  2. DFD-Lab: A Modular Audio-Visual Deepfake Detection Pipeline

    Sep 20, 2026Jan Rybarczyk, Mateusz Roszkowski, Jacek KomorowskiAudio Deepfake DetectionDeepfake Detection

  3. MoSTE: Mixture of Spatio-Temporal Experts for Deepfake Detection in Videos

    Nov 9, 2026Charbel Yahchouchi, Nabyl Quignon, Noemi Roggero +2Deepfake DetectionMixture of Experts