Intersectional Invariant Learning for Fair Deepfake Detection

Uzoamaka Ezeakunne; Taiwo Olabamipe; Deevesh Jain; Xiuwen Liu

cs.CVOct 27, 2026

Intersectional Invariant Learning for Fair Deepfake Detection

Authors: Uzoamaka Ezeakunne, Taiwo Olabamipe, Deevesh Jain, Xiuwen Liu

Organizations: Department of Computer Science, Florida State University · Department of Computer Science, University of Central Florida

Abstract

Deepfake detection models achieve high overall accuracy but exhibit large performance disparities across demographic groups, causing some groups to experience substantially higher false-positive rates than others. Existing fairness-aware methods address this by constraining model outputs to equalize group predictions, but do not tackle the underlying feature-level bias. As a result, models may still rely on demographic characteristics, such as skin texture, as indirect cues for detecting manipulated content. To address this problem, we propose Intersectional Invariant Learning (IIL), a framework that improves feature learning rather than only adjusting model outputs. IIL alternates between identifying the most challenging demographic partitions and learning features that remain consistent across those partitions. The framework also includes a false-positive-rate fairness objective and group-balanced sampling across six gender x skin-tone intersectional groups. We evaluate IIL on a large-scale fairness benchmark containing over one million images, 10 baseline detectors, and 21 fairness and utility metrics. Results show that IIL achieves Pareto-optimal performance, simultaneously attaining an AUC above 99%, a false-positive rate under 2.5%, and the lowest intersectional fairness disparity (FMEO) among all compared methods. IIL outperforms prior fairness-aware methods on the large majority of the 21 evaluation metrics, while reducing both false-positive rates and intersectional fairness disparities by several-fold compared with the strongest competing method.

Figures & tables

Component	Prior work (INV-REG) [ 22 ]	IIL
Step 1: REx [ 15 ]	$\checkmark$	$\checkmark$
Step 2: IRMv1 [ 2 ]	$\checkmark$	$\checkmark$
Classification loss $\mathcal{L}_{\text{cls}}$	Supcon. [ 14 ]	BCE
Partition level	Identity	Gender ${\times}$ Skin-tone
$K$ (partitions)	2	6
FPR parity loss	$\times$	$\checkmark$

Table 1: IIL components compared with prior invariant-learning work [ 22 ] .

\checkmark

= present;

\times

= absent.

		Detector
Attr.	Metric	Xception [ 5 ]	EffB4 [ 30 ]	F3Net [ 26 ]	SPSL [ 19 ]	SRM [ 21 ]	UCF [ 36 ]	CORE [ 24 ]	DAW [ 13 ]	DAG [ 13 ]	PG-FDD [ 17 ]	IIL (ours)
Skin Tone	FMEO $\downarrow$	8.836	8.300	19.938	8.055	10.002	17.325	10.779	14.118	6.551	6.465	0.730
	FDP $\downarrow$	9.751	6.184	12.876	9.379	10.897	12.581	10.317	10.706	8.617	9.746	8.468
	FOAE $\downarrow$	1.271	4.377	2.818	1.135	0.915	1.883	1.332	1.667	1.388	0.882	0.770
	FEO $\downarrow$	12.132	11.062	23.708	9.789	14.239	21.920	13.069	16.604	7.383	9.115	1.537
Gender	FMEO $\downarrow$	3.975	5.385	4.717	4.411	6.271	5.074	5.795	5.510	5.910	3.190	0.566

Table 2: Fairness and utility results on the AI-Face test set. Benchmark values from Table 4 of [ 18 ] . All values in %.

\downarrow

lower is better;

\uparrow

higher is better. Bold = best across all 11 detectors. EffB4 = EfficientNet-B4; DAW = DAW-FDD; DAG = DAG-FDD. Gray column = IIL (ours).

Figure 1: Per-group FPR (left) and TPR (right) across the six gender

{\times}

skin-tone demographic partitions (Female/Male

\times

Skin-Tone 1–3) for IIL (ours), EfficientNet-B4, DAG-FDD, and PG-FDD. IIL achieves near-uniform FPR across all six gender

{\times}

skin-tone groups (max 3.4%, range 1.5 pp) compared to EffB4 (max 33.0%, range 24.1 pp), DAG (max 22.8%), and PG-FDD (max 16.9%), while maintaining TPR above 97% for every group. The dominant disparity axis in baselines is skin tone; gender contributes a secondary gap within each skin-tone tier.

Panel of Figure 2 — Figure 2: Fairness–utility analysis.

	Components
Variant	Balanced Sampler	FPR Loss	IRM/ REx	AUC $\uparrow$	FPR $\downarrow$	inter-FMEO $\downarrow$	skin-FMEO $\downarrow$	$S$ $\downarrow$
A	$\times$	$\times$	$\times$	98.23	12.18	17.18	6.58	54.12
B	$\checkmark$	$\times$	$\times$	97.98	12.74	14.32	5.63	44.43
C †	$\checkmark$	$\checkmark$	$\times$	93.94	100.0	—	—	—
D	$\checkmark$	$\checkmark$	$\checkmark$	98.26	9.15	6.49	1.98	24.43
E ( $P_{\mathrm{fix}}$ )	$\checkmark$	$\checkmark$	$\checkmark$	98.22	9.68	6.97	3.39	26.62

Table 3: Component ablation (15-epoch run, 50 000 training samples, paper hyperparameters).

\checkmark

= enabled;

\times

= disabled; P fix = fixed identity partition instead of learned.

S

= fairness score (Eq. 3 ;

\downarrow

). Bold = best non-collapsed value.

\dagger

collapsed:

\mathrm{FPR}{=}100\%

, all images predicted fake.

Figure 3: UMAP [ 23 ] embeddings for EffB4, DAG, PG-FDD, and IIL (ours). Top row : colored by Real/Fake label. EffB4 spreads into multiple elongated thread-like strands; DAG produces two large compact blobs; PG-FDD yields serpentine filamentous curves; IIL achieves the tightest compact clusters with the sharpest Real/Fake boundary. All four models achieve clear Real/Fake separation, consistent with AUC values above 98%. Bottom row : colored by intersectional demographic group.

Figure 4: GradCAM [ 29 ] attention maps across the six gender ${\times}$ skin-tone groups and four models. Left block: real images; right block: fake images.

	Celeb-DF			DFD			DFDC			FF++
Model	AUC $\uparrow$	FPR $\downarrow$	FMEO $\downarrow$	AUC $\uparrow$	FPR $\downarrow$	FMEO $\downarrow$	AUC $\uparrow$	FPR $\downarrow$	FMEO $\downarrow$	AUC $\uparrow$	FPR $\downarrow$	FMEO $\downarrow$
Xception [ 5 ]	92.85	28.75	87.50	85.42	31.40	16.80	75.24	13.56	61.35	86.57	17.33	82.18
EffB4 [ 30 ]	71.85	85.34	21.74	72.98	87.11	16.69	64.90	55.06	81.71	71.55	35.04	75.79
F3Net [ 26 ]	89.06	9.26	93.33	81.82	16.30	23.75	73.54	9.03	47.46	81.84	6.03	49.89
SPSL [ 19 ]	93.46	14.47	87.50	84.25	51.62	10.18	76.66	15.16	68.12	90.66	17.12	69.09
SRM [ 21 ]	87.64	5.40	37.50	84.60	19.76	12.91	81.67	16.49	71.43	81.75	26.22	59.49

Table 4: Generalization across datasets. Evaluated on AI-Face demographically annotated subsets [ 18 ] (8K–36K images per dataset). AUC

\uparrow

and FPR

\downarrow

(%); FMEO

\downarrow

(%) is the intersectional Max Equalized Odds gap across the six gender

{\times}

skin-tone groups, as defined in § 3.1 . Bold = best per metric.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

		Detector
Attr.	Metric	Xception	EffB4	F3Net	SPSL	SRM	UCF	CORE	DAW	DAG	PG-FDD	IIL
Skin Tone	FMEO $\downarrow$	8.851	8.276	19.935	8.353	9.998	17.321	10.775	14.122	6.555	6.468	0.730
	FDP $\downarrow$	9.759	6.195	12.876	9.479	10.894	12.582	10.318	10.707	8.618	9.747	8.468
	FOAE $\downarrow$	1.270	4.367	2.814	1.163	0.911	1.878	1.329	1.669	1.388	0.881	0.770
	FEO $\downarrow$	12.144	11.061	23.703	10.049	14.233	21.921	13.068	16.607	7.389	9.120	1.537
Gender	FMEO $\downarrow$	3.978	5.366	4.707	4.488	6.264	5.076	5.805	5.507	5.912	3.188	0.566

Appendix

Table 5: Full-test-split re-evaluation of all ten baselines and IIL, same layout as Table 2. Bold = best per row. Deviation from Table 2’s published baseline values (210 cells: 10 baselines

\times

21 metrics) is rounding-level throughout: max

|\Delta|

= 0.772 pp (Intersection FEO, SPSL), mean

|\Delta|

= 0.026 pp overall. Per-attribute max/mean

|\Delta|

: Skin Tone 0.298/0.021, Gender 0.077/0.009, Age 0.615/0.053, Intersection 0.772/0.036 (N=40 each), Utility 0.027/0.003 (N=50). No cell deviates by more than 0.8 pp, confirming rounding-level agreement with the published values.

Panel of Figure 5 — Figure 5: FPR (left) and fairness score $S$ (right) across the full $8\times 5$ grid at $K{=}6$ /20k samples/8 epochs. Black box = selected point $(\lambda_{\text{IRM}}{=}0.07,\lambda_{\text{REx}}{=}5.0)$ .

$(\lambda_{\text{IRM}},\lambda_{\text{REx}})$	$S\downarrow$	AUC $\uparrow$	ACC $\uparrow$	FPR $\downarrow$	inter	skin	gen	age
(0.07, 5.00)	45.5	95.49	90.58	18.28	10.89	5.09	4.96	8.62
(0.06, 3.00)	50.3	96.24	90.94	18.90	11.71	5.70	3.58	11.86
(0.05, 4.00)	52.3	96.32	91.10	14.18	13.14	6.52	1.11	11.87
(0.01, 2.00)	55.6	95.93	90.48	16.17	13.65	7.92	2.66	9.82
(0.02, 5.00)	58.1	95.44	90.66	19.75	14.27	7.10	3.79	11.55
(0.01, 4.00)	66.6	96.22	91.40	18.23	23.08	4.48	2.58	8.89

Appendix

Table 6: Full

8\times 5

sweep grid at

K{=}6

/20k samples/8 epochs, all 40 combinations, sorted by fairness score

S

(Eq. 3; bold = selected point, lowest

S

). AUC/ACC/FPR in %; FMEO columns in % (inter/skin/gen/age, left to right).

Figure 6: IIL training curves over 15 epochs (

\lambda_{\text{IRM}}{=}0.07

\lambda_{\text{REx}}{=}5.0

K{=}6

). Left: FPR. Centre: inter- and skin-FMEO. Right: AUC. Star = best checkpoint (epoch 13,

S{=}6.09

Figure 7: Variant C (FPR+DP loss, no IRM/REx) at small (50k samples) and full deployed (300k samples) scale, per-epoch over 15 epochs. Left: FPR — collapsed (100%) at every epoch, at both scales. Right: AUC — rises with training at both scales despite the FPR collapse, since AUC measures ranking quality of raw scores, not the collapsed classification threshold. Fairness score

S

and FMEO are omitted: both are trivially 0 under complete FPR collapse (uniform failure, not a real parity result), matching Table 3’s “—” convention for this variant.

Partition	Child	Young	Adult	Middle	Senior
F, skin 1	9.18	35.11	42.61	10.66	2.44
F, skin 2	4.98	25.69	48.78	18.38	2.18
F, skin 3	4.58	20.01	53.06	18.74	3.60
M, skin 1	8.34	20.25	44.94	20.45	6.02
M, skin 2	2.38	10.08	49.08	33.76	4.70
M, skin 3	1.28	8.18	50.92	35.52	4.10

Appendix

Table 7: Row-normalized Age-bracket composition (%) within each of the six gender

\times

skin-tone partitions used by

\mathcal{L}_{\text{FPR}}

\mathcal{L}_{\text{DP}}

/IRM. Each row sums to 100%.

Model	Intersection (6-way)	Skin (3-way)	Gender (2-way)
Majority-class baseline	17.0%	33.9%	50.7%
EffB4	55.1 $\pm$ 1.0%	75.6 $\pm$ 0.7%	71.7 $\pm$ 1.3%
DAG-FDD	37.7 $\pm$ 1.1%	62.9 $\pm$ 1.2%	61.5 $\pm$ 2.0%
PG-FDD	22.1 $\pm$ 1.2%	41.9 $\pm$ 1.0%	54.4 $\pm$ 1.5%
IIL (ours)	41.1 $\pm$ 0.6%	60.1 $\pm$ 0.8%	66.3 $\pm$ 0.9%

Appendix

Table 8: Demographic-group classifiability from learned features (accuracy, mean

\pm

std over 5 folds; lower

=

more invariant). Majority-class baseline given for reference.

Deepfake Detection

Deepfake detection identifies manipulated or AI-generated media to support trustworthy communication, identity verification, and forensic analysis.

See all 246 papers

Toward Calibrated, Fair, and accurate Deepfake Detection
Jun 3, 2026Ryan Brown, Chris RussellAlgorithmic Fairness Deepfake Detection
University of Oxford
FairReL: Deepfake Detection using Fairness-Aware Representation Learning
Aug 28, 2026Xiaoman Lu, Jiaqi Li, Shuntian Zheng +2Deepfake Detection Algorithmic Fairness
Department of Computer Science University of Warwick Coventry, United Kingdom · School of Computer Science University of Birmingham Birmingham, United Kingdom
FYC: FPR-Disparity Weight Perturbation and Youden Calibration for Fair Deepfake Detection
Oct 27, 2026Uzoamaka Ezeakunne, Taiwo Olabamipe, Deevesh Jain +1Deepfake Detection Algorithmic Fairness
Department of Computer Science, Florida State University · Department of Computer Science, University of Central Florida

Jun 3, 2026J/K move · Enter open · S save

Toward Calibrated, Fair, and accurate Deepfake Detection

Ryan Brown, Chris Russell

University of Oxford

Algorithmic Fairness Deepfake Detection Model Calibration

Open paper AskPDF arXiv

Deepfake detectors show large performance gaps across demographic groups. Existing fairness approaches require demographic labels, retraining, or sacrifice accuracy. We introduce Face-Fairness (FF), a plug-and-play framework for bias mitigation. Our primary contribution, Face-Feature Tuning (FFT), is the first demographic label-free fairness method demonstrated for deepfake detection: a lightweight calibrator that performs a logit remapping conditioned on frozen face embeddings. We complement FFT with two variants: FF-Max, which maximizes worst-group accuracy when demographics are available, and FF-Discover, which does the same with embedding-discovered groups. Across in-domain and cross-dataset test settings, FF consistently reduces FPR/TPR gaps and improves minimum group accuracy while maintaining (often improving) overall accuracy. The approach is detector-agnostic, adds negligible runtime overhead, and requires no access to identity attributes.

Abstract

Deepfake Detection

Explore similar work

Toward Calibrated, Fair, and accurate Deepfake Detection

FairReL: Deepfake Detection using Fairness-Aware Representation Learning

FYC: FPR-Disparity Weight Perturbation and Youden Calibration for Fair Deepfake Detection