Fusion Under Component Failure: Negative Results and Failure Modes in Ensemble AI-Generated Image Detection

Suraj Singh; Tushar Verma; Pragyan Singh; Shaurya Bhav; Shivam Kumar

cs.CVSep 23, 2026

Fusion Under Component Failure: Negative Results and Failure Modes in Ensemble AI-Generated Image Detection

Authors: Suraj Singh, Tushar Verma, Pragyan Singh, Shaurya Bhav, Shivam Kumar

Abstract

We built an ordinary stacking ensemble for AI-generated image detection -- three open detectors producing five scores, fused by a gradient-boosted meta-learner that treats a detector's failure as missing data -- deployed it, and then evaluated it against three controls it should have faced first. This paper reports what the controls found, including where they overturned our own earlier conclusions. Fusion is worth its cost only when refitted on the target domain. The shipped meta-learner, fitted on a separate corpus, does not beat its best single member on 2000 StyleGAN faces (AUC 0.9896 vs 0.9961; McNemar p = 1.000). But a stacker refitted in-domain beats that member plus a post-hoc calibrator (Delta-AUC = +0.0025, [+0.0014, +0.0038]; p = 3.4e-4). An earlier draft claimed the calibrated single detector won outright; that comparison mixed regimes and we correct it here. One corpus is not an evaluation. On 80 screenshots every model's AUC interval contains 0.5. We can say nothing stronger: the difference between the ensemble's drop and its best member's is [-0.185, +0.115]. Abstention is real, correlated, and mishandled. With four of five detectors silent and the survivor reporting "real", the system returns P(AI) = 0.9985, because an all-NaN input scores 0.9995 in a learner never fitted with missingness. The three AIDE checkpoints fail together, sharing one preprocessing path. A quorum rule requiring two distinct architectures prevents both failures with no retraining. We also find Corpus A carries a class-conditional JPEG bias severe enough to separate the classes from the header alone, which limits every in-domain number we report. Code, harness, hash-identified artifacts and all corrections are released.

Figures & tables

Detector	Basis	Scores
Swin-B [ 16 ]	semantic classifier	VQGAN
Nonescape	frozen DINOv2 [ 19 ]	NONESCAPE
	+ EfficientNetV2 attention
AIDE [ 23 ]	SRM residuals [ 6 ]	SD14 , PROGAN ,
	over DCT-selected patches	GENIMAGE

Table 1: The three third-party detectors and the five scores they produce. None is our work; we integrate and evaluate them.

Model	Acc	Prec	Rec	F1	AUC	AP	$t^{\star}$	Acc ⋆
AIDE-SD14	0.7455	0.9168	0.5400	0.6797	0.8821	0.8842	0.154	0.7910
VQGAN/Swin	0.5695	0.6304	0.3360	0.4384	0.6189	0.6089	0.347	0.5895
AIDE-PROGAN	0.9470	0.9923	0.9010	0.9444	0.9961	0.9962	0.322	0.9700
AIDE-GENIMAGE	0.7420	0.9764	0.4960	0.6578	0.9208	0.9254	0.059	0.8535
NONESCAPE	0.7960	0.7139	0.9880	0.8289	0.9567	0.9520	0.881	0.8965
Quorum (shipped)	0.9470	0.9357	0.9600	0.9477	0.9896	0.9899	0.555	0.9515

Table 2: Corpus A (

2000

StyleGAN faces), zero-shot.

t^{\star}

and Acc ⋆ are whole-corpus (oracle) Youden fits , retained only to expose the ranking/calibration gap; they are optimistically biased and are not the basis of any claim. The unbiased nested figures are in § 6 below. No detector abstained.

Figure 1: Corpus A ROC. AIDE-PROGAN (0.9961) sits above the fused curve (0.9896).

Figure 2: Corpus A reliability, including the single detector raw and Platt-scaled.

Model	Regime	AUC	Acc	ECE [95% CI]
AIDE-PROGAN (raw)	zero-shot	0.9961	0.9470	0.1272 [0.1195, 0.1349]
Quorum (shipped)	zero-shot	0.9896	0.9470	0.0416 [0.0348, 0.0512]
AIDE-PROGAN + Platt	in-domain	0.9960	0.9660	0.0319 [0.0272, 0.0392]
AIDE-PROGAN + isotonic	in-domain	0.9946	0.9665	0.0083 [0.0074, 0.0175]
XGBoost (refit)	in-domain	0.9985	0.9805	0.0096 [0.0063, 0.0163]

Table 3: Corpus A. Calibrators are fitted strictly out-of-fold on Corpus A , so they are in-domain models and belong in the lower block. ECE intervals are bootstrap over 2000 image resamples.

Model	A (AUC)	B (AUC)	B 95% CI
AIDE-SD14	0.8821	0.6293	[0.4993, 0.7393]
VQGAN/Swin	0.6189	0.4817	[0.3462, 0.6119]
AIDE-PROGAN	0.9961	0.4563	[0.3311, 0.5857]
AIDE-GENIMAGE	0.9208	0.4227	[0.2894, 0.5584]
NONESCAPE	0.9567	0.4620	[0.3289, 0.5958]
Quorum	0.9896	0.4107	[0.2730, 0.5474]

Table 4: The same fixed models on both corpora. Corpus B intervals are percentile bootstrap over

2000

resamples. Every Corpus B interval contains

0.5

	real (FFHQ)	AI (StyleGAN)
median quant.-table sum	1483	375
median bytes/pixel	0.2399	0.4253
distinct quant. tables	1	1

Table 5: Corpus A JPEG encoding by class, first 250 images each. Lower quantisation sums mean higher quality. Each class carries exactly one distinct quantisation table.

Input	Outcome	Abst.	$P(\text{AI})$
baseline RGB	scored	0/5	0.0313
$1{\times}1$ px	scored, degraded	4/5	0.9985
$4096{\times}3$	scored, degraded	3/5	0.9991
prose as .jpg	all detectors failed	5/5	—
zero bytes	all detectors failed	5/5	—
corrupt CRC	all detectors failed	5/5	—

Table 6: Induced failures.

14

constructed inputs; the table lists the outcomes that were not "scored, nothing abstained".

Case	shipped $P$	live architectures	rule output
$1{\times}1$ px	0.9985	nonescape	abstain
$4096{\times}3$	0.9991	nonescape, swin	0.2535

Table 7: Supervisory rules applied to the two partial-failure cases. Quorum: abstain unless at least two distinct architectures report. Survivor-mean: average the live scores instead of consulting the learner’s defaults.

Substitute	mean $P(\text{AI})$	$\|$ dist. from prior $\|$
all-zero	0.0006	0.4994
all-column-mean	0.5990	0.0990
all- NaN	0.9995	0.4995

Table 8: Corpus A prior is

0.5000

; a policy meaning "no evidence" should land near it.

Rate	Policy	AUC	Acc	mean $P(\text{AI})$
0%	any	0.9985	0.9805	0.4994
10%	native (NaN)	0.9881	0.9375	0.5462
10%	zero-imputed	0.9824	0.9325	0.4472
10%	mean-imputed	0.9953	0.9605	0.5120
25%	native (NaN)	0.9580	0.8625	0.6226
25%	zero-imputed	0.9511	0.8390	0.3497

Table 9: Corpus A. Fixed model trained on complete scores; abstentions injected MCAR at inference.

Rate	Policy	AUC	Acc
0%	any	0.9985	0.9805
10%	native (NaN)	0.9972	0.9720
10%	zero-imputed	0.9970	0.9755
10%	mean-imputed	0.9974	0.9710
25%	native (NaN)	0.9947	0.9575
25%	zero-imputed	0.9936	0.9595

Table 10: Corpus A. Model refitted under each policy at each rate — the like-for-like comparison. Table 9 characterises the shipped artifact; this characterises the design.

Figure 3: Corpus A score distributions per detector, real vs AI, with the

t=0.5

line. AIDE-SD14 and AIDE-GENIMAGE compress both classes below the line — the “sink class” effect — so they appear to fail at

0.5

while still separating the classes.

Rule	Regime	AUC	95% CI	Acc	95% CI	ECE
Best single (AIDE-PROGAN)	untrained	0.9961	[0.9944, 0.9974]	0.9470	[0.9370, 0.9565]	0.1272
Max fusion	untrained	0.9628	[0.9548, 0.9702]	0.7185	[0.6985, 0.7375]	0.2589
Mean fusion	untrained	0.9913	[0.9886, 0.9938]	0.8710	[0.8570, 0.8855]	0.2279
Majority vote	untrained	0.9685	[0.9626, 0.9742]	0.8550	[0.8395, 0.8695]	0.1544
Quorum (shipped)	zero-shot	0.9896	[0.9861, 0.9925]	0.9470	[0.9365, 0.9565]	0.0416
Logistic regression	in-domain CV	0.9989	[0.9982, 0.9994]	0.9815	[0.9755, 0.9870]	0.0112

Table 11: Corpus A fusion rules. Regimes are never ranked against each other. Brackets are 95% bootstrap CIs over 2000 resamples.

Removed	AUC	$\Delta$ AUC
none (all 5)	0.9985	—
AIDE-SD14	0.9985	$-0.0001$
VQGAN/Swin	0.9983	$-0.0003$
AIDE-GENIMAGE	0.9985	$-0.0000$
NONESCAPE	0.9962	$-0.0023$
AIDE-PROGAN	0.9879	$\mathbf{-0.0106}$

Table 12: Leave-one-detector-out, Corpus A, in-domain 5-fold CV.

Detector	Dev.	Load	Single	Batch/img
VQGAN/Swin	MPS	2.6	0.029	0.015
Nonescape	MPS	3.9	0.057	0.032
AIDE $\times$ 3	CPU	23.8	9.178	4.244

Table 13: Per-detector cost in seconds, Apple M4 Max. Medians over 3 rounds (

n=24

timed single images, 3 batch runs of 16). AIDE runs on CPU following the S1 device probe.

Model	FPR	50%	10%	1%	0.1%
AIDE-PROGAN	0.0070	0.9923	0.9346	0.5652	0.1141
Quorum	0.0660	0.9357	0.6178	0.1281	0.0144
XGB (refit)	0.0200	0.9800	0.8450	0.3313	0.0468

Table 14: Corpus A precision at deployment prevalence (columns), from measured TPR/FPR at

t=0.5

xgboost_production_model.pkl 120,067 B

133961b097df89a764c9eff5599a490755cef63827d11c185941c55f1f6c0880

xgboost_production_features.pkl 108 B

cc819d11a25e1db608a0907a8a55b0428babcefcc9faa4cdd1dc8edb3dbb8fae

xgboost_production_model.json 120,736 B

be1518a6e78f4d52b793af0fe9b0f568e9a9afd5fde9b9759a5fbab18161a6c3

Corpus A scores (final_master_dataset.csv) 130,098 B

Table 18

Adversarial Attacks on AI-Generated Image Detectors

See all 11 papers

How Fragile Are Training-Free AI-Generated Image Detectors? A Controlled Audit of Score Direction, Preprocessing, and Compression
Jun 18, 2026Jingwen Zhou, Mingzhe WangImage Corruption Robustness AI-Generated Image Detection
Xidian University, Xi’an, China
SPARED: Reasoning-Based AI-Generated Image Detection via Adversarially Edited Data
Aug 13, 2026Yicheng Bao, Xiahui Guo, Xuhong Wang +1Adversarial Training Adversarial Examples
1East China Normal University · 2Shanghai Artificial Intelligence Laboratory
BIAS-ID: A Framework for Analyzing Transformation Biases in AI-Generated Image Detectors
May 29, 2026Jonas Ricker, Asja Fischer, Erwin QuiringAlgorithmic Bias AI-Generated Image Detection
Ruhr University Bochum · Bochum, Germany · _fbeta +1

Jun 18, 2026J/K move · Enter open · S save

How Fragile Are Training-Free AI-Generated Image Detectors? A Controlled Audit of Score Direction, Preprocessing, and Compression

Jingwen Zhou, Mingzhe Wang

Xidian University, Xi’an, China

Image Corruption Robustness AI-Generated Image Detection

Open paper AskPDF arXiv

Training-free detectors of AI-generated images promise generator-agnostic deployment without classifier training, yet their reported numbers are rarely compared under a single controlled protocol. We audit two representative training-free scores -- an autoencoder-reconstruction score (AEROBLADE-style) and a noise-perturbation feature-similarity score (RIGID-style) -- plus a naive feature-kNN control, on a common 1,500-image GenImage-derived benchmark spanning seven generators and JPEG compression at quality 70 and 50. The audit yields three cautionary findings. (i) Implementation details masquerade as method differences: replacing the LPIPS backbone (AlexNet -> VGG-16) changes overall AUROC by +0.085, and switching between resize-to-512 and native-resolution preprocessing flips per-generator conclusions by up to 0.38 AUROC. (ii) Score direction is not a property of the method but of its hyperparameters: the RIGID-style score is inverted (AUROC < 0.5) on SD1.5 and Wukong at noise level sigma=0.05, recovers to >0.5 for every generator at sigma=0.01, and collapses to 0.15 at sigma=0.3. (iii) Dataset format bias inflates robustness claims: without unified re-encoding, AUROC under JPEG-50 exceeds the clean condition for the AlexNet-backbone reconstruction score; after bias correction the residual anomaly localizes to a single generator (BigGAN). The audited scores have complementary per-generator failure sets, but naive z-score fusion does not beat the best single score, indicating that exploiting complementarity requires direction-aware combination.

Abstract

Adversarial Attacks on AI-Generated Image Detectors

Explore similar work

How Fragile Are Training-Free AI-Generated Image Detectors? A Controlled Audit of Score Direction, Preprocessing, and Compression

SPARED: Reasoning-Based AI-Generated Image Detection via Adversarially Edited Data

BIAS-ID: A Framework for Analyzing Transformation Biases in AI-Generated Image Detectors