Non-identifiability of Explanations from Model Behavior in Deep Networks of Image Authenticity Judgments

Icaro Re Depaolini; Uri Hasson

cs.CVApr 8, 2026

Non-identifiability of Explanations from Model Behavior in Deep Networks of Image Authenticity Judgments

Authors: Icaro Re Depaolini, Uri Hasson

Organizations: Center for Mind/Brain Sciences, The University of Trento, Trento, Italy

Abstract

Deep neural networks can predict human judgments, but this does not imply that they rely on human-like information or reveal the cues underlying those judgments. Prior work has addressed this issue using attribution heatmaps, but their explanatory value in itself depends on robustness. Here we tested the robustness of such explanations by evaluating whether models that predict human authenticity ratings also produce consistent explanations within and across architectures. We fit lightweight regression heads to multiple frozen pretrained vision models and generated attribution maps using Grad-CAM, LIME, and multiscale pixel masking. Several architectures predicted ratings well, reaching about 80% of the noise ceiling. VGG models achieved this by tracking image quality rather than authenticity-specific variance, limiting the relevance of their attributions. Among the remaining models, attribution maps were generally stable across random seeds within an architecture, especially for EfficientNetB3 and Barlow Twins, and consistency was higher for images judged as more authentic. Crucially, agreement in attribution across architectures was weak even when predictive performance was similar. To address this, we combined models in ensembles, which improved prediction of human authenticity judgments and enabled image-level attribution via pixel masking. We conclude that while deep networks can predict human authenticity judgments well, they do not produce identifiable explanations for those judgments. More broadly, our findings suggest that post hoc explanations from successful models of behavior should be treated as weak evidence for cognitive mechanism.

Figures & tables

Figure 1: Explanation consistency analysis: within- and across-architecture. For each image and each architecture, 10 explainability maps were generated from 10 independently trained model variants. Within-architecture consistency was computed as the mean pairwise similarity among the 10 heatmaps per image. Across-architecture consistency was computed by first averaging the 10 maps per architecture into a single prototype, then correlating prototypes across all architecture pairs, producing one agreement score per image per pair.

Setting	Base fit	Model sel.	Final eval.	Implementation
Exp1	70%	20%	10%	Train/validation used for early stopping; the selected model is evaluated on a held-out test set.
Exp2	70%	20%	10%	Train/validation used for early stopping; pruning is optimized on the test partition (used here as an explanation-optimization set).
Exp3
Bagging	70%	10%	20%	A fixed 20% test split is held out. 60 variants are trained on different train/validation splits of the remaining 80%; ensemble prediction obtained by averaging test predictions across models and evaluated on the held-out test set.
Exp3
Stacking	70%	10%	see Imp.	A fixed 20% partition is used for stacking. It is split internally for 5-fold CV (meta-train/meta-holdout); reported performance is averaged over held-out folds.

Table 1: Dataset partition roles across experimental settings. Percentages refer to the full dataset (

N=1367

). For Exp3 (Bagging and Stacking), percentages describe the split applied to each individual base model.

Figure 2: Distribution and relationships of human authenticity and quality ratings. (A) Per-image mean quality and authenticity ratings. Quality showed a bimodal distribution while authenticity did not. (B) Positive correlation between per-image mean authenticity and quality ratings ( $r=0.84$ ). (C) Relationship between mean authenticity and inter-rater standard deviation: images rated as less authentic showed greater rater agreement.

	Prediction metrics			Reliability
Architecture	RMSE	PLCC	SRCC	Model reliability	PLCC Ceiling
Barlow Twins	$\bm{7.10\pm 0.11}$	$0.62\pm 0.01$	$0.63\pm 0.01$	$0.91$	$0.78$
ResNet152	$9.85\pm 0.39$	$0.51\pm 0.02$	$0.53\pm 0.02$	$0.91$	$0.78$
DenseNet161	$7.48\pm 0.22$	$0.63\pm 0.02$	$0.62\pm 0.02$	$0.90$	$0.78$
EfficientNetB3	$8.82\pm 0.26$	$0.50\pm 0.02$	$0.51\pm 0.03$	$0.84$	$0.75$
VGG16	$8.62\pm 0.43$	$0.54\pm 0.03$	$0.57\pm 0.03$	$0.81$	$0.74$

Table 2: Predictive performance and reliability across architectures. Mean

\pm

SD across 10 random seeds. Models were trained to minimize MSE, and best performance is bold-faced. PLCC: Pearson linear correlation coefficient. SRCC: Spearman rank correlation coefficient. Model reliability: mean pairwise Pearson correlation between prediction vectors across random initializations (45 pairs). PLCC ceiling: estimated upper bound on the observable correlation between a single trained instance and the human MOS vector, accounting for reliability of both human ratings and model predictions.

Architecture	$r(A,\hat{A})$ (PLCC)	$r(Q,\hat{A})$	Partial $r(A,\hat{A}\mid Q)$
Barlow Twins	$0.62\pm 0.01$	$0.60\pm 0.02$	$0.24^{*}\pm 0.03$
ResNet152	$0.51\pm 0.02$	$0.51\pm 0.01$	$0.15^{*}\pm 0.02$
DenseNet161	$0.63\pm 0.02$	$0.65\pm 0.02$	$0.16^{*}\pm 0.03$
EfficientNetB3	$0.50\pm 0.02$	$0.47\pm 0.02$	$0.20^{*}\pm 0.03$
VGG19	$0.56\pm 0.03$	$0.63\pm 0.02$	$0.03\pm 0.07$
VGG16	$0.54\pm 0.03$	$0.61\pm 0.03$	$0.01\pm 0.06$

Table 3: The relation between authenticity predictions and quality ratings . Correlations between true Authenticity scores (

A

), predicted Authenticity scores (

\hat{A}

), and true Quality scores (

Q

) across model variants. Entries are mean

\pm

SD across 10 variants per architecture. The partial correlation indicates the residual correlation between true and predicted Authenticity after partialling out

Q

from both. Stars (*) indicate that the mean partial correlation differs significantly from zero (one-sample

t

-test against zero,

p<.05

Architecture	Consistency (Corr.)	IoU @ 5%	IoU @ 15%	IoU @ 25%	Pred. similarity (Corr.)	RSM similarity (Corr.)
Barlow Twins	$0.99\pm 0.01$	$0.77\pm 0.13$	$0.84\pm 0.09$	$0.88\pm 0.07$	$0.93$	$0.78$
ResNet152	$0.99\pm 0.01$	$0.77\pm 0.14$	$0.85\pm 0.10$	$0.88\pm 0.08$	$0.91$	$0.83$
DenseNet161	$0.78\pm 0.20$	$0.41\pm 0.26$	$0.52\pm 0.21$	$0.58\pm 0.17$	$0.90$	$0.77$
EfficientNetB3	$0.85\pm 0.14$	$0.46\pm 0.24$	$0.56\pm 0.19$	$0.61\pm 0.16$	$0.85$	$0.82$
VGG19	$0.99\pm 0.01$	$0.77\pm 0.11$	$0.83\pm 0.07$	$0.86\pm 0.06$	$0.84$	$0.74$
VGG16	$0.99\pm 0.01$	$0.81\pm 0.10$	$0.86\pm 0.06$	$0.89\pm 0.05$	$0.84$	$0.64$

Table 4: Within-architecture explanation-consistency (Grad-CAM). Test-set Grad-CAM within-architecture consistency (mean

\pm

SD) measured by correlations between attributions maps and using IoU at three saliency thresholds (top 5%, 15%, 25% pixels). Prediction similarity: mean pairwise Pearson correlation between prediction vectors. RSM similarity: mean pairwise Pearson correlation between upper-triangle entries of cosine-similarity representational similarity matrices, computed from penultimate regression-head embeddings (test set).

Architecture	Consistency (Corr.)	IoU @ 5	IoU @ 15	IoU @ 25
Barlow Twins	$0.88\pm 0.03$	$0.52\pm 0.13$	$0.59\pm 0.08$	$0.63\pm 0.07$
ResNet152	$0.86\pm 0.04$	$0.52\pm 0.14$	$0.57\pm 0.08$	$0.61\pm 0.07$
DenseNet161	$0.85\pm 0.03$	$0.47\pm 0.12$	$0.53\pm 0.07$	$0.59\pm 0.06$
EfficientNetB3	$0.76\pm 0.06$	$0.39\pm 0.13$	$0.45\pm 0.08$	$0.50\pm 0.07$
VGG16	$0.85\pm 0.03$	$0.43\pm 0.11$	$0.51\pm 0.07$	$0.57\pm 0.06$
VGG19	$0.83\pm 0.04$	$0.44\pm 0.12$	$0.51\pm 0.07$	$0.56\pm 0.06$

Table 5: Within-architecture explanation consistency (LIME). Mean

\pm

SD across images of within-architecture Pearson’s correlation (mean pairwise across variants) and IoU overlap between the top-

K

most positive LIME coefficients.

Figure 3: Across-architecture explanation agreement. Each cell shows the mean Spearman correlation between prototype explanation maps for a pair of architectures, averaged across test-set images. Panel A: Grad-CAM. Panel B: Multiscale Pixel Masking (MPM). Panel C: LIME. Higher values indicate greater similarity. SDs across images are provided in Appendix Figure A.3 .

Figure 4: Examples of across-architecture Grad-CAM agreement. Each column shows the attribution map produced by one architecture for the same image. Upper row: an image with relatively high across-architecture agreement. Lower row: an image with low across-architecture agreement. Note that VGG architectures systematically produced attribution maps that differed from the other architectures (see text).

Architecture	Test				Train
	Corr.-Based	Top 5%	Top 15%	Top 25%	Corr.-Based	Top 5%	Top 15%	Top 25%
Barlow Twins	0.31*	0.11	0.15	0.13	0.25*	0.16	0.22	0.26*
ResNet152	0.12	0.11	0.11	0.12	0.08	0.11	0.16	0.16
DenseNet161	0.01	$-0.12$	$-0.03$	0.07	0.11	0.10	0.12	0.14
EfficientNetB3	0.25*	0.21	0.25*	0.29*	0.24*	0.14	0.20	0.23*
VGG19	0.05	$-0.01$	0.08	0.09	0.03	$-0.02$	0.01	0.05

Table 6: Images perceived as more authentic are associated with more consistent explanations . Pearson correlations between explanation consistency and mean human authenticity ratings. Train and test sets are reported separately. Bold values indicate

p<.05

(uncorrected). Stars (*) indicate correlations that survive FDR correction for 48 multiple comparisons (

p<.0067

Architecture	RMSE (mean $\pm$ SD) [best variant]	PLCC	SRCC
Barlow Twins	$7.01\pm 0.25$ [6.65]	$0.64\pm 0.03$	$0.65\pm 0.03$
ResNet152	$9.14\pm 0.31$ [8.71]	$0.48\pm 0.03$	$0.50\pm 0.03$
DenseNet161	$7.43\pm 0.25$ [6.95]	$0.63\pm 0.02$	$0.62\pm 0.02$
EfficientNetB3	$8.89\pm 0.33$ [8.33]	$0.47\pm 0.01$	$0.48\pm 0.01$
VGG16	$8.57\pm 0.16$ [8.29]	$0.52\pm 0.02$	$0.55\pm 0.02$
VGG19	$8.37\pm 0.35$ [7.79]	$0.55\pm 0.03$	$0.59\pm 0.03$

Table 7: Ensembles outperform all individual models. For base architectures,

\mathrm{RMSE}

is reported as mean

\pm

SD across 10 unpruned variants, with the best pruned variant (lowese RMSE) in brackets.

\mathrm{PLCC}

and

\mathrm{SRCC}

are mean

\pm

SD across the ten variants. All results on held-out data.

Figure 5: MPM attributions provided for Ensemble explanations. For two target images, MPM attribution maps are shown for the Bagging Ensemble and for each of the six individual architectures. Single-architecture maps are averaged across 10 model variants. Warm colors indicate regions whose occlusion reduces predicted authenticity; cool colors indicate regions whose occlusion increases it.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A.1: Authenticity score distributions for excluded versus retained images. Per-image mean authenticity scores for images excluded ( Removed , orange; $n=1033$ ) and retained ( Kept , blue; $n=1367$ ) based on prompt-metadata criteria. Removed images received significantly lower authenticity ratings (see text).

Figure A.2: LIME explanation of authenticity judgments Left: input image with SLIC superpixel boundaries overlaid. In the LIME workflow, each superpixel is treated as a binary feature. In each perturbation, a random subset of superpixels is masked (replaced by the image mean RGB) and the resulting image is passed through the model for inference. Right: LIME explanation heatmap showing the contribution of each superpixel to the model’s predicted authenticity score. Red: superpixel presence increases predicted authenticity. Blue: superpixel presence decreases predicted authenticity.

Figure A.3: Across-architecture Grad-CAM agreement with variability. Supplementary to Figure 3 . Each cell shows the mean Spearman correlation between prototype Grad-CAM maps for a pair of architectures, averaged across test-set images, with the SD across images in parentheses.

Figure A.4: Across-architecture similarity (IoU) . Each cell shows average Intersection-Over-Union overlap of top 5% (and top 25%) pixels over images with standard deviation over images.

Figure A.5: Examples of across-architecture MPM agreement. Each row shows MPM attribution maps for the same image across all six architectures. Upper rows: images with relatively high across-architecture agreement. Lower rows: images with low agreement.

Model	Target Layer
Barlow Twins	features.7.2.conv3
ResNet152	features.7.2.conv3
DenseNet161	features.denseblock4.denselayer24.conv2
EfficientNetB3	features.8.0
VGG16	features.28
VGG19	features.34

Appendix

Table A.1: Target convolutional layers for SBS pruning and Grad-CAM attribution.

Architecture	$R^{2}$ (mean $\pm$ SD)	min–max
Barlow Twins	$0.537\pm 0.091$	0.243–0.816
DenseNet161	$0.478\pm 0.081$	0.274–0.773
EfficientNetB3	$0.437\pm 0.076$	0.226–0.773
ResNet152	$0.439\pm 0.083$	0.205–0.794
VGG16	$0.603\pm 0.077$	0.289–0.820
VGG19	$0.562\pm 0.079$	0.302–0.787

Appendix

Table A.2: LIME surrogate fidelity by architecture. For each architecture, we report the locality-weighted

R^{2}

of the surrogate fit vs. the predictions of the original model, aggregated across all explained images and variants.

Backbone / model	Regression head	# Reg head. params
VGG16	$4096\rightarrow 128\rightarrow 1$	524,545
VGG19	$4096\rightarrow 128\rightarrow 1$	524,545
EfficientNet-B3	$1536\rightarrow 512\rightarrow 128\rightarrow 1$	852,737
DenseNet-161	$2208\rightarrow 512\rightarrow 128\rightarrow 1$	1,196,801
ResNet-152	$2048\rightarrow 512\rightarrow 128\rightarrow 1$	1,114,881
Barlow Twins (ResNet50 encoder)	$2048\rightarrow 512\rightarrow 128\rightarrow 1$	1,114,881

Appendix

Table A.3: Regression head architectures and trainable parameter counts. Each head maps the frozen backbone’s output to a scalar authenticity score.

Architecture	Consistency (Corr.)	IoU @ 5%	IoU @ 15%	IoU @ 25%
Barlow Twins	$0.10$	$-0.02$	$-0.02$	$0.02$
ResNet152	$-0.01$	$-0.04$	$-0.04$	$-0.09$
DenseNet161	$-0.08$	$-0.05$	$-0.09$	$-0.12$
EfficientNetB3	$-0.13$	$-0.14$	$-0.11$	$-0.08$
VGG19	$0.04$	$-0.06$	$-0.12$	$-0.03$
VGG16	$-0.01$	$0.05$	$0.08$	$0.01$

Appendix

Table A.4: Explanation consistency does not reliably predict image-level prediction error. Pearson correlations between per-image Grad-CAM explanation consistency and mean absolute prediction error (MAE). IoU-based consistency at top-5%, 15%, and 25% saliency thresholds. All correlations are non-significant.

Architecture	Baseline			Pruned			Red. (%)
	RMSE	PLCC	SRCC	RMSE	PLCC	SRCC
Barlow Twins	$7.10\pm 0.11$	$0.62\pm 0.01$	$0.63\pm 0.01$	$\bm{5.29\pm 0.20}$	$\bm{0.80\pm 0.02}$	$\bm{0.80\pm 0.02}$	$28.15\pm 4.06$
ResNet152	$9.85\pm 0.39$	$0.51\pm 0.02$	$0.53\pm 0.02$	$8.53\pm 0.34$	$0.54\pm 0.03$	$0.55\pm 0.03$	$8.23\pm 0.68$
DenseNet161	$7.48\pm 0.22$	$0.63\pm 0.02$	$0.62\pm 0.02$	$7.13\pm 0.18$	$0.65\pm 0.02$	$0.64\pm 0.02$	$40.83\pm 4.30$
EfficientNetB3	$8.82\pm 0.26$	$0.50\pm 0.02$	$0.51\pm 0.03$	$7.56\pm 0.39$	$0.61\pm 0.03$	$0.61\pm 0.03$	$8.20\pm 0.78$
VGG16	$8.62\pm 0.43$	$0.54\pm 0.03$	$0.57\pm 0.03$	$7.43\pm 0.30$	$0.63\pm 0.02$	$0.65\pm 0.02$	$28.14\pm 3.42$

Appendix

Table A.5: Performance before and after optimizing a pruning solution for the test-set images. Values are reported as mean

\pm

standard deviation across 10 random initializations. PLCC = Pearson correlation; SRCC = Spearman correlation. Red. (%) indicates the percentage of feature maps removed.

Explainability Evaluation

Explainability evaluation tests whether AI explanations are faithful, stable, understandable, useful, and reliable across models and settings.

See all 321 papers

May 11, 2026

Scaling Vision Models Does Not Consistently Improve Localisation-Based Explanation Quality

Artificial intelligence models are increasingly scaled to improve predictive accuracy, yet it remains unclear whether scale improves the quality of post-hoc explanations. We investigate this relationship by evaluating 11 computer vision models representing increasing levels of depth and complexity within the ResNet, DenseNet, and Vision Transformer families, trained from scratch or pretrained, across three image datasets with ground-truth segmentation masks. For each model, we generate explanations using five post-hoc explainable AI methods and quantify mask alignment using two localisation metrics: Relevance Rank Accuracy (Arras et al., 2022) and the proposed Dual-Polarity Precision, which measures positive attributions inside the class mask and negative attributions outside it. Across datasets and methods, increasing architectural depth and parameter count does not improve explanation quality in most statistical comparisons, and smaller models often match or exceed deeper variants. While pretraining typically improves predictive performance and increases the dependence of explanations on learned weights, it does not consistently increase localisation scores. We also observe scenarios in which models achieve strong predictive performance while localisation precision is near zero, suggesting that performance metrics alone may not indicate whether predictions are based on the annotated regions. These results indicate that larger models do not reliably provide higher-quality explanations, and that explainability should therefore be assessed explicitly during model selection for safety-sensitive deployments.

Mateusz Cedro, Marcin Chlebus

University of Antwerp, Antwerp, 2000, Belgium · University of Warsaw, Warsaw, 00-241, Poland

Neural Network Interpretability Explainability Evaluation Explanation Evaluation

Ask PDF

May 7, 2026

Concept-Based Abductive and Contrastive Explanations for Behaviors of Vision Models

Concept-based explanations offer a promising approach for explaining the predictions of deep neural networks in terms of high-level, human-understandable concepts. However, existing methods either do not establish a causal connection between the concepts and model predictions or are limited in expressivity and only able to infer causal explanations involving single concepts. At the same time, the parallel line of work on formal abductive and contrastive explanations computes the minimal set of input features causally relevant for model outcomes but only considers low-level features such as pixels. Merging these two threads, in this work, we propose the notion of concept-based abductive and contrastive explanations that capture the minimal sets of high-level concepts causally relevant for model outcomes. We then present a family of algorithms that enumerate all minimal explanations while using concept erasure procedures to establish causal relationships. By appropriately aggregating such explanations, we are not only able to understand model predictions on individual images but also on collections of images where the model exhibits a user-specified, common behavior. We evaluate our approach on multiple models, datasets, and behaviors, and demonstrate its effectiveness in computing helpful, user-friendly explanations.

Ronaldo Canizales, Divya Gopinath, Corina Păsăreanu +1

Colorado State University, Fort Collins CO, USA · KBR Inc., NASA Ames, Moffett Field CA, USA · Carnegie Mellon University, Pittsburgh PA, USA

Abductive Reasoning Concept-Based Explanations Neural Network Interpretability Explainable Image Classification Causal Interpretability

Ask PDF

Oct 15, 2026

Deep Learning for Deepfake Image Detection: Comparative Model Evaluation and Gradient-Based Visual Attribution

Synthetic facial imagery has made the distinction between authentic and generated content increasingly important for digital forensics and online trust. This study examines an explainable deep-learning workflow for image-level deepfake detection using two contrasting classification strategies: a compact convolutional model trained from scratch and an ImageNet-pretrained MobileNetV2 model adapted through two-stage transfer learning. Experiments were conducted with the balanced 140k Real and Fake Faces dataset, comprising 140,000 facial images. Under the recorded validation protocol, the augmented from-scratch model reached 84.15% accuracy (loss 0.3625), compared with 95.90% accuracy (loss 0.1074) for the fine-tuned pretrained model. Gradient-weighted class activation mapping (Grad-CAM) was used to visualize image regions associated with predictions, and a Flask-based prototype was developed for side-by-side inspection. These observations support the practical value of pretrained visual representations in the studied setting while underscoring the need to interpret validation performance alongside dataset provenance, explanation limitations, and evaluation scope. The reported results are validation results, not held-out test estimates.

Saad Korchi, Abderrahim Waga

Euromed University of Fez, Fez, Morocco

Deepfake Detection Explainable Deepfake Detection Transfer Learning

Ask PDF

Abstract

Explainability Evaluation

Explore similar work

Scaling Vision Models Does Not Consistently Improve Localisation-Based Explanation Quality

Concept-Based Abductive and Contrastive Explanations for Behaviors of Vision Models

Deep Learning for Deepfake Image Detection: Comparative Model Evaluation and Gradient-Based Visual Attribution