Do CNNs Internally Represent Real and Fake Images Differently? A Hidden-Layer Analysis

Moumita Sen Sarma; Pascal Hitzler; Eugene Y. Vasserman

cs.CVAug 12, 2026

Do CNNs Internally Represent Real and Fake Images Differently? A Hidden-Layer Analysis

Authors: Moumita Sen Sarma, Pascal Hitzler, Eugene Y. Vasserman

Organizations: Department of Computer Science, Kansas State University, Manhattan, Kansas, USA

Abstract

Fake/synthetic images are increasingly prevalent, but it remains unclear whether Convolutional Neural Networks (CNNs) process real and fake images in the same internal manner. This work examines the hypothesis that CNNs represent real and fake images differently, such that fake images induce different hidden-layer activation patterns even when semantic content is preserved. The hypothesis is evaluated in scene recognition settings using trained CNN models. Dense-layer activations are extracted, and neurosymbolic methods assign semantic labels to selected neurons. For each real test image, corresponding fake images are generated with similar semantic content using object-label-guided text-to-image and image-to-image generation based on Stable Diffusion variants. Paired real-fake activation patterns are then compared statistically. Additional experiments with another dataset, CNN architecture, generative model, and JPEG/blur degradation analysis assess robustness. Results suggest that fake images evoke different hidden-neuron activations, and these differences are not explained only by simple image degradation. Overall, the findings indicate that real and fake images differ in CNN hidden-layer activation behavior at least in some settings, which opens the door for follow-up work on making use of this different behavior to improve fake image detection.

Figures & tables

Figure 1: Stable Diffusion Model Architecture. Adapted from Ritika (2023)

Figure 2: Workflow for comparing real and generated images through hidden-layer neuron activations. In this approach, fake images are generated using both text-only and image along with text settings before activation-level statistical comparison.

Figure 3: Sample fake output images using SUN2012 and Stable Diffusion variants; (a): from Text-Guided Fake Image Generation using ground-truth object annotations; (b): from Structure-Guided Fake Image Generation. Additional examples are depicted in Figure 11 in Appendix D.1 .

Hypothesis	Neuron Subset	Selection Criteria	Metric Compared	Interpretation
H1	All	Activated ( $\geq 0.8\times\max$ ), zero-filtered	Count	Real images trigger broader neural activity than fake
H2	Relevant confirmed	i) Semantically aligned or ii) activated	Value	Real images activate relevant neurons more strongly
H3	Activated for real	Activation threshold	Shared Count	Fake images fail to preserve real activation patterns
H4	Activated & confirmed	Confirmed with activation threshold	Count	Real images better activate semantically meaningful neurons

Table 1: Summary of the filtering criteria and interpretation for the four hypotheses.

Neuron ID	ECII Concepts	TLA %	Non-TLA %	Target Median	Non-Target Median	Target Mean	Non-Target Mean	$z$ -score	$p$ -value
0	snowy_mountain	95	54.44	7.05	0.25	6.12	1.04	-6.57	$<0.00001$
7	sky_and_snowy_mountain	95	38.81	2.81	0.00	2.72	0.56	-5.92	$<0.00001$
9	fence_and_central	100	62.30	4.03	0.76	3.97	1.46	-5.54	$<0.00001$
…	…	…	…	…	…	…	…	…	$<0.00001$
41	skyscraper	80	69.92	1.21	1.03	2.05	1.46	-1.40	$\mathbf{0.15536}$
…	…	…	…	…	…	…	…	…	$<0.00001$

Table 2: Statistical evaluation results of concepts of confirmed neurons from concept induction using Mann-Whitney U. Bold rows represent neurons with

p

-value

\geq 0.05

, where the null hypothesis cannot be rejected . The complete table is represented in Figure 7 in Appendix A .

Statistic	Text-Guided Fake Image Generation	Structure-Guided Fake Image Generation
Images paired	793	793
Nonzero diffs used	402	348
Median(diff)	1.000	1.000
Mean(diff)	0.542	0.787
Prop(real $>$ fake)	0.600	0.681
$p$ -value	$2.749528\times 10^{-6}$	$2.939991\times 10^{-12}$

Table 3: Hypothesis 1 statistical results with SUN2012 dataset, InceptionV3 CNN model, and Stable Diffusion image generation model.

Statistic	Text-Guided Fake Generation (Concept Alignment)	Structure-Guided Fake Generation (Concept Alignment)	Text-Guided Fake Generation (Activation-Based Relevancy)	Structure-Guided Fake Generation (Activation-Based Relevancy)
Images paired	890	885	244	249
Nonzero diffs used	890	885	244	249
Median (real - fake)	0.7047	1.0429	2.5363	2.2209
Mean (real - fake)	0.7421	1.2023	2.6552	2.5786
Prop(real - fake)	0.657	0.740	0.959	0.924
$p$ -value	$4.681\times 10^{-28}$	$5.839\times 10^{-66}$	$4.507\times 10^{-41}$	$2.117\times 10^{-40}$

Table 4: Hypothesis 2 statistical results with SUN2012 dataset, InceptionV3 CNN model, and Stable Diffusion image generation model.

Statistic	Text-Guided Fake Image Generation	Structure-Guided Fake Image Generation
Images paired	793	793
Nonzero diffs used	299	264
Mean(real_count)	0.803	0.803
Mean(fake_count)	0.057	0.158
Median (real_count - fake_count)	1.000	1.000
Prop(real_count $>$ fake_count)	1.000	1.000

Table 5: Hypothesis 3 statistical results with SUN2012 dataset, InceptionV3 CNN model, and Stable Diffusion image generation model.

Statistic	Text-Guided Fake Image Generation	Structure-Guided Fake Image Generation
Images paired	793	793
Nonzero diffs used	212	189
Median (real-fake) counts	1.000	1.000
Prop(real $>$ fake)	0.646	0.709
$p$ -value	$6.272851\times 10^{-6}$	$1.724670\times 10^{-8}$
Effect size ( $r_{rb})$	0.292	0.418

Table 6: Hypothesis 4 statistical results with SUN2012 dataset, InceptionV3 CNN model, and Stable Diffusion image generation model.

Figure 4: Suppression in neurons and their mean activation drop for SUN2012 dataset, InceptionV3 CNN model, and Stable Diffusion image generation model. (a) Text-guided fake image generation (all neurons), (b) Structure-guided fake image generation (all neurons), (c) Text-guided generation for confirmed neurons, and (d) Structure-guided generation for confirmed neurons. Complete table is represented in Appendix B .

Figure 5: Structural Distortion Effects in Structure-Guided Fake Image Generation

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

Neuron ID	ECII Concepts	TLA %	Non-TLA %	Target Median	Non-Target Median	Target Mean	Non-Target Mean	$z$ -score	$p$ -value
0	snowy_mountain	95	54.44	7.05	0.25	6.12	1.04	-6.57	$<0.00001$
7	sky_and_snowy_mountain	95	38.81	2.81	0.00	2.72	0.56	-5.92	$<0.00001$
9	fence_and_central	100	62.30	4.03	0.76	3.97	1.46	-5.54	$<0.00001$
11	bathtub	100	51.98	4.79	0.05	4.69	0.75	-7.23	$<0.00001$
12	coffee_and_bouquet	95	55.16	1.37	0.20	1.60	0.78	-3.82	$0.00006$
16	skyscraper_and_building	95	45.95	2.68	0.00	2.45	0.59	-5.57	$<0.00001$

Appendix

Table 7: Statistical evaluation results of concepts of confirmed neurons from concept induction. Bold rows represent neurons with

p

-value

\geq 0.05

, where the null hypothesis cannot be rejected .

Figure 6: Suppression in neurons and their mean activation drop for SUN2012 dataset, InceptionV3 CNN model, and Stable Diffusion image generation model. (a) Text-guided fake image generation (all neurons), (b) Structure-guided fake image generation (all neurons), (c) Text-guided generation for confirmed neurons, and (d) Structure-guided generation for confirmed neurons.

Statistic	Real Images with JPEG Compression
Images paired	793
Nonzero diffs used	304
Median(real - compressed_real)	-1.000
Mean(real - compressed_real)	-0.280
Prop(real $>$ compressed_real)	0.418
$p$ -value	0.9972669

Appendix

Table 8: Hypothesis 1 statistical results for real images with JPEG compression

Statistic	Concept Alignment	Activation-Based Relevancy
Images paired	966	260
Nonzero diffs used	966	260
Median(real - compressed_real)	-0.0383	0.3315
Mean(real - compressed_real)	-0.0656	0.4086
Prop(real - compressed_real $>$ 0)	0.478	0.677
$p$ -value	0.9676	$9.09\times 10^{-13}$

Appendix

Table 9: Hypothesis 2 statistical results for real images with JPEG compression

Statistic	Real Images with JPEG Compression
Images paired	793
Nonzero diffs used	168
Mean(real_count)	0.803
Mean(compressed_real_count)	0.480
Median(real_count - compressed_real_count)	1.000
Prop(real_count $>$ compressed_real_count)	1.000

Appendix

Table 10: Hypothesis 3 statistical results for real images with JPEG compression

Statistic	Real Images with JPEG Compression
Images paired	793
Nonzero diffs used	151
Median(real - compressed_real) counts	-1.000
Prop(real $>$ compressed_real)	0.457
$p$ -value	0.8666343
Effect size ( $r_{rb}$ )	-0.086

Appendix

Table 11: Hypothesis 4 statistical results for real images with JPEG compression

Figure 7: Suppression in neurons and their mean activation drop for JPEG compression. (a) Results for all neurons, (b) Results for confirmed neurons.

Statistic	Mild Blur	Medium Blur
Images paired	793	793
Nonzero diffs used	295	302
Median(real - blurred_real)	1.000	1.000
Mean(real - blurred_real)	-0.044	0.331
Prop(real $>$ blurred_real)	0.515	0.606
$p$ -value	0.6090308	$4.809483\times 10^{-4}$

Appendix

Table 12: Hypothesis 1 statistical results for real images with mild and medium blurred real images

Statistic	Mild Blur		Medium Blur
	Concept Alignment	Activation-Based Relevancy	Concept Alignment	Activation-Based Relevancy
N total	967	260	960	260
Nonzero diffs used	967	260	960	260
Median(real - blurred_real)	0.0226	0.4866	0.1731	0.7375
Mean(real - blurred_real)	0.0261	0.5052	0.1861	0.7686
Prop(real $>$ blurred_real)	0.508	0.754	0.571	0.762

Appendix

Table 13: Hypothesis 2 statistical results for real images with mild and medium blurred real images

Statistic	Mild Blur	Medium Blur
Images paired	793	793
Nonzero diffs used	189	229
Median(real_count - blurred_real_count)	1.000	1.000
Prop(real_count $>$ blurred_real_count)	1.000	1.000
$p$ -value	$1.085851\times 10^{-35}$	$2.100855\times 10^{-42}$
Effect size ( $r_{rb}$ )	1.000	1.000

Appendix

Table 14: Hypothesis 3 statistical results for real images with mild and medium blurred real images

Statistic	Mild Blur	Medium Blur
Images paired	793	793
Nonzero diffs used	155	162
Median(real - blurred_real)	1.000	1.000
Prop(real $>$ blurred_real)	0.523	0.605
$p$ -value	0.3883456	$1.639224\times 10^{-3}$
Effect size ( $r_{rb}$ )	0.045	0.210

Appendix

Table 15: Hypothesis 4 statistical results for real images with mild and medium blurred real images

Figure 8: Suppression in neurons and their mean activation drop. (a) Mild Blurred Image (all neurons), (b) Medium Blurred Image (all neurons), (c) Mild Blurred Image (confirmed neurons), and (d) Medium Blurred Image (confirmed neurons).

Statistic	Text-Guided Fake Image Generation	Structure-Guided Fake Image Generation
Images paired	1370	1370
Nonzero diffs used	399	339
Median(real-fake)	1.000	1.000
Mean(real-fake)	0.574	0.209
Prop(real $>$ fake)	0.667	0.558
$p$ -value	$1.452988\times 10^{-10}$	$1.203812\times 10^{-2}$

Appendix

Table 16: Hypothesis 1 statistical results with ADE20K dataset, ResNet50V2 CNN model, and Stable Diffusion image generation model.

Statistic	Text-Guided Fake Generation (Concept Alignment)	Structure-Guided Fake Generation (Concept Alignment)	Text-Guided Fake Generation (Activation-Based Relevancy)	Structure-Guided Fake Generation (Activation-Based Relevancy)
Images paired	888	880	269	285
Nonzero diffs used	888	880	269	285
Median(real - fake)	1.4678	0.1044	4.2830	0.8398
Mean(real - fake)	1.3775	0.4081	4.3264	1.4776
Prop(real $>$ fake)	0.718	0.536	0.970	0.775
$p$ -value	$4.977\times 10^{-49}$	$8.824\times 10^{-7}$	$1.252\times 10^{-45}$	$3.940\times 10^{-27}$

Appendix

Table 17: Hypothesis 2 statistical results with ADE20K dataset, ResNet50V2 CNN model, and Stable Diffusion image generation model.

Statistic	Text-Guided Fake Image Generation	Structure-Guided Fake Image Generation
Images paired	1370	1370
Nonzero diffs used	329	215
Mean (real_count)	0.431	0.431
Mean (fake_count)	0.014	0.196
Median(real_count - fake_count)	1.000	1.000
Prop(real_count $>$ fake_count)	1.000	1.000

Appendix

Table 18: Hypothesis 3 statistical results with ADE20K dataset, ResNet50V2 CNN model, and Stable Diffusion image generation model.

Statistic	Text-Guided Fake Image Generation	Structure-Guided Fake Image Generation
Images paired	1370	1370
Nonzero diffs used	232	205
Median (real-fake) counts	1.000	1.000
Prop(real $>$ fake)	0.828	0.546
$p$ -value	$1.044829\times 10^{-19}$	$1.176608\times 10^{-2}$
Effect size ( $r_{rb}$ )	0.655	0.093

Appendix

Table 19: Hypothesis 4 statistical results with ADE20K dataset, ResNet50V2 CNN model, and Stable Diffusion image generation model.

Figure 9: Suppression in neurons and their mean activation drop with ADE20K dataset, ResNet50V2 CNN model, and Stable Diffusion image generation model. (a) Text-guided fake image generation (all neurons), (b) Structure-guided fake image generation (all neurons), (c) Text-guided generation for confirmed neurons, and (d) Structure-guided generation for confirmed neurons.

Statistic	Text-Guided Fake Image Generation	Structure-Guided Fake Image Generation
Images paired	1370	1370
Nonzero diffs used	400	329
Median(real - fake)	1.000	1.000
Mean(real - fake)	0.818	0.356
Prop(real $>$ fake)	0.718	0.626
$p$ -value	$2.304215\times 10^{-17}$	$3.436013\times 10^{-5}$

Appendix

Table 20: Hypothesis 1 statistical results with ADE20K dataset, ResNet50V2 CNN model, and FLUX image generation model.

Statistic	Text-Guided Fake Image Generation (Concept Alignment)	Structure-Guided Fake Image Generation (Concept Alignment)	Text-Guided Fake Image Generation (Activation-Based Relevancy)	Structure-Guided Fake Image Generation (Activation-Based Relevancy)
Images paired	925	905	316	292
Nonzero diffs used	925	905	316	292
Median(real - fake)	1.1181	0.1640	3.3038	0.8045
Mean(real - fake)	1.0456	0.1993	3.4708	0.8583
Prop(real $>$ fake)	0.692	0.572	0.984	0.757
$p$ -value	$2.017\times 10^{-43}$	$1.966\times 10^{-8}$	$1.251\times 10^{-53}$	$8.404\times 10^{-24}$

Appendix

Table 21: Hypothesis 2 statistical results with ADE20K dataset, ResNet50V2 CNN model, and FLUX image generation model.

Statistic	Text-Guided Fake Image Generation	Structure-Guided Fake Image Generation
Images paired	1370	1370
Nonzero diffs used	321	229
Mean (real_count)	0.431	0.431
Mean (fake_count)	0.021	0.191
Median(real_count - fake_count)	1.000	1.000
Prop(real_count $>$ fake_count)	1.000	1.000

Appendix

Table 22: Hypothesis 3 statistical results with ADE20K dataset, ResNet50V2 CNN model, and FLUX image generation model.

Statistic	Text-Guided Fake Image Generation	Structure-Guided Fake Image Generation
Images paired	1370	1370
Nonzero diffs used	256	212
Median(real - fake)	1.000	1.000
Prop(real $>$ fake)	0.758	0.580
$p$ -value	$1.963471\times 10^{-14}$	$2.452926\times 10^{-2}$
Effect size ( $r_{rb}$ )	0.516	0.160

Appendix

Table 23: Hypothesis 4 statistical results with ADE20K dataset, ResNet50V2 CNN model, and FLUX image generation model.

Figure 10: Suppression in neurons and their mean activation drop with ADE20K dataset, ResNet50V2 CNN model, and FLUX image generation model. (a) Text-guided fake image generation (all neurons), (b) Structure-guided fake image generation (all neurons), (c) Text-guided generation for confirmed neurons, and (d) Structure-guided generation for confirmed neurons.

Figure 11: Additional Sample fake output images using SUN2012 and Stable Diffusion variants; (a): from Text-Guided Fake Image Generation; (b): from Structure-Guided Fake Image Generation.

Figure 12: Additional Sample fake output images using ADE20K and Stable Diffusion variants; (a): from Text-Guided Fake Image Generation; (b): from Structure-Guided Fake Image Generation.

Figure 13: Additional Sample fake output images using ADE20K and FLUX; (a): from Text-Guided Fake Image Generation; (b): from Structure-Guided Fake Image Generation.

Explore similar work

SimLBR: Learning to Detect Fake Images by Learning to Detect Real Images
Feb 23, 2026Aayush Dhakal, Subash Khanal, Srikumar Sastry +4AI-Generated Image Detection OOD Generalization
Washington University in St. Louis · Oak Ridge National Laboratory
Comparative Evaluation of Deep Learning Models for Fake Image Detection
May 20, 2026Akhitha Pakala, Mohammed Mahir Rahman, Shahzad Memon +1Image Forgery Detection Digital Image Forensics
University of East London, London E16 2RD, UK
Hierarchical Channel Stacking: A Structured Decision Framework for AI-Generated Image Detection
Aug 27, 2026Saifullah Shoaib, Akash Borigi, Rupendra Lekkala +5Hierarchical Classification Explainable Image Classification
Missouri University of Science & Technology, Rolla MO 65409, USA · University of Houston, Houston TX 77479, USA · Aug Cranium AI, Inc., Short Hills NJ 07078, USA

Feb 23, 2026J/K move · Enter open · S save

SimLBR: Learning to Detect Fake Images by Learning to Detect Real Images

Aayush Dhakal, Subash Khanal, Srikumar Sastry, Jacob Arndt+3

Washington University in St. Louis · Oak Ridge National Laboratory

AI-Generated Image Detection OOD Generalization One-Class Classification

Open paper AskPDF arXiv

The rapid advancement of generative models has made the detection of AI-generated images a critical challenge for both research and society. Recent works have shown that most state-of-the-art fake image detection methods overfit to their training data and catastrophically fail when evaluated on curated hard test sets with strong distribution shifts. In this work, we argue that it is more principled to learn a tight decision boundary around the real image distribution and treat the fake category as a sink class. To this end, we propose SimLBR, a simple and efficient framework for fake image detection using Latent Blending Regularization (LBR). Our method significantly improves cross-generator generalization, achieving up to +24.85% accuracy and +69.62% recall on the challenging Chameleon benchmark. SimLBR is also highly efficient, training orders of magnitude faster than existing approaches. Furthermore, we emphasize the need for reliability-oriented evaluation in fake image detection, introducing risk-adjusted metrics and worst-case estimates to better assess model robustness. All code and models will be released on HuggingFace and GitHub.

Abstract

Explore similar work

SimLBR: Learning to Detect Fake Images by Learning to Detect Real Images

Comparative Evaluation of Deep Learning Models for Fake Image Detection

Hierarchical Channel Stacking: A Structured Decision Framework for AI-Generated Image Detection