Unmasking Biases and Reliability Concerns in Convolutional Neural Networks Analysis of Cancer Pathology Images

Michael Okonoda; Eder Martinez; Abhilekha Dalal; Lior Shamir

eess.IVMar 12, 2026

Unmasking Biases and Reliability Concerns in Convolutional Neural Networks Analysis of Cancer Pathology Images

Authors: Michael Okonoda, Eder Martinez, Abhilekha Dalal, Lior Shamir

Organizations: Department of Computer Science, Kansas State University, Manhattan, KS 66506, USA

Abstract

Convolutional Neural Networks have shown promising effectiveness in identifying different types of cancer from radiographs. However, the opaque nature of CNNs makes it difficult to fully understand the way they operate, limiting their assessment to empirical evaluation. Here we study the soundness of the standard practices by which CNNs are evaluated for the purpose of cancer pathology. Thirteen highly used cancer benchmark datasets were analyzed, using four common CNN architectures and different types of cancer, such as melanoma, carcinoma, colorectal cancer, and lung cancer. We compared the accuracy of each model with that of datasets made of cropped segments from the background of the original images that do not contain clinically relevant content. Because the rendered datasets contain no clinical information, the null hypothesis is that the CNNs should provide mere chance-based accuracy when classifying these datasets. The results show that the CNN models provided high accuracy when using the cropped segments, sometimes as high as 93%, even though they lacked biomedical information. These results show that some CNN architectures are more sensitive to bias than others. The analysis shows that the common practices of machine learning evaluation might lead to unreliable results when applied to cancer pathology. These biases are very difficult to identify, and might mislead researchers as they use available benchmark datasets to test the efficacy of CNN methods.

Figures & tables

Figure 1: Example of 20 $\times$ 20 cropped images from the ISIC 2017 Skin cancer diagnosis dataset and five sections of the original image: upper-left, upper-right, center, bottom-left, and bottom-right. These small sub-image contain no medical information, and therefore are not expected to be useful for identifying cancer to a level beyond mere chance. The empirical classification accuracy of each section is compared with that of the classification accuracy observed when using the original image.

Figure 2: Sample images from the different classes of the DermaMNIST dataset and their respective 20 $\times$ 20 cropped sections that have no meaningful medical content. Since they do not have any medically relevant information, these non-informative sub-images are not expected to be able to identify cancer.

PathMNIST	Histopatology	Multi-Class (9)	1. Cancer-Associated Stroma, 2. Colorectal Adenocarcinoma Epithelium = Cancer Present / 3. Adipose Tissue, 4. Background, 5. Debris, 6. Lymphocytes, 7. Mucus, 8. Smooth Muscle and Normal Colon Mucosa = Cancer Absent	107,180	95,435	11,745
DermaMNIST	Dermatoscope	Multi-Class (7)	1. Melanoma 2. Basal cell carcinoma = Cancer Present / 3. Melanocytic nevi, 4. Benign keratosis-like lesions, 5. Dermatofibroma, 6. Vascular lesions 7. Actinic keratoses = Cancer Absent	10,015	656	9,359
BreastMNIST	Breast Ultrasound	Multi-Class (3)	1. Malignant = Cancer Present / 2. Normal, 3. Benign = Cancer Absent	780	570	210
NoduleMNIST3D	Chest CT	Binary-Class (2)	1. Malignant = Cancer Present / 2. Benign = Cancer Absent	1,633	401	1232

Table 1: The description, class modification, and distribution of samples of the MedMNIST Dataset.

BreakHis Dataset
Magnification	Modality	Initial Class	Modified Binary Class	Total Samples	Cancer Present Samples	Cancer Absent Samples
$40\times$	Microscopy Histopathology	Multi-Class (8)	1 to 4 = Cancer Absent/ 5 to 8 = Cancer Present	1995	1370	652
$100\times$				2081	1437	644
$200\times$				2013	1390	623
$400\times$				1820	1232	588

Table 2: The Description, class modification, and distribution of samples of the BreakHis Dataset. Figures assigned to each class are as follows: Adenosis = 1; Fibroadenoma = 2; Phyllodes tumor = 3; Tubular adenoma = 4; Ducta carcinoma = 5; Lobular carcinoma = 6; Mucinous carcinoma = 7; and Papillary carcinoma = 8.

The ISIC (2016–2019) Dataset
Dataset	Data Modality	Initial Class	Modified Binary Class	Total Samples	Cancer Present Samples	Cancer Absent Samples
ISIC-2016	Dermatoscope	Binary Class	1. Melanoma = Cancer Present; 2. Benign = Cancer Absent	1279	248	1031
ISIC-2017	Dermatoscope	Multi-Class (3)	1. Melanoma = Cancer Present; 2. Seborrheic_keratosis; 3. Nevus = Cancer Absent	11,527	3736	7790
ISIC-2018	Dermatoscope	Multi-Class (7)	1. Melanoma; 2. Basal cell carcinoma = Cancer Present; 3. Melanocytic nevi; 4. Benign keratosis-like lesions; 5. Dermatofibroma; 6. Vascular lesions; 7. Actinic keratoses = Cancer Absent	10,015	7007	1003
ISIC-2019	Dermatoscope	Multi-Class (8)	1. Malanoma; 2. Besal Cell Carcinoma; 3. Antinic Keratosis; 4. Squamous Cell Carcinoma = Cancer Present; 5. Melanocytic Nevus; 6. Benign Keratosis; 7. Dermotafibroma; 8. Vascular Lesion = Cancer Absent	780	546	78

Table 3: The description, class modification, and distribution of samples of the ISIC dataset.

Breast Histopathology Image (IDC) Dataset
Dataset	Modality	Initial Class	Modified Binary Class	Total Samples	Cancer Present Samples	Cancer Absent Samples
Breast Histopathology Image (IDC)	Microscopy Histopathology	Binary	IDC-Nagative = Cancer Absent/IDC-Positive = Cancer Present	277,524	78,786	198,738

Table 4: Distribution of samples of the Breast Histopathology Image (IDC) Dataset.

Figure 3: Sample images across the MedMNIST datasets, showing the original images and their 20 $\times$ 20 cropped images. Mostly, the cropped images contain little or no medical content useful for correct diagnostics.

Figure 4: The classification accuracy of the BreastMNIST original image dataset alongside cropped 20 $\times$ 20 pixel sections. As expected, CNNs models can classify the original images. Surprisingly, all CNN models performed better than the random chance accuracy of 50%, when applied to the cropped sections, highlighting that these models can classify images that lack lesion-specific features.

Model	Sub Dataset	Accuracy	Recall	F 1	Precision
DENSENET121	Bottom Left	85.8372	85.5634	81.9552	78.7535
DENSENET121	Bottom Right	62.8590	4.4366	7.4707	28.3412
DENSENET121	Center	62.7399	3.9437	6.8230	35.0489
DENSENET121	Original Images	61.2442	6.5141	10.0572	64.2105
DENSENET121	Upper Left	62.9782	4.6479	7.9069	57.9132
DENSENET121	Upper Right	62.8855	4.7887	7.9559	42.9699

Table 5: Accuracy, precision, recall, and F-1 results of the analysis of the balanced BreastMNIST.

Figure 5: The classification accuracy, precision, recall, and F-1 scores for the DermaMNIST dataset when each class contains 458 images.

Figure 6: The classification accuracy of the original image alongside a cropped 20 $\times$ 20 pixel of the DermaMNIST datasets. All CNN models achieved a consistent classification accuracy of 93.42% on all four corner-cropped-image datasets, even when they contained only skin-toned background images. However, performance is slightly higher and varied on center-cropped images across all models; these images may contain small fractions of the lesion structure.

Figure 7: The classification accuracy performance of CNN models on the NoduleMNIST original images and their 20 $\times$ 20 pixel subset images. Unexpectedly, neural networks can identify positive and negative cases from the cropped-image datasets, despite the lack of sufficient nodule structures to make adequate diagnostics.

Figure 8: The classification accuracy performance of CNN models on the PathMNIST original dataset and its 20 $\times$ 20 pixel subsets, suggesting that these models may be classifying histopathology images as either cancerous or noncancerous based on non-informative medical cues.

Figure 9: The classification accuracy, precision, recall, and F-1 scores for the PathMNIST dataset when the size of each class is set to 9366 images.

Model	Sub Dataset	Accuracy	Recall	F-1	Precision
DENSENET121	Original Images	98.5376	98.8588	99.3495	99.1036
DENSENET121	Bottom Left	82.7994	82.6044	99.8973	90.4315
DENSENET121	Bottom Right	82.8830	82.6653	99.9144	90.4751
DENSENET121	Center	82.9109	82.6795	99.9315	90.4906
DENSENET121	Upper Left	82.7994	82.5952	99.9144	90.4330
DENSENET121	Upper Right	82.7855	82.6481	99.7946	90.4156

Table 6: Accuracy, precision, recall, and F-1 results of the analysis of the balanced PathMNIST.

Figure 10: Sample images across different magnification levels of the BreakHis datasets, showing the original images and their 20 $\times$ 20 cropped images. Cropped images show staining protocols that often lack clear histopathological structures.

Figure 11: The classification accuracy of the BreakHis $40\times$ magnification level of the original images and its corresponding 20 $\times$ 20 cropped sections, showing that CNNs can classify images that contain non-informative medical content.

Figure 12: All the CNN models can accurately classify the original images as well as all the 20 $\times$ 20 cropped sections when applied using BreakHis $100\times$ , suggesting that neural networks may be learning from staining color or texture pattern not related to the actual disease.

Figure 13: The results from BreakHis $200\times$ show that CNN models can classify cancerous or noncancerous tumors from both the original images and their 20 $\times$ 20 cropped sections.

Figure 14: The classification accuracy score of all the CNN architectures across BreakHis $400\times$ datasets (the original image and cropped sections) is far higher than accuracy when using only random chance.

Model	Sub Dataset	Accuracy	Recall	F-1	Precision
DENSENET121	Original Images	97.69	97.97	98.30	98.64
DENSENET121	Bottom Left	55.48	51.58	49.49	64.49
DENSENET121	Bottom Right	55.35	50.48	48.87	65.09
DENSENET121	Center	56.41	52.64	50.93	84.84
DENSENET121	Upper Left	55.93	52.25	50.27	85.12
DENSENET121	Upper Right	54.74	49.95	48.53	65.76

Table 7: Summary of the results of the analysis of the BreakHis dataset with 40

\times

magnification.

Figure 15: The classification accuracy, precision, recall, and F-1 scores for the BreakHis dataset with 40

\times

magnification when each class contains exactly 405 images.

Figure 16: Sample images across the ISIC datasets, showing the original images and their 20 $\times$ 20 cropped images. Mostly, the cropped images contain a black or skin-toned background. Sometimes, center-cropped images include lesion structures that lack meaningful diagnostic features for an expert dermatologist.

Figure 17: All CNN models can classify the ISIC_2017 cropped-image datasets with an accuracy higher than the 50% of mere chance. VGG16 consistently performs slightly better on the cropped images than on the original images.

Figure 18: The classification accuracy of the ISIC_2016 original image dataset alongside its 20 $\times$ 20 cropped sections; VGG16 exhibited consistent accuracy performance across all cropped datasets, suggesting that neural networks may be learning from background artifacts during training.

Figure 19: The classification accuracy scores on the ISIC_2018 original images and the cropped sections show that CNN models can also identify skin cancer from a small fraction of the full dermatoscope image containing only skin-toned background.

Model	Sub Dataset	Accuracy	Recall	F-1	Precision
DENSENET121	Original Images	85.8372	85.5634	81.9552	78.7535
DENSENET121	Bottom Right	62.8590	4.4366	7.4707	28.3412
DENSENET121	Bottom Left	62.7399	3.9437	6.8230	35.0489
DENSENET121	Center	61.2442	6.5141	10.0572	64.2105
DENSENET121	Upper Left	62.9782	4.6479	7.9069	57.9132
DENSENET121	Upper Right	62.8855	4.7887	7.9559	42.9699

Table 8: Accuracy, precision, recall, and F-1 results of the analysis of the balanced ISIC-2018 dataset.

Model	Sub Dataset	Accuracy	Recall	F-1	Precision
DENSENET121	Original Images	0.68	0.53	0.78	0.63
DENSENET121	Bottom Left	0.48	0.38	0.80	0.52
DENSENET121	Bottom Right	0.48	0.38	0.82	0.52
DENSENET121	Center	0.48	0.30	0.40	0.34
DENSENET121	Upper Left	0.48	0.38	0.82	0.52
DENSENET121	Upper Right	0.49	0.39	0.84	0.53

Table 9: Accuracy, precision, recall, and F-1 results of the analysis of the balanced ISIC-2019 dataset.

Figure 20: Classification accuracy scores on the ISIC_2019 original images and cropped sections. Performance reveals a significant drop compared to previous years, but CNN can still classify cropped images with an accuracy slightly higher than mere chance of 50%.

Figure 21: Classification accuracy of ISIC-2016 using different parts of the image with or without using transfer learning. The number of epochs was set to 15 in both cases.

Figure 22: Precision observed when using different deep neural network architectures when using ISIC-2016 with different parts of the image, with and without using transfer learning.

Figure 23: Recall of different architectures when using ISIC-2016 with different parts of the image, with or without using transfer learning.

Figure 24: F-1 of different architectures when using ISIC-2016 with different parts of the image, with or without using transfer learning.

Figure 25: Learning as a function of the number of epochs for different CNN architectures.

Figure 26: The classification accuracy scores on the breast histopathology original images and cropped images show that the models can accurately detect the classes, whether IDC-positive or IDC-negative, on cropped images containing little or no medical content.

Explore similar work

Sep 13, 2026

Evaluating Contextual Bias in CNN Image Classification: Evidence from Agricultural Benchmark Datasets

Convolutional neural networks (CNNs) are typically evaluated using held-out classification accuracy, an approach that presupposes predictions are based primarily on the intended object of interest rather than incidental surrounding context. We test this assumption in CNN-based agricultural image classification by comparing model performance on original images with performance on background-dominated patches extracted from the same images across eight publicly available agricultural benchmark datasets and four widely used CNN architectures. Background-dominated patches were classified above dataset-specific random chance for six of the eight datasets, and substantially above chance for four of them, indicating that contextual information contributes to model predictions for the majority of datasets evaluated. For these four datasets, we further evaluated whether this behavior reflected genuine class-discriminative information or was primarily attributable to class imbalance using macro-averaged precision, recall, and F1 together with class-balanced test subsets. The results show that contextual reliance does not admit a single explanation: class imbalance accounts for a substantial portion of the observed signal for some datasets and architectures, whereas above-chance contextual classification persists after balancing for others. Together with previous evidence from curated object recognition and cancer pathology imaging, these findings support the growing view that contextual bias is a recurring characteristic of CNN-based image classification rather than a phenomenon confined to a single application domain. More broadly, this work provides a systematic framework for quantifying contextual bias across heterogeneous image datasets by combining dataset-specific random-chance baselines, contextual bias categorization, macro-averaged evaluation, and class-balanced robustness analysis.

Abhilekha Dalal, Michael Okonoda, Eder Martinez +1

Image Classification Convolutional Neural Networks

Ask PDF

Jun 23, 2026

BenchX: Benchmarking AI Models for Cancer Detection and Localization with Demographic and Protocol Biases

Artificial intelligence (AI) has achieved remarkable success in medical imaging, but it is widely recognized that these models often perform inconsistently across real-world clinical settings. Such inconsistencies occur when patient demographics and imaging protocols vary, for example, in detecting small tumors, analyzing scans from different contrast phases, or evaluating patients of different ages or sexes. To quantify these inconsistencies, we develop a large-scale, open benchmark of 85,355 CT scans that systematically evaluates 12 tumor-detection AI models across tumor size, location, patient subgroup, and imaging protocol. We leverage large language models (LLMs) to extract and organize subgroup information from clinical data, which makes the analysis both scalable and reproducible. Our benchmark reveals that current state-of-the-art AI models, optimized for average accuracy, perform poorly in rare or underrepresented subgroups, such as young, female African Americans. However, collecting sufficient annotated data for these rare cases is often impractical. The benchmark provides a foundation for building more reliable and robust AI models for tumor detection and highlighting the need for rigorous, subgroup-level evaluation in medical imaging and computer vision. Datasets, code

Qi Chen, Wenxuan Li, Pedro R. A. S. Bassi +14

Johns Hopkins University · German Cancer Research Center · University Hospital Basel +10

Distribution Shift Medical Image Analysis Algorithmic Fairness Oncology Computed Tomography Medical Image Localization+1

Ask PDF

Sep 10, 2026

A Comparative Evaluation of Pre-trained Convolutional Neural Networks for Melanoma Detection

Early diagnosis of melanoma is critical for improving patient survival rates. However, accurately distinguishing melanoma from other skin lesions remains a significant clinical challenge due to the high visual similarity among lesion types and variability in image acquisition conditions. Artificial intelligence, particularly machine learning, has emerged as a promising tool to support dermatological diagnosis by automating feature extraction from medical images. Among the available approaches, convolutional neural networks (CNNs) have demonstrated strong performance in image classification tasks, making them well-suited for analyzing both dermatoscopic and histopathological images, given their ability to capture hierarchical visual patterns relevant to lesion characterization. Nevertheless, despite numerous pre-trained CNN architectures having been proposed, selecting the most appropriate one for a given imaging modality remains an open challenge. In this study, we evaluate pre-trained convolutional neural networks (CNNs) for skin lesion classification using dermatoscopic and histopathological image datasets. Experiments were conducted on the HAM10000, ISIC 2018, and CR-AI4SkIN datasets, evaluating the ResNet50, VGG16, VGG19, MobileNet, and InceptionV3 architectures under the same training protocol. The experimental evaluation showed that the models achieved accuracies ranging from 71% (InceptionV3 on ISIC 2018) to 84% (ResNet50 on HAM10000) on dermatoscopic images. For histopathological images, accuracies ranged from 72% (VGG19) to 83% (ResNet50) on the CR-AI4SkIN dataset. The results demonstrate that model performance differs between dermatoscopic and histopathological image modalities, showing that architectures exhibiting similar performance on dermatoscopic images exhibit different performance on histopathological data.

Wagner Moreno Schmitz, Marco Antonio de Castro Barbosa, Thiago Magalhães Amaral +2

Universidade Federal do Vale do São Francisco (Univasf), Petrolina, Pernambuco, Brasil · Universidade Tecnológica Federal do Paraná (UTFPR), Pato Branco, Paraná, Brasil

Skin Lesion Classification Medical Image Classification Convolutional Neural Networks Histopathology Image Classification

Ask PDF