DeepHistoViT: An Interpretable Vision Transformer Framework for Histopathological Cancer Classification

Ravi Mosalpuri; Mohammed Abdelsamea; Ahmed Karam Eldaly

cs.CVMar 12, 2026

DeepHistoViT: An Interpretable Vision Transformer Framework for Histopathological Cancer Classification

Authors: Ravi Mosalpuri, Mohammed Abdelsamea, Ahmed Karam Eldaly

Abstract

Histopathology remains the gold standard for cancer diagnosis because it provides detailed cellular-level assessment of tissue morphology. However, manual histopathological examination is time-consuming, labour-intensive, and subject to inter-observer variability, creating a demand for reliable computer-assisted diagnostic tools. Recent advances in deep learning, particularly transformer-based architectures, have shown strong potential for modelling complex spatial dependencies in medical images. In this work, we propose DeepHistoViT, a transformer-based framework for automated classification of histopathological images. The model employs a customized Vision Transformer architecture with an integrated attention mechanism designed to capture fine-grained cellular structures while improving interpretability through attention-based localization of diagnostically relevant regions. The framework is evaluated on three publicly available histopathology datasets covering lung cancer, colon cancer, and acute lymphoblastic leukaemia. Experimental results demonstrate state-of-the-art performance across all datasets, with classification accuracy, precision, recall, F1-score, and ROC-AUC reaching 100 percent on the lung and colon cancer datasets, and 99.85 percent, 99.84 percent, 99.86 percent, 99.85 percent, and 99.99 percent respectively on the acute lymphoblastic leukaemia dataset. All performance metrics are reported with 95 percent confidence intervals. These results highlight the effectiveness of transformer-based architectures for histopathological image analysis and demonstrate the potential of DeepHistoViT as an interpretable computer-assisted diagnostic tool to support pathologists in clinical decision-making.

Figures & tables

Figure 1: Example histopathological images from the datasets used in this study. Top row: Colon tissue samples from the LC25000 dataset showing (h) benign colon tissue and (i) colon adenocarcinoma. Middle row: Lung tissue samples showing (a) benign lung tissue, (b) squamous cell carcinoma, and (c) adenocarcinoma. Bottom row: Peripheral blood smear images from the acute lymphoblastic leukaemia (ALL) dataset showing (d) benign cells, (e) early Pre-B lymphoblasts, (f) Pre-B lymphoblasts, and (g) Pro-B lymphoblasts. All histopathological images are stained using hematoxylin and eosin (H&E).

Figure 2: Overview of the proposed Vision Transformer (ViT - 16) based pipeline for histopathology image classification. The framework consists of three stages: (i) Preprocessing, where raw histopathology images are resized to 256 pixels, augmented through random resizing, horizontal flipping, rotation, and colour jitter, followed by ImageNet normalisation; (ii) Feature Extraction, leveraging a ViT - 16 base model pre - trained on ImageNet to obtain the CLS token representation; and (iii) Custom Classification Head, comprising sequential dense layers (512 and 256 units) with batch normalization, ReLU activation, and configurable dropout, culminating in a sigmoid - activated output layer. This architecture integrates transformer - based global feature learning with a tailored classification head to enhance performance on medical image analysis tasks.

Class	Precision	Recall	F1-Score	Support
Benign	1.0000	1.0000	1.0000	1000
ACC	1.0000	1.0000	1.0000	1000
SCC	1.0000	1.0000	1.0000	1000
Accuracy			1.0000	3000
Macro Avg	1.0000	1.0000	1.0000	3000
Weighted Avg	1.0000	1.0000	1.0000	3000

Table 1: Classification performance of the proposed DeepHistoViT model on the LC25000 lung cancer dataset. Precision, recall, and F1-score values are reported for each class, along with overall accuracy.

Class	Precision	Recall	F1-Score	Support
Benign	1.0000	1.0000	1.0000	1000
ACC	1.0000	1.0000	1.0000	1000
Accuracy			1.0000	2000
Macro Avg	1.0000	1.0000	1.0000	2000
Weighted Avg	1.0000	1.0000	1.0000	2000

Table 2: Classification performance of the proposed DeepHistoViT model on the LC25000 colon cancer dataset. Precision, recall, and F1-score values are reported for each class, along with overall accuracy.

Class	Precision	Recall	F1-Score	Support
Benign	1.0000	0.9901	0.9950	101
Early	0.9949	1.0000	0.9975	197
Pre	1.0000	1.0000	1.0000	193
Pro	1.0000	1.0000	1.0000	161
Accuracy			0.9985	652
Macro Avg	0.9987	0.9975	0.9981	652

Table 3: Classification performance of the proposed DeepHistoViT model on the ALL dataset. Precision, recall, and F1-score values are reported for each class, along with overall accuracy.

Panel of Figure 3 — Figure 3: Confusion matrices for the three evaluated datasets: lung cancer (LC25000), colon cancer (LC25000), and acute lymphoblastic leukaemia (ALL). The lung and colon datasets show perfect classification performance, while the ALL dataset shows near-perfect classification with minimal misclassification.

Panel of Figure 4 — Figure 4: Training and validation accuracy and loss curves across datasets, demonstrating stable convergence and consistent generalisation performance.

Panel of Figure 5 — Figure 5: Attention visualisation generated by the proposed DeepHistoViT model. For each example, the original histopathology image is shown alongside the corresponding attention map, highlighting regions that contribute most strongly to the classification decision. The attention maps demonstrate spatial localisation of diagnostically relevant morphological features.

Study	Model	Accuracy (%)	Precision (%)	Recall (%)	F1-Score (%)	ROC-AUC
Kumar et al. Kumar et al. (2024) (2024)	ML3CNet	99.72	99.64	99.66	99.65	99.78
Attallah et al. Attallah (2025) (2025)	Multi-CNN	99.78	99.78	99.78	99.78	–
Shahadat et al. Shahadat et al. (2024) (2025)	DL	100.00	100.00	100.00	100.00	100.00
Proposed	DeepHistoViT	100.00	100.00	100.00	100.00	100.00

Table 4: Comparison of classification performance for lung cancer (LC25000) dataset across recent studies and the proposed DeepHistoViT model.

Study	Model	Accuracy (%)	Precision (%)	Recall (%)	F1-Score (%)	ROC-AUC
Ochoa-Ornelas et al. Ochoa-Ornelas et al. (2025) (2025)	EfficientNet-B3	99.39	99.39	99.39	99.39	–
Attallah et al. Attallah (2025) (2025)	Multi-CNN	99.78	99.78	99.78	99.78	–
Shahadat et al. Shahadat et al. (2024) (2025)	DL	100.00	100.00	100.00	100.00	100.00
Proposed	DeepHistoViT	100.00	100.00	100.00	100.00	100.00

Table 5: Comparison of classification performance for colon cancer (LC25000) dataset across recent studies and the proposed DeepHistoViT model.

Study	Model	Accuracy (%)	Precision (%)	Recall (%)	F1-Score (%)	ROC-AUC
Muhammad et al. Muhammad et al. (2025) (2025)	EfficientNet-B7	96.78	97.00	96.25	96.57	99.85
Basaran et al. Başaran et al. (2025) (2025)	DarkNet19 ESA	98.52	98.40	98.22	98.30	99.95
Thiriveedhi et al. Thiriveedhi et al. (2025) (2025)	ALL-Net	99.32	99.25	99.50	99.50	–
Proposed	DeepHistoViT	99.85	99.85	99.85	99.85	99.99

Table 6: Performance comparison of recent models on the acute lymphoblastic leukaemia dataset.

Explore similar work

DSVTLA: Deep Swin Vision Transformer-Based Transfer Learning Architecture for Multi-Type Cancer Histopathological Cancer Image Classification
Apr 10, 2026Muazzem Hussain Khan, Tasdid Hasnain, Md. Jamil khan +4Histopathology Image Classification Hybrid CNN-Transformer Architectures
Department of Computer Science and Engineering, Metropolitan University, Sylhet-3104 · Department of Statistics, Pabna University of Science and Technology, Pabna-6600, Bangladesh · Department of Computer Science and Engineering, Rajshahi University of Engineering and Technology, Rajshahi-6204, Bangladesh +1
MagViT: Interpretable Multi-Magnification Transformers with Patient-Level Model Selection for Breast Histopathology
Aug 16, 2026Nabil Ashab, Soumit Kumar Kundu, Saif Mahmud Parvez +3Histopathology Image Classification Vision Transformer
Department of Computer Science and Engineering, Dhaka International University, Dhaka, Bangladesh · Department of Microbiology, TMSS Medical College, Bogura, Bangladesh
Comprehensive Benchmarking of Deep Learning Architectures for Lung Cancer Histopathology
Aug 16, 2026Hadi Hasan, Safaa Salman, Lama Sleem +2Histopathology Image Classification Computational Pathology
Electrical and Computer Engineering American University of Beirut Beirut, Lebanon · Computational Science American University of Beirut Beirut, Lebanon · Industrial Engineering American University of Beirut Beirut, Lebanon

Apr 10, 2026J/K move · Enter open · S save

DSVTLA: Deep Swin Vision Transformer-Based Transfer Learning Architecture for Multi-Type Cancer Histopathological Cancer Image Classification

Muazzem Hussain Khan, Tasdid Hasnain, Md. Jamil khan, Ruhul Amin+3

Department of Computer Science and Engineering, Metropolitan University, Sylhet-3104 · Department of Statistics, Pabna University of Science and Technology, Pabna-6600, Bangladesh · Department of Computer Science and Engineering, Rajshahi University of Engineering and Technology, Rajshahi-6204, Bangladesh+1

Histopathology Image Classification Hybrid CNN-Transformer Architectures Vision Transformer Transfer Learning

Open paper AskPDF arXiv

In this study, we proposed a deep Swin-Vision Transformer-based transfer learning architecture for robust multi-cancer histopathological image classification. The proposed framework integrates a hierarchical Swin Transformer with ResNet50-based convolution features extraction, enabling the model to capture both long-range contextual dependencies and fine-grained local morphological patterns within histopathological images. To validate the efficiency of the proposed architecture, an extensive experiment was executed on a comprehensive multi-cancer dataset including Breast Cancer, Oral Cancer, Lung and Colon Cancer, Kidney Cancer, and Acute Lymphocytic Leukemia (ALL), including both original and segmented images were analyzed to assess model robustness across heterogeneous clinical imaging conditions. Our approach is benchmarked alongside several state-of-the-art CNN and transfer models, including DenseNet121, DenseNet201, InceptionV3, ResNet50, EfficientNetB3, multiple ViT variants, and Swin Transformer models. However, all models were trained and validated using a unified pipeline, incorporating balanced data preprocessing, transfer learning, and fine-tuning strategies. The experimental results demonstrated that our proposed architecture consistently gained superior performance, reaching 100% test accuracy for lung-colon cancer, segmented leukemia datasets, and up to 99.23% accuracy for breast cancer classification. The model also achieved near-perfect precision, f1 score, and recall, indicating highly stable scores across divers cancer types. Overall, the proposed model establishes a highly accurate, interpretable, and also robust multi-cancer classification system, demonstrating strong benchmark for future research and provides a unified comparative assessment useful for designing reliable AI-assisted histopathological diagnosis and clinical decision-making.

Abstract

Explore similar work

DSVTLA: Deep Swin Vision Transformer-Based Transfer Learning Architecture for Multi-Type Cancer Histopathological Cancer Image Classification

MagViT: Interpretable Multi-Magnification Transformers with Patient-Level Model Selection for Breast Histopathology

Comprehensive Benchmarking of Deep Learning Architectures for Lung Cancer Histopathology