Abnormalities and Disease Detection in Gastro-Intestinal Tract Images

Zeshan Khan; Muhammad Atif Tahir

eess.IVMar 23, 2026

Abnormalities and Disease Detection in Gastro-Intestinal Tract Images

Authors: Zeshan Khan, Muhammad Atif Tahir

Organizations: NATIONAL UNIVERSITY OF COMPUTER AND EMERGING SCIENCES FAST School of Computing

Abstract

Gastrointestinal (GI) tract image analysis plays a crucial role in medical diagnosis. This research addresses the challenge of accurately classifying and segmenting GI images for real-time applications, where traditional methods often struggle due to the diversity and complexity of abnormalities. The high computational demands of this domain require efficient and adaptable solutions. This PhD thesis presents a multifaceted approach to GI image analysis. Initially, texture-based feature extraction and classification methods were explored, achieving high processing speed (over 4000 FPS) and strong performance (F1-score: 0.76, Accuracy: 0.98) on the Kvasir V2 dataset. The study then transitions to deep learning, where an optimized model combined with data bagging techniques improved performance, reaching an accuracy of 0.92 and an F1-score of 0.60 on the HyperKvasir dataset, and an F1-score of 0.88 on Kvasir V2. To support real-time detection, a streamlined neural network integrating texture and local binary patterns was developed. By addressing inter-class similarity and intra-class variation through a learned threshold, the system achieved 41 FPS with high accuracy (0.99) and an F1-score of 0.91 on HyperKvasir. Additionally, two segmentation tools are proposed to enhance usability, leveraging Depth-Wise Separable Convolution and neural network ensembles for improved detection, particularly in low-FPS scenarios. Overall, this research introduces novel and adaptable methodologies, progressing from traditional texture-based techniques to deep learning and ensemble approaches, providing a comprehensive framework for advancing GI image analysis.

Figures & tables

GI	Gastrointestinal
GI-Tract	Gastrointestinal Tract
UC	Ulcerative Colitis
BBPS	Boston Bowel Preparation Scale
EAD	Endoscopic Artefact Detection
ICPR	International Conference on Pattern Recognition
ICPR EndoTech	ICPR Task of Endoscopic Abnormality Detection

Table 1

Figure 1.1: Example of Feature Selection.

Figure 1.2: Flowchart of Genetic Algorithm.

Figure 1.3: Flow of GA based feature selection method using wrapper approach.

Table 2.1: Image and Local Binary Patterns

Sr.	Paper	Technique Used	Features Used	Dataset	Acc.	F1
1	Z. Guo’s LBP [ 40 ]	LDSMT, CLBPC, CLBP-s and CLBP-m	LBP [ 53 ]	Clinically-Obtained	X	X
2	Polyp-alert [ 82 ]	Multi derivative Edge [ 82 ]	Texture	Self-Captured Videos	0.98
3	Normal and cancerous colonic mucosa detection [ 83 ]	Section Profile Calculation	Texture	Camera Images	0.90
4	Neural network-based colonoscopic diagnosis [ 84 ]	Step-wise discriminant Analysis	Texture	N/A	N/A	N/A
5	Texture and NN based Classification [ 85 ]	Neural Nets	Texture	Clinically-Obtained	N/A	N/A
6	Cold [ 86 ]	CoLD	Texture	Clinically-Obtained	0.95	N/A

Table 2.4: Analysis of the Approaches using texture features

Sr.	Paper	Technique Used	Features Used	Dataset	Acc.	F1
1	Deep Features Ensemble [ 95 ]	AlexNet [ 13 ] , GoogleLeNet [ 17 ] , SVM	Deep Features	Train: 6776, Test: 4166	0.82	N/A
2	Two stage Polyp Detection CAD [ 97 ]	Random Forest	Texture and Color Features	18900 Images	0.50	N/A
3	Part Based Derivative [ 98 ]	SVM, RBF	Texture Features	69 Videos	0.81	N/A
4	Color and Position and Texture for SVM [ 99 ]	SVM	Position versus Texture Features	4620 images	0.94	N/A
5	Elliptical shape features based polyps detection [ 100 ]	Ellipse Fitting on Edges	Elliptical Shape Features	8621 images	0.81	N/A
6	Sector Accumulation-Depth of Valleys Accumulation [ 101 ]	(SA-DOVA)	Segmentation Features	Custom	0.97	0.89

Table 2.5: Comparative Assessment of BCC Approaches for GI-Track Analysis

Sr.	Paper	Technique Used	Features Used	Dataset	Acc.	F1
1	Fusion of Deep CNN and geometric features [ 111 ]	VGG16, VGG19, Geometric Features	KNN	Clinically-Obtained	0.99	0.99
2	Finetune of the CaffeNet [ 112 ]	CaffeNet	NA	Private: 1930 Images	0.91	0.94

Table 2.6: Assessment of TCC Approaches for GI-Track Analysis

Sr.	Paper	Technique Used	Features Used	Acc.	F1
1	Deep With Global Features Approach [ 49 ]	RTC, RFC [ 116 ] and LMTC [ 117 ]	Global and Deep (ResNet50, Inception V3, Custom CNN)	0.96	0.83
2	SCL-UMD [ 113 ]	SVM	Baseline, Inception-V3, VGGNet features	0.96	0.85
3	Inception-like CNN [ 114 ]	CNN	Deep Features	0.94	0.76
4	HKBU Approach [ 19 ]	SVM	Bidirectional Marginal Fisher Analysis	0.93	0.70
5	Ensemble of Texture and Deep Learning Features [ 115 ]	Ensemble Learning	Texture and Deep Features	0.83	0.82

Table 2.7: Analysis of the Approaches on Kvasir V1 Dataset [ 2 ]

Sr.	Paper	Technique Used	Features Used	Acc.	F1
1	Weighted Discriminant Embedding (WDE) [ 23 ]	cost-sensitive nearest neighbor (CS-NN)	NA	0.95	0.48
2	Deep Features Selection [ 119 ]	Ant Colony Optimization, SVM		0.99	NA
3	Global Features With Deep Learning Approach [ 120 ]	Deep Learning	Global Texture Features	0.95	0.95
4	Preprocessing based Approach [ 127 ]	Inpainting based preprocessing and Deep Learning	Deep Features	0.99	0.94
5	Hyper Parameter Optimization [ 122 ]	Hyper Parameter Optimization	NA	0.93	0.92
6	Transfer learning with prioritized classification [ 124 ]	Transfer Learning	NA	0.98	0.82

Table 2.8: Analysis of the Approaches on Kvasir V2 Dataset [ 1 ]

Sr.	Paper	Technique Used	Features Used	Acc.	F1
1	FixMatch Semisupervised Learning Approach [ 128 ]	FixMatch semi-supervised learning algorithm	NA	0.91	NA
2	Convolutional-capsule Network [ 129 ]	CNN	NA	0.86	NA
3	Multi-Feature Fusion [ 130 ]	Fusion of Features	Visual Features	0.98	0.97
4	Explainable Deep Learning [ 131 ]	ResNet152	NA	0.93	NA

Table 2.9: Analysis of the Approaches on Hyper Kvasir Dataset [ 132 ]

Figure 3.1: Class Distribution of Kvasir 2018 dataset

Panel of Figure 3 — Figure 3.2: Some images from ICPR 2020 Dataset

Class Label	Number of Images	Class Label	Number of Images
barretts	41	barretts-short-segment	53
bbps-0-1	646	bbps-2-3	1148
cecum	1009	dyed-lifted-polyps	1002
dyed-resection-margins	989	esophagitis-a	403
esophagitis-b-d	260	hemorrhoids	6
ileum	9	impacted-stool	131

Table 3.1: Description of Hyper Kvasir Dataset [ 132 ]

Figure 3.3: Class Distribution of ICPR Endo Tech 2020 dataset

Dataset	Classes	Training Samples	Testing Samples	Class Distribution
Kvasir V1 [ 2 ]	8	4000	4000	Equal
Kvasir V2 [ 1 ]	16	5293	8740	Un-equal
Hyper Kvasir [ 132 ]	23	10662	721	Un-equal
Kvasir 2017 (10 Versions) [ 136 ]	8	72000	8000	Equal
EAD 2019 [ 137 ]	7	2147	195	Un-equal
EAD 2020 [ 138 ]	8	2200	99	Un-equal

Table 3.2: Abstract Analysis of the GI-Tract image datasets

Figure 4.1: Proposed Model for VGG19 Finetune For Deep Features Extraction

Figure 4.2: Proposed Model for Heterogeneous Approach for GI-Tract Abnormalities Detection

Figure 4.3: Reflections on the images of the Kvasir V2 [ 1 ] polyps category.

Figure 4.4: Data preprocessing for the reflection removal and data augmentation for the class imbalance problem.

Figure 4.5: Methodology of the Realtime Detection of GI-Tract Abnormalities.

Figure 4.6: Deep feature extraction methodology.

Figure 4.7: Methodology of TreeNet Architecture.

Figure 4.8: In-site of the layers of TreeNet Model.

Sr. No.	Characteristics	Decider Models	Strength/ Weakness	Details
1	Layered Processing	Neural Network Architecture	Strength	Feature Hierarchy and Abstraction
2	Feature Transformation	Neural Network Architecture	Strength	Feature Hierarchy and Abstraction
3	Model Complexity	Neural Network Architecture	Weakness	Affects Inference and Training Time
4	Hyper-parameters	Neural Network Architecture	Weakness	High Training time and Error probability based on Hyper Parameters
5	Differential Mode (Weight Adjustment)	Neural Network Architecture	Weakness	High Training Time
6	Gradient Descent	Neural Network Architecture	Weakness	Local Minima and Plateaus

Table 4.1: Selection of the various characteristics of machine learning deciders

Sr. No.	Characteristics	Decider Models
1	Layered Processing	Neural Network Architecture
2	Feature Transformation	Neural Network Architecture
3	Generalization	Ensemble Learning Model
4	Diversity-driven Performance	Ensemble Learning Model
5	Input Feature Manipulation	Ensemble Learning Model
6	Output Manipulation	Ensemble Learning Model

Table 4.2: Characteristics of Proposed Model

Panel of Figure 4 — Figure 4.9: Process Flow

	Accuracy	F1	MCC
Ensemble on 60 Images	0.956	0.625	0.614
Ensemble on 300 Images	0.957	0.587	0.603
Ensemble on 5293 Images	0.954	0.549	0.572
Selected Features 60 Images	0.961	0.611	0.597
Selected Features 300 Images	0.976	0.745	0.741
Selected Features 5293 Images	0.979	0.752	0.756

Table 5.1: Accuracy, F1, and MCC on different execution setups of the testing

Sr. No.	Measures	Value
1	True Positives	435
2	True Negatives	15576
3	False Positives	286
4	False Negatives	286
5	Precision	0.60
6	Recall/Sensitivity	0.60

Table 5.2: Accuracy Measures on Test Data

Predicted Actual	All	non-All
ALL	7271	1469
non-All	1469	129631

Table 5.3: Confusion matrix of all classes. There are total 16 classes and summary of all classes is shown

Predicted Actual	df-polyps	non df-polyps
dyed-lifted-polyps	339	236
non-dyed-lifted-polyps	217	7948

Table 5.4: Confusion matrix for class dyed-lifted-polyps versus non dyed-lifted-polyps. df=dyed-lifted.

Predicted class Actual class	dr-margins	non-dr-margins
dyed-resection-margins	387	232
non-dyed-resection-margins	177	7944

Table 5.5: Confusion matrix for class dyed-resection-polyps versus non dyed-resection-polyps. dr=dyed-resection.

Sr. No.	Approach	Acc.	F1-score	MCC	FPS
1	HKBU17 [ 19 , 156 ]	0.93	0.70	0.66	2
2	Ensemble17 [ 11 , 156 ]	0.94	0.77	0.74	2
3	Inception17 [ 114 ]	0.94	0.76	0.72	1
4	SCL-UMD17 [ 113 ]	0.96	0.85	0.83	1
5	DLGF17 [ 49 , 156 ]	0.96	0.83	0.79	46
6	Lire-CNN	0.99	0.90	0.90	41

Table 5.6: Analysis of some of the approaches for classification of abnormalities and landmarks detection on the Kvasir V1 dataset [ 136 ] .

Sr. No.	Approach	Acc.	F1-score	MCC	FPS
1	EEIC19 [ 169 ]	0.99	0.95	0.94	23
1	EEIC19 Fast [ 169 ]	0.99	0.88	0.89	3
2	TLCSS18 [ 125 , 156 ]	0.98	0.87	0.89	3
2	TLCSS18 Fast [ 125 , 156 ]	0.99	0.91	0.90	27
3	FRCSS18 [ 118 , 156 ]	0.99	0.94	0.94	23
4	DSTL18 [ 123 , 156 ]	0.99	0.89	0.89	1015

Table 5.7: Analysis of various approaches for classification of abnormalities and landmarks detection on the Kvasir V2 dataset [ 1 ] .

Sr. No.	Approach	Acc.	F1-score	MCC	FPS
1	HMTA [ 170 ]	0.99	0.87	0.86	20
2	HLNN [ 171 ]	0.98	0.90	0.89	129
3	EDTDE [ 172 ]	0.93	0.76	0.76	49
4	Lire-CNN	0.99	0.91	0.91	41

Table 5.8: Analysis of several approaches for classification of abnormalities and landmarks detection on the Kvasir V3 (Hyper Kvasir) dataset [ 132 ] .

Dataset	Original	Telea Ref.	Image Crop
Kvasir V1 [ 136 ]	0.70	0.71	0.52
Kvasir V2 [ 1 ]	0.81	0.82	0.47
Kvasir V3 [ 132 ]	0.82	0.82	0.61
DowPK	0.62	0.68	0.63

Table 5.9: Impact of reflection removal on the abnormalities detection F1-score.

Dataset	Original	Augmentation
Kvasir V1 [ 136 ]	0.71	0.71
Kvasir V2 [ 1 ]	0.82	0.84
Kvasir V3 [ 132 ]	0.82	0.83
Dow	0.68	0.79

Table 5.10: Impact of data augmentation on the detection F-1 Score.

Figure 5.1: Images of Z-line and Esophagitis with high inter class similarities. (Row1: Z-line, Row2: Esophagitis)

Class Label	Threshold
retroflex-rectum	0.6
out-of-patient	0.2
ulcerative-colitis	0.4
normal-cecum	0.4
normal-z-line	0.3
dyed-lifted-polyps	0.6

Table 5.11: Thresholds computed for the Kvasir V2 [ 1 ] Dataset Classes.

Dataset	Kvasir V1 [ 136 ]	Kvasir V2 [ 1 ]	Hyper Kvasir [ 132 ]	DowPK [ 168 ]
No Preprocessing	0.74	0.75	0.79	0.61
No Augmentation (Reflection Removed)	0.76	0.76	0.79	0.70
Individual Feature AutoColorCorrelogram	0.77	0.81	0.80	0.79
Individual Feature ColorLayout	0.67	0.75	0.74	0.75
Individual Feature EdgeHistogram	0.59	0.64	0.70	0.60
Individual Feature Gabor	0.38	0.30	0.32	0.31

Table 5.12: Impact on F1-score by applying various additions in the procedure on various datasets.

Sr. No.	Chunk	Size	Time	Acc	P	R	MCC	F1
1	1	4000	2506	0.75	0.78	0.75	0.72	0.74
2	0.9	3600	1886	0.75	0.78	0.75	0.72	0.74
3	0.8	3200	1918	0.75	0.78	0.75	0.72	0.74
4	0.7	2800	1631	0.74	0.76	0.74	0.71	0.73
5	0.6	2400	1515	0.73	0.75	0.73	0.70	0.72
6	0.5	2000	1428	0.73	0.75	0.73	0.70	0.72

Table 5.13: Results on Benchmark Dataset of Kvasir V1 using various data proportions [ 136 , 2 ]

Sr. No.	Chunk	Size	Time	Acc	P	R	MCC	F1
1	1	5293	7428	0.71	0.87	0.71	0.70	0.77
2	0.5	2645	3866	0.70	0.87	0.69	0.68	0.76
3	0.4	2117	3482	0.68	0.87	0.68	0.67	0.75
4	0.3	1588	2931	0.66	0.87	0.66	0.66	0.73
5	0.2	1060	2172	0.58	0.87	0.58	0.59	0.69
6	0.1	536	1506	0.52	0.85	0.52	0.52	0.61

Table 5.14: Results on Benchmark Dataset of Kvasir V2 using various data proportions [ 1 ]

Figure 5.2: F1-score of the Hyper Kvasir dataset using various chunks of the training data [ 132 ]

Figure 5.3: F1-score of the DowPK dataset using various chunks of the training data

Model	Recall	Precision	Dice	mIoU
Unet	75.23%	84.52%	71.91%	59.53%
resunet++	64.97%	89.81%	78.35%	69.48%
sepv_conv_resunet++	60.55%	93.31%	77.25%	67.56%
dsapp_resunet++	69.72%	82.62%	76.66%	66.71%
dsapp_relu_resunet++	61.54%	92.33%	74.63%	66.03%

Table 5.15: Test Data Results

Model	Params	GFLOPs
Unet	3,588,997	7,165,148
resunet++	4,371,265	8,718,068
sepv_conv_resunet++	3,047,265	6,070,057
dsapp_resunet++	5,024,705	10,024,898
dsapp_relu_resunet++	5,024,705	10,024,898

Table 5.16: Model Size

Figure 5.5: Best segmentation results from MediaEval 2021 dataset [ 174 , 177 ]

Figure 5.6: Erroneous segmentation results from MediaEval 2021 dataset [ 174 , 177 ]

Explore similar work

Apr 18, 2026

A Two-Stage Deep Learning Framework for Segmentation of Ten Gastrointestinal Organs from Coronal MR Enterography

Accurate segmentation of gastrointestinal (GI) organs in magnetic resonance enterography (MRE) is critical for diagnosing inflammatory bowel disease (IBD). However, anatomical variability, class imbalance, and low tissue contrast hinder reliable automation. This study proposes a dual-stage deep learning framework for organ-specific segmentation of GI structures from coronal MRE images to address these challenges. A publicly available MRE dataset of 3,195 coronal T2-weighted HASTE slices from 114 IBD patients was used. Initially, a DenseNet201-UNet++ model generated coarse masks for ROI extraction. A DenseNet121-SelfONN-UNet model was then trained on organ-specific patches. Extensive data augmentation, normalization, five-fold cross-validation, and class-specific weighting were applied to mitigate severe class imbalance, particularly for the appendix. The initial stage achieved strong organ localization but underperformed for the appendix; class weighting improved its DSC from 6.76% to 85.76%. The second-stage DenseNet121-SelfONN-UNet significantly enhanced segmentation across all GI structures, with notable DSC gains (cecum +23.62%, sigmoid +18.57%, rectum +17.99%, small intestine +16.06%). Overall, the framework achieved mDSC of 88.99%, mIoU of 84.76%, and mHD95 of 6.94 mm, outperforming all baselines. This framework demonstrates the effectiveness of a coarse-to-fine, organ-aware segmentation strategy for intestinal MRE. Despite higher computational cost, it shows strong potential for clinical translation and enables anatomically informed diagnostic tools in gastroenterology.

Ashiqur Rahman, Md. Abu Sayed, Md Sharjis Ibne Wadud +3

Department of Biomedical Physics and Technology, University of Dhaka, Dhaka 1000, Bangladesh. · Department of Biomedical Engineering, Jashore University of Science and Technology, Jashore 7408, Bangladesh. · Department of Radiology, Hamad Medical Corporation, Doha, Qatar. +1

Magnetic Resonance Imaging Medical Image Segmentation

Ask PDF

Oct 11, 2023

Precise localization within the GI tract by combining classification of CNNs and time-series analysis of HMMs

This paper presents a method to efficiently classify the gastroenterologic section of images derived from Video Capsule Endoscopy (VCE) studies by exploring the combination of a Convolutional Neural Network (CNN) for classification with the time-series analysis properties of a Hidden Markov Model (HMM). It is demonstrated that successive time-series analysis identifies and corrects errors in the CNN output. Our approach achieves an accuracy of

98.04\%

on the Rhode Island (RI) Gastroenterology dataset. This allows for precise localization within the gastrointestinal (GI) tract while requiring only approximately 1M parameters and thus, provides a method suitable for low power devices

Julia Werner, Christoph Gerum, Moritz Reiber +2

Department of Computer Science, University of Tübingen, Germany. · Department of Mathematics, ETH Zürich, Switzerland.

Endoscopy Medical Imaging Medical Image Localization Hidden Markov Models

Ask PDF

Sep 4, 2026

MultiAttenGastro: Multi-Dimensional Attention Augmentation for Gastrointestinal Endoscopy Classification

Automated gastrointestinal (GI) endoscopy classification requires models that generalize across diverse modalities and class distributions, often far from natural-image pretraining. We propose MultiAttenGastro, a plug-and-play attention framework with parallel 1-D channel, 2-D spatial, and 3-D contextual heads, and present the first systematic cross-dataset evaluation across eight CNN and transformer backbones on five public GI datasets (80 backbone--dataset runs). We find that attention effectiveness is not universal but tracks the representational gap between ImageNet features and the target distribution: MultiAttenGastro improves 6 of 8 backbones on Kvasir-Capsule (14-class WCE, large gap; best macro F1 98.33%), is uniformly negative on the small-gap Kvasir-v2 benchmark (0/8), and shows mixed outcomes on datasets with intermediate gap. Five-seed ablation on the strongest case (Kvasir-Capsule, ConvNeXt-Tiny) shows this improvement is directionally consistent, but not statistically decisive (paired

t

p=0.47

; Wilcoxon:

p=0.63

), and that individual attention heads are not uniformly beneficial in isolation only their combination yields a positive mean effect. Centered Kernel Alignment (CKA) analysis links this pattern to representational redundancy: low inter-head CKA under large domain gaps coincides with the framework's only consistent gains, while high redundancy under small gaps coincides with its losses. We report these results, including the non-significant margins, as evidence for when and why multi-dimensional attention helps GI endoscopy classification, rather than as a claim that MultiAttenGastro is a strictly superior architectural choice.

Sadhana Devarajan, Praveen Kumar Chandaliya, Dhruvin Jashvant Kumar Shah +2

Sardar Vallabhbhai National Institute of Technology, Surat, Gujarat, India · Swasthyam Gastro and Liver Hospital, Surat, Gujarat, India · Norwegian University of Science and Technology, Norway

Medical Image Classification Attention Mechanisms Endoscopy Hybrid Attention Visual Representation Learning

Ask PDF