Computer-aided detection (CADe) systems for colonoscopy promise to reduce clinical miss rates, yet reliable real-world deployment remains elusive. This translational gap stems in part from a structural flaw in model development: the reliance on curated datasets that under-represent the long negative stretches and procedure-related artifacts characteristic of routine examinations. Training and evaluating architectures strictly on these lesion-centric benchmarks creates an illusion of success, since such benchmarks cannot capture clinically crucial metrics. To expose this gap, we establish TRUE-Colon, a standardized benchmarking protocol that measures key deployment characteristics alongside localization accuracy, and evaluate four real-time architectures (Faster R-CNN, YOLOv8, YOLOv11, RT-DETR) across curated benchmarks (SUN, PICCOLO) and 60 unedited, full-length procedures (REAL-Colon). We observe a consistent transfer asymmetry: models trained strictly on curated clips suffer a severe performance collapse when evaluated on full procedures, whereas procedure-trained models substantially improve rejection of non-polyp content on REAL-Colon, and largely retain their accuracy on curated benchmarks. Beyond transferability, we find that the Transformer detector attains the strongest sensitivity and the earliest, most persistent detections, while the convolutional detectors stay competitive at a higher throughput. Together, these results indicate that both training and benchmarking for deployable CADe should shift from curated, lesion-centric clips toward full-procedure data and deployment-relevant operating points. Source code is available at https://github.com/sdoerrich97/true-colon.
Figures & tables
Figure 1: Qualitative YOLOv11 results on representative colonoscopy frames. The examples illustrate varying levels of difficulty, including clear polyp visibility, specular highlights, motion blur, and challenging background artifacts. Green bounding boxes show the ground-truth, while red bounding boxes indicate predictions.
Dataset
Split
POS
NEG
Total
Neg (%)
REAL-Colon [ 1 ]
Train
230,935
1,475,700
1,706,635
86.47
(CC BY 4.0)
Val
29,067
330,236
359,303
91.91
Test
82,106
489,979
572,085
85.65
SUN [ 15 ]
Train
34,915
58,839
93,754
62.76
(Research Agreement)
Val
5,792
23,845
29,637
80.46
Test
8,429
26,869
35,298
76.12
Table 1: Dataset distribution including positive/negative frame counts per split and proportion of negative frames. REAL-Colon maintains the highest proportion of negative frames ( >85% ) in contrast to the curated benchmarks SUN and PICCOLO.
Dataset (Train → Test)
Model
τ⋆
mAP 50:95
mAP 50
TPR
FPR
SUN → SUN
Faster R-CNN
0.07
0.408
0.738
0.739
0.044
YOLOv8
0.07
0.396
0.694
0.731
0.046
YOLOv11
0.10
0.411
0.724
0.735
0.047
RT-DETR
0.12
0.389
0.689
0.725
0.050
PICCOLO → PICCOLO
Faster R-CNN
–
0.531
0.743
–
–
YOLOv8
–
0.487
0.673
–
–
Table 2: Curated in-domain baselines on SUN and PICCOLO. Frame-level TPR/FPR are reported for SUN only; PICCOLO frame-level metrics are omitted because the official test split contains only one negative image. The convolutional detectors achieve high accuracy on both curated domains, while RT-DETR underperforms on PICCOLO.
Dataset (Train → Test)
Model
τ⋆
mAP 50:95
mAP 50
TPR
FPR
Curated → Procedure (Test: REAL-Colon)
SUN → REAL-Colon
Faster R-CNN
0.20
0.131
0.220
0.290
0.043
YOLOv8
0.25
0.105
0.177
0.261
0.045
YOLOv11
0.04
0.097
0.164
0.240
0.040
RT-DETR
0.25
0.132
0.225
0.310
0.040
PICCOLO → REAL-Colon
Faster R-CNN
0.99
0.080
0.154
0.271
0.060
Table 3: Cross-distribution transfer between curated benchmarks (SUN, PICCOLO) and full procedures (REAL-Colon). For curated → REAL-Colon, thresholds are chosen to approximately match the evaluated frame-level FPR; reverse transfer reports detection-level generalization back to curated domains.
Metric
F. R-CNN
YOLOv8
YOLOv11
RT-DETR
mAP50 ( ↑ )
0.324±0.03
0.413±0.00
0.384±0.01
0.488±0.02
A: Det.
mAP50:95 ( ↑ )
0.196±0.02
0.272±0.01
0.245±0.01
0.327±0.01
TPR τ ( ↑ )
0.463±0.06
0.508±0.02
0.453±0.00
0.720±0.04
FPR τ ( ↓ )
0.046±0.02
0.020±0.00
0.017±0.00
0.082±0.03
TPR τ⋆ ( ↑ )
0.463±0.07
0.605±0.03
0.573±0.01
0.651±0.05
B: Frame
FPR τ⋆ ( ↓ )
0.046±0.02
0.044±0.01
0.043±0.00
0.043±0.02
Table 4: REAL-Colon test performance at a fixed ( τ=0.2 ) and a normalized ( τ⋆ targeting FPR≈4 – 5% , selected post-hoc on the test split for cross-model normalization) threshold. Bold marks the best mean per row; lesion-level differences (block C, n=21 ) fall within seed variance and are not claimed as significant.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Video
Range
Video
Range
Video
Range
1_1
920–45137
1_2
1620–25974
1_3
1967–39372
1_4
850–45385
1_5
360–28544
1_6
786–48217
1_7
1050–37920
1_8
1165–34891
1_9
2031–39884
1_10
2075–46215
1_11
380–23390
1_12
530–40842
1_13
1996–22417
1_14
347–28711
1_15
910–30831
2_1
1–25421
2_2
414–24916
2_3
1–47191
Appendix
Table 0.B1: Retained inside-patient frame-ID intervals for the 60 REAL-Colon videos (labeled patient_video). Frames outside the listed range are discarded as out-of-patient.
Cases 1–57
Cases 58–113
Case ID
Type
Split
#Images
Case ID
Type
Split
#Images
1
POS
train
527
58
POS
train
267
2
POS
train
1313
59
POS
train
646
3
POS
train
292
60
POS
train
146
4
POS
train
80
61
POS
train
679
5
POS
train
930
62
POS
train
351
Appendix
Table 0.B2: SUN case-level image counts and train/val/test assignment (positive cases IDs 1–100, negative cases 101–113). Splitting by case rather than by frame prevents patient-level leakage into the evaluation folds.
Faster R-CNN
YOLOv8
YOLOv11
RT-DETR
mAP50
0.324±0.032
0.413±0.002
0.384±0.009
0.488±0.019
mAP75
0.216±0.023
0.299±0.010
0.263±0.005
0.359±0.009
mAP50:95
0.196±0.021
0.272±0.005
0.245±0.005
0.327±0.009
AR1
0.295±0.014
0.376±0.003
0.357±0.005
0.415±0.011
AR10
0.387±0.030
0.466±0.009
0.453±0.012
0.585±0.007
AR100
0.394±0.035
0.471±0.012
0.457±0.013
0.672±0.008
Appendix
Table 0.C1: Extended detection-level performance on REAL-Colon (mean ± std over three seeds). mAPt denotes mean average precision at IoU threshold t , mAP50:95 averages over IoU 0.50 – 0.95 , and ARk is average recall allowing k detections per image. RT-DETR leads at every threshold and recall budget.
Faster R-CNN
YOLOv8
YOLOv11
RT-DETR
APsmall
0.000
0.000
0.000
0.002
APmedium
0.012±0.006
0.049±0.009
0.054±0.005
0.044±0.014
APlarge
0.207±0.021
0.284±0.005
0.255±0.005
0.343±0.010
Appendix
Table 0.C2: Average precision by object size on REAL-Colon, following the COCO definition (small ≤322 , medium 322 – 962 , large ≥962 pixels), as mean ± std over three seeds. Small-polyp AP is near zero for all detectors, so localization is driven by medium and large lesions.
Figure 0.C1: AFROC curves on REAL-Colon. Left: model-specific confidence thresholds ( FPR≈4 – 5% ). Right: fixed τ=0.2 for all models. The y -axis is the fraction of ground-truth lesion boxes detected and the x -axis is the fraction of negative frames raising at least one false-positive alert. RT-DETR retains the highest sensitivity at the lowest false-positive fractions.
Metric
Faster R-CNN
YOLOv8
YOLOv11
RT-DETR
Shared confidence τ=0.20
Precision
0.524±0.086
0.701±0.022
0.715±0.011
0.398±0.095
Recall
0.463±0.066
0.508±0.022
0.453±0.004
0.720±0.041
Model-specific thresholds (FPR ≈ 4–5%)
Precision
0.524±0.086
0.523±0.029
0.516±0.017
0.577±0.094
Recall
0.463±0.066
0.605±0.030
0.573±0.009
0.651±0.049
Appendix
Table 0.C3: Detection-level precision and recall under three confidence-threshold strategies (shared τ=0.2 , model-specific, and per-seed; mean ± std over three seeds). A shared threshold penalizes RT-DETR, which over-fires at low τ ; model-specific thresholds place every detector at a comparable false-positive rate.
Metric
Faster R-CNN
YOLOv8
YOLOv11
RT-DETR
Model-specific thresholds (FPR ≈ 4–5%)
Sensitivity / TPR
0.463±0.066
0.605±0.030
0.573±0.009
0.651±0.049
Specificity
0.954±0.017
0.956±0.006
0.957±0.003
0.957±0.017
FPR
0.046±0.017
0.044±0.006
0.043±0.003
0.043±0.017
Precision
0.637±0.086
0.698±0.023
0.693±0.020
0.723±0.070
F1 score
0.532±0.047
0.648±0.015
0.627±0.013
0.681±0.001
Appendix
Table 0.C4: Frame-level performance under the model-specific and per-seed threshold strategies (mean ± std over three seeds), extending the frame-level block of Table 4 with specificity, precision, F1 , and F2 . At matched operating points RT-DETR attains the best F1 and F2 . Per-seed refitting only reduces variance.
Faster R-CNN
YOLOv8
YOLOv11
RT-DETR
Fixed confidence threshold τ=0.20
Lesions detected ( ≥1 match)
21.0
21.0
21.0
21.0
Lesions detected ( ≥25% frames)
19.0
19.3
18.7
21.0
Lesions detected ( ≥50% frames)
11.7
12.7
12.7
19.7
Detected within 1 s
1 1.0
1 1.7
1 1.0
1 7.7
Detected within 3 s
1 7.0
1 5.3
1 5.3
12.0
Appendix
Table 0.C5: Lesion-level consistency and latency on REAL-Colon ( n=21 lesions, means over three seeds), under a fixed τ=0.20 and under model-specific thresholds. Rows report how many lesions are detected at all and on ≥25% / ≥50% of their visible frames, how many are first detected within 1 / 3 / 5 s, and the latency to the first detected frame. RT-DETR detects lesions more persistently and earlier than the convolutional detectors.
Figure 0.C2: Per-lesion detection persistence on REAL-Colon at model-specific operating thresholds. Blue bars: mean detected frames. Orange bars: mean missed frames. Dotted lines: min/max across seeds. The number above each bar is the first-detection frame index. Persistence varies widely by lesion, and the zero-shot SSL/TSA lesions are among the least consistently detected.
Figure 0.C3: Representative frames for the zero-shot histological subtypes in the test set. Top: Lesion 7 (SSL). Middle: Lesion 14 (SSL). Bottom: Lesion 21 (TSA). Green boxes indicate ground-truth annotations. These subtypes are absent from the training set; Lesion 14 (SSL) shows the highest miss rates across all architectures.
Detectron2
Model
Inf [ms]
Eval [ms]
Total [ms]
FPS inf
FPS total
Faster R-CNN
25.07±2.72
0.20±0.00
26.70±2.52
39.9
37.5
Appendix
Table 0.C6: Runtime on REAL-Colon at batch size 1 on one NVIDIA H100 NVL GPU (mean ± std). Times are per image for pre-processing, inference, and post-processing. FPSinf counts inference only and FPStotal the full pipeline. All detectors exceed the 30 FPS real-time threshold.
Resolution
conf
mAP50:95
mAP50
TPR
FPR
FPSinf
FPStotal
224×224
0.01
0.184
0.289
0.444
0.044
212.9
183.4
640×640
0.05
0.246
0.384
0.584
0.043
192.3
163.9
1024×1024
0.03
0.255
0.400
0.576
0.044
184.3
143.8
Appendix
Table 0.C7: Effect of input resolution on YOLOv11-M (REAL-Colon). 640×640 gives the best accuracy-to-throughput trade-off and is the resolution used in the main paper.
Model
conf
mAP50:95
mAP50
TPR
FPR
Faster R-CNN
0.200
0.203
0.330
0.497
0.043
YOLOv8-M
0.013
0.221
0.342
0.524
0.048
YOLOv11-M
0.015
0.194
0.297
0.504
0.046
RT-DETR
0.250
0.290
0.432
0.634
0.050
SSD-300 (author setup)
–
0.216
0.338
0.505
0.054
Appendix
Table 0.C8: Comparison at 300×300 following the original REAL-Colon author setup [ 1 ] on the full dataset. Because the original IoU and confidence thresholds are unspecified, TPR and FPR are for observational reference only.
Model / Config
conf
mAP50:95
mAP50
TPR
FPR
YOLOv11-S
0.040
0.221
0.353
0.570
0.045
YOLOv11-M
0.050
0.246
0.384
0.584
0.043
YOLOv11-L
0.007
0.276
0.420
0.628
0.044
Neg. ratio 1:0.15
0.27
0.209
0.328
0.482
0.048
Neg. ratio 1:0.50
0.27
0.155
0.254
0.450
0.047
Neg. ratio 1:1.00
0.10
0.253
0.401
0.602
0.048
Appendix
Table 0.C9: Effect of model capacity, negative-sampling ratio, and optimizer on YOLOv11 (REAL-Colon, 640×640 unless noted). A balanced 1:1 negative ratio and larger capacity help most, while the optimizer barely matters.
Metric
GI Genius v1
GI Genius v2
EndoAID A
EndoAID B
EndoMind
Sensitivity / TPR
0.5063
0.6785
0.6560
0.5295
0.6022
Specificity
0.9694
0.9577
0.9719
0.9930
0.9589
FPR
0.0275
0.0380
0.0252
0.0063
0.0369
Precision
0.5966
0.5701
0.6658
0.8720
0.5499
F1 score
0.5915
0.6391
0.6943
0.7177
0.5960
First detection (ms)
1510
607
659
1316
1083
Appendix
Table 0.C10: Frame-level performance of commercial CADe systems and the open-source EndoMind detector ( IoU>0 , τ=0.2 ), as reported in [ 22 ] . Values come from a different dataset and pipeline, so they contextualize rather than directly compare with our detectors.
Figure 0.C4: Challenging REAL-Colon test frames on which all evaluated architectures produce false negatives or false positives. From left to right: specular highlights, motion blur, instrument occlusion, and a flat non-polypoid lesion. These conditions are the main shared failure mode across detectors.
Figure 0.C5: Qualitative detection results across all four architectures on representative REAL-Colon test frames. Each row is one frame under varying visual conditions. The columns are Faster R-CNN, YOLOv8, YOLOv11, and RT-DETR (left to right). Green boxes are ground-truth annotations and red boxes are model predictions. Predictions agree on clear polyps and diverge on flat and NBI frames.
Medical image benchmarks compare AI methods on curated clinical images, tasks, and evaluation protocols to measure accuracy, robustness, and clinical usefulness.
Accurate polyp segmentation in colonoscopy is essential for early colorectal cancer detection, yet real-world clinical environments pose persistent challenges such as motion blur, specular reflections, and illumination instability. Most existing methods are optimized on clean benchmark images and suffer noticeable performance degradation when deployed in authentic surgical scenarios. We propose DepthPolyp, a lightweight and robust segmentation framework based on pseudo-depth-guided multi-task learning and efficient feature modulation. The architecture combines hierarchical Ghost factorization for compact feature generation, Interleaved Shuffle Fusion for low-cost cross-scale interaction, and Dynamic Group Gating for adaptive group-wise feature weighting. Extensive experiments demonstrate that DepthPolyp achieves strong cross-dataset generalization when trained on degraded data and evaluated on both clean and noisy target domains, consistently outperforming lightweight baselines and remaining competitive with substantially larger models. In real surgical video evaluation on PolypGen, DepthPolyp achieves better segmentation performance than models up to 20× larger while preserving real-time inference speed. With only 3.57M parameters and 0.86 GMACs, the proposed method runs at over 180 FPS on mobile devices, making it well suited for real-time deployment in resource-constrained clinical environments. Code and pretrained weights are available at: https://github.com/ReaganWu/DepthPolyp/
Zhuoyu Wu, Wenhui Ou, Lexi Zhang +5
CyPhi AI Lab, Monash University, Malaysia Campus, Malaysia · Department of Electronic & Computer Engineering, Hong Kong University of Science & Technology, Hong Kong, P.R. China · Harbin Institute of Technology, Harbin, P.R. China +1
Progress in colonoscopy polyp segmentation is routinely reported through leaderboard comparisons on a small set of public benchmarks. We argue that this apparent progress is difficult to verify: a systematic audit of 27 papers published between 2015 and 2026 reveals three structural problems in how the community evaluates models. First, 25 of 27 papers omit the Hausdorff distance. Hausdorff distance is a boundary-accuracy metric with direct clinical relevance for detecting flat or small polyps, and is a standard in radiotherapy segmentation. Second, at least five incompatible train/test split protocols co-exist across papers reporting results on the same two datasets (Kvasir-SEG and CVC-ClinicDB), making published Dice scores non-comparable even when they appear in the same leaderboard column. Third, 26 of 27 papers make performance claims without any statistical significance test. Strikingly, four papers published after the Metrics Reloaded framework~\cite{metricsreloaded2024} (Maier-Hein et al., Nature Methods 2024) perpetuate these same problems, suggesting that general-purpose metric guidance has not yet reached the colonoscopy sub-community. To show these problems are not merely cosmetic, we re-evaluate five representative models under three controlled protocols with a single uniform scorer, and find that the reported metric conceals large boundary and recall failures, that the ``best'' model changes with the metric, and that near-tied rankings reverse across random splits. We propose a five-point Polyp Segmentation Reporting Checklist~(PSRC) as a lightweight, domain-adapted corrective.
Aisha Urooj, Zain Ul Abdien, Neelu Madan
Lunit, South Korea · RPTU Kaiserslautern, Germany · Aalborg University & Pioneer Center, Denmark
Monocular depth estimation (MDE) for colonoscopy is hampered by the domain gap between simulated and real-world images. Existing image-to-image translation methods, which use depth as a posterior constraint, often produce structural distortions and specular highlights by failing to balance realism with structure consistency. To address this, we propose a Structure-to-Image paradigm that transforms the depth map from a passive constraint into an active generative foundation. We are the first to introduce phase congruency to colonoscopic domain adaptation and design a cross-level structure constraint to co-optimize geometric structures and fine-grained details like vascular textures. In zero-shot evaluations conducted on a publicly available phantom dataset, the MDE model that was fine-tuned on our generated data achieved a maximum reduction of 44.18% in RMSE compared to competing methods. Our code is available at https://github.com/YyangJJuan/PC-S2I.git.
Juan Yang, Yuyan Zhang, Han Jia +2
College of Computer Science, Sichuan University, Chengdu, China · Department of Gastroenterology and Hepatology, West China Hospital, Sichuan University, Chengdu, China