Development and evaluation of CADe systems in low-prevalence setting: The RARE25 challenge for early detection of Barrett's neoplasia

Tim J. M. Jaspers; Francisco Caetano; Cris H. B. Claessens; Carolus H. J. Kusters; Rixta A. H. van Eijck van Heslinga; Floor Slooter; Jacques J. Bergman; Peter H. N. De With

cs.CVApr 13, 2026

Development and evaluation of CADe systems in low-prevalence setting: The RARE25 challenge for early detection of Barrett's neoplasia

Authors: Tim J. M. Jaspers, Francisco Caetano, Cris H. B. Claessens, Carolus H. J. Kusters, Rixta A. H. van Eijck van Heslinga, Floor Slooter, Jacques J. Bergman, Peter H. N. De With,

Organizations: Department of Electrical Engineering, Architectures for Reliable Image Analysis Lab, Eindhoven University of Technology, Eindhoven, Netherlands · Department of Gastroenterology and Hepatology, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, The Netherlands

Abstract

Computer-aided detection (CADe) of early neoplasia in Barrett's esophagus is a low-prevalence surveillance problem in which clinically relevant findings are rare. Although many CADe systems report strong performance on balanced or enriched datasets, their behavior under realistic prevalence remains insufficiently characterized. The RARE25 challenge addresses this gap by introducing a large-scale, prevalence-aware benchmark for neoplasia detection. It includes a public training set and a hidden test set reflecting real-world incidence. Methods were evaluated using operating-point-specific metrics emphasizing high sensitivity and accounting for prevalence. Eleven teams from seven countries submitted approaches using diverse architectures, pretraining, ensembling, and calibration strategies. While several methods achieved strong discriminative performance, positive predictive values remained low, highlighting the difficulty of low-prevalence detection and the risk of overestimating clinical utility when prevalence is ignored. All methods relied on fully supervised classification despite the dominance of normal findings, indicating a lack of prevalence-agnostic approaches such as anomaly detection or one-class learning. By releasing a public dataset and a reproducible evaluation framework, RARE25 aims to support the development of CADe systems robust to prevalence shift and suitable for clinical surveillance workflows.

Figures & tables

Figure 1: Overview of the RARE25 Challenge timeline, illustrating the three phases: Open Development Phase, Closed Testing Phase, and Post-MICCAI Phase.

Figure 2: Overview of the RARE25 Challenge setup, highlighting the dataset class imbalance and the evaluation protocol.

Figure 3: Visual examples of images included in the public training and closed test set.

Team	Affiliation	Architecture	Ens.	Add. data	Pretrain	IV	Calib.
IMSY	German Cancer Research Center (DKFZ), Division of Intelligent Medical Systems, Germany	ResNet-50 ViT-L (DINOv3)	40-model ensemble	No	GastroNet-5M DINOv3 (LoRA)	5-fold	Yes
UT	The University of Tokyo, Japan	MaxViT-T	No	No	ImageNet	5-fold	No
Jmees-inc	Jmees-inc, Japan	ViT-B + ViT-L (seg.)	2-model ensemble	Yes	Segmentation pretraining	5-fold	No
MEDAI	Konkuk Univ.; Pukyong Natl. Univ.; Seoul St. Mary’s Hospital, South Korea	ResNet-50	No	External val set	GastroNet-5M	External data [ 40 ]	Yes
AJLG_ETRI	Ajou University; ETRI, South Korea	ConvNeXt-B	No	No	ImageNet	5-fold	No
sk	Muroran Institute of Technology; Fujita Health University, Japan	ConvNeXt-B	10-model ensemble	No	ImageNet-21k	50/50	No

Table 1: Overview of participating teams and their final submissions. Abbreviations: IV = internal-validation split; Calib. = explicit post-hoc calibration.

Panel of Figure 4 — Figure 4: Results of all teams in the Closed Testing Phase. Performance is reported for (a) PPV@90%RECALL, (b) AUROC, and (c) AUPRC. PPV@90% recall was defined as the primary challenge metric and used to determine the final ranking, as it reflects performance at a clinically relevant operating point under real-world low-prevalence conditions.

Figure 5: Representative endoscopic images containing neoplastic lesions, ordered from very subtle to severe appearance (left to right). In the first three examples, localized neoplastic regions are indicated by blue dotted outlines, whereas in the last two examples neoplasia involves a large portion of the visible mucosal surface. For each image, the classification outcome of the submitted methods is indicated, showing whether each team correctly classified the image.

Panel of Figure 6 — Figure 6: Team rankings in the Closed Testing Phase under different evaluation metrics. Each panel shows the relative ranking of teams when ordered by (a) PPV@90%RECALL, (b) AUROC, and (c) AUPRC. Blob size is proportional to the frequency of the rank achieved based on bootstrapping (N = 1000) and the black lines indicate the 95% confidence intervals.

Medical Image Analysis

Medical image analysis uses computational methods to interpret clinical images, measure anatomy and disease, and support diagnosis and treatment.

See all 291 papers

TRUE-Colon: Exposing a Consistent Transfer Asymmetry in Real-Time Polyp Detection
Aug 13, 2026Sebastian Doerrich, Andreas Franz Schwab, Francesco Di Salvo +3Medical Image Benchmarks Endoscopy
XAI Lab Bamberg, University of Bamberg, Bamberg, Germany
RISED: A Pre-Deployment Evaluation Framework for High-Stakes AI Decision-Support Systems, with Application to Healthcare
May 13, 2026Rohith Reddy Bellibatlu, Manpreet Singh, Yash Jajoo +2Distribution Shift Robustness Healthcare
Florida International University, Miami, FL, USA. · Boston University, Boston, MA, USA. · New York University, New York, NY, USA. +2
Gaze responses to false-positive computer-aided detection prompts during colonoscopy: a paired-video and real-time eye-tracking study
Sep 22, 2026Te Luo, Yan Zhu, Peiyao Fu +5Endoscopy Medical Image Analysis
Digital Medical Research Center, School of Basic Medical Sciences, Fudan University, Shanghai, China · Shanghai Key Laboratory of MICCAI, Shanghai, China · Endoscopy Center and Endoscopy Research Institute, Zhongshan Hospital, Fudan University, Shanghai, China +4

Aug 13, 2026J/K move · Enter open · S save

TRUE-Colon: Exposing a Consistent Transfer Asymmetry in Real-Time Polyp Detection

Sebastian Doerrich, Andreas Franz Schwab, Francesco Di Salvo, Shyam Nandan Rai+2

XAI Lab Bamberg, University of Bamberg, Bamberg, Germany

Medical Image Benchmarks Endoscopy Real-Time Object Detection Domain Generalization

Open paper AskPDF arXiv

Computer-aided detection (CADe) systems for colonoscopy promise to reduce clinical miss rates, yet reliable real-world deployment remains elusive. This translational gap stems in part from a structural flaw in model development: the reliance on curated datasets that under-represent the long negative stretches and procedure-related artifacts characteristic of routine examinations. Training and evaluating architectures strictly on these lesion-centric benchmarks creates an illusion of success, since such benchmarks cannot capture clinically crucial metrics. To expose this gap, we establish TRUE-Colon, a standardized benchmarking protocol that measures key deployment characteristics alongside localization accuracy, and evaluate four real-time architectures (Faster R-CNN, YOLOv8, YOLOv11, RT-DETR) across curated benchmarks (SUN, PICCOLO) and 60 unedited, full-length procedures (REAL-Colon). We observe a consistent transfer asymmetry: models trained strictly on curated clips suffer a severe performance collapse when evaluated on full procedures, whereas procedure-trained models substantially improve rejection of non-polyp content on REAL-Colon, and largely retain their accuracy on curated benchmarks. Beyond transferability, we find that the Transformer detector attains the strongest sensitivity and the earliest, most persistent detections, while the convolutional detectors stay competitive at a higher throughput. Together, these results indicate that both training and benchmarking for deployable CADe should shift from curated, lesion-centric clips toward full-procedure data and deployment-relevant operating points. Source code is available at https://github.com/sdoerrich97/true-colon.

Abstract

Medical Image Analysis

Explore similar work

TRUE-Colon: Exposing a Consistent Transfer Asymmetry in Real-Time Polyp Detection

RISED: A Pre-Deployment Evaluation Framework for High-Stakes AI Decision-Support Systems, with Application to Healthcare

Gaze responses to false-positive computer-aided detection prompts during colonoscopy: a paired-video and real-time eye-tracking study