Evaluating the Effects of Inter-Observer and Model Variability on Radiological Peritoneal Cancer Index Assessment

Peritoneal metastases (PM) are staged using the surgically determined Peritoneal Cancer Index (sPCI), which requires invasive laparoscopic assessment. Although CT is routinely used for preoperative evaluation, imaging-based assessment of PM extent remains challenging and is often less structured than surgical PCI scoring. A recent consensus study defined radiological PCI (rPCI) regions for cross-sectional imaging. We present the first deep learning approach to automatically segment 13 rPCI regions on CT. 62 contrast-enhanced CT scans were retrospectively collected across the full PCI range. Each scan was annotated into non-overlapping rPCI regions by one researcher, reviewed by a second, with disagreements resolved by a radiologist. Using five-fold cross-validation, we compared nnU-Net and Swin UNETR with Dice, 95th-percentile Hausdorff distance (HD95) and Average Surface Distance (ASD). We introduce an anatomically constrained pipeline that trains on merged super-regions and splits them during post-processing using TotalSegmentator landmarks at the hips and the ligament of Treitz. On this 62-scan cohort, the baseline nnU-Net reached an overall Dice of 0.81 and outperformed Swin UNETR (0.76). The proposed pipeline improved the overall Dice to 0.84 and reduced boundary error (HD95 13.7 to 11.8 mm; ASD 4.1 to 3.4 mm), with the largest gains in the small-bowel regions, approaching the interobserver Dice of 0.87. Automated rPCI region segmentation on CT is feasible and approaches interobserver agreement. Encoding anatomical boundary constraints substantially improves segmentation quality in the most challenging regions. This provides a reproducible foundation for non-invasive, imaging-based PCI assessment. The main limitations are the single-center cohort and the small interobserver subset.

In real-time colonoscopy, ground-truth annotations are unavailable at inference, so polyp segmentation models can fail silently. We propose Referee-Based Quality Estimation (RBQE), a reference-free framework measuring agreement between a primary segmentation model and an independently trained referee on the same image. RBQE is evaluated on a standardized 1,223-image external benchmark drawn from four public datasets, using four referee configurations chosen to separate two design axes: referee independence and architectural diversity. Using a common Agreement Dice descriptor, a same-architecture referee differing from the primary model only in random initialization already yields a useful reliability signal (ROC-AUC = 0.923), showing that independent training alone is sufficient. Cross-architecture referees improve further: SegFormer-B0 achieves the strongest performance (ROC-AUC = 0.960), significantly outperforming the same-architecture control and UNet++, and exceeding a representative Test-Time Augmentation baseline by 0.055 ROC-AUC under an identical protocol, whereas a prompt-coupled MedSAM referee underperforms despite maximal architectural diversity. Because empty-mask agreement is trivially separable, we also report a restricted evaluation excluding such cases: ROC-AUC falls to 0.876 (SegFormer-B0, 1,046 images) and 0.783 (same-architecture control, 975 images), yet RBQE's margin over both baselines widens on this identical subset. RBQE additionally increases the mean Dice of retained predictions as low-agreement cases are progressively rejected, supporting selective prediction, and requires only one additional deterministic referee forward pass at inference. Our study therefore supports cross-model agreement as a practical, interpretable reliability framework for automated polyp segmentation.

Surgical resection remains the only potentially curative treatment for pancreatic ductal adenocarcinoma (PDAC), and eligibility depends on accurate assessment of vascular invasion (VI), i.e., tumor extension into adjacent critical vessels. Despite its importance for preoperative staging and surgical planning, computational VI assessment remains underexplored. Two major challenges are the lack of public datasets and the diagnostic ambiguity at the tumor-vessel interface, which leads to substantial inter-rater variability even among expert radiologists. To address these limitations, we introduce the CURVAS-PDACVI Dataset and Challenge, an open benchmark for uncertainty-aware AI in PDAC staging based on a densely annotated dataset with five independent expert annotations per scan. We also propose a multi-metric evaluation framework that extends beyond spatial overlap to include probabilistic calibration and VI assessment. Evaluation of six state-of-the-art methods shows that strong global volumetric overlap does not necessarily translate into reliable performance at clinically critical tumor-vessel interfaces. In particular, methods optimized for binary segmentation perform competitively on average overlap metrics, but often degrade in high-complexity cases with low expert consensus, either collapsing in volume or overextending at uncertain boundaries. In contrast, methods that model inter-rater disagreement produce better calibrated probabilistic maps and show greater robustness in these ambiguous cases. The benchmark highlights the limitations of volumetric accuracy as a proxy for localized surgical utility, motivating uncertainty-aware probabilistic models for preoperative decision-making.