CHIMERA Challenge: Biochemical Recurrence Prediction in Prostate Cancer Patients using multimodal datasets
Authors: Robert N. Spaans, Catherine Chia, Tongjie Wang, Adam Kowalewski, Parandzem Khachatryan, Domingos Oliveira, Khrystyna Faryna, Jean-Paul A. van Basten, Geert Litjens, Nadieh Khalili
Organizations: Radboud University Medical Center, Department of Pathology, Research Institute for Medical Innovation, Nijmegen, The Netherlands · Canisius Wilhelmina Hospital, Department of Urology, Nijmegen, The Netherlands · Oncode Institute, Utrecht, The Netherlands · Erasmus University Medical Center, Department of Pathology, Rotterdam, The Netherlands · Erasmus University Medical Center, Department of Dermatology, Rotterdam, The Netherlands · Erasmus University Medical Center, Department of Urology, Rotterdam, The Netherlands · Bydgoszcz University of Science and Technology, Faculty of Medicine, Bydgoszcz, Poland · Yerevan State Medical University, Yerevan, Armenia · IMP Diagnostics, Research and Development Unit, Porto, Portugal
Biochemical recurrence (BCR), defined as any detectable prostate-specific antigen level after prostatectomy with confirmatory elevation, is widely used as a surrogate endpoint and typically assessed using clinical and pathological variables. Currently, no standardized benchmark exists for multimodal prognostic modeling in urological cancers, partly because curating heterogeneous multimodal data remains challenging. We developed the CHIMERA Challenge, a multimodal benchmark integrating preoperative mpMRI, post-prostatectomy histopathology, patient characteristics, and clinician-derived variables from 267 patients across two institutions. The dataset comprises 801 MRI sequences, 13 clinical variables per case, and 942 WSIs. Training (n=95), validation (n=23), and test (n=149) splits were established and hosted on the Grand Challenge platform. Baseline clinical and pathological characteristics did not differ significantly across splits. Models were evaluated on predicting time to BCR using the C-index. Post-challenge analyses tested how each model type performed when clinician-derived variables were withheld or randomized. Unimodal clinical models achieved the highest test C-index of 0.7402 but proved sensitive to the integrity of these variables, with performance collapsing toward chance (C approximately 0.50) when they were randomized. Multimodal models retained near-baseline performance when these variables were withheld (delta C at most 0.04), indicating their ability to recover prognostic signal directly from imaging data. CHIMERA is the first public, standardized multimodal benchmark for prostate cancer prognosis. Although models using only patient characteristics and clinician-derived variables yielded the highest leaderboard performance, multimodal models demonstrated greater robustness in clinically realistic scenarios where complete expert annotation is not guaranteed.
Figures & tables
Figure 1: Data distribution of prostate cancer patients who have had a radical prostatectomy between 2012 and 2021 for Task 1 of the CHIMERA challenge.
Figure 2: Flowchart of available data types provided in CHIMERA challenge.
Figure 3: Overview of the CHIMERA challenge and study setup. The global competition attracted 336 participants from 43 countries. The challenge was organized into three phases. Participants could download the open-source training set for algorithm development and submit candidate algorithms for evaluation on the hidden validation and test sets through the Grand Challenge platform. Each participant was allowed up to five successful submissions to the validation set and one successful submission to the final test set. After the challenge ended, algorithms that were successfully submitted to the test phase were included in additional analyses, including feature importance and modality importance analyses.
WL
SMILE
TIA-Pegasus
Randomized Variable
Cr
Δ C
q
Cr
Δ C
q
Cr
Δ C
q
Reference
C0=0.7294
C0=0.7280
C0=0.7402
Capsular Penetration
0.7493
−0.0198 [-0.0454, 0.0031]
0.185
0.7424
−0.0144 [-0.0372, 0.0093]
0.228
0.7527
−0.0126 [-0.0297, 0.0030]
0.138
Tertiary Gleason
0.7276
0.0018 [-0.0049, 0.0092]
0.749
0.7161
0.0119 [-0.0016, 0.0294]
0.137
0.7402
0.0000 [0, 0]
–
Pre-operative PSA
0.7282
0.0012 [-0.0074, 0.0118]
0.834
0.7163
0.0117 [0.0006, 0.0249]
0.080
0.7274
0.0128 [-0.0004, 0.0299]
0.116
pT Stage
0.7337
−0.0043 [-0.0289, 0.0173]
0.778
0.7078
0.0203 [-0.0097, 0.0501]
0.201
0.7333
0.0069 [-0.0065, 0.0208]
0.309
Table 3: Overview of performance per unimodal team for randomized variables. Models were bootstrapped (10,000 samples). Baseline performance is denoted by C0 and post-randomization performance by Cr ; Δ C-index ( C0−Cr ) is reported with 95% bootstrap confidence intervals. Benjamini–Hochberg (BH) FDR-adjusted two-sided p -values ( q ) are shown separately. Values in bold denote q<0.05 .
Team
Modalities
C0
Cr
Δ C [95% CI]
p
IU CompPath
Clin + MRI + WSI
0.7197
0.6869
0.0328 [-0.0006, 0.0693]
0.056
VCMI
Clin + MRI + WSI
0.7153
0.6762
0.0391 [-0.0105, 0.0949]
0.124
OHSU-Cedar
Clin + WSI
0.6885
0.6881
0.0004 [-0.1027, 0.1053]
0.993
Table 4: Overview of performance per multimodal model when pathology-derived variables are removed. Models were bootstrapped (10,000 samples). Baseline performance is denoted by C0 and post-variable removal performance by Cr . Δ C-index ( C0−Cr ) is reported with 95% bootstrap confidence intervals.
Configuration
Cr
Δ C-index
q
WSI + MRI + Clinical (full) - Reference
C0=0.7153
-
-
WSI + Clinical (full)
0.7076
0.0077
0.5435
WSI + Clinical (filtered)
0.6981
0.0172
0.5435
WSI + MRI
0.6902
0.0251
0.4254
Clinical only (full)
0.6831
0.0322
0.4402
WSI + MRI + Clinical (filtered)
0.6762
0.0391
0.2488
Table 5: Ablation analysis for VCMI under different modality and clinical feature configurations. Baseline (full model) performance is denoted by C0 and the ablated performance by Cr . Δ C-index ( C0−Cr ) is reported as the difference between baseline and ablation performance. Benjamini–Hochberg (BH) FDR-adjusted two-sided p -values are reported as q .
Supplementary Figure S1: Forest plot of the performance per unimodal team for randomized variables per cohort.
Supplementary Figure S2: Forest plot of the performance per multimodal team when pathology-derived variables were removed, split per origin center of the data.
Supplementary Figure S3: Forest plot of the ablation analysis for the VCMI model under different modality and clinical feature configurations per cohort
Biochemical recurrence (BCR) after radical prostatectomy is a critical endpoint in prostate cancer, yet risk stratification relies almost entirely on variables dominated by Gleason grade. Whether H&E whole slide images (WSIs) carry prognostic signal beyond grade, and whether multiple instance learning (MIL) can recover it, remains unsettled. A key obstacle is that many pipelines select model checkpoints on the evaluation fold, artificially inflating concordance. We construct a rigorous benchmark on TCGA-PRAD (487 patients, 101 BCR events) using strict out-of-fold scoring over five-fold cross-validation repeated across five seeds. The choice of MIL aggregator (ABMIL, CLAM, TransMIL, PatchGCN) has little effect (C-index 0.61-0.64 with UNI2-h), while the feature extractor is the dominant factor (ResNet50 0.566 versus pathology foundation models up to 0.639). A clinical Cox model on grade, stage, and age reaches 0.687; no imaging-only model significantly outperforms it (p > 0.10). We introduce Grade-Disentangled MIL (GD-MIL), a gated-attention MIL encoder trained with a gradient-reversal grade adversary that encourages the slide representation to be invariant to Gleason grade before late fusion with clinical variables. GD-MIL achieves C-index 0.704, significantly outperforming both the clinical baseline (delta-c = +0.029, p = 0.0005) and the best imaging-only model (delta-c = +0.062, p = 0.039), suggesting H&E morphology contains prognostic information complementary to grade. A median risk split yields log-rank p < 0.0001 separation in BCR-free survival (~20% vs ~70% at five years).
High-risk non-muscle-invasive bladder cancer (HR-NMIBC) carries substantial risks of recurrence and progression, while current clinical risk stratification remains limited. CHIMERA was established as a multimodal AI challenge to benchmark prediction in HR-NMIBC under standardized evaluation. Task BRS predicts RNA-seq-defined BCG Response Subtypes from histopathology and structured clinicopathological data, whereas Task Progression models time-to-progression using histopathology, structured data, and RNA sequencing. A multimodal dataset of 368 patients was divided into public training and hidden validation and test sets. In total, 159 submissions were made, and 13 top-performing models were selected for benchmarking. The best models achieved a weighted F1 score of 0.73 for Task BRS and a C-index of 0.68 for Task Progression. Post-challenge analyses revealed task-dependent modality contributions, cohort-dependent performance degradation, and sensitivity to missing structured data. In Task BRS, histopathology partly compensated for pathology-derived structured variables, whereas progression models showed greater dependence on complementary inputs. Cross-model error analysis further identified patients that were consistently difficult across different architectures, with T1 substage associated with prediction difficulty. These findings highlight barriers to transportability and the importance of missingness-aware modeling and independent multi-institutional validation. CHIMERA provides a standardized multimodal benchmark for bladder cancer and a framework for studying not only model performance, but also robustness, information sufficiency, and patient-level prediction failure.
Accurate prediction of biochemical recurrence (BCR) after radical prostatectomy is critical for guiding adjuvant treatment and surveillance decisions in prostate cancer. However, existing clinicopathological risk models reduce complex morphology to relatively coarse descriptors, leaving substantial prognostic information embedded in routine histopathology underexplored. We present a deep learning-based biomarker that predicts continuous, patient-specific risk of BCR directly from H&E-stained whole-slide prostatectomy specimens. Trained end-to-end on time-to-event outcomes and evaluated across four independent international cohorts, our model demonstrates robust generalization across institutions and patient populations. When integrated with the CAPRA-S clinical risk score, the deep learning risk score consistently improved discrimination for BCR, increasing concordance indices from 0.725-0.772 to 0.749-0.788 across cohorts. To support clinical interpretability, outcome-grounded analyses revealed subtle histomorphological patterns associated with recurrence risk that are not captured by conventional clinicopathological risk scores. This multicohort study demonstrates that deep learning applied to routine prostate histopathology can deliver reproducible and clinically generalizable biomarkers that augment postoperative risk stratification, with potential to support personalized management of prostate cancer in real-world clinical settings.
Clément Grisi, Khrystyna Faryna, Nefise Uysal +11
Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands · Department of Molecular and Translational Medicine, University of Brescia, Brescia, Italy · Surgical Pathology Unit, Verona University Hospital Trust, Verona, Italy +4