The Loss or the Architecture? What Matters in Survival Analysis with Competing Risks

Julie Alberge; Gaël Varoquaux; Judith Abécassis

stat.MLOct 16, 2026

The Loss or the Architecture? What Matters in Survival Analysis with Competing Risks

Authors: Julie Alberge, Gaël Varoquaux, Judith Abécassis

Organizations: SODA Team, Inria Saclay, France

Abstract

Often, eg in healthcare and predictive maintenance, what matters is which of several competing causes will strike first, and with what probability: survival analysis with competing risks. Corresponding deep learning methods are developed as bundles: a new architecture together with a new training loss, tested on one to five datasets against a handful of baselines. Thus, a state-of-the-art result cannot be attributed to the architecture, the loss, or the evaluation protocol. The coupling is not accidental: with one incidence curve per cause, a network's head is tied to the losses expressible on it. We vary the two axes independently, three losses, two of them strictly proper, on four backbones, across two modalities that tease the axes apart: 20 tabular and 3 pathology-image datasets. One loss wins in both: modelling cumulative incidence directly under censoring weighting. It brings twice as much as benefits a good choice of backbone on competing-risks data, and no more than it on single-event data: the loss is decisive for competing risks. The backbone matters too, and must match the modality: tabular networks lead on tables, pretrained vision encoders on images. The two choices compound rather than substitute. Proper scoring rules reveal these dynamics; the concordance index (C-index), which is not proper, does not separate models: neither axis moves it, not even tuning on the loss built to improve it. This is an avenue of progress: composed, the two give a model nobody has tried, RealMLP or TabM trained with the cumulative-incidence loss, which takes the best Brier score in our benchmark.

Figures & tables

	# Real Life CR	#	# CR
Publication	Datasets	baselines	metrics
Ishwaran et al. 2014	5	5	2
Lee et al. 2018	1	3	1
Wang & Sun 2022	1	6	1
Jeanselme et al. 2023	3	5	3
Alberge et al. 2025	1	7	3

Table 1: Competing-risks methods are introduced on very thin evidence. Datasets, baselines and metrics used by the papers proposing methods, against this study.

Figure 1: Any backbone becomes a competing-risks model with two changes. One prediction head per cause, plus a survival head for losses that model the cumulative incidence directly, and a discretized time grid at the output. Everything upstream is left as the original architecture, which is what lets us vary loss and backbone independently.

Competing Risks
Dataset	Samples	Cens.
Framingham	4,240	58%
Metabric (CR)	3,237	33%
MIMIC IV	65,366	58%
MIMIC III	38,511	54%
SEER Breast	476,746	63%

Table 2: Our 20 tabular survival datasets, spanning three orders of magnitude in sample size and censoring rates from 19% to 95%. The three image datasets are described in Sec. 5.1 . Appendix A gives the full characteristics and the access procedure for each one.

Panel of Figure 2 — Figure 2: A competitive competing-risks model needs a backbone matched to the modality and a loss that models cumulative incidence under censoring; neither substitutes for the other. Both panels are grouped and coloured by loss, with the backbone named beneath each box. In both modalities CRNLL comes first within every backbone. What changes is which backbone wins: on tables the backbones built for tables, RealMLP and TabM, lead the plain MLP, while on images the pretrained encoders lead it by a wide margin and the MLP is also the least stable. The two panels have different backbones, baselines and vertical axes, and are not to be compared cell by cell.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset name	CR	Samples	Nb features	Censoring rate	Github
Framingham	✓	4,240	24	58%	$\sim$
Metabric (competing events)	✓	3,237	73	33%	✗
MIMIC IV	✓	65,366	172	58%	✗
MIMIC III	✓	38,511	81	54%	✗
SEER Breast Cancer	✓	476,746	18	63%	✗
SEER Digestion Cancer	✓	887,552	304	19%	✗

Appendix

Table A.1: Characteristics of the 20 tabular datasets. The CR column indicates whether the dataset has competing events. The number of features is counted after applying skrub’s TableVectorizer. The Github column indicates whether the dataset is directly available from GitHub; Sec. A.1 explains how we obtained the others.

Package	Language	used for
pycox	Python	Downloading datasets, DeepHit Architecture
lifelines	Python	Aalen–Johansen estimator ( Aalen & Johansen, 1978 )
auton-survival	Python	Downloading Datasets
hazardous	Python	Computing CR metrics (IBS, C-index, calibrations), SurvivalBoost
cbioportal	R	Downloading Datasets
cmprsks	R	Fine and Gray estimator ( Fine & Gray, 1999 ) , RSF

Appendix

Table C.1: Open-source libraries used in this benchmark.

model	parameter	range
Backbones (each trained under all three losses)
MLP	batch size	128, 256
	dropout	None, 0.1, 0.2, 0.3
	epochs	randint (256, 1024)
	lr	loguniform (1e-05, 0.001)
	nb output times	40, 60, 80, 100

Appendix

Table C.2: Hyper-parameter search space for every backbone and baseline. Each cell is searched with RandomizedSearchCV for 150 iterations over 3 stratified splits. randint / loguniform / uniform denote the scipy.stats distributions; a list denotes a categorical choice. The ranking weight

\alpha

and temperature

\sigma

are searched only for the HLRanking loss, which is the only loss that uses them.

Selection	Backbone	Median	$Q_{1}$	$Q_{3}$	Min	Max
IBS	MLP	$0.56$	$0.56$	$0.82$	$0.20$	$0.95$
	RealMLP	$0.66$	$0.52$	$0.82$	$0.09$	$0.98$
	TabM	$0.58$	$0.20$	$0.73$	$0.04$	$0.90$
	SurvTRACE	$0.66$	$0.46$	$0.74$	$0.15$	$0.99$
	All backbones	$0.59$	$0.42$	$0.79$	$0.04$	$0.99$
C-index	MLP	$0.46$	$0.20$	$0.55$	$0.03$	$0.83$

Appendix

Table C.3: Ranking weight

\alpha

selected for HLRanking , by selection criterion and backbone: median, quartiles and range.

	$n$ cells	median Cox vs. KM	median stratified vs. KM
Stable	24	$+0.0\%$	$+0.4\%$
Blown up ( $>20\%$ worse)	11	$+139.4\%$	$+0.2\%$

Appendix

Table C.4: A Cox censoring model is unreliable here, while a covariate-stratified one reproduces the marginal estimate. CRNLL training-time IPCW, Cox against Kaplan-Meier against stratified Kaplan-Meier, on the 35 (dataset, backbone) cells where all three schemes produced a usable fit, split at

|

IBS gain over Kaplan-Meier

|=20\%

Datasets	Loss effect		Backbone effect		Ratio
	median	IQR	median	IQR
Competing risks	0.053	[0.020, 0.084]	0.027	[0.021, 0.062]	1.93
Single event	0.072	[0.045, 0.132]	0.086	[0.039, 0.208]	0.83
All 20	0.065	[0.030, 0.100]	0.059	[0.023, 0.136]	1.10

Appendix

Table D.1: The loss separates the models twice as much as the backbone on competing-risks data, and slightly less than the backbone on single-event data. Median spread in IBS gain, with the interquartile range across groups. Pooling the two families averages two opposite regimes and hides the effect, which is why the main text reports them apart.

Dataset	Type	Loss spread	Backbone spread	Best loss
framingham	CR	0.036	0.059	CRNLL
metabric_competing	CR	0.068	0.022	CRNLL
mimic	CR	0.012	–	CRNLL
mimiciii	CR	0.011	0.020	CRNLL
seer	CR	0.043	0.021	CRNLL
seer_digestion	CR	0.029	–	CRNLL

Appendix

Table D.2: Per-dataset view behind Tab. D.1 . For each dataset, the median spread over the three losses and over the four backbones, and which loss attains the best median gain. The loss spread exceeds the backbone spread on most competing-risks datasets and on fewer of the single-event ones.

Figure D.1: IBS results. (reproducing Fig. 2(a) of the main text for ease of read) IBS gain over the Aalen–Johansen estimator on all 20 tabular datasets.

Figure D.2: The second proper scoring rule orders the losses as the Brier score does. Integrated negative log-likelihood against the Aalen–Johansen estimator. CRNLL is lowest on every backbone.

Figure D.3: The C-index does not separate well the models. Gain in C-index over the Aalen–Johansen estimator, higher is better. The boxes overlap across losses, backbones and baselines alike.

Figure D.4: Integrated AUC also gives a muddied picture. Gain in integrated time-dependent AUC over the Aalen–Johansen estimator, higher is better. As for the C-index, models are not well separated.

Figure D.5: On raw marginal calibration too, CRNLL is the best calibrated and HLRanking the worst. Area between the Aalen–Johansen estimator ( Aalen & Johansen, 1978 ) and one model.

	C-index spread			C-index / IBS spread
Spread across	CR	single event	$p$	CR	single event
the 3 losses	$0.032$ [0.015, 0.067]	$0.035$ [0.015, 0.068]	$0.43$	$0.63$ [0.46, 1.25]	$0.50$ [0.16, 1.05]
the 4 backbones	$0.054$ [0.026, 0.113]	$0.038$ [0.024, 0.075]	$0.69$	$1.09$ [0.76, 4.09]	$0.44$ [0.14, 1.40]
all cells of a dataset	$0.113$ [0.061, 0.182]	$0.082$ [0.048, 0.133]	$0.67$	$1.11$ [0.62, 1.76]	$0.60$ [0.18, 0.74]

Appendix

Table D.3: The C-index separates models about as little on competing risks (CR) as on single-event data. Median spread in C-index gain at the median time, with interquartile range, over the groups of Sec. D.1 ;

p

: one-sided Mann–Whitney test of a smaller spread on CR. Right: the same spread divided by the IBS spread of the same group.

Figure D.7: The ordering is unchanged on the competing-risks datasets alone. IBS gain over the Aalen–Johansen estimator, restricted to the 10 competing-risks datasets. Layout as in Fig. 2(a) .

	Competing risks (10 datasets)				Single event (10 datasets)
Loss	MLP	RealMLP	TabM	SurvTRACE	MLP	RealMLP	TabM	SurvTRACE
CRNLL	0.261	0.250	0.351	0.258	0.146	0.150	0.152	0.125
HazardLikelihood	0.117	0.198	0.268	0.120	0.084	0.043	0.128	0.117
HLRanking	0.120	0.093	0.180	0.120	0.063	0.049	0.126	0.117

Appendix

Table D.4: The advantage of the cumulative-incidence loss is far wider on competing-risks data. Median IBS gain over Aalen–Johansen for each cell of the matrix, best backbone per row in bold. On competing risks CRNLL leads the nearest hazard-based loss by a large margin on every backbone; on single-event data the three losses fall much closer together.

Figure D.8: Under heavy censoring the advantage of the CRNLL loss persists and widens. IBS gain over the Aalen–Johansen estimator on the datasets censored above

90\%

, grouped by backbone so that the three losses can be compared within each block; higher is better. CRNLL is ahead of both hazard-based losses on every backbone, while those two remain close to each other, as in Fig. 2(a) . Two contrasts with the full collection are worth noting: the random survival forest, which is competitive overall, falls well behind the neural models trained with CRNLL here, and one cell (SurvTRACE with the HazardLikelihood loss) degrades to worse than the Aalen–Johansen reference.

Panel of Figure 21 — Figure D.9: C-index at given times separates neither the loss nor the backbone Gain in C-index over the Aalen–Johansen estimator for different time horizons. Layout as in Fig. 2(a) .

Backbone	$n$	med. $\Delta$ IBS	IBS worse	med. $\Delta$ C-idx	C-idx better
MLP	22	$+0.0069$	17/22	$+0.0018$	13/22
RealMLP	19	$+0.0150$	14/19	$+0.0018$	12/19
TabM	20	$+0.0142$	17/20	$+0.0020$	13/20
SurvTRACE	13	$+0.0082$	9/13	$+0.0006$	7/13
Pooled	74	$+0.0117$	57/74 (77%)	$+0.0018$	45/74 (61%)

Appendix

Table D.5: Tuning HLRanking on the C-index buys almost no C-index and costs a great deal of IBS. IBS-selected against C-index-selected hyperparameters, paired per (dataset, backbone), with

\alpha

and

\sigma

searched jointly (Sec. 2.2 ).

n

= number of datasets with both selections available for that backbone. Median

\Delta

= C-index-selected minus IBS-selected.

Figure D.10: The CRNLL advantage does not come from its extra survival head. IBS gain over the Aalen–Johansen estimator for CRNLL trained with the dedicated

(K+1)

-th survival head (dark) and with

K

heads renormalized without one (light), paired per backbone; higher is better. Each point is one dataset, the box spans the interquartile range and the whiskers reach 1.5 IQR. The two variants overlap on all four backbones and neither is systematically ahead, so “loss” and “output parameterization” are separable here and it is the loss that carries the effect.

Panel of Figure 24 — Figure D.11: The survival head does not change calibration either. Same ablation and same layout as Fig. D.10 , scored on the two competing-risks calibration criteria of Sec. 2.3 . On AJ-calibration both variants concentrate near zero on every backbone, with the same handful of poorly calibrated datasets in each. D-calibration is far more dispersed, but the paired boxes still overlap heavily and the better median alternates between the two variants across backbones, which is what one expects when the parameterization is immaterial rather than beneficial.

Survival Analysis

Survival analysis models when events such as disease progression, failure, or death occur, accounting for incomplete follow-up and censoring.

See all 106 papers

Jul 31, 2026

A reproducible and extensible framework for benchmarking competing risks survival models

A wide range of statistical and machine learning methods have been proposed for survival analysis with competing risks, where the occurrence of one event (i.e., cancer death) precludes the occurrence of other events (i.e., cardiovascular disease death). Despite these methodological advances, their systematic evaluation and adoption are limited by the lack of comprehensive, reproducible and extensible benchmarking frameworks. We developed an open-source benchmarking framework for competing risks models that enables their systematic comparison across multiple datasets under different aspects of performance; calibration, discrimination, overall prediction error and clinical utility. We additionally introduce an extension of SHAP for competing risks, allowing model-agnostic interpretability of covariates contributions over time. All our code is publicly available via GitHub:https://github.com/BBolosSierra/CompRisksBenchmark

Begoña B. Sierra, Colin McLean, Peter S. Hall +2

Cancer Research UK Scotland Centre · Institute of Genetics and Cancer · University of Edinburgh +2

Benchmark Design Survival Analysis

Ask PDF

Jun 8, 2026

A Guide to Estimating Conditional Average Treatment Effects in Competing Risks Settings

Conditional average treatment effects (CATEs) are central to treatment decision-making in personalized medicine. In competing risks settings, estimating CATEs from survival data allows for patient-specific assessments of treatment effectiveness for a specific event of interest while properly accounting for alternative event types. This distinction is essential in the presence of comorbidities, where competing causes of death may otherwise confound the therapeutic benefit. Focusing on right-censored survival times with binary treatment, we examine CATEs defined as covariate-conditional differences in the absolute risk for the event of interest at a fixed time. To this end, we study meta-learners which adapt machine learning algorithms for CATE estimation in competing risks scenarios. We systematically compare six meta-learners, combining Cox regression or random survival forests for risk modeling with elastic net regression or random forests for direct CATE modeling. To provide practical guidance on model selection, we evaluate their performance in multiple simulation settings, that differ in hazard complexity, treatment heterogeneity, treatment assignment, event type distribution and censoring. To facilitate applied use, we provide the R package, crsurvlearners, which implements all considered approaches.

Daniel Klippert, Sarah Friedrich, Markus Pauly

Department of Statistics, TU Dortmund University · Research Center Trustworthy Data Science and Security, University Alliance Ruhr (UA Ruhr) · Institute for Mathematics, University of Augsburg

Causal Effect Estimation Meta-Learning Conditional Average Treatment Effect Estimation Survival Analysis Individualized Treatment Effect Estimation

Ask PDF

Jul 18, 2026

Value-Monotonicity Matters: A Concordance Loss for Deep Survival Prediction

Deep survival models are evaluated almost exclusively by the concordance index (C-index), yet they are commonly trained using likelihood objectives such as the Cox partial likelihood, discrete-time negative log-likelihood, and DeepHit likelihood. This mismatch is usually considered acceptable because the C-index can be recomputed on validation data during training. However, for end-to-end training of high-capacity encoders on small, heavily censored oncology cohorts, frequent C-index evaluation is computationally expensive, making the loss value itself an important signal for monitoring, early stopping, and model selection. We show that likelihood losses are unreliable for this purpose and propose a value-monotone concordance loss. We prove that every strictly proper survival likelihood admits directions where the loss decreases while the C-index remains unchanged, causing the loss value to decouple from ranking performance. We then study a sigmoid concordance loss (SCL), whose value approximates one minus the C-index up to a temperature term, ensuring that lower loss corresponds to higher C-index during optimization. The loss is architecture agnostic and reduces to a convex survival ranking support vector machine for linear models. Across eighteen datasets from four modalities using a unified five-fold cross-validation protocol, SCL achieves discrimination comparable to standard likelihood losses and is the best or within one standard deviation of the best C-index. Unlike likelihood losses, SCL maintains a strong correlation between loss value and C-index during training, with rank correlations of 0.96 to 0.99 compared with -0.03 to 0.53 for likelihood losses. Calibration measured by the integrated Brier score is comparable. SCL provides a value-monotone optimization objective whose value can serve as a reliable surrogate for the C-index during expensive end-to-end training.

Meixu Chen, Kai Wang, Jing Wang

Department of Radiation Oncology, University of Colorado School of Medicine, Aurora, CO, USA · Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, TX, USA

Survival Analysis

Ask PDF

Abstract

Survival Analysis

Explore similar work

A reproducible and extensible framework for benchmarking competing risks survival models

A Guide to Estimating Conditional Average Treatment Effects in Competing Risks Settings

Value-Monotonicity Matters: A Concordance Loss for Deep Survival Prediction