Off-Policy Evaluation and Learning for Survival Outcomes under Censoring

Kohsuke Kubota; Mitsuhiro Takahashi; Yuta Saito

stat.MEMar 24, 2026

Off-Policy Evaluation and Learning for Survival Outcomes under Censoring

Authors: Kohsuke Kubota, Mitsuhiro Takahashi, Yuta Saito

Organizations: NTT DOCOMO, INC. · Hanjuku-kaso, Co, Ltd.

Abstract

Optimizing survival outcomes, such as patient survival or customer retention, is a critical objective in data-driven decision-making. Off-Policy Evaluation~(OPE) provides a powerful framework for assessing such decision-making policies using logged data alone, without the need for costly or risky online experiments in high-stakes applications. However, typical estimators are not designed to handle right-censored survival outcomes, as they ignore unobserved survival times beyond the censoring time, leading to systematic underestimation of the true policy performance. To address this issue, we propose a novel framework for OPE and Off-Policy Learning~(OPL) tailored for survival outcomes under censoring. Specifically, we introduce IPCW-IPS and IPCW-DR, which employ the Inverse Probability of Censoring Weighting technique to explicitly deal with censoring bias. We theoretically establish that our estimators are unbiased and that IPCW-DR achieves double robustness, ensuring consistency if either the propensity score or the outcome model is correct. Furthermore, we extend this framework to constrained OPL to optimize policy value under budget constraints. We demonstrate the effectiveness of our proposed methods through simulation studies and illustrate their practical impacts using public real-world data for both evaluation and learning tasks.

Figures & tables

Figure 1: Illustration of the underestimation bias inherent in naive OPE. While the true event time for User B is unobserved due to censoring, naive estimators treat the censoring time as the event time, leading to a systematic underestimation of the survival time.

Panel of Figure 2 — Figure 2: OPE performance comparison across key experimental factors. The figure illustrates how MSE changes as we vary (a) the logged data size $n$ , (b) the censoring rate $\rho_{1}$ , and (c) the divergence between the logging and evaluation policies $\epsilon$ .

Panel of Figure 3 — Figure 3: OPL performance comparison. The improvement ratio $V^{\tau}(\hat{\pi})/V^{\tau}(\pi_{0})$ quantifies how effectively each method learns a superior policy compared to the logging policy optimalities.

Estimator	MSE	Squared bias	Variance
DM	30.7	29.2	1.5
IPS	216.7	213.7	3.1
DR	190.1	188.5	1.6
IPCW-IPS (Ours)	11.8	1.4	10.5
IPCW-DR (Ours)	10.7	4.4	6.4

Table 1: OPE results on the Starbucks dataset. The best and second-best results are highlighted in bold and underlined , respectively.

Learner	RMST (Mean $\pm$ Std) $\downarrow$	Cost (Mean $\pm$ Std)	Feasible Rate $\uparrow$
Regression	48.86 $\pm$ 1.21	2.54 $\pm$ 0.30	0.80
IPS	39.61 $\pm$ 1.90	2.44 $\pm$ 0.20	0.95
DR	39.11 $\pm$ 2.06	2.51 $\pm$ 0.23	0.90
IPCW-IPS (Ours)	39.41 $\pm$ 1.73	2.43 $\pm$ 0.20	0.99
IPCW-DR (Ours)	38.16 $\pm$ 1.41	2.53 $\pm$ 0.20	0.95

Table 2: Performance comparison of constrained OPL methods averaged over 100 trials with budget

B=2.82

. Feasible rate denotes the proportion of trials satisfying the constraint. The symbols

\downarrow

and

\uparrow

indicate that lower and higher values are better, respectively. The best and second-best results are highlighted in bold and underlined , respectively.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 4: OPL performance comparison under varying

\beta

. The improvement ratio

V^{\tau}(\hat{\pi})/V^{\tau}(\pi_{0})

quantifies how effectively each method learns a superior policy compared to the logging policy optimalities.

Off-Policy Evaluation

Off-policy evaluation estimates a policy’s value from data collected by another policy, using weighting, models, and uncertainty-aware methods.

See all 74 papers

Proper Scoring Rules for Right-Censored Survival Data
Jun 4, 2026Jef Jonkers, Glenn Van Wallendael, Luc Duchateau +1Survival Analysis Probabilistic Forecasting
IDLab Department of Electronics and Information Systems Ghent University, Belgium · IDLab Department of Electronics and Information Systems Ghent University - imec, Belgium · Biometrics Research Group Department of Morphology, Imaging, Orthopedics, Rehabilitation and Nutrition Ghent University, Belgium
Adaptive Experimentation for Censored Survival Outcomes
May 18, 2026Yuxin Wang, Dennis Frauen, Jonas Schweisthal +3Causal Effect Estimation Survival Analysis
LMU Munich Munich Center of Machine Learning (MCML)
Off-Policy Evaluation for Missingness-Aware Policies in MDPs with Rewards Missing Not at Random
Jun 18, 2026Ziheng Wei, Annie Qu, Rui MiaoReinforcement Learning Off-Policy Evaluation
Department of Statistics, University of Michigan at Ann Arbor · Department of Statistics and Applied Probability, University of California at Santa Barbara · Department of Mathematical Sciences, University of Texas at Dallas

Jun 4, 2026J/K move · Enter open · S save

Proper Scoring Rules for Right-Censored Survival Data

Jef Jonkers, Glenn Van Wallendael, Luc Duchateau, Sofie Van Hoecke

IDLab Department of Electronics and Information Systems Ghent University, Belgium · IDLab Department of Electronics and Information Systems Ghent University - imec, Belgium · Biometrics Research Group Department of Morphology, Imaging, Orthopedics, Rehabilitation and Nutrition Ghent University, Belgium

Survival Analysis Probabilistic Forecasting Proper Scoring Rules

Open paper AskPDF arXiv

Proper scoring rules provide a rigorous theoretical basis for the training and evaluation of probabilistic forecasts. However, in the presence of right censoring, the event time is only partially observed, rendering conventional scoring rules inapplicable in their standard form. We propose a framework for proper scoring of right-censored survival outcomes based on a simple idea: first, map the predictive distribution through the censoring mechanism, then apply the underlying proper score on the induced observed-data law. This yields localized scores for fixed censoring times and marginalized scores when the censoring time is random or only partially observed. The resulting construction recovers familiar right-censored likelihood and IPCW-type criteria within a coherent framework, while also yielding right-censored versions of the CRPS, pinball loss, Brier score, and energy score. We show that the marginalized score is proper under conditional independent censoring and strictly proper on the identifiable region. The same principle also leads to censored engression, a sample-based learning objective for multivariate right-censored survival modeling. In experiments, our scores correctly rank the oracle forecast across several censoring regimes, whereas forecast-dependent plug-in weighted scores can exhibit ranking reversals. Censored engression likewise substantially improves over naive training on censored outcomes.

Abstract

Off-Policy Evaluation

Explore similar work

Proper Scoring Rules for Right-Censored Survival Data

Adaptive Experimentation for Censored Survival Outcomes

Off-Policy Evaluation for Missingness-Aware Policies in MDPs with Rewards Missing Not at Random