Mask-Conditional Conformal Prediction: Valid Uncertainty For All Missing Data Mechanisms

Conformal prediction (CP) provides a distribution-free framework for uncertainty quantification with finite-sample marginal coverage guarantees. Yet, missing values introduce distributional heterogeneity, where marginal coverage can hide systematic undercoverage for certain missingness patterns. At the same time, existing methods based on distances or smoothness require a continuous representation of partially observed covariates. Although imputation restores continuity, it obscures the missingness patterns and introduces task-dependent overhead. We develop a new CP framework that is adaptive to missingness patterns, without imputing covariates. We rely on data augmentation by feature embedding that accounts for missingness. Our framework represents each input with missing values by a tree embedding together with its raw missingness mask, and then learns a regularized quantile score function. Varying the choices for the penalty function and the augmentation enables us to characterize three practically relevant validity levels: exact coordinate-wise validity, exact validity on observed masks, and approximate missingness-conditional guarantees for general mechanisms (MCAR, MAR, MNAR). Experiments on synthetic and real datasets support these guarantees and show that the proposed method reduces miscoverage risk while maintaining informative intervals, especially in non-MCAR settings. These results highlight the practical value of uncertainty quantification that adapts to missingness.

Conformal prediction provides rigorous distribution-free finite-sample guarantees for marginal coverage under the assumption of exchangeability, but may exhibit systematic undercoverage or overcoverage for specific subpopulations. Assessing conditional validity is challenging, as standard stratification methods suffer from the curse of dimensionality. We propose Conformal Prediction Assessment (CPA), a framework that reframes the evaluation of conditional coverage as a supervised learning task by training a reliability estimator that predicts instance-level coverage probabilities. Building on this estimator, we introduce the Conditional Validity Index (CVI), which decomposes reliability into safety (undercoverage risk) and efficiency (overcoverage cost). We establish convergence rates for the reliability estimator and prove the consistency of CVI-based model selection. Extensive experiments on synthetic and real-world datasets demonstrate that CPA effectively diagnoses local failure modes and that CC-Select, our CVI-based model selection algorithm, consistently identifies predictors with superior conditional coverage performance.

A predictor with small overall bias can still exhibit systematic bias conditional on its predicted value, limiting its reliability for downstream uncertainty quantification and candidate screening. We propose Isotonic Conformal Inference, a unified framework for prediction and multiple testing that separates calibration from inference and exploits the level set structure of an isotonic predictor. This structure enables the computation of localized conformal quantiles and $p$-values while avoiding repeated isotonic refitting across candidate responses. For prediction, we introduce Split Isotonic Conformal Prediction (sICP) and Transductive Isotonic Conformal Prediction (tICP), establish prediction-conditional coverage guarantees for both methods, and characterize their finite-sample and asymptotic self-calibration properties. For selection, we introduce Isotonic Conformal Selection (ICS), which provides finite-sample stratum-conditional false discovery rate control. Experiments on simulated regression problems, a healthcare-utilization dataset, and a high-throughput ADMET drug discovery dataset demonstrate the practical performance of the framework and substantially lower computational costs relative to full transductive baselines.