Conformal Robustness in Prediction-Driven Decision-Making

Lingjie Zhao; Hansheng Jiang; Wei Qi

stat.MLSep 19, 2026

Conformal Robustness in Prediction-Driven Decision-Making

Authors: Lingjie Zhao, Hansheng Jiang, Wei Qi

Organizations: Department of Industrial Engineering, Tsinghua University, Beijing 100084, China · Rotman School of Management, University of Toronto, Toronto, Ontario M5S 3E6, Canada · Desautels Faculty of Management, McGill University, Montreal, Quebec H3A 1G5, Canada

Abstract

Modern prediction-driven decision systems often rely on black-box predictors, but a point forecast alone does not provide the uncertainty scale required for robust downstream decision-making. We build a score-calibrated robustness framework that converts any fixed point predictor into a decision-relevant uncertainty representation through distribution-free conformal calibration. We use the conformal score, rather than a particular uncertainty set, as the primitive unit of robustness. The same score determines coverage-calibrated uncertainty sets for reliability-based robust optimization and normalizes target violations in a target-oriented formulation, Conformal Robust Satisficing. This formulation induces a conformal fragility measure that quantifies how rapidly performance deteriorates as the realized parameter departs from the forecast on the conformal score scale. For objective-uncertainty problems under standard convexity and duality conditions, we show that the reliability-based and target-oriented formulations parameterize the same score-calibrated robust decision frontier. This equivalence yields a data-driven mapping between reliability levels and acceptable targets and characterizes the marginal cost of robustness. Synthetic experiments validate the theoretical guarantees and illustrate the reliability-target correspondence. A real-data online-grocery case study demonstrates how the interface combines deep-learning demand forecasts with tractable inventory optimization, thereby improving reliability and reducing operational costs. Overall, our work shows that conformal scores endow fixed black-box predictors with an interpretable uncertainty scale for downstream decision-making while enabling reliability guarantees, acceptable-target selection, and fragility analysis within a unified framework.

Figures & tables

Panel of Figure 1 — Figure 1: Predict-Calibrate-Solve as a score-calibrated robustness interface.

Geometry	Conformal score $s(\bm{z},\bm{d})$	Resulting uncertainty set $\mathcal{U}_{\alpha}(\bm{z})$
Box	$\displaystyle\max_{j\in[J]}\frac{\|d_{j}-\hat{f}_{j}(\bm{z})\|}{\hat{h}_{j}(\bm{z})}$	$\displaystyle\left\{\bm{d}:\|d_{j}-\hat{f}_{j}(\bm{z})\|\leq\eta_{\alpha}\hat{h}_{j}(\bm{z}),\ j\in[J]\right\}$
Ellipsoid	$\displaystyle\frac{\\|\bm{L}(\bm{d}-\hat{f}(\bm{z}))\\|_{2}}{\hat{g}(\bm{z})},\,\hat{\bm{\Sigma}}^{-1}=\bm{L}^{\top}\bm{L}$	$\displaystyle\left\{\bm{d}:(\bm{d}-\hat{f}(\bm{z}))^{\top}\hat{\bm{\Sigma}}^{-1}(\bm{d}-\hat{f}(\bm{z}))\leq\eta_{\alpha}^{2}\hat{g}^{2}(\bm{z})\right\}$
Budget	$\displaystyle\left\\|\frac{\bm{d}-\hat{f}(\bm{z})}{\bm{\beta}\odot\hat{\bm{h}}(\bm{z})}\right\\|_{1}$	$\displaystyle\left\{\bm{d}:\sum_{j=1}^{J}\frac{\|d_{j}-\hat{f}_{j}(\bm{z})\|}{\beta_{j}\hat{h}_{j}(\bm{z})}\leq\eta_{\alpha}\right\}$

Table 1: Correspondence between uncertainty set geometries and conformal score designs.

Model	Decision-maker input	Objective	Uncertainty anchor	Decision philosophy
ConfRO	Coverage level $\alpha$	Minimize cost $V_{\mathrm{ConfRO}}$	Calibrated set $\mathcal{U}_{\alpha}(\bm{z})$	Reliability-based
ConfRS	Acceptable target $\tau$	Minimize fragility $\kappa^{*}(\tau)$	Score deviation $\mathfrak{s}(\bm{d},\hat{\bm{d}})$	Target-oriented

Table 2: Key conceptual components of ConfRO and ConfRS.

Panel of Figure 2 — Figure 2: Geometric intuition of the ConfRO–ConfRS correspondence.

Table 3: Out-of-sample performance comparison of ConfRO and benchmarks.

Method	Overall			High-Volume SKUs			Low-Volume SKUs
Method	WAPE	WPE	MAE	WAPE	WPE	MAE	WAPE	WPE	MAE
TFT	29.2%	5.4%	0.363	22.8%	2.1%	0.644	38.1%	10.0%	0.267
DLinear	30.6%	6.6%	0.381	23.5%	2.7%	0.666	40.6%	12.0%	0.283
SSA	31.1%	-1.2%	0.387	26.0%	-4.0%	0.739	38.5%	2.70%	0.267

Table 4: Predictive performance comparison of different predictors.

Figure 5: Operational cost and empirical coverage: ConfRO framework versus baselines.

Score class	Set geometry	Duality mechanism	Used in the paper
Weighted box / $L_{\infty}$	Polyhedral interval set	Linear-programming duality under relative-interior feasibility	Score-design examples
Weighted budget / $L_{1}$	Polyhedral budget set	Linear-programming duality under relative-interior feasibility	Fractional knapsack and online-grocery case study
Mahalanobis / ellipsoidal	Ellipsoid or second-order-cone set	Conic or convex-quadratic strong duality under Slater feasibility	Fractional knapsack and online-grocery case study
Asymmetric weighted budget	Polyhedral set with direction-specific slopes	Linear-programming duality under relative-interior feasibility	Score-design examples

Table H.1 : Common score geometries satisfying strong duality for affine objective uncertainty.

Paper	Contextual predictor	Parameter- uncertainty robust optimization	Parameter- uncertainty robust satisficing	Decision equivalence	Robustness object	Robustness interpretation
Conformal robust optimization and risk-sensitive linear programs ( Sun et al. 2023 , Patel et al. 2024 , Chenreddy and Delage 2024 , Cai et al. 2025 )	Arbitrary	Yes	No	No	Calibrated uncertainty set	Score not used as a unifying decision primitive
Joint estimation and robustness optimization ( Zhu et al. 2022 )	Structured †	Yes	No ‡	No	Estimation error in input parameters	Tailored to specific estimation procedures
Robust satisficing and equivalence ( Long et al. 2023 , Wang et al. 2025b )	Not applied	No ∗	No ∗	Yes ∗	Distributional ambiguity	Not prediction-centered
Prediction-based robust satisficing and fortification ( Sim et al. 2024 )	Structured †	No	Partial	No ∗	Residual distribution and prediction coefficients	Prediction-centered distributionally robust satisficing (DRS) with estimation fortification
This paper	Arbitrary	Yes	Yes	Yes	Calibrated prediction-error score and induced fragility	Decision-level equivalence; interplay of score radius, reliability, and target

Table I.1 : Positioning relative to closely related robustness frameworks from a prediction-driven perspective.

Score geometry	Hidden-layer widths	Activation	Output size	Predicted residual scale
Box/Budget	$64,32$	GELU	$20$	$\|c_{i}-\hat{c}_{i}\|$ for each $i\in[n]$
Ellipsoid	$64,16$	ReLU	$1$	$\\|\bm{c}-\hat{\bm{c}}\\|_{2}$

Table J.1 : Residual-scale quantile models used for different uncertainty-set geometries in ConfRO.

Coverage	ConfRO- Box	ConfRO- Ellipsoid	ConfRO- Budget	KMeans-RO	KNN-RO	PTO	Imp. vs KMeans-RO	Imp. vs KNN-RO	Imp. vs PTO
0.60	1307.7	1296.9	1309.4	898.9(29.40%)	1030.5(37.40%)	1311.2	45.66%	27.06%	-0.14%
0.70	1308.5	1296.0	1310.2	845.9(18.40%)	1021.0(25.20%)	1311.2	54.88%	28.32%	-0.08%
0.80	1309.5	1294.5	1310.4	904.9(7.60%)	1063.8(13.60%)	1311.2	44.81%	23.18%	-0.06%
0.85	1309.5	1293.7	1310.6	843.2(7.60%)	1044.0(10.20%)	1311.2	55.43%	25.54%	-0.05%
0.90	1308.7	1292.9	1310.8	918.3(4.80%)	1051.1(5.40%)	1311.2	42.74%	24.71%	-0.03%
0.95	1309.9	1291.2	1310.5	662.8(4.80%)	1002.4(3.00%)	1311.2	97.73%	30.74%	-0.06%

Table J.2 : Out-of-sample performance comparison in the fractional knapsack experiment.

Coverage	ConfRO-Box	ConfRO-Ellipsoid	ConfRO-Budget	PTO	Imp.
0.60	1084.0	1097.0	1082.4	1082.4	1.35%
0.70	1083.2	1096.7	1084.2	1082.4	1.32%
0.80	1086.4	1096.2	1082.5	1082.4	1.28%
0.85	1085.2	1096.2	1081.4	1082.4	1.28%
0.90	1085.3	1096.1	1082.5	1082.4	1.27%
0.95	1086.7	1095.4	1080.7	1082.4	1.20%

Table J.3 : Performance comparison under shifted evaluation with downside utility perturbations.

Method	Model specification	Training and forecasting settings
TFT	Attention-based multi-horizon model using static and time-varying covariates; hidden size $64$ , four attention heads, dropout $0.1$	Lookback $63$ , horizon $7$ , QuantileLoss, batch size $64$ , $30$ epochs, Ranger optimizer
DLinear	Trend–seasonal decomposition with moving-average window $28$ and seven input channels; dropout $0.05$	Lookback $62$ , horizon $7$ , mean absolute error loss, batch size $1024$ , $6$ epochs

Table J.4: Forecasting implementations for the inventory case study.

Coverage level	Best ConfRO	KNN-RO	Ellipsoid-RO	PTO	Imp. vs KNN-RO	Imp. vs Ellipsoid-RO	Imp. vs PTO
0.5	49.653	52.405	60.630	54.539	5.25%	18.10%	8.96%
0.6	49.672	53.328	61.697	54.539	6.86%	19.49%	8.92%
0.7	49.755	55.080	63.075	54.539	9.67%	21.12%	8.77%
0.8	49.938	57.933	65.132	54.539	13.80%	23.33%	8.44%
0.9	50.318	63.192	68.884	54.539	20.37%	26.95%	7.74%

Table J.5 : Out-of-sample cost reduction of ConfRO in the case study.

		Feasible ratio (%)				Realized relative cost				Improvement over DRS (%)
$\rho$	$N(\rho)$	TFT	DLinear	SSA	DRS	TFT	DLinear	SSA	DRS	TFT	DLinear	SSA
1.05	71	67.2	63.4	72.6	14.8	1.461	1.561	1.642	2.322	37.1	32.8	29.3
1.10	93	73.6	70.4	78.0	19.0	1.419	1.497	1.573	2.174	34.7	31.1	27.6
1.15	113	78.0	75.8	84.2	22.8	1.399	1.448	1.519	2.067	32.3	29.9	26.5
1.20	140	81.8	79.4	87.0	28.4	1.388	1.427	1.499	1.977	29.8	27.8	24.2
1.30	200	90.0	86.8	92.6	40.2	1.384	1.393	1.438	1.813	23.7	23.2	20.7

Table J.6 : The feasibility and realized performance of the three ConfRS variants and DRS.

Table 18

Figure J.1 : Performance gaps of different methods under various coverage levels.

Robust Optimization

Robust optimization studies decisions that remain effective when data, models, or operating conditions are uncertain or adversarial.

See all 49 papers

May 8, 2026

Learning Polyhedral Conformal Sets for Robust Optimization

Robust optimization (RO) provides a principled framework for decision-making under uncertainty, but its performance critically depends on the choice of the uncertainty set. While large sets ensure reliability, they often lead to overly conservative decisions, whereas small sets risk excluding the true outcome. Recent data-driven approaches, particularly conformal prediction, offer finite-sample validity guarantees but remain largely task-agnostic, ignoring the downstream decision structure. In this paper, we propose a decision-aware conformal framework that learns uncertainty sets tailored to robust optimization objectives. Our approach parameterizes a flexible family of polyhedral sets via data-driven hyperplanes and learns their geometry by directly minimizing the induced robust loss, while preserving statistical validity through conformal calibration. To correct for data-dependent selection, we incorporate a re-calibration step on an independent dataset to restore coverage. The resulting sets capture directional and anisotropic uncertainty aligned with the decision objective while remaining computationally tractable. We provide finite-sample coverage guarantees and bounds on the sub-optimality gap to an oracle decision. This work bridges the gap between statistical validity and decision optimality, providing a principled framework for data-driven robust optimization.

Shuyi Chen, Wenbin Zhou, Shixiang Zhu

Carnegie Mellon University

Conformal Prediction Robust Optimization Decision-Making under Uncertainty

Ask PDF

Aug 30, 2026

A Unified Perspective on Conformal Prediction and Wasserstein Distributionally Robust Optimization for Uncertainty Quantification

Uncertainty quantification from finite data is central to machine learning, optimization, and automation systems, where decisions must remain reliable under limited samples and test-time distribution shift. Conformal prediction (CP) and distributionally robust optimization (DRO) offer two complementary approaches: CP constructs data-dependent prediction sets with distribution-free finite-sample validity under exchangeability, while DRO optimizes worst-case performance over an ambiguity set around an empirical distribution. We develop a unified probabilistic perspective on CP and DRO by viewing both as ways to turn finite calibration data into a data-dependent quantile estimator that a test score falls below with high probability. From this perspective, CP and DRO correct the empirical quantile along two coordinates of the same family of estimators: CP inflates the quantile level, whereas DRO shifts the quantile value through an ambiguity radius. Both methods provide the same calibration-conditional guarantee for the true distribution, requiring the target coverage to hold with high probability over the calibration sample. Their constructions differ, however: CP uses a closed-form, distribution-free level correction, while DRO uses a value-space correction whose certified radius depends on properties of the unknown distribution and additionally guarantees coverage uniformly over the ambiguity set. This distinction emerges in the tails of the score distribution. Because CP relies on sparse upper-tail order statistics of the calibration samples, its level inflation barely moves the estimator when those samples are dense near the target quantile but overshoots when they are sparse, whereas a well-chosen DRO radius corrects in value space and may avoid this overshoot.

Kehan Long, Yiqi Zhao, Pol Mestres +3

Contextual Robotics Institute, University of California San Diego, La Jolla, CA, USA · Thomas Lord Department of Computer Science, University of Southern California, Los Angeles, CA, USA · Department of Mechanical and Civil Engineering, California Institute of Technology, Pasadena, CA, USA +1

Distribution Shift Robustness Uncertainty Quantification Conformal Prediction Wasserstein Distributionally Robust Optimization

Ask PDF

Jul 2, 2026

Prediction Sets for Counterfactual Decisions: Coverage, Optimality, and Conformal Prediction

Predictions are increasingly used to guide high-stakes decisions, from treatment selection to policy making. To ensure reliability with imperfect predictions, uncertainty quantification methods such as conformal prediction build prediction sets with coverage guarantees. However, statistical validity alone does not immediately determine the decisions to take, nor the optimality thereof. This gap is especially delicate in counterfactual settings where the outcome that materializes depends on the action taken, so uncertainty cannot be specified independently of the decision rule. We develop a decision-theoretic framework for uncertainty-informed counterfactual decisions. We identify a novel notion of policy-coupled coverage -- namely, coverage of the realized outcome under the action induced by the prediction sets themselves -- as the optimal and lossless interface between uncertainty and action. It plays three roles. First, it justifies acting via a natural max-min rule as minimax-optimal under distributional ambiguity. Second, optimizing prediction sets under policy-coupled coverage is equivalent both to a stronger universal-coverage formulation and to the direct risk-averse optimization over policies and utility certificates; this equivalence yields the explicit form of the population-optimal prediction sets. Third, it admits a two-stage procedure, Policy-Coupled Risk-Averse Conformal Prediction (PC-RACP), that approximates these optimal sets with rigorous finite-sample coverage. Simulations and a real email-marketing experiment confirm that PC-RACP delivers higher utility than existing approaches while maintaining valid coverage, and that ignoring the counterfactual structure of the decision problem is suboptimal for both validity and utility.

Yurui Zheng, Ying Jin

School of Mathematical Sciences, Peking University · Department of Statistics and Data Science, University of Pennsylvania

Set Prediction Conformal Prediction Counterfactual Reasoning Decision-Making under Uncertainty

Ask PDF

Abstract

Robust Optimization

Explore similar work

Learning Polyhedral Conformal Sets for Robust Optimization

A Unified Perspective on Conformal Prediction and Wasserstein Distributionally Robust Optimization for Uncertainty Quantification

Prediction Sets for Counterfactual Decisions: Coverage, Optimality, and Conformal Prediction