Conformal Robustness in Prediction-Driven Decision-Making

Lingjie Zhao; Hansheng Jiang; Wei Qi

stat.MLSep 19, 2026

Conformal Robustness in Prediction-Driven Decision-Making

Authors: Lingjie Zhao, Hansheng Jiang, Wei Qi

Organizations: Department of Industrial Engineering, Tsinghua University, Beijing 100084, China · Rotman School of Management, University of Toronto, Toronto, Ontario M5S 3E6, Canada · Desautels Faculty of Management, McGill University, Montreal, Quebec H3A 1G5, Canada

Abstract

Modern prediction-driven decision systems often rely on black-box predictors, but a point forecast alone does not provide the uncertainty scale required for robust downstream decision-making. We build a score-calibrated robustness framework that converts any fixed point predictor into a decision-relevant uncertainty representation through distribution-free conformal calibration. We use the conformal score, rather than a particular uncertainty set, as the primitive unit of robustness. The same score determines coverage-calibrated uncertainty sets for reliability-based robust optimization and normalizes target violations in a target-oriented formulation, Conformal Robust Satisficing. This formulation induces a conformal fragility measure that quantifies how rapidly performance deteriorates as the realized parameter departs from the forecast on the conformal score scale. For objective-uncertainty problems under standard convexity and duality conditions, we show that the reliability-based and target-oriented formulations parameterize the same score-calibrated robust decision frontier. This equivalence yields a data-driven mapping between reliability levels and acceptable targets and characterizes the marginal cost of robustness. Synthetic experiments validate the theoretical guarantees and illustrate the reliability-target correspondence. A real-data online-grocery case study demonstrates how the interface combines deep-learning demand forecasts with tractable inventory optimization, thereby improving reliability and reducing operational costs. Overall, our work shows that conformal scores endow fixed black-box predictors with an interpretable uncertainty scale for downstream decision-making while enabling reliability guarantees, acceptable-target selection, and fragility analysis within a unified framework.

Figures & tables

Panel of Figure 1 — Figure 1: Predict-Calibrate-Solve as a score-calibrated robustness interface.

Geometry	Conformal score $s(\bm{z},\bm{d})$	Resulting uncertainty set $\mathcal{U}_{\alpha}(\bm{z})$
Box	$\displaystyle\max_{j\in[J]}\frac{\|d_{j}-\hat{f}_{j}(\bm{z})\|}{\hat{h}_{j}(\bm{z})}$	$\displaystyle\left\{\bm{d}:\|d_{j}-\hat{f}_{j}(\bm{z})\|\leq\eta_{\alpha}\hat{h}_{j}(\bm{z}),\ j\in[J]\right\}$
Ellipsoid	$\displaystyle\frac{\\|\bm{L}(\bm{d}-\hat{f}(\bm{z}))\\|_{2}}{\hat{g}(\bm{z})},\,\hat{\bm{\Sigma}}^{-1}=\bm{L}^{\top}\bm{L}$	$\displaystyle\left\{\bm{d}:(\bm{d}-\hat{f}(\bm{z}))^{\top}\hat{\bm{\Sigma}}^{-1}(\bm{d}-\hat{f}(\bm{z}))\leq\eta_{\alpha}^{2}\hat{g}^{2}(\bm{z})\right\}$
Budget	$\displaystyle\left\\|\frac{\bm{d}-\hat{f}(\bm{z})}{\bm{\beta}\odot\hat{\bm{h}}(\bm{z})}\right\\|_{1}$	$\displaystyle\left\{\bm{d}:\sum_{j=1}^{J}\frac{\|d_{j}-\hat{f}_{j}(\bm{z})\|}{\beta_{j}\hat{h}_{j}(\bm{z})}\leq\eta_{\alpha}\right\}$

Table 1: Correspondence between uncertainty set geometries and conformal score designs.

Model	Decision-maker input	Objective	Uncertainty anchor	Decision philosophy
ConfRO	Coverage level $\alpha$	Minimize cost $V_{\mathrm{ConfRO}}$	Calibrated set $\mathcal{U}_{\alpha}(\bm{z})$	Reliability-based
ConfRS	Acceptable target $\tau$	Minimize fragility $\kappa^{*}(\tau)$	Score deviation $\mathfrak{s}(\bm{d},\hat{\bm{d}})$	Target-oriented

Table 2: Key conceptual components of ConfRO and ConfRS.

Panel of Figure 2 — Figure 2: Geometric intuition of the ConfRO–ConfRS correspondence.

Table 3: Out-of-sample performance comparison of ConfRO and benchmarks.

Method	Overall			High-Volume SKUs			Low-Volume SKUs
Method	WAPE	WPE	MAE	WAPE	WPE	MAE	WAPE	WPE	MAE
TFT	29.2%	5.4%	0.363	22.8%	2.1%	0.644	38.1%	10.0%	0.267
DLinear	30.6%	6.6%	0.381	23.5%	2.7%	0.666	40.6%	12.0%	0.283
SSA	31.1%	-1.2%	0.387	26.0%	-4.0%	0.739	38.5%	2.70%	0.267

Table 4: Predictive performance comparison of different predictors.

Figure 5: Operational cost and empirical coverage: ConfRO framework versus baselines.

Score class	Set geometry	Duality mechanism	Used in the paper
Weighted box / $L_{\infty}$	Polyhedral interval set	Linear-programming duality under relative-interior feasibility	Score-design examples
Weighted budget / $L_{1}$	Polyhedral budget set	Linear-programming duality under relative-interior feasibility	Fractional knapsack and online-grocery case study
Mahalanobis / ellipsoidal	Ellipsoid or second-order-cone set	Conic or convex-quadratic strong duality under Slater feasibility	Fractional knapsack and online-grocery case study
Asymmetric weighted budget	Polyhedral set with direction-specific slopes	Linear-programming duality under relative-interior feasibility	Score-design examples

Table H.1 : Common score geometries satisfying strong duality for affine objective uncertainty.

Paper	Contextual predictor	Parameter- uncertainty robust optimization	Parameter- uncertainty robust satisficing	Decision equivalence	Robustness object	Robustness interpretation
Conformal robust optimization and risk-sensitive linear programs ( Sun et al. 2023 , Patel et al. 2024 , Chenreddy and Delage 2024 , Cai et al. 2025 )	Arbitrary	Yes	No	No	Calibrated uncertainty set	Score not used as a unifying decision primitive
Joint estimation and robustness optimization ( Zhu et al. 2022 )	Structured †	Yes	No ‡	No	Estimation error in input parameters	Tailored to specific estimation procedures
Robust satisficing and equivalence ( Long et al. 2023 , Wang et al. 2025b )	Not applied	No ∗	No ∗	Yes ∗	Distributional ambiguity	Not prediction-centered
Prediction-based robust satisficing and fortification ( Sim et al. 2024 )	Structured †	No	Partial	No ∗	Residual distribution and prediction coefficients	Prediction-centered distributionally robust satisficing (DRS) with estimation fortification
This paper	Arbitrary	Yes	Yes	Yes	Calibrated prediction-error score and induced fragility	Decision-level equivalence; interplay of score radius, reliability, and target

Table I.1 : Positioning relative to closely related robustness frameworks from a prediction-driven perspective.

Score geometry	Hidden-layer widths	Activation	Output size	Predicted residual scale
Box/Budget	$64,32$	GELU	$20$	$\|c_{i}-\hat{c}_{i}\|$ for each $i\in[n]$
Ellipsoid	$64,16$	ReLU	$1$	$\\|\bm{c}-\hat{\bm{c}}\\|_{2}$

Table J.1 : Residual-scale quantile models used for different uncertainty-set geometries in ConfRO.

Coverage	ConfRO- Box	ConfRO- Ellipsoid	ConfRO- Budget	KMeans-RO	KNN-RO	PTO	Imp. vs KMeans-RO	Imp. vs KNN-RO	Imp. vs PTO
0.60	1307.7	1296.9	1309.4	898.9(29.40%)	1030.5(37.40%)	1311.2	45.66%	27.06%	-0.14%
0.70	1308.5	1296.0	1310.2	845.9(18.40%)	1021.0(25.20%)	1311.2	54.88%	28.32%	-0.08%
0.80	1309.5	1294.5	1310.4	904.9(7.60%)	1063.8(13.60%)	1311.2	44.81%	23.18%	-0.06%
0.85	1309.5	1293.7	1310.6	843.2(7.60%)	1044.0(10.20%)	1311.2	55.43%	25.54%	-0.05%
0.90	1308.7	1292.9	1310.8	918.3(4.80%)	1051.1(5.40%)	1311.2	42.74%	24.71%	-0.03%
0.95	1309.9	1291.2	1310.5	662.8(4.80%)	1002.4(3.00%)	1311.2	97.73%	30.74%	-0.06%

Table J.2 : Out-of-sample performance comparison in the fractional knapsack experiment.

Coverage	ConfRO-Box	ConfRO-Ellipsoid	ConfRO-Budget	PTO	Imp.
0.60	1084.0	1097.0	1082.4	1082.4	1.35%
0.70	1083.2	1096.7	1084.2	1082.4	1.32%
0.80	1086.4	1096.2	1082.5	1082.4	1.28%
0.85	1085.2	1096.2	1081.4	1082.4	1.28%
0.90	1085.3	1096.1	1082.5	1082.4	1.27%
0.95	1086.7	1095.4	1080.7	1082.4	1.20%

Table J.3 : Performance comparison under shifted evaluation with downside utility perturbations.

Method	Model specification	Training and forecasting settings
TFT	Attention-based multi-horizon model using static and time-varying covariates; hidden size $64$ , four attention heads, dropout $0.1$	Lookback $63$ , horizon $7$ , QuantileLoss, batch size $64$ , $30$ epochs, Ranger optimizer
DLinear	Trend–seasonal decomposition with moving-average window $28$ and seven input channels; dropout $0.05$	Lookback $62$ , horizon $7$ , mean absolute error loss, batch size $1024$ , $6$ epochs

Table J.4: Forecasting implementations for the inventory case study.

Coverage level	Best ConfRO	KNN-RO	Ellipsoid-RO	PTO	Imp. vs KNN-RO	Imp. vs Ellipsoid-RO	Imp. vs PTO
0.5	49.653	52.405	60.630	54.539	5.25%	18.10%	8.96%
0.6	49.672	53.328	61.697	54.539	6.86%	19.49%	8.92%
0.7	49.755	55.080	63.075	54.539	9.67%	21.12%	8.77%
0.8	49.938	57.933	65.132	54.539	13.80%	23.33%	8.44%
0.9	50.318	63.192	68.884	54.539	20.37%	26.95%	7.74%

Table J.5 : Out-of-sample cost reduction of ConfRO in the case study.

		Feasible ratio (%)				Realized relative cost				Improvement over DRS (%)
$\rho$	$N(\rho)$	TFT	DLinear	SSA	DRS	TFT	DLinear	SSA	DRS	TFT	DLinear	SSA
1.05	71	67.2	63.4	72.6	14.8	1.461	1.561	1.642	2.322	37.1	32.8	29.3
1.10	93	73.6	70.4	78.0	19.0	1.419	1.497	1.573	2.174	34.7	31.1	27.6
1.15	113	78.0	75.8	84.2	22.8	1.399	1.448	1.519	2.067	32.3	29.9	26.5
1.20	140	81.8	79.4	87.0	28.4	1.388	1.427	1.499	1.977	29.8	27.8	24.2
1.30	200	90.0	86.8	92.6	40.2	1.384	1.393	1.438	1.813	23.7	23.2	20.7

Table J.6 : The feasibility and realized performance of the three ConfRS variants and DRS.

Table 18

Figure J.1 : Performance gaps of different methods under various coverage levels.

Robust Optimization

Robust optimization studies decisions that remain effective when data, models, or operating conditions are uncertain or adversarial.

See all 49 papers

Learning Polyhedral Conformal Sets for Robust Optimization
May 8, 2026Shuyi Chen, Wenbin Zhou, Shixiang ZhuConformal Prediction Robust Optimization
Carnegie Mellon University
A Unified Perspective on Conformal Prediction and Wasserstein Distributionally Robust Optimization for Uncertainty Quantification
Aug 30, 2026Kehan Long, Yiqi Zhao, Pol Mestres +3Distribution Shift Robustness Uncertainty Quantification
Contextual Robotics Institute, University of California San Diego, La Jolla, CA, USA · Thomas Lord Department of Computer Science, University of Southern California, Los Angeles, CA, USA · Department of Mechanical and Civil Engineering, California Institute of Technology, Pasadena, CA, USA +1
Prediction Sets for Counterfactual Decisions: Coverage, Optimality, and Conformal Prediction
Jul 2, 2026Yurui Zheng, Ying JinSet Prediction Conformal Prediction
School of Mathematical Sciences, Peking University · Department of Statistics and Data Science, University of Pennsylvania

May 8, 2026J/K move · Enter open · S save

Learning Polyhedral Conformal Sets for Robust Optimization

Shuyi Chen, Wenbin Zhou, Shixiang Zhu

Carnegie Mellon University

Conformal Prediction Robust Optimization Decision-Making under Uncertainty

Open paper AskPDF arXiv

Robust optimization (RO) provides a principled framework for decision-making under uncertainty, but its performance critically depends on the choice of the uncertainty set. While large sets ensure reliability, they often lead to overly conservative decisions, whereas small sets risk excluding the true outcome. Recent data-driven approaches, particularly conformal prediction, offer finite-sample validity guarantees but remain largely task-agnostic, ignoring the downstream decision structure. In this paper, we propose a decision-aware conformal framework that learns uncertainty sets tailored to robust optimization objectives. Our approach parameterizes a flexible family of polyhedral sets via data-driven hyperplanes and learns their geometry by directly minimizing the induced robust loss, while preserving statistical validity through conformal calibration. To correct for data-dependent selection, we incorporate a re-calibration step on an independent dataset to restore coverage. The resulting sets capture directional and anisotropic uncertainty aligned with the decision objective while remaining computationally tractable. We provide finite-sample coverage guarantees and bounds on the sub-optimality gap to an oracle decision. This work bridges the gap between statistical validity and decision optimality, providing a principled framework for data-driven robust optimization.

Abstract

Robust Optimization

Explore similar work

Learning Polyhedral Conformal Sets for Robust Optimization

A Unified Perspective on Conformal Prediction and Wasserstein Distributionally Robust Optimization for Uncertainty Quantification

Prediction Sets for Counterfactual Decisions: Coverage, Optimality, and Conformal Prediction