Proactive Routing to Interpretable Surrogates with Distribution-Free Safety Guarantees

Iqtedar Uddin; Mazin Khider; André Bauer

cs.LGMar 15, 2026

Proactive Routing to Interpretable Surrogates with Distribution-Free Safety Guarantees

Authors: Iqtedar Uddin, Mazin Khider, André Bauer

Organizations: Illinois Institute of Technology

Abstract

Model routing determines whether to use an accurate black-box model or a simpler surrogate that approximates it at lower cost or greater interpretability. In deployment settings, practitioners often wish to restrict surrogate use to inputs where its degradation relative to a reference model is controlled. We study proactive (input-based) routing, in which a lightweight gate selects the model before either runs, enabling distribution-free control of the fraction of routed inputs whose degradation exceeds a tolerance τ. The gate is trained to distinguish safe from unsafe inputs, and a routing threshold is chosen via Clopper-Pearson conformal calibration on a held-out set, guaranteeing that the routed-set violation rate is at most α with probability 1-δ. We derive a feasibility condition linking safe routing to the base safe rate π and risk budget α, along with sufficient AUC thresholds ensuring that feasible routing exists. Across 35 OpenML datasets and multiple black-box model families, gate-based conformal routing maintains controlled violation while achieving substantially higher coverage than regression conformal and naive baselines. We further show that probabilistic calibration primarily affects routing efficiency rather than distribution-free validity.

Figures & tables

Figure 1: Feasibility landscape: critical AUC $\Phi_{c}(\pi,\alpha)$ as a function of base safe rate $\pi$ and risk budget $\alpha$ . Darker regions require higher AUC for guaranteed feasibility. Stars mark our six $\tau$ operating points at $\alpha=0.2$ , colored by empirical outcome: green if routing achieves positive coverage, red if coverage is zero. At $\tau\leq 0$ (low $\pi$ ), the gate falls in the hard region and correctly abstains. At $\tau=0.5$ and $\tau=1.0$ , routing succeeds despite AUC falling below $\Phi_{c}$ , illustrating that the sufficient condition is conservative. At $\tau=2.0$ , AUC exceeds $\Phi_{c}$ and feasibility is guaranteed.

$\tau$	$\pi$	AUC	ECE	$\Phi_{c}$	$\Phi_{c}^{*}$	Conf. ( $\alpha\!=\!0.2$ )
		(med.)				Cov.	Viol.
$-1.5$	0.13	0.71	0.07	1.00	0.92	0.000	—
$-1.0$	0.16	0.70	0.09	1.00	0.95	0.000	—
$0.0$	0.43	0.54	0.25	1.00	0.88	0.003	0.03
$0.5$	0.66	0.57	0.20	1.00	0.76	0.243	0.15
$1.0$	0.74	0.58	0.17	0.70	0.65	0.446	0.14

Table 1: Gate properties across tolerance levels.

\pi

: base safe rate; AUC: gate ranking quality (median [IQR]); ECE: calibration error.

\Phi_{c}

\Phi_{c}^{*}

: sufficient AUC thresholds from Theorems 1 / 2 . Coverage and mean violation at

\alpha=0.2

averaged over 35 datasets.

$\alpha$	Gate Conf.	Reg. Conf.	Naive
0.10	7/34 0 (21%)	31/50 0 (62%)	151/172 (88%)
0.20	8/66 0 (12%)	57/102 (56%)	128/172 (74%)
0.30	6/82 00 (7%)	80/144 (56%)	106/172 (62%)
0.50	8/112 0 (7%)	64/181 (35%)	0 62/172 (36%)

Table 2: Fraction of (dataset,

\tau

) pairs where the realized violation rate exceeds target

\alpha

. Gate conformal is broadly consistent with

\delta=0.10

. Baselines violate substantially more often.

Figure 2: Coverage vs. violation rate at

\alpha=0.2

. Each point corresponds to a

\tau

value averaged over 35 datasets. Gate conformal remains below

\alpha

. Regression conformal and naive operate above it.

Figure 3: Left: gate ECE varies from 0.07 to 0.28 across

\tau

. Right: despite high ECE, conformal violation remains below

\alpha=0.1

. Naive violations are 4–8

\times

higher.

$\tau$	$\alpha$	#DS	Higher Cov.		Violates $\alpha$
			Gate	Reg	Gate	Reg
0.5	0.2	23	10	11	2	15
0.5	0.3	32	16	14	2	18
1.0	0.2	29	18	7	4	11
1.0	0.3	32	20	7	1	12
2.0	0.2	32	18	6	2	7

Table 3: Per-dataset comparison at practical operating points. Gate conformal achieves higher coverage on more datasets while violating

\alpha

on far fewer.

Figure 4: Fraction of (dataset,

\tau

) pairs violating target

\alpha

across the full

\alpha

range. Gate conformal remains near the

\delta=0.10

line. Baselines converge only at very permissive

\alpha

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

$\tau$	$\alpha$	Gate Cov.	Gate Viol.	Reg Cov.	Reg Viol.
0.5	0.10	0.096	0.021	0.073	0.098
0.5	0.20	0.243	0.059	0.175	0.187
0.5	0.30	0.431	0.134	0.335	0.320
1.0	0.10	0.132	0.015	0.122	0.064
1.0	0.20	0.446	0.091	0.320	0.160
1.0	0.30	0.638	0.145	0.550	0.253

Appendix

Table 4: Gate conformal vs. regression conformal: mean coverage and violation across 35 datasets at representative operating points.

Comparison	$\tau$ , $\alpha$	Diff.	$p$ -value	$n$
Gate vs. Naive (violation rate):
	0.5, 0.2	$-$ 0.076	0.007	14
	1.0, 0.2	$-$ 0.060	0.002	23
	2.0, 0.2	$-$ 0.025	0.009	28
Gate vs. Reg. Conformal (coverage):
	0.5, 0.2	$+$ 0.256	0.005	13

Appendix

Table 5: Significance tests (Wilcoxon signed-rank, two-sided). To evaluate operational tradeoffs, tests are conditioned on problem instances where both methods successfully found a valid routing threshold (coverage ¿ 0). Under these conditions, Gate Conformal achieves significantly lower violation rates than the naive baseline and significantly higher coverage than regression conformal.

$\tau$	$\pi$	AUC (median)	ECE
-1.5	0.135	0.50	0.137
-1.0	0.173	0.50	0.170
0.0	0.459	0.51	0.302
0.5	0.639	0.58	0.213
1.0	0.717	0.56	0.160
2.0	0.799	0.57	0.136

Appendix

Table 6: Gate properties for XGBoost vs. Decision Tree across

\tau

. AUC is reported as median across datasets.

$\tau$	$\pi$	AUC (median)	ECE
-1.5	0.300	0.688	0.122
-1.0	0.368	0.627	0.154
0.0	0.629	0.544	0.221
0.5	0.723	0.570	0.194
1.0	0.770	0.557	0.153
2.0	0.830	0.535	0.112

Appendix

Table 7: Gate properties for MLP vs. Decision Tree.

	Calibration points routed ( $n$ )
$k$ unsafe	50	100	200	300	500
0	0.045	0.023	0.011	0.008	0.005
1	0.076	0.038	0.019	0.013	0.008
5	0.178	0.091	0.046	0.031	0.018
10	0.291	0.150	0.076	0.051	0.031
20	—	0.261	0.133	0.089	0.054

Appendix

Table 8: Clopper–Pearson upper bounds

\mathrm{UCB}_{0.10}(k,n)

for selected calibration set sizes. With

n\approx 300

, the bound is sufficiently tight for practical risk control.

$\tau$	Recalibration	ECE	Coverage	Violation
0.5	None (Platt)	0.215	0.243	0.059
0.5	Beta	0.066	0.243	0.059
0.5	Temperature	0.077	0.243	0.059
0.5	Isotonic	0.050	0.216	0.043
1.0	None (Platt)	0.184	0.446	0.091
1.0	Beta	0.060	0.421	0.085

Appendix

Table 9: Effect of post-hoc recalibration at

\alpha=0.2

, averaged over 35 datasets (RF vs. decision tree). Recalibration substantially reduces ECE but does not increase coverage, indicating the efficiency gap is dominated by ranking limitations.

$\alpha$	Method	Violations	Mean Viol.	Mean Cov.
0.10	Gate Conf.	7/34 (21%)	0.071	0.595
	HGBR Conf.	93/119 (78%)	0.385	0.164
	Naive 0.5	151/172 (88%)	0.404	0.692
0.20	Gate Conf.	8/66 (12%)	0.129	0.717
	HGBR Conf.	128/177 (72%)	0.406	0.249
	Naive 0.5	128/172 (74%)	0.404	0.692

Appendix

Table 10: Guarantee check across

(\text{dataset},\tau)

pairs when regression-conformal uses HistGradientBoostingRegressor . For each target

\alpha

, we report the fraction of pairs whose realized violation rate exceeds

\alpha

, along with mean violation rate and mean coverage.

		AUC		Conf. ( $\alpha{=}0.2$ )
$\tau$	$\pi$	(med.)	ECE	Cov.	Viol.
$-1.5$	0.18	0.53	0.06	0.000	—
$-1.0$	0.19	0.52	0.05	0.000	—
$0.0$	0.43	0.52	0.16	0.048	0.20
$0.5$	0.69	0.52	0.14	0.452	0.05
$1.0$	0.73	0.51	0.11	0.462	0.04

Appendix

Table 11: Gate properties and conformal routing performance across tolerance levels on 22 general-purpose OpenML datasets. Coverage and violation at

\alpha=0.2

averaged over datasets with positive coverage.

$\alpha$	Gate Conf.	Reg. Conf.	Naive
0.10	0/35 (0.0%)	20/62 (32.3%)	64/91 (70.3%)
0.20	1/38 (2.6%)	48/88 (54.5%)	58/91 (63.7%)
0.30	2/43 (4.7%)	50/105 (47.6%)	53/91 (58.2%)
0.50	4/66 (6.1%)	50/112 (44.6%)	39/91 (42.9%)

Appendix

Table 12: Fraction of (dataset,

\tau

) pairs where the realized violation rate exceeds target

\alpha

on general-purpose OpenML datasets. Gate conformal maintains strong safety control. Baselines violate substantially more often.

Explore similar work

Threshold-Aware Conformal Routing
Oct 1, 2026Shiwei Tan, Huzefa Rangwala, Danielle C. MaddixConformal Prediction Regression Prediction Intervals
Rutgers University · Siemens
CopRoute: Dependence-Aware Conformal Routing for Language Models
Oct 9, 2026Siva Rajesh Kasa, Santhosh Kumar Kasa, Karan GuptaLLM Routing Efficient Language Model Inference
RACER: Risk-Aware Calibrated Efficient Routing for Large Language Models
Feb 20, 2026Sai Hao, Hao Zeng, Hongxin Wei +1LLM Routing Adaptive Model Routing
Southern University of Science and Technology, China · The Chinese University of Hong Kong, Shenzhen, China · Shenzhen Loop Area Institute, China

Oct 1, 2026J/K move · Enter open · S save

Threshold-Aware Conformal Routing

Shiwei Tan, Huzefa Rangwala, Danielle C. Maddix

Rutgers University · Siemens

Conformal Prediction Regression Prediction Intervals

Open paper AskPDF arXiv

High-fidelity simulations are essential to scientific and engineering design, but can be expensive to run repeatedly. Learned surrogates offer a faster alternative, yet their higher errors may alter downstream decisions. This accuracy-speed tradeoff creates a need to determine whether a surrogate can be used or the full simulator remains necessary. We study decisions determined by whether a scalar quantity of interest lies above or below a fixed threshold. For each input, we use the surrogate when its conformal interval lies entirely on one side of the threshold and route the input to simulation when the interval intersects it. Standard conformal prediction constructs intervals without reference to the downstream decision threshold: even a narrow interval near the threshold can cross it and trigger simulation, whereas a wider interval farther away can remain entirely on one side and require no simulation. We introduce Threshold-Aware Conformal Routing (TACR), which learns an input-dependent scale using a threshold-aware objective that concentrates interval tightness near the decision boundary. Exact split-conformal calibration on held-out data preserves distribution-free marginal coverage, which also upper-bounds the probability of an incorrect threshold decision that is not routed. Across various scientific and engineering datasets, TACR reduces simulator deferrals by 14-75% relative to standard conformal prediction at the same coverage target. Against a variant without threshold-local weighting but with similar predictor accuracy, TACR further reduces deferrals by 10-24% on four datasets. These results show that optimizing interval allocation for routing can reduce simulator calls without weakening the standard conformal guarantee.

Abstract

Explore similar work

Threshold-Aware Conformal Routing

CopRoute: Dependence-Aware Conformal Routing for Language Models

RACER: Risk-Aware Calibrated Efficient Routing for Large Language Models