CopRoute: Dependence-Aware Conformal Routing for Language Models

Siva Rajesh Kasa; Santhosh Kumar Kasa; Karan Gupta

cs.LGOct 9, 2026

CopRoute: Dependence-Aware Conformal Routing for Language Models

Authors: Siva Rajesh Kasa, Santhosh Kumar Kasa, Karan Gupta

Abstract

Set-based language model routing aims to increase the coverage of the event where at least one model answers a query correctly while controlling the number of inference calls made. Existing risk-controlled routing policies bound the all-miss risk (the probability that every model in the selected set fails) by calibrating a single conformal threshold and applying it to a nested family of model sets. Because each model is scored in isolation, these policies ignore the dependence structure of model failures. Models that share training data or architecture tend to fail on the same queries, so two individually strong models can be jointly redundant while a weaker but complementary model covers the queries that the others miss. To account for this dependence, which existing risk-controlled policies ignore, we propose CopRoute, which orders models by how many new queries each one answers correctly beyond those already selected, decides per query how far down that order to go, and then picks the smallest cutoff that still meets the target risk. We show that CopRoute's ordering attains, on the design data, the best guarantee any efficient fixed-ordering algorithm can give for the average number of calls to the first correct model, and that a separate calibration step certifies the all-miss risk of the deployed sets. On RouterBench and RouterEval, CopRoute attains the fewest certified calls on nine of ten datasets at a target risk of 0.05 and on eight of ten at 0.10, making up to 79 percent fewer calls (median 24 percent) than the state-of-the-art risk-controlled policy at the same certified all-miss risk.

Figures & tables

Table 1: Certified mean calls at matched target risk, mean (dispersion) over 20 redraws; the dispersion is the standard error, or the standard deviation across five training seeds for FlexRouter and BaRP. CopRoute is the default policy CopRoute-Ind. The best value in each row is shaded, with exact ties shaded together, and the next distinct value is underlined . The last column is the exact-independence control, which orders models by router score alone and is reported but not ranked. RouterBench datasets (first four rows) use the 11-model pool and RouterEval datasets the 100-model pool.

Figure 1: Certified mean calls against the target risk, 100-model pool, logarithmic vertical axis; lower is better. Markers are open where fewer than 90 percent of redraws certify, and a missing point means none certifies.

Table 4

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Meaning	Note
Basic routing objects
$\mathcal{M}=\{1,\ldots,m\}$	Pool of candidate language models.	$m$ is the number of models.
$i,j,k,\ell$	Indices for a labeled query, a model, a prefix depth, and a position inside a prefix, respectively.	The range follows the equation in which an index appears.
$X,x$	A random query and one realized query, respectively.	Uppercase denotes a random variable.
$Y_{j}$ , $Y=(Y_{1},\ldots,Y_{m})$	Binary indicator that model $j$ answers $X$ correctly, and the vector of all model outcomes.	$Y_{j}\in\{0,1\}$ .
$G(X),G_{i}$	Models that answer query $X$ , or labeled query $i$ , correctly.	Both are subsets of $\mathcal{M}$ .

Appendix

Table 4: Notation used in the paper.

		$\alpha=0.05$						$\alpha=0.10$
dataset	pool	Ind	Dep	G	P	Flex.	BaRP	Ind	Dep	G	P	Flex.	BaRP
BBH-boolean	200	2.34	2.34	4.26	4.72	4.67	2.70	1.12	1.12	2.19	3.09	1.11	1.28
	300	2.45	2.45	4.61	5.69	2.11	3.03	1.27	1.27	2.16	3.33	1.23	1.34
	500	2.79	2.79	5.78	5.81	2.86	4.50	1.40	1.40	2.89	3.95	1.40	1.49
BBH-navigate	200	5.02	5.02	20.7	20.1	6.34	9.14	2.39	2.39	4.14	4.30	4.94	5.39
	300	6.82	6.82	11.7	11.0	5.77	8.94	2.09	2.09	5.22	5.86	2.45	6.09

Appendix

Table 5: Certified mean calls on the RouterEval benchmarks at the 200-, 300-, and 500-model pools, completing Table 1 . A dagger marks cells whose certification rate is below 0.9; a dash means no redraw certifies.

Figure 4: Relative call efficiency at the 200-, 300-, and 500-model pools of the six RouterEval datasets, one panel per pool size and target risk. Each value is the best certified calls on the axis divided by the policy’s certified calls, so the per-axis winner lies on the rim at 1; FlexRouter and BaRP are averaged over five training seeds. CopRoute is on the rim on 29 of the 36 axes and FlexRouter on the remaining seven. The 500-model panels are repeated in Figure 3 .

pool	.15	.20
100	0.00	0.00
200	0.00	0.00
300	0.00	0.15
500	0.05	0.30

Appendix

Table 6: MATH-intermediate-algebra certification rates for CopRoute-Ind (identical for every policy in each cell). No policy certifies at the targets of Table 1 , so the dataset is reported here rather than in the main comparison.

	$\alpha=0.05$		$\alpha=0.10$
dataset	Dep $-$ Ind	marginal $-$ Ind	Dep $-$ Ind	marginal $-$ Ind
ARC	+ 0.00 (tie)	+0.12 [-0.06, +0.42]	+ 0.00 (tie)	+ 0.00 (tie)
HellaSwag	+ 0.00 (tie)	+0.61 [+0.49, +0.73]	+ 0.00 (tie)	+0.02 [ + 0.00, +0.04]
MMLU	+ 0.00 (tie)	+1.69 [+1.41, +1.93]	+ 0.00 (tie)	+0.78 [+0.65, +0.89]
WinoGrande	+0.01 [ + 0.00, +0.02]	+2.57 [+2.29, +2.85]	+ 0.00 [-0.01, +0.02]	+0.95 [+0.70, +1.22]
BBH-boolean	+ 0.00 (tie)	+0.16 [-0.30, +0.77]	+ 0.00 (tie)	+0.13 [ + 0.00, +0.28]

Appendix

Table 7: Ablation of the two components on the cells of Table 1 . Dep

-

Ind keeps the tally order and replaces the independence-based stopping rule by the dependence-corrected rule of CopRoute-Dep; marginal

-

Ind keeps the stopping rule and replaces the tally order by the design-marginal order, which ranks models by design accuracy alone. Entries are paired differences in certified mean calls with bootstrap 95 percent intervals over redraws; bold intervals exclude zero, and a superscript gives the number of jointly certified redraws when it is below 20.

	independence-based stopping						dependence-corrected stopping
order	all	p11	p100	p200	p300	p500	all	p11	p100	p200	p300	p500
Bernoulli mixture	+0.42	+0.10	-0.65	-0.32	+0.67	+2.01	+0.36	+0.08	-0.39	+0.49	+0.47	+0.94
Student-t copula	+1.77	+0.42	-0.68	-0.35	+3.94	+4.63	+1.35	+0.39	-0.67	-0.48	+3.25	+3.55
Gaussian copula	+1.97	+0.54	-0.09	+1.08	+3.35	+4.13	+1.81	+0.55	-0.10	+1.59	+2.97	+3.33
rank correlation	+2.04	+0.54	+0.26	+0.71	+3.10	+4.76	+1.83	+0.54	+0.20	+1.31	+2.96	+3.44
double fault	+1.59	+0.03	+0.13	+0.38	+3.22	+3.40	+1.13	+0.02	+0.00	+0.04	+2.79	+2.21

Appendix

Table 8: Complete order-by-pool grid of extra certified calls relative to the tally order under the same stopping rule, weighted by certified redraws over the ten datasets and seven targets; negative favors the alternative.

independence-based stopping
order	ARC	HSwag	MMLU	WinoG	BBH-b	BBH-n	GPQA	IFEval	MATH-a	MuSR
Bernoulli mixture	+0.16	+0.15	+0.09	+0.00	-0.04	-2.51	-0.40	+4.70	-0.16	+1.72
Student-t copula	+0.11	+0.32	+0.37	+0.82	-0.46	-1.78	-1.26	+4.94	-0.21	+12.65
Gaussian copula	+0.14	+0.34	+0.47	+1.18	-0.03	-1.17	-0.79	+5.26	-0.28	+11.83
rank correlation	+0.15	+0.33	+0.45	+1.16	-0.03	-1.59	-1.09	+5.52	-0.23	+13.01
double fault	+0.08	+0.05	-0.01	+0.01	+1.06	+3.67	-7.31	+5.71	+2.61	+5.57

Appendix

Table 9: Dataset-wise extra certified calls relative to the tally order under the same stopping rule, weighted by certified redraws over each dataset’s pools and the seven targets; negative (bold) favors the alternative. RouterBench columns use the 11-model pool and RouterEval columns aggregate pools 100 to 500.

	RouterBench (pool 11)				RouterEval (pool 100)
	$\alpha=0.05$		$\alpha=0.10$		$\alpha=0.05$		$\alpha=0.10$
policy	ID	OOD	ID	OOD	ID	OOD	ID	OOD
CopRoute-Ind	3.65	5.29	1.90	2.69	23.44	33.26	6.60	17.23
CopRoute-Dep	3.65	5.30	1.90	2.69	24.52	35.28	6.53	17.96
RACER-G	4.81	5.44	2.89	2.91	31.91	34.81	13.73	18.50
RACER-P	5.11	5.86	3.35	3.36	32.20	34.62	15.59	18.46

Appendix

Table 10: Mean certified calls in domain (ID) and out of domain (OOD, averaged over every directed source-target pair in the suite), recalibrated on the target.

	fit call premium		source-calibrated risk inflation
policy	RouterBench	RouterEval	RouterBench	RouterEval
CopRoute-Ind	+0.89	+16.53	+0.010	+0.222
CopRoute-Dep	+0.90	+18.63	+0.010	+0.222
RACER-G	+0.13	+5.38	+0.016	+0.169
RACER-P	+0.06	-2.04	+0.015	+0.165
FlexRouter	+0.19	+0.09	+0.002	+0.140

Appendix

Table 11: Cross-dataset transfer summary of the mean extra certified calls after out-of-domain fitting with target recalibration (fit premium), and the mean excess of realized risk over the target when a source-calibrated cutoff is reused (risk inflation).

Figure 5: Extra certified calls after out-of-domain fitting for every policy, recalibrated on the target, RouterEval pool 100 at target risk 0.10. Cell values are exact; the color scale is shared and capped at 12. The two pairs within the BBH family were deliberately not run (n/r), so that no transfer is near in-domain.

$\tau$	pool 100	pool 200	pool 300	pool 500
0.0	10.00 / 0.934	10.00 / 0.935	10.00 / 0.952	10.00 / 0.960
0.01	10.00 / 0.934	10.00 / 0.935	10.00 / 0.952	10.00 / 0.960
0.05	10.00 / 0.934	10.00 / 0.935	10.00 / 0.952	10.00 / 0.960
0.1	10.00 / 0.934	10.00 / 0.935	10.00 / 0.952	10.00 / 0.960
0.2	10.00 / 0.934	10.00 / 0.935	10.00 / 0.952	9.88 / 0.958
0.3	9.99 / 0.934	9.99 / 0.935	9.76 / 0.951	8.70 / 0.947

Appendix

Table 12: Native FlexRouter adaptive stopping on IFEVAL. Entries are mean calls / Success, averaged over five training seeds and 40 redraws per seed. No risk certificate is claimed for this native rule.

$\tau$	pool 100	pool 200	pool 300	pool 500
0.0	10.00 / 0.796	10.00 / 0.767	10.00 / 0.783	10.00 / 0.859
0.01	10.00 / 0.796	10.00 / 0.767	10.00 / 0.783	10.00 / 0.859
0.05	10.00 / 0.796	10.00 / 0.767	10.00 / 0.783	10.00 / 0.859
0.1	10.00 / 0.796	10.00 / 0.767	10.00 / 0.783	10.00 / 0.859
0.2	10.00 / 0.796	10.00 / 0.767	10.00 / 0.783	9.80 / 0.857
0.3	9.99 / 0.796	10.00 / 0.767	9.66 / 0.782	8.90 / 0.851

Appendix

Table 13: Native FlexRouter adaptive stopping on MUSR. Entries are mean calls / Success, averaged over five training seeds and 40 redraws per seed. No risk certificate is claimed for this native rule.

Figure 6: Native BaRP quality-cost sensitivity after training on the three nondegenerate RouterBench task families. These single-model source metrics are supplementary to the common set-routing evaluation.

quality wt.	cost wt.	HellaSwag	MBPP	macro
1.0	0.0	0.799 / 1.356	0.575 / 5.810	0.687 / 3.583
0.9	0.1	0.799 / 1.356	0.575 / 5.810	0.687 / 3.583
0.8	0.2	0.713 / 0.359	0.573 / 5.755	0.643 / 3.057
0.7	0.3	0.654 / 0.159	0.459 / 0.243	0.556 / 0.201
0.6	0.4	0.563 / 0.149	0.449 / 0.227	0.506 / 0.188
0.5	0.5	0.461 / 0.135	0.471 / 0.200	0.466 / 0.167

Appendix

Table 14: Native BaRP preference sensitivity. Entries are official quality / mean cost multiplied by

10^{3}

. The macro column averages HellaSwag and MBPP. Values aggregate five training seeds.

	$\alpha=0.05$			$\alpha=0.10$
policy	calls	accuracy (%)	cost (USD/1k)	calls	accuracy (%)	cost (USD/1k)
CopRoute-Ind	7.75 (0.27)	80.44 (0.19)	3.41 (0.10)	2.99 (0.05)	80.97 (0.18)	1.90 (0.05)
CopRoute-Dep	7.75 (0.27)	80.44 (0.19)	3.41 (0.10)	2.99 (0.05)	80.97 (0.18)	1.90 (0.05)
RACER-G	8.66 (0.10)	80.19 (0.21)	4.24 (0.01)	5.64 (0.07)	80.18 (0.21)	3.55 (0.03)
RACER-P	8.64 (0.04)	80.19 (0.21)	4.22 (0.01)	6.45 (0.04)	80.13 (0.22)	3.66 (0.02)
exact independence	9.45 (0.18)	80.19 (0.21)	4.28 (0.03)	3.87 (0.07)	80.19 (0.21)	2.83 (0.03)

Appendix

Table 15: Cost and final-answer accuracy on RouterBench MMLU at matched certified risk; entries are means over 20 redraws with standard errors in brackets, and the best calibrated value per column is shaded. The last two rows are fixed references reported without calibration.

		cost, USD per 1k queries			cost-weighted minus CopRoute-Ind
dataset	$\alpha$	cost-weighted	CopRoute-Ind	RACER-G	$\Delta$ cost [CI95]	$\Delta$ calls [CI95]
ARC	0.05	0.183	0.996	1.049	-0.813 [-0.891, -0.756]	+2.73 [+2.53, +2.92]
	0.10	0.104	0.924	0.893	-0.820 [-0.829, -0.812]	+1.82 [+1.62, +2.00]
HellaSwag	0.05	1.089	2.623	3.684	-1.533 [-1.663, -1.409]	+2.66 [+2.46, +2.87]
	0.10	0.431	2.277	2.176	-1.845 [-1.861, -1.830]	+1.44 [+1.35, +1.53]
MMLU	0.05	3.046	3.405	4.238	-0.359 [-0.469, -0.247]	+0.71 [+0.45, +0.96]

Appendix

Table 16: Cost-weighted stopping on RouterBench, reporting the certified mean cost per 1,000 queries in USD, and paired differences against count-optimized CopRoute-Ind with bootstrap 95 percent intervals; every cost interval excludes zero.

	$\alpha=0.05$		$\alpha=0.10$
dataset	cascade	CopRoute-Ind	cascade	CopRoute-Ind
ARC (pool 11)	0.093 ∗	0.032	0.099	0.033
GPQA (pool 100)	0.076 ∗	0.066 ∗	0.139 ∗	0.121 ∗
IFEval (pool 100)	0.100 ∗	0.049	0.315 ∗	0.095
MATH-algebra (pool 100)	0.139 ∗	0.065 ∗	0.350 ∗	0.110 ∗

Appendix

Table 17: Realized all-miss risk of the pre-response cascade against matched CopRoute-Ind on identical splits; a star marks values above the target. The cascade carries no distribution-free certificate in any cell.

policy	intervention	frozen certificates surviving	certified after refit	mean call change
CopRoute-Ind	remove one model	3.4% (25 of 729)	63.1%	+1.07
CopRoute-Ind	add one model	100.0% (729 of 729)	73.6%	+0.07
CopRoute-Dep	remove one model	2.2% (16 of 729)	63.1%	+0.98
CopRoute-Dep	add one model	100.0% (729 of 729)	73.6%	-0.02

Appendix

Table 18: Pool composition shift over 51 dataset, pool, and target cells of RouterBench and RouterEval (1,020 redraws, of which 729 certify on the unshifted pool, a rate of 71.5 percent). A frozen certificate survives when the unshifted order and cutoff still certify on the shifted pool without any refitting. The last two columns refit the order and recalibrate the cutoff on the shifted pool and report the fraction of all 1,020 redraws that then certify and the mean change in certified calls relative to the unshifted pool.

RouterBench MMLU, 11 models
score transformation	CopRoute-Ind	exact independence	RACER-G
temperature scaling	-0.01 [-0.03, +0.01]	+0.03 [+0.01, +0.05]	-0.11 [-0.14, -0.08]
systematic overconfidence	+0.38 [+0.32, +0.44]	+0.15 [+0.08, +0.21]	+0.46 [+0.41, +0.52]
systematic underconfidence	+0.05 [+0.02, +0.09]	+0.17 [+0.10, +0.23]	-0.51 [-0.57, -0.45]
Gaussian noise, $\sigma=0.05$	-0.01 [-0.02, +0.01]	-0.01 [-0.05, +0.04]	+0.01 [-0.02, +0.05]
Gaussian noise, $\sigma=0.10$	+ 0.00 [-0.01, +0.02]	-0.03 [-0.08, +0.02]	-0.01 [-0.04, +0.02]

Appendix

Table 19: Router degradation at

\alpha=0.10

, measured as the change in certified mean calls relative to the unmodified KNN router when its scores are transformed and nothing else is refitted, paired over 20 redraws with bootstrap 95 percent intervals; bold intervals exclude zero. The realized test all-miss risk stays within sampling error of the target in every cell (0.085 to 0.103 across the three policies), so degradation is paid in calls rather than in risk.

ground-truth regime	tally	pairwise count	error corr.	latent normal	query-local
conditional independence, mixed difficulty	0/15	0/15	0/15	0/15	0/15
pooled correlation, no residual dependence	0/15	0/15	0/15	0/15	0/15
redundant families (positive residual)	10/15	14/15	12/15	10/15	13/15
complementary specialists (negative residual)	8/15	6/15	6/15	0/15	15/15
higher-order, pairwise-invisible	3/15	0/15	0/15	2/15	0/15

Appendix

Table 20: Synthetic mechanism map, giving the cells out of 15 (three pool sizes, five design budgets) where the ordering beats the design-marginal reference with a bootstrap interval excluding zero, per ground-truth regime.

Explore similar work

RACER: Risk-Aware Calibrated Efficient Routing for Large Language Models
Feb 20, 2026Sai Hao, Hao Zeng, Hongxin Wei +1LLM Routing Adaptive Model Routing
Southern University of Science and Technology, China · The Chinese University of Hong Kong, Shenzhen, China · Shenzhen Loop Area Institute, China
FlexRouter: Learning Complementary Model Sets for Flexible LLM Routing
Sep 29, 2026Wang Wei, Harry Yang, Tiankai Yang +6Adaptive Model Routing LLM Routing
Virginia Tech · Independent Researcher · University of Southern California +2
Federate the Router: Learning Language Model Routers with Sparse and Decentralized Evaluations
Jan 29, 2026Baris Askin, Shivam Patel, Anupam Nayak +4LLM Routing Federated Learning
Carnegie Mellon University.

Feb 20, 2026J/K move · Enter open · S save

RACER: Risk-Aware Calibrated Efficient Routing for Large Language Models

Sai Hao, Hao Zeng, Hongxin Wei, Bingyi Jing

Southern University of Science and Technology, China · The Chinese University of Hong Kong, Shenzhen, China · Shenzhen Loop Area Institute, China

LLM Routing Adaptive Model Routing

Open paper AskPDF arXiv

Efficiently routing queries to the optimal large language model (LLM) is crucial for optimizing the cost-performance trade-off in multi-model systems. However, most existing routers rely on single-model selection, making them susceptible to misrouting. In this work, we formulate LLM routing as the

α

-VOR problem to minimize expected set size while controlling the misrouting risk, and propose a novel method -- RACER, extending base routers to output model sets that can be subsequently aggregated for improved output. In particular, RACER constructs nested model sets via augmented scoring and utilizes finite-sample concentration bounds to calibrate a threshold that allows for both variable set sizes and abstention. We theoretically prove that RACER achieves rigorous distribution-free risk control on unseen test data in a post-hoc and model-agnostic manner. Extensive experiments verify our theoretical guarantees and demonstrate that RACER consistently enhances downstream accuracy across a wide range of benchmarks.

Abstract

Explore similar work

RACER: Risk-Aware Calibrated Efficient Routing for Large Language Models

FlexRouter: Learning Complementary Model Sets for Flexible LLM Routing

Federate the Router: Learning Language Model Routers with Sparse and Decentralized Evaluations