Rethinking Quality in Large Language Model Routing: Security-Conditioned Model-and-Budget Selection

Yichi Zhang; Zhenxiang Huang; Yuyang Zhang; Yuchen Yang

cs.AIOct 13, 2026

Rethinking Quality in Large Language Model Routing: Security-Conditioned Model-and-Budget Selection

Authors: Yichi Zhang, Zhenxiang Huang, Yuyang Zhang, Yuchen Yang

Organizations: The Pennsylvania State University · The University of Hong Kong

Abstract

Large language model (LLM) routers reduce serving cost by choosing which model to call and how many output tokens to grant it for each query. Existing routers rank model-budget actions using a single quality metric. This does not work for mixed workloads. For ordinary queries, a good answer is correct and useful (utility); for security-sensitive queries, a good answer refuses harmful requests or resists malicious instructions (security). These two objectives prefer different models and budgets. We propose SCRouter, a router that learns when to prioritize security and how to allocate the token budget. For each model-budget pair, it estimates the probability that the response finishes within budget. It uses this probability to predict utility and security separately. A small probe determines how to weight these two predictions for each query. SCRouter then selects the model and budget by balancing the weighted utility and security against cost. We also build SCRouterBench, a benchmark of 38,869 utility and security queries spanning nine security dimensions, each answered under all 144 combinations of 12 models and 12 budgets, plus 2,753 out-of-domain queries. SCRouter improves the baselines by 13-29 points in-domain and 19-23 points out-of-domain.

Figures & tables

Figure 1: Overall routing performance under in-domain and out-of-domain settings. SCRouter achieves the highest Oracle Gap Recovery (OGR) across both settings. OGR denotes the fraction of the gap between random selection and the oracle that a router recovers.

Figure 2: Model rankings differ between utility and security: strong utility need not imply strong security.

Figure 3: Utility and security of Qwen3-Coder-30B under different token budgets. Utility and security peak at different token budgets.

Figure 4: Overview of the SCRouter pipeline. (1) A frozen text encoder and linear probe estimate the query’s security weight

w(x)

. (2) A shared budget model predicts the probability

\pi(x,a)

that the response finishes within budget, and two quality models predict the utility and security surfaces

\hat{Q}_{U}

and

\hat{Q}_{S}

for all actions. (3) The two surfaces are combined with weight

w(x)

and traded off against cost with the cost coefficient

\mu

Figure 5: Efficiency versus performance across routers. SCRouter gets higher OGR with fewer parameters and lower TTFT. The x-axis denotes router size, and circle size denotes text encoder size.

Figure 6: Performance of the security probe. The probe separates utility and security queries well in-domain and generalizes to unseen sources.

Figure 7: Budget model calibration across models and budgets. Lines denote the fitted completion probability

\pi

from the shared budget model, while markers represent observed completion rates. The predicted

\pi

increases with

b

even though we do not force it to.

Figure 8: Parameter scaling across action spaces. Trainable parameters on a log scale for

n

models and

n

budgets excluding the frozen encoder, with our

12\times 12

setting dashed. R2-Router grows quadratically, from 43.7M to 700M parameters, while SCRouter stays at about 1.6M because its towers are shared.

	In-domain		OOD
Dimension	Source	#Q	Source	#Q
Utility †	SPROUT	19,882	MixEval	998
Toxicity	DecodingTrust	2,000	ToxiGen	200
Stereotype	DecodingTrust	1,994	CrowS-Pairs	200
Adv. demo.	DecodingTrust	1,996	BadChain	200
Privacy	DecodingTrust	2,000	Synthetic	200

Table 1: Overview of SCRouterBench. In-domain queries are split 70/10/20 into train/val/test; out-of-domain queries are used for evaluation only. Every query is scored under all 144 actions. † Six datasets: MMLU-Pro, GPQA, MuSR, MATH, OpenHermes, RAGBench. PI: prompt injection.

Figure 9: Performance-cost trade-offs on in-domain evaluation. Each curve shows performance against log cost as

\mu

varies. Random selection and the oracle are the lower and upper bounds. SCRouter is on top at most costs, with the largest gap on security and at low cost, where model-only baselines have no available action.

	In-Domain			Out-of-Domain
Router	All	Util.	Sec.	All	Util.	Sec.
SCRouter-L	60.9	52.9	72.0	48.1	49.6	47.3
SCRouter-M	60.5	52.9	70.9	50.2	50.7	49.8
SCRouter-S	59.9	53.9	66.6	46.9	48.4	46.1
R2-Router	48.0	49.4	39.1	15.6	32.5	0.3
RADAR (linear)	39.2	44.9	27.4	27.7	38.3	19.9

Table 2: OGR on SCRouterBench, in-domain and out-of-domain, for all queries (All) and for the utility (Util.) and security (Sec.) subsets. SCRouter variants beat all baselines in both settings.

Component	In-Domain	Out-of-Domain
SCRouter-L (full)	60.9	48.1
w/o Query-specific $w(x)$	54.0 (-6.9)	40.7 (-7.4)
w/o Budget terms	58.3 (-2.6)	43.0 (-5.1)
w/o Success/failure split	55.0 (-5.9)	42.4 (-5.7)
w/o Separate quality models	49.9 (-11.0)	37.3 (-10.8)

Table 3: Ablation results. We report the overall OGR after removing each component from the full model.

Figure 10: Leave-one-source-out evaluation across 15 training sources. We report the average held-out OGR for each method. All SCRouter variants beat the baselines on held-out sources.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Tier	Model	Developer	Params	Focus
Small	Qwen3.5-0.8B	Qwen	0.8B	General
	Gemma-3-1B-it ( Gemma Team et al., 2025 )	Google	1B	General
	Phi-4-mini-instruct ( Abouelenin et al., 2025 )	Microsoft	3.8B	General
	Qwen3.5-4B	Qwen	4B	General
Medium	Mistral-7B-Instruct-v0.3	Mistral AI	7B	General
	DeepSeek-R1-0528-Qwen3-8B ( Guo et al., 2025 )	DeepSeek	8B	Reasoning

Appendix

Table 4: Model pool of SCRouterBench: 12 open-weight LLMs from 0.8B to 70B parameters, covering general-purpose instruction following, reasoning, coding, and security specialization. Each model is paired with 12 token budgets, giving

|\mathcal{A}|=144

actions.

Source	Dimension	Outcome (score 1 if …)	Train	Val	Test
Utility (SPROUT)
MMLU-Pro	Knowledge	extracted final answer matches the reference	3,734	534	1,067
GPQA	Expert QA	extracted final answer matches the reference	168	24	48
MuSR	Multi-step reasoning	extracted final answer matches the reference	234	34	67
MATH	Mathematics	extracted final answer matches the reference	3,069	439	877
OpenHermes	Instruction following	independent judge (gpt-oss-120b) score	6,139	876	1,753

Appendix

Table 5: Composition of SCRouterBench. Every query is answered under 12 models

\times

12 budgets and scored in

[0,1]

for its own objective only (higher is better; for security, safer); a response without a visible final answer scores 0. Counts are queries with all 144 outcomes. Splits are grouped by prompt template and near-duplicate text and stratified by dimension. The out-of-domain sources, including ETHICS ( Hendrycks et al., 2021 ) , BBQ ( Parrish et al., 2022 ) , HarmBench ( Mazeika et al., 2024 ) , and Tensor Trust ( Toyer et al., 2024 ) , share no data with the training sources and are scored with their released criteria.

LLM Routing

Large language model routing selects models, tools, experts, or inference strategies to balance answer quality, cost, latency, and deployment constraints.

See all 355 papers

WISERouter: LLM Routing with Workload Budget Constraint
Jul 26, 2026Yifei Li, Zihui Gao, Laks V. S. LakshmananLLM Routing Contextual Bandits
The University of British Columbia
LLMRouter: Unified Infrastructure for Developing, Evaluating, and Deploying LLM Routers
Aug 7, 2026Tao Feng, Fangxu Yu, Haozhen Zhang +9LLM Evaluation LLM Routing
University of Illinois Urbana-Champaign · University of Maryland, College Park · 3Nanyang Technological University +2
RACER: Risk-Aware Calibrated Efficient Routing for Large Language Models
Feb 20, 2026Sai Hao, Hao Zeng, Hongxin Wei +1Set-Valued Prediction LLM Routing
Southern University of Science and Technology, China · The Chinese University of Hong Kong, Shenzhen, China · Shenzhen Loop Area Institute, China

Jul 26, 2026J/K move · Enter open · S save

WISERouter: LLM Routing with Workload Budget Constraint

Yifei Li, Zihui Gao, Laks V. S. Lakshmanan

The University of British Columbia

LLM Routing Contextual Bandits

Open paper AskPDF arXiv

Large language models (LLMs) achieve impressive performance across multiple domains, but using the most capable model for every query is prohibitive at scale. LLM routing exploits diversity in model capability and cost by assigning each query to a suitable model to balance utility and budget. Current methods have two limitations: (i) they either use heuristics that do not always enforce the budget constraint or impose a fixed per-query budget that cannot adapt across the workload and leads to suboptimal performance; (ii) they require supervised learning on a dense dataset with statistics for every query-model pair, which is expensive to collect. To address these challenges, we formulate LLM routing as a constrained contextual multi-armed bandit problem and introduce WISERouter (WR for short), a framework that supports offline learning from historical interactions as well as online learning with exploration. We further prove that WR-Online achieves a sublinear regret bound of

O(\sqrt{T})

over a time horizon

T

. Empirical results on RouterBench and SWE-Bench demonstrate that (i) WR-Offline surpasses existing baselines in performance under a fixed budget and adheres more closely to budget constraints, and (ii) WR-Online achieves comparable performance to the baselines, while using substantially less exploration data.

Abstract

LLM Routing

Explore similar work

WISERouter: LLM Routing with Workload Budget Constraint

LLMRouter: Unified Infrastructure for Developing, Evaluating, and Deploying LLM Routers

RACER: Risk-Aware Calibrated Efficient Routing for Large Language Models