ROUTEAUDIT: Interaction-Aware Identification for Budgeted Multi-Verifier Routing

Miaobo Hu; Shuhao Hu; Xiaobo Guo; Xin Wang; Bokun Wang; Tianshu Fu; Daren Zha

cs.AIOct 2, 2026

ROUTEAUDIT: Interaction-Aware Identification for Budgeted Multi-Verifier Routing

Authors: Miaobo Hu, Shuhao Hu, Xiaobo Guo, Xin Wang, Bokun Wang, Tianshu Fu, Daren Zha

Organizations: School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China

Abstract

Adaptive multi-verifier systems are commonly compared through endpoint quality-cost gaps, even when the verifier catalog, availability, accounting, information filtration, or scorer changes with the policy. We formulate verifier routing as a contract-conditioned identification problem. The contract records request support, verifier catalog, realized availability, resource accounting, online filtration, and post-trace scoring; a matched route contrast changes only the policy coordinate. ROUTEAUDIT adds three measurable objects to this contract. A contract lattice averages coordinate increments over every admissible bridge order and reports the resulting attribution together with its path sensitivity. A policy-independent response tape identifies paired sequential contrasts when adaptive policies reveal different observations. For incomplete matching, request-level bounds use whichever potential outcome remains observed and give a sharp finite-population interval. The protocol commits paid observations and ledger events before the oracle join and returns an attribution certificate for each comparison. On two held-out raw-tail caches, matched static SF+SA equals the cascade, assigning the apparent gains of 0.1797 and 0.1250 over full static to the verifier-set edge. On 1,319 held-out task requests, the learned and RLVR studies report quality 0.9522 and 0.9553 versus 0.9484 for matched static; the RLVR-static paired difference is +0.0068 with a request-paired interval $[0.0015,0.0122]$ and a training-seed-by-request hierarchical interval $[0.0006,0.0131]$ . Controlled attribution recovery yields route mean absolute error 0.0011 and endpoint reconstruction error 0.0004. Factorial, bridge-order, and stochastic-provider studies evaluate the certificate interface; RLVR supplies a learned-policy stress test under the same identification contract.

Figures & tables

family	outcomes	gate	budget trace	matched set	post-trace audit	task/live path
Selective prediction	task outputs	learned	aggregate	optional	risk–coverage	task
LLM routing	frozen or live outputs	learned	cost summary	benchmark dependent	aggregate	task
Process verification	step labels	verifier policy	verifier calls	verifier dependent	oracle join	task
Offline RL evaluation	trajectories	policy	support/cost	dataset dependent	replay	held-out task
RouteAudit	frozen traces	fixed or learned	per-call ledger	required	replay + mutation	task + serving

Table 1: Mechanism-level comparison. “Trace” means that per-call observations and budget events are retained; “matched” means that a static policy uses the same verifier set and split as the adaptive policy. The final row is the RouteAudit contract used in this paper.

Figure 1: RouteAudit’s identification workflow. The online policy observes only the frozen record, paid verifier outputs, and the remaining ledger; the trace is frozen before the post-freeze oracle join. The audit then reports quality, cost, verifier-set contrasts, and acceptance gates from the same committed trace.

Route	Queried verifiers	Fusion rule	Tie / abstention	Final-answer source
SF+SA	SF, SA	Eq. 5	SA tie; single-label fallback	verifier vote
SF+LF	SF, LF	LF-terminal rule; SF fallback	LF preferred on disagreement; single-label fallback; abstain if both missing	LF if available, otherwise SF
SF+SA+LF	SF, SA, LF	LF-terminal rule after the SF+SA vote	LF preferred; SF+SA vote fallback; abstain if all labels are missing	LF if available, otherwise SF+SA verifier vote

Table 2: Verifier fusion semantics. The SF+SA rule is explicit in Eq. 5 ; the remaining route fields are recorded as contract columns for the corresponding replay.

Component	Definition
Task / dataset	task-v3 ; normalized exact-answer reasoning score
Public benchmark / revision	GSM8K / test-v1 ; the public split or revision identifier is bound in the run manifest
Generators	Qwen2.5-7B / Qwen2.5-32B; checkpoint identities bound by the run record
Candidates	cand-r4 ; prompt, decoding, response count, and filtering are fixed
Verifier semantics	SF: small-response parser; SA: small–large agreement; LF: large-response parser
Router I/O	Paid labels/confidences, availability, budget; query, stop, or abstain

Table 3: Experimental system and comparison unit. Generator identities and candidate construction are bound independently for the task study and historical cache diagnostics.

Policy	Task quality $\uparrow$	Calls $\downarrow$	Cost $\downarrow$	p95 (ms) $\downarrow$
Matched static	0.9484	2.0000	1.000	1084
Fixed cascade	0.9484	1.7031	0.797	892
Calibrated cascade	0.9477	1.5815	0.736	824
Learned router	0.9522	1.4936	0.693	756
RLVR router	0.9553	1.4466	0.680	773

Table 4: Matched task and serving comparison on 1,319 held-out requests. Quality is the reported full-set task score; calls are per request. Cost is normalized to matched static; p95 is live-serving latency. Best values within each metric are bold.

Figure 2: Task quality and serving cost (left), with reported paired task-quality differences and 95% intervals relative to matched static (right). The points use the five-seed task summary and seven-seed serving summary. Intervals apply to paired differences; the static reference is zero by definition.

Appendix figures & tables68 assets

Supplementary material from the paper’s appendix.

Appendix

cache / contrast	policy	quality	calls / units
baseline-128	static SF+SA+LF	0.7656	3.00 / 10
	cascade SF $\rightarrow$ SA	0.9453	2.00 / 6
	matched static SF+SA	0.9453	2.00 / 6
	$\Delta_{\mathrm{route}}$	0.0000	0.00 / 0
	$\Delta_{\mathrm{set}}$ (matched minus full)	$+0.1797$	$-1.00$ / $-4$
corrected-2B-32	static SF+SA+LF	0.8438	3.00 / 10

Appendix

Table 5: Matched decomposition on the raw-tail held-out caches. Values are proxy accuracy / mean calls / declared units per example. The route contrast compares cascade with static SF+SA; the set contrast subtracts full static SF+SA+LF from matched static SF+SA.

Catalog	Policy	Quality	Calls	Norm. cost	$\Gamma_{\mathcal{V}}$
matched	static	0.9484	2.0000	1.000	–
matched	adaptive	0.9553	1.4466	0.680	–
expanded	static	0.9561	3.0000	1.000	–
expanded	adaptive	0.9602	1.7500	0.646	$-0.0027$

Appendix

Table 6: Contract interaction cells on the held-out task. Quality, calls, and normalized cost use the common request denominator;

\Gamma_{\mathcal{V}}

is the adaptive-minus-static route contrast under the expanded catalog minus the corresponding contrast under the matched catalog.

Figure 3: Raw-tail proxy quality under full static, matched static, and cascade controls. The matched policies coincide on both caches, assigning the apparent full-static gap to verifier-set choice.

Figure 4: Predeclared gate across the threshold grid. The mixed-path region violates quality non-inferiority; the high threshold restores quality by eliminating route diversity and savings.

Cache	$n$	Unmatched discord.	Matched discord.	Route $\Delta$	$p$	95% CI
baseline-128	128	23	0	0.0000	1.0000	$[0.0000,0.0000]$
corrected-2B-32	32	4	0	0.0000	1.0000	$[0.0000,0.0000]$

Appendix

Table 7: Paired raw-tail audit. Unmatched discordances compare full static with cascade; matched discordances compare SF+SA static with cascade. The interval is a sample-paired 95% bootstrap interval and

p

is the exact two-sided McNemar value.

Field	Fixed experimental contract
splits	Train / calibration / held-out splits stay disjoint by sample ID; split sizes are $4096/256/1319$ .
caches	Independent model and verifier caches are bound to each split; cache identities and hashes are recorded in the run manifest.
calibration	Calibrated confidence and correlated-error evaluation run on the calibration split before any held-out scoring; calibration seed count is $5$ .
methods	Matched static verifier set; fixed cascade; calibrated cascade; learned trust/router; RLVR router where applicable.
budgets	Matched verifier-call, token, and compute budgets are fixed before evaluation; exact ceilings are 3 calls / 10 ledger units / 4096 tokens / 1.0 GPU-s per request.
seeds	Independent training, calibration, and evaluation seeds are predeclared; seed count is $5$ .

Appendix

Table 8: Task protocol. Policies share the request split, verifier access, and resource ceilings; calibration precedes held-out scoring.

Policy	Quality	Accepted quality	Coverage	Abstain	Route diversity	Calls
Matched static	0.9484	0.9484	1.0000	0.0000	0.0000	2.0000
Fixed cascade	0.9484	0.9528	0.9955	0.0045	0.2969	1.7031
Calibrated cascade	0.9477	0.9513	0.9962	0.0038	0.4185	1.5815
Learned router	0.9522	0.9537	0.9985	0.0015	0.5064	1.4936
RLVR router	0.9553	0.9560	0.9992	0.0008	0.5534	1.4466

Appendix

Table 9: Task outcomes on 1,319 held-out requests (five-seed study). The full-coverage score equals the task-quality column for every policy in the source. Coverage and abstention are complementary fractions. Diversity is the reported one-call path fraction.

Policy	USD / request	Tokens	p50 (ms)	p95 (ms)	Throughput (req/s)	GPU-h
Matched static	$0.0231	1.82k	78	139	75.8	0.96
Fixed cascade	$0.0184	1.61k	65	118	90.4	0.81
Calibrated cascade	$0.0170	1.52k	61	108	96.1	0.75
Learned router	$0.0160	1.46k	59	103	99.3	0.72
RLVR router	$0.0157	1.43k	62	107	96.7	0.74

Appendix

Table 10: Task-run resource measurements. Tokens are generation tokens (k denotes thousands); p50/p95 are task-run milliseconds. GPU-hours cover the complete run. Serving timings are reported separately in Table 13 .

Policy	Reported $\Delta$ and 95% interval
Matched static	$0.0000\,[0.0000,0.0000]$
Fixed cascade	$0.0000\,[-0.0030,0.0030]$
Calibrated cascade	$-0.0008\,[-0.0045,0.0031]$
Learned router	$+0.0038\,[-0.0003,0.0079]$
RLVR router	$\mathbf{+0.0068}\,[0.0015,0.0122]$

Appendix

Table 11: Paired task-quality differences against matched static. Values and request-paired 95% intervals retain the precision of the paired aggregation. The intervals quantify differences, not marginal accuracies.

Field	Fixed experimental contract
splits	Train / calibration / held-out splits stay disjoint by sample ID; split sizes are $4096/256/1319$ .
caches	Independent model and verifier caches are bound to train, calibration, and held-out splits.
calibration	Calibrated confidence and correlated-error evaluation run on the calibration split before any latency measurement; calibration seed count is $5$ .
methods	Matched static verifier set; fixed cascade; calibrated cascade; learned trust/router; RLVR router where applicable.
budgets	Matched verifier-call, token, and compute budgets are fixed before evaluation; exact ceilings are 3 calls / 10 ledger units / 4096 tokens / 1.0 GPU-s per request.
seeds	Independent deployment and measurement seeds are predeclared; seed count is $7$ .

Appendix

Table 12: Serving protocol. Each policy is measured on the same held-out requests with fixed resource ceilings and separate deployment seeds.

Policy	p50 (ms)	p95 (ms)	Throughput (req/s)	GPU-h	Normalized cost
Matched static	622	1084	12.8	1.04	1.000
Fixed cascade	515	892	15.6	0.86	0.797
Calibrated cascade	476	824	16.8	0.80	0.736
Learned router	438	756	18.2	0.78	0.693
RLVR router	446	773	17.9	0.81	0.680

Appendix

Table 13: Live-serving measurements on the matched request set (seven measurement seeds). Task quality, accepted quality, full-coverage quality, coverage, abstention, diversity, calls, money/tokens, and paired intervals coincide with Tables 9 – 11 .

Figure 5: Live-serving p50 and p95 (left) and throughput (right) for the five matched policies. Horizontal connectors link the two latency quantiles, not confidence intervals. Values summarize the seven-seed serving study.

condition	policy	proxy	coverage	calls	ledger units
baseline, test-1319	static	0.9416	1.00	2.00	6.00
baseline, test-1319	cascade	0.9416	1.00	2.00	6.00
large unavailable	cascade	0.9204	1.00	1.00	2.00
tight budget	cascade	0.9204	1.00	1.00	2.00

Appendix

Table 14: Real-model cache replay. Proxy matches are joined after routing; the final column is verifier-ledger units and the end-to-end cost fields are defined in Appendix E .

track	policy	checks	calls / units
catalog	static	cheap (2) + contract (12)	2.00 / 14.0
catalog	cascade	cheap (2), conditional contract (12)	1.625 / 9.5
raw-tail	static	SF (2) + SA (4) + LF (4)	3.00 / 10.0
raw-tail	cascade	SF (2) $\rightarrow$ SA (4)	2.00 / 6.0

Appendix

Table 15: Verifier-call contracts in the two evidence tracks. Calls and declared units are ledger quantities; the raw-tail labels name provider checks, not additional model-generation runs. The tracks have different verifier IDs, costs, and budgets.

policy	proxy-oracle acc.	selective acc.	coverage	calls	units
static ensemble	1.0000	1.0000	1.00	2.000	14.0
cascade	1.0000	1.0000	1.00	1.625	9.5

Appendix

Table 16: Historical mechanism screen on the 16-case frozen catalog. Quality columns are proxy-oracle joins; calls and declared verifier-cost units are per-request ledger measurements. Source mappings and hashes are recorded in the run manifest.

cache	policy	proxy	calls	units
baseline-128	static SF+LF	0.7656	2	6
baseline-128	static SF+SA	0.9453	2	6
baseline-128	cascade	0.9453	2	6
corrected-2B-32	static SF+LF	0.8438	2	6
corrected-2B-32	static SF+SA	0.9688	2	6
corrected-2B-32	cascade	0.9688	2	6

Appendix

Table 17: Budget-matched raw-tail controls. All rows use the same held-out records and declared 2-call/6-unit budget; proxy labels are joined only after each trace is frozen.

cache	$n$	static	cascade	$\Delta$	McNemar $p$
baseline-128	128	0.7656 [0.6852,0.8306]	0.9453 [0.8914,0.9733]	+0.1797	$2.38\times 10^{-7}$
corrected-2B-32	32	0.8438 [0.6825,0.9314]	0.9688 [0.8426,0.9945]	+0.1250	0.125

Appendix

Table 18: Paired frozen-cache audit. Static and cascade predictions are paired by sample ID; Wilson intervals and the exact two-sided McNemar test are descriptive checks on proxy labels after online trace validation.

cache	shift	policy	proxy	units	calls	budget
baseline-128	baseline	static	0.7656	10	3	0.0000
baseline-128	baseline	cascade	0.9453	6	2	0.0000
baseline-128	price shift	static	0.9453	8	2	1.0000
baseline-128	price shift	cascade	0.9453	8	2	0.0000
corrected-2B-32	baseline	static	0.8438	10	3	0.0000
corrected-2B-32	baseline	cascade	0.9688	6	2	0.0000

Appendix

Table 19: Selected held-out raw-tail replay rows. Proxy accuracy is joined after the routing trace is frozen; declared units and calls are ledger quantities in the separate SF/SA/LF catalog. The price-shift rows are stress diagnostics; unavailable and tight-budget rows are retained in the evidence metrics but omitted here.

cache	shift	policy	proxy	units	calls	ms/call	unavail./budget
baseline-128	baseline	static	0.7422	10	3	0.0113	0.0 / 0.0
baseline-128	baseline	cascade	0.9375	6	2	0.0118	0.0 / 0.0
baseline-128	large unavailable	cascade	0.9062	2	1	0.0108	1.0 / 0.0
baseline-128	tight budget	cascade	0.9062	2	1	0.0112	0.0 / 1.0
baseline-128	price shift	cascade	0.9375	8	2	0.0116	0.0 / 0.0
corrected-2B-32	baseline	cascade	0.9375	6	2	0.0109	0.0 / 0.0

Appendix

Table 20: On-demand local-provider replay over validated model caches. Proxy accuracy is joined after routing; declared units are the raw-tail ledger, and mean call time is local CPU function wall-time. End-to-end network and service fields are defined separately in Appendix E .

Figure 6: On-demand local-provider diagnostics at the selected threshold on baseline-128. Panel (a) compares static and cascade proxy quality against declared units; panel (b) shows cascade quality under availability, budget, and price shifts. Points are the corresponding frozen-cache measurements in Table 20 .

Figure 7: Primary catalog accounting. Separate panels show mean verifier calls and declared cost units per example, preserving their distinct units. The panels are the frozen CPU mechanism ledger; serving latency is reported in Appendix E .

Figure 8: Verifier-cost and oracle-accuracy diagnostic generated from the frozen catalog replay. The same ledger fields feed the trained-router and independent-verifier extensions.

Figure 9: Held-out raw-tail replay across both validated caches. The left panel compares full-static and cascade proxy accuracy; the right panel compares declared verifier units. The matched SF+SA static control shares the cascade quality and cost in both caches (Table 17 ).

Figure 10: Historical confidence diagnostics. SF uses 64 records and SA uses 13 non-abstaining records. Reported mean confidence, empirical accuracy, and expected calibration error are distinct statistics; fusion weights remain fixed.

field group	fields	scoring rule
identity	sample ID, split, cache hash, request hash	joins paired policies by exact ID
candidate	candidate IDs, order, availability	records the paid-read set and order
observation	verifier ID, label, confidence, reason	route input and calibration feature
budget	calls, ledger units, token ceiling, remaining budget	conservation check after every event
decision	answer, accept/abstain, final route	task scorer and selective metrics
oracle join	canonical answer, normalized answer, error class	joined after trace freeze

Appendix

Table 21: Task-row contract used by the matched evaluator. Fields are grouped by their visibility at route time and by their use during post-trace scoring.

Split	$n$	Proxy correct	Task correct	Both	Proxy only	Task only
calibration	256	243	245	241	2	4
validation	256	242	244	240	2	4
held-out	1319	1242	1251	1237	5	14

Appendix

Table 22: Exact-string proxy and normalized-task agreement counts. Each split has its own denominator; proxy-only and task-only count disagreements between the two scorers.

$\tau_{a}$	$\tau_{v}$	$Q_{\mathrm{proxy}}$	$U$	$T$ (ms)	Paths	Gate
0.55	0.20	0.9258	3.6875	0.31	148/108	FAIL Q
0.55	0.30	0.9297	3.8125	0.33	140/116	FAIL Q
0.55	0.40	0.9336	3.9375	0.34	132/124	FAIL Q
0.65	0.20	0.9375	4.0625	0.34	124/132	FAIL Q
0.65	0.30	0.9414	4.1719	0.36	117/139	FAIL Q
0.65	0.40	0.9414	4.2813	0.37	110/146	FAIL Q

Appendix

Table 23: Complete Selection256 calibration grid.

U

is mean ledger units,

T

is local parser ms, and paths are one-call/two-call counts (sum 256).

Q

denotes the quality gate; SELECTED marks the selected operating pair.

Figure 11: Selection256 calibration response surface over the 15 threshold pairs. Panels show proxy quality and mean ledger units; the outlined cell is the selected pair $(0.85,0.30)$ . Every cell summarizes 256 calibration requests; the heatmap shows point values.

Cache	Pair	$p_{j}$	$p_{k}$	$p_{jk}$	$j\mid k$ / $k\mid j$	$\phi$
baseline-128	SF–SA	0.0781	0.0547	0.0313	0.5714 / 0.4000	0.4421
baseline-128	SF–LF	0.0781	0.2344	0.0469	0.2000 / 0.6000	0.2513
corrected-2B-32	SF–SA	0.0625	0.0313	0.0313	1.0000 / 0.5000	0.6956
corrected-2B-32	SF–LF	0.0625	0.1563	0.0313	0.2000 / 0.5000	0.2444

Appendix

Table 24: Error dependence on the two held-out caches.

p_{j},p_{k}

are marginal error rates,

p_{jk}

is joint error, and conditional columns are

p(e_{j}\mid e_{k})

and

p(e_{k}\mid e_{j})

. The last column is the reported

\phi

coefficient.

Factor	Level	Quality	$\Delta$	Fail-closed fraction	Units
availability	SF off	0.9136	-0.0348	0.1835	3.44
	SA off	0.9272	-0.0212	0.0713	2.00
	LF off	0.9484	0.0000	0.0045	4.81
label noise	0.00	0.9484	0.0000	0.0045	4.81
	0.05	0.9409	-0.0076	0.0053	4.82
	0.10	0.9325	-0.0159	0.0068	4.84

Appendix

Table 25: One-factor sensitivity results.

\Delta

is the quality change from the unperturbed reference, fail-closed is a fraction, and units are mean ledger units. S1–S4 preserve the source order for the four signed confidence perturbations with magnitudes 0.05 and 0.10.

Figure 12: Robustness under label noise (left) and call-budget ceilings (right). Points are the one-factor sweep values on the matched request set; lines connect tested levels. The curves show quality degradation under noise and the two-call budget plateau.

Component	Measurement	Mean	p50	p95	Share
generation	Tokens	1473	1432	2198	0.566
verifier	Ledger units	4.326	6.000	6.000	0.224
network	Network time (ms)	71.8	63.2	138.7	0.096
queue	Queue time (ms)	26.4	18.6	72.1	0.071
post-process	Post-process (ms)	0.86	0.78	1.49	0.043
total	Normalized cost	1.000	0.812	1.431	1.000

Appendix

Table 26: Component ledger from the serving result summary. Raw measurements retain their row-specific units. Shares are normalized cost shares; the normalized-total row is a separate cost statistic. Raw token, unit, and time rows are never summed as latency.

Policy	Quality	Coverage	Diversity	Cost	p95 (s)	GPU-h
matched static	0.9484	1.0000	0.000	1.000	1.084	1.04
fixed cascade	0.9484	0.9955	0.2969	0.797	0.892	0.86
calibrated cascade	0.9477	0.9962	0.4185	0.736	0.824	0.80
learned router	0.9522	0.9985	0.5064	0.693	0.756	0.78
RLVR router	0.9553	0.9992	0.5534	0.680	0.773	0.81

Appendix

Table 27: Quality–cost frontier for the serving comparison. Task quality, coverage, and diversity are fractions; cost is normalized to matched static, latency is seconds, and GPU-h is aggregate run consumption.

Family	Mutation	$N$	Detected	Missed
sample identity	duplicate/drop/relabel ID	256	256	0
ordering	swap adjacent events	256	256	0
paid read	hide or add provider event	256	256	0
availability	flip provider status	248	247	1
budget	change cost or remaining units	256	256	0
reason	alter escalation reason	244	242	2

Appendix

Table 28: Trace-integrity mutation results.

N

is the number of mutated traces; detected and missed partition that count. Clean FP counts false positives on unmodified traces.

Fault source	Mutation family	$N$	Detected	Missed
Known generator	single-coordinate	256	256	0
Unseen generator	single-coordinate	256	253	3
Unseen generator	two-coordinate compound	248	247	1
Unseen generator	three-coordinate compound	244	244	0

Appendix

Table 29: Blind and compound trace faults. Detected and missed counts partition the mutated traces; clean false positives are measured on unmodified traces.

Semantic corruption	Cases	Detected	Missed
Scorer swap	128	128	0
Stale cache substitution	128	126	2
Availability drift	128	127	1
Held-out threshold selection	128	124	4
Response-tape mismatch	128	128	0
Matched-ID / response mismatch	128	127	1

Appendix

Table 30: Blind semantic audit. Cases, detections, misses, and clean false positives are reported for scientifically invalid but schema-valid comparisons.

Class	Trigger	Count	Rate	$\Delta$	Median units
malformed response	parser cannot extract marker	6	0.0045	-0.0015	2.0
parser disagreement	SF/SA/LF labels differ	14	0.0106	-0.0030	6.0
correlated error	shared wrong normalized answer	26	0.0197	-0.0106	6.0
availability	provider unavailable or timeout	8	0.0061	-0.0045	2.0
budget	escalation rejected by ledger	5	0.0038	-0.0038	2.0
calibration drift	confidence bin changes	9	0.0068	-0.0023	4.0

Appendix

Table 31: Held-out error taxonomy. Counts and rates use 1,319 requests; paired

\Delta

values and median units are reported within the diagnostic classes. Classes may include secondary tags, so their deltas are not added as an overall effect.

field	meaning	consumer
schema version	row and event schema identifier	replay checker
source revision	code revision and dirty-state field	provenance audit
config digest	policy, budget, threshold, and scorer config	selection256
model/cache hash	candidate and verifier cache identity	split binding
sample digest	ordered sample IDs and split role	paired statistics
seed	generation, calibration, or training seed	uncertainty analysis

Appendix

Table 32: Artifact manifest fields. Values are hashes or relative handles; local machine paths and credentials are excluded from the public package.

Result family	Source	Consumer
Historical cache audit	public summary	Tables 5 , 17
Historical trace replay	replay manifest	Paired and threshold diagnostics
Task and serving results	parsed result ledger	Tables 4 – 11
Correlation and robustness	parsed result ledger	Tables 24 , 25
Efficiency and mutations	parsed result ledger	Tables 26 , 28
Experiment result receipt	immutable result receipt	All reported numerical rows

Appendix

Table 33: Result-to-artifact index. The result records store exact source rows and artifact hashes.

$\tau$	One-call fraction	Quality	Matched $\Delta$	Units	Calls	Gate
0.55	0.296875	0.921875	-0.0625	4.8125	1.703125	FAIL Q
0.65	0.296875	0.921875	-0.0625	4.8125	1.703125	FAIL Q
0.75	0.296875	0.921875	-0.0625	4.8125	1.703125	FAIL Q
0.85	0.296875	0.921875	-0.0625	4.8125	1.703125	FAIL Q
0.95	0.000000	0.984375	0.0000	6.0000	2.000	FAIL D,S

Appendix

Table 34: Raw-tail threshold sweep on 64 calibration records.

Q

denotes the quality gate;

D,S

denote diversity and strict savings. The four mixed-path rows share the same observed route assignment.

Cache	Policy	Proxy	Task	Calls	Units
baseline-128	static SF+LF	0.7656	0.7813	2	6
baseline-128	static SF+SA	0.9453	0.9531	2	6
baseline-128	cascade SF $\rightarrow$ SA	0.9453	0.9531	2	6
baseline-128	static SF+SA+LF	0.7656	0.7813	3	10
corrected-2B-32	static SF+LF	0.8438	0.8750	2	6
corrected-2B-32	static SF+SA	0.9688	0.9688	2	6

Appendix

Table 35: Fixed-operating-point controls. Proxy and normalized task scores use the same cache IDs; calls and units refer to the full two-call cascade operating point. Mixed-threshold path fractions are reported separately below.

Case	Observed route	Ledger transition	Post-trace outcome
lowest-cost match	SF label/confidence	$B_{0}\rightarrow B_{1}$	quality = 1.000
largest discordance	SF, SA, LF labels	calls/units = 3/10	quality delta = +1.000
unavailable provider	SF then availability event	Fallback: UNAVAILABLE_CHEAP_ONLY	Answer: sf_normalized
tight budget	SF then budget event	remaining units = 0	abstention = false

Appendix

Table 36: Four representative route fixtures. The case description names the observation sequence, ledger outcome, and post-trace result; request identifiers remain de-identified.

Field	Learned router	RLVR router
Dataset / generator / candidate revision	task-v3 / Qwen2.5-7B+32B / cand-r4	task-v3 / Qwen2.5-7B+32B / cand-r4
Request encoder and frozen parameters	frozen 384-d request encoder + 14 telemetry scalars	frozen 384-d request encoder + 14 telemetry scalars
Policy layers / width / activation	2-layer MLP / 256,128 / GELU	2-layer shared trunk / 256,128 / GELU
Policy/value sharing and parameter count	policy-only; $0.136$ M	shared trunk + separate policy/value heads; $0.136$ M
Initialization and reference policy	Xavier-uniform; supervised calibration checkpoint	learned-router warm start; frozen reference = initialization
Training label / reward coefficients	training-only feasible-route label; CE $+\,0.10\,C_{\rm cost}$	$R^{\lambda}=z-\sum_{j}\lambda_{j}(C_{j}-\bar{C}_{j})$ ; adaptive duals

Appendix

Table 37: Implementation binding for the measured learned and RLVR policies. Each field is linked to a configuration or checkpoint receipt; the constrained formulation alone does not determine these implementation choices.

Policy	Quality	Paired $\Delta$	Calls	Norm. cost	p95 ms	Diversity
Matched static	0.9484	0.0000	2.0000	1.000	1084	0.0000
Fixed cascade	0.9484	0.0000	1.7031	0.797	892	0.2969
Calibrated cascade	0.9477	-0.0008	1.5815	0.736	824	0.4185
Random matched	0.9448	-0.0036	1.4466	0.689	804	0.5534
Cost-sensitive linear	0.9496	+0.0012	1.5682	0.724	817	0.4318
Cost-sensitive MLP	0.9514	+0.0030	1.5211	0.704	793	0.4789

Appendix

Table 38: Matched learned-routing baselines under the common verifier and resource contract. Quality is full-set task quality; paired

\Delta

is against matched static; cost is normalized serving cost and diversity is the one-call fraction.

Comparator	Mean $\Delta$	Request 95% CI	Hierarchical 95% CI	Wins	Losses	Ties
Bayesian Thompson	$+0.0023$	$[-0.0016,\,0.0062]$	$[-0.0021,\,0.0069]$	5	2	1312
Cost-constrained online	$+0.0015$	$[-0.0021,\,0.0052]$	$[-0.0026,\,0.0058]$	4	2	1313
Learned router	$+0.0030$	$[-0.0017,\,0.0077]$	$[-0.0020,\,0.0084]$	7	3	1309

Appendix

Table 39: Direct paired comparison of RLVR with the strongest learned baselines. Differences are RLVR minus the named comparator; intervals are request-paired and hierarchical seed-by-request 95% intervals on the same held-out requests.

Violation	Broken condition	$\Delta_{\mathrm{matched}}$	$\Delta_{\mathrm{naive}}$	$E_{\mathrm{attr}}$	Detected?
Verifier-set mismatch	C2	+0.0068	+0.1296	0.1228	Yes
Budget mismatch	C3	+0.0068	+0.0419	0.0351	Yes
Availability mismatch	C2	+0.0068	-0.0217	0.0285	Yes
Cache mismatch	C1	+0.0068	+0.0556	0.0488	Yes
Scorer mismatch	C1	+0.0068	+0.0307	0.0239	Yes
Held-out threshold selection	C4	+0.0068	+0.0245	0.0177	Yes

Appendix

Table 40: Controlled identification stress tests.

\Delta_{\mathrm{matched}}

is the fully matched route contrast,

\Delta_{\mathrm{naive}}

is the contrast after the stated violation, and

E_{\mathrm{attr}}=|\Delta_{\mathrm{naive}}-\Delta_{\mathrm{matched}}|

Policy	Quality	Calls	Norm. cost	Diversity	Paired $\Delta$
Math reasoning: GSM8K / test-v1
Matched static	0.9412	2.0000	1.000	0.0000	–
RLVR	0.9491	1.4587	0.684	0.5413	$+0.0079$
Knowledge / factual QA: TriviaQA / unfiltered-v1
Matched static	0.8876	2.0000	1.000	0.0000	–
RLVR	0.8952	1.4314	0.671	0.5686	$+0.0076$

Appendix

Table 41: Multi-task validation under the shared observed-only routing contract. Each task family has its own benchmark and split record; quality, calls, cost, route diversity, and the paired RLVR-minus-static contrast use that family-specific held-out denominator.

Method	Route MAE	Contract MAE	Interaction MAE	Endpoint error
Endpoint only	0.0187	0.0241	0.0118	0.0164
Single bridge order	0.0102	0.0127	0.0084	0.0061
Path mean, no interaction	0.0046	0.0053	0.0079	0.0028
RouteAudit lattice	0.0011	0.0013	0.0015	0.0004

Appendix

Table 42: Ground-truth attribution recovery. Absolute errors compare each analysis with the known finite-population route, contract, and interaction contributions.

Figure 13: Controlled attribution recovery. Route mean absolute error and endpoint reconstruction error are shown for the four analysis methods on the common request population (Table 42 ); smaller values are better. The plotted point values are the reported aggregate errors, without additional uncertainty estimates.

$V$	$P$	$A$	Static quality	Adaptive quality	$\Delta_{\rm route}$	Calls	Norm. cost
0	0	0	0.9298	0.9319	+0.0021	1.3207	0.591
0	0	1	0.9364	0.9411	+0.0047	1.3479	0.604
0	1	0	0.9349	0.9399	+0.0050	1.4216	0.641
0	1	1	0.9417	0.9488	+0.0071	1.4578	0.659
1	0	0	0.9393	0.9456	+0.0063	1.4862	0.677
1	0	1	0.9458	0.9554	+0.0096	1.5274	0.693

Appendix

Table 43: Complete contract factorial over verifier set (

V

), budget (

P

), and availability (

A

). The common denominator and scorer make each cell a bridge node for the lattice.

Coordinate	$\phi_{j}$	Min	Max	$\Omega_{j}$
Verifier set $V$	0.0053	0.0042	0.0064	0.0022
Budget $P$	0.0034	0.0024	0.0044	0.0020
Availability $A$	0.0027	0.0021	0.0033	0.0012

Appendix

Table 44: Order-robust contract attribution.

\phi_{j}

is the permutation-average contribution, and

\Omega_{j}

is the range over admissible bridge orders.

Replay	Task	Naive $\Delta$	Matched $\Delta$	Largest attribution	Certificate	Changed
FrugalGPT	GSM8K	$+0.0284$	$+0.0061$	Verifier/model set ( $+0.0187$ )	partially identified	Yes
RouteLLM	MMLU	$+0.0176$	$+0.0054$	Verifier/model set ( $+0.0098$ )	partially identified	Yes
SLARouter	Arena-style QA	$+0.0109$	$+0.0088$	Budget ( $+0.0017$ )	identified	No

Appendix

Table 45: Contract re-audit of controlled routing-system replays. A changed interpretation assigns the largest part of the endpoint gap to a contract coordinate under the replay contract.

Coordinate	Level	Policy	Quality	Calls	Norm. cost	Route $\Gamma$
verifier catalog	matched	static	0.9484	2.0000	1.000	–
verifier catalog	matched	adaptive	0.9553	1.4466	0.680	–
verifier catalog	expanded	static	0.9561	3.0000	1.000	–
verifier catalog	expanded	adaptive	0.9602	1.7500	0.646	$-0.0027$
budget	reference	static	0.9484	2.0000	1.000	–
budget	relaxed	adaptive	0.9575	1.6127	0.742	$+0.0023$

Appendix

Table 46: Contract-interaction factorial. Each cell uses the common held-out request denominator;

\Gamma

columns are route–contract interactions relative to the matched reference.

Verifier set
Path order	Min	Max	Mean	$\Omega$
$\mathcal{V}\to\mathcal{P}\to\mathcal{A}$	0.0039	0.0051	0.0045	0.0012
$\mathcal{V}\to\mathcal{A}\to\mathcal{P}$	0.0036	0.0048	0.0042	0.0012
$\mathcal{P}\to\mathcal{V}\to\mathcal{A}$	0.0032	0.0044	0.0038	0.0012
$\mathcal{P}\to\mathcal{A}\to\mathcal{V}$	0.0030	0.0042	0.0036	0.0012
$\mathcal{A}\to\mathcal{V}\to\mathcal{P}$	0.0042	0.0056	0.0049	0.0014

Appendix

Table 47: Path-sensitive bridge attribution. Each row is a predeclared bridge order; coordinate columns report the path minimum, maximum, mean attribution, and sensitivity

\Omega

Removal	$\epsilon$	Matched $\Delta$	Observed $\Delta$	Envelope width	Coverage	Tightness
Random	0.00	0.0068	0.0068	0.0000	1.0000	1.0000
Random	0.05	0.0068	0.0065	0.2000	1.0000	0.9875
Difficulty-biased	0.10	0.0068	0.0089	0.4000	1.0000	0.9812
Verifier-dependent	0.20	0.0068	$-0.0037$	0.8000	1.0000	0.9726
Verifier-dependent	0.30	0.0068	$-0.0114$	1.2000	1.0000	0.9651

Appendix

Table 48: Incomplete-match stress test. Width, coverage, and tightness are reported for the conservative envelope over repeated removals; they are distinct from the request-level sharp bound.

Provider regime	Mean $\Delta$	SD	CI width	Route disagreement	Certificate match	Seeds
Frozen deterministic cache	0.0068	0.0019	0.0107	0.0842	1.0000	5
Shared response tape	0.0067	0.0023	0.0114	0.0876	1.0000	7
Independent response draws	0.0061	0.0049	0.0198	0.1164	0.9986	7

Appendix

Table 49: Provider-coupling regimes. Mean

\Delta

is the route contrast; the final columns record paired stability and certificate validity under the declared provider realization.

Variant	Quality	Calls	Units	Norm. cost	Diversity	Violations
Quality-only RLVR	0.9570	1.9285	5.942	0.914	0.0948	83
Fixed scalar penalty	0.9529	1.5934	4.508	0.754	0.4147	12
Without call constraint	0.9562	2.1179	6.734	0.887	0.0819	97
Without unit constraint	0.9558	1.6742	5.812	0.801	0.3596	74
Without latency constraint	0.9550	1.4759	3.904	0.701	0.5347	42
Without budget state	0.9518	1.5276	4.113	0.719	0.4724	26

Appendix

Table 50: RLVR component ablation on the matched held-out task. Units are declared ledger units; constraint violations count requests exceeding a declared ceiling.

Update	Quality	Calls	Units	Violations	KL	Entropy	$\lambda_{N}$	$\lambda_{U}$
0	0.9488	1.9874	5.934	126	0.0000	0.996	0.000	0.000
128	0.9507	1.7389	4.982	64	0.0087	0.923	0.031	0.044
256	0.9528	1.5794	4.331	26	0.0159	0.861	0.057	0.078
384	0.9544	1.4912	3.996	7	0.0218	0.803	0.074	0.103
512	0.9553	1.4466	3.786	0	0.0264	0.758	0.083	0.116

Appendix

Table 51: RLVR training dynamics at predeclared update checkpoints. Entries report seed-aggregated means; quality is task reward, calls and units are per request, and violations count requests above the declared ceilings.

Figure 14: Training-checkpoint trajectories on the shared held-out contract. Quality and resource means use the five declared training seeds; the violation panel retains the reported counts. Lines connect the five recorded checkpoints and carry no additional uncertainty estimate.

Policy	Run	Quality	Calls	Cost	Diversity
Learned	1	0.9509	1.5072	0.700	0.4928
Learned	2	0.9534	1.4815	0.687	0.5185
Learned	3	0.9521	1.4958	0.694	0.5042
Learned	4	0.9516	1.5011	0.697	0.4989
Learned	5	0.9530	1.4824	0.687	0.5176
RLVR	1	0.9541	1.4598	0.688	0.5402

Appendix

Table 52: Per-training-run record on the common held-out requests. Run indices enumerate independent fits; their random seed IDs and checkpoints are bound in the implementation record.

Policy	Quality	Calls	Norm. cost	Hierarchical 95% CI for $\Delta$
Cost-sensitive MLP	0.9514	1.5211	0.704	$[-0.0018,\,0.0077]$
Contextual bandit	0.9521	1.4879	0.696	$[-0.0011,\,0.0086]$
Bayesian router	0.9530	1.4693	0.687	$[-0.0003,\,0.0095]$
Learned router	0.9522	1.4936	0.693	$[-0.0010,\,0.0087]$
RLVR	0.9553	1.4466	0.680	$[0.0006,\,0.0131]$

Appendix

Table 53: Training-seed stability. Quality, calls, and cost are means over the held-out requests; the last column combines training-seed and request resampling.

Figure 15: Paired learned-router and RLVR operating points for the five declared training seeds. Segments join the same seed ID on the shared 1,319-request evaluation set. These are individual run summaries; request-level and hierarchical intervals are reported separately in the tables.

$K$	Policy	Quality	Calls	Norm. cost	$H_{\mathrm{route}}$	$\Gamma_{\mathcal{V}}$
2	Static	0.9484	2.0000	1.000	0	–
2	RLVR	0.9553	1.4466	0.680	0.9918	0.0000
3	Static	0.9561	3.0000	1.000	0	–
3	RLVR	0.9602	1.7500	0.646	0.9545	$-0.0027$
4	Static	0.9583	4.0000	1.000	0	–
4	RLVR	0.9624	1.9800	0.621	0.9078	$-0.0028$

Appendix

Table 54: Scaling with verifier-catalog size. Adaptive and static policies use the same catalog at each

K

; quality and cost use the full held-out request denominator. The final column is the route–catalog interaction measured against the matched static row.

Train / evaluation scorer	Full set	Scorer- agree	Proxy-only	Task-only
Proxy / proxy	0.9544	0.9600	0.9400	0.8600
Proxy / task	0.9512	0.9585	0.8550	0.9220
Task / task	0.9553	0.9606	0.8900	0.9300
Task / proxy	0.9561	0.9612	0.9450	0.8680

Appendix

Table 55: Scorer robustness of RLVR. The complete held-out set is partitioned into scorer-agree, proxy-only, and task-only strata before aggregation. Bold values compare training rewards within the same evaluation scorer.

Policy	Quality	Cost	Violations	Regret
Contextual bandit	0.9519	$21.26	6	8.31
Bayesian Thompson	0.9530	$21.03	4	6.77
Constrained online	0.9541	$20.86	2	4.92
RLVR	0.9556	$20.71	0	3.18

Appendix

Table 56: Streaming adaptation under delayed feedback. Quality is cumulative task score divided by processed requests; cost is cumulative paid cost. Regret uses the post-trace feasible-route oracle with the same request budgets.

Protocol	False attribution rate	Ledger violations detected	Oracle leaks detected	Replay match
Unmatched aggregate comparison	0.4219	–	–	0.7422
Matched verifier set	0.2344	–	–	0.8516
Budget and availability matching	0.1094	0.6875	–	0.9219
Trace freeze	0.0469	0.8750	0.9063	0.9688
Provenance binding	0.0156	0.9375	0.9375	0.9922
Full RouteAudit	0.0000	1.0000	1.0000	1.0000

Appendix

Table 57: Audit-component ablation. A false attribution is a reported route effect that differs from the fully matched reference because a non-policy factor changed.

stage	stored fields	invariant checked before release
setup	split, request, cache, budget	hashes and IDs bind the run to one contract
route	verifier, availability, confidence, budget	no oracle field enters the online state
close	label, reason, terminal state	event order and budget conservation hold
join	target, task score, latency	joins occur only after trace closure
compare	pair, interval, gate	matched IDs and verifier observations agree

Appendix

Table 58: Operational contract used by every RouteAudit run. The first three stages are online; scoring and uncertainty are post-trace joins.

Object	Contents
Numerical source	immutable experiment result receipt
Parsed rows and provenance	parsed rows and source map
Plot regeneration	vector PDF and 600 dpi PNG files
Data checks	row counts, identities, and marker scan
Historical replay	replay manifest with online events and oracle joins
Delivery receipt	source/PDF hashes and page audit

Appendix

Table 59: Reproducible supplement objects. The build receipt supplies concrete hashes and checks for the delivered source/PDF pair.

Adaptive Model Routing

Adaptive model routing selects models or specialized experts per input to balance prediction quality, computational cost, latency, and resource constraints.

See all 91 papers

VStress: Correlation-Aware Auditing and Adaptive Budget Allocation for Repeated Verifiers
Sep 29, 2026Miaobo Hu, Shuhao Hu, Xiaobo Guo +4Algorithmic Auditing Black-Box Model Auditing
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China
COVER: Identifiable Evaluation of Coalition Routing
Aug 28, 2026Raghul Sugumar, Amrit GopinathAI Agent Evaluation LLM Agent Routing
Sri Sivasubramaniya Nadar College of Engineering
RouteGuard: Certifying Routing Gain in LLM Multi-Agent Systems When Complementarity Is Not Enough
Aug 5, 2026Anchen Sun, Kaiqi YangMulti-Agent LLM Systems AI Agent Benchmarks
Department of Electrical and Computer Engineering, University of Miami, Coral Gables, FL 33146 USA · Department of Mathematics, University of Miami, Coral Gables, FL 33146 USA

Sep 29, 2026J/K move · Enter open · S save

VStress: Correlation-Aware Auditing and Adaptive Budget Allocation for Repeated Verifiers

Miaobo Hu, Shuhao Hu, Xiaobo Guo, Xin Wang+3

School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China

Algorithmic Auditing Black-Box Model Auditing

Open paper AskPDF arXiv

Repeated verifier calls are useful only when they contribute conditional information. We introduce VStress, an auditable replay contract, and VStress-CA, a correlation-aware allocation policy that estimates the conditional marginal information of an unqueried verifier on a sealed calibration split, discounts uncertainty, normalizes by call cost, and stops or abstains when the next call is not informative. The controller freezes its decision and cost ledger before joining the clean oracle; a dependence-shift alarm disables channel preference and falls back to exact-stop. The controlled audit gives the mechanism boundary: at 35% symmetric corruption, majority-5 improves balanced accuracy from 0.6578 to 0.7739, whereas at 65% it loses 0.1226 points. In the matched fixed-budget comparison, breadth, redundancy, and adaptive allocation obtain balanced accuracies 0.6048, 0.6375, and 0.6538, with 3.4216 calls per item and an RLVR score of 0.6417 for VStress-CA. Dependence diagnostics also increase from same-model repeats to cross-family channels, with conditional marginal gains of 0.0126, 0.0462, and 0.0913. These measurements turn correlation from a post-hoc warning into an auditable allocation decision.

Abstract

Adaptive Model Routing

Explore similar work

VStress: Correlation-Aware Auditing and Adaptive Budget Allocation for Repeated Verifiers

COVER: Identifiable Evaluation of Coalition Routing

RouteGuard: Certifying Routing Gain in LLM Multi-Agent Systems When Complementarity Is Not Enough