Organizations: School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China
Adaptive multi-verifier systems are commonly compared through endpoint quality-cost gaps, even when the verifier catalog, availability, accounting, information filtration, or scorer changes with the policy. We formulate verifier routing as a contract-conditioned identification problem. The contract records request support, verifier catalog, realized availability, resource accounting, online filtration, and post-trace scoring; a matched route contrast changes only the policy coordinate. ROUTEAUDIT adds three measurable objects to this contract. A contract lattice averages coordinate increments over every admissible bridge order and reports the resulting attribution together with its path sensitivity. A policy-independent response tape identifies paired sequential contrasts when adaptive policies reveal different observations. For incomplete matching, request-level bounds use whichever potential outcome remains observed and give a sharp finite-population interval. The protocol commits paid observations and ledger events before the oracle join and returns an attribution certificate for each comparison. On two held-out raw-tail caches, matched static SF+SA equals the cascade, assigning the apparent gains of 0.1797 and 0.1250 over full static to the verifier-set edge. On 1,319 held-out task requests, the learned and RLVR studies report quality 0.9522 and 0.9553 versus 0.9484 for matched static; the RLVR-static paired difference is +0.0068 with a request-paired interval [0.0015,0.0122] and a training-seed-by-request hierarchical interval [0.0006,0.0131]. Controlled attribution recovery yields route mean absolute error 0.0011 and endpoint reconstruction error 0.0004. Factorial, bridge-order, and stochastic-provider studies evaluate the certificate interface; RLVR supplies a learned-policy stress test under the same identification contract.
Figures & tables
family
outcomes
gate
budget trace
matched set
post-trace audit
task/live path
Selective prediction
task outputs
learned
aggregate
optional
risk–coverage
task
LLM routing
frozen or live outputs
learned
cost summary
benchmark dependent
aggregate
task
Process verification
step labels
verifier policy
verifier calls
verifier dependent
oracle join
task
Offline RL evaluation
trajectories
policy
support/cost
dataset dependent
replay
held-out task
RouteAudit
frozen traces
fixed or learned
per-call ledger
required
replay + mutation
task + serving
Table 1: Mechanism-level comparison. “Trace” means that per-call observations and budget events are retained; “matched” means that a static policy uses the same verifier set and split as the adaptive policy. The final row is the RouteAudit contract used in this paper.
Figure 1: RouteAudit’s identification workflow. The online policy observes only the frozen record, paid verifier outputs, and the remaining ledger; the trace is frozen before the post-freeze oracle join. The audit then reports quality, cost, verifier-set contrasts, and acceptance gates from the same committed trace.
Route
Queried verifiers
Fusion rule
Tie / abstention
Final-answer source
SF+SA
SF, SA
Eq. 5
SA tie; single-label fallback
verifier vote
SF+LF
SF, LF
LF-terminal rule; SF fallback
LF preferred on disagreement; single-label fallback; abstain if both missing
LF if available, otherwise SF
SF+SA+LF
SF, SA, LF
LF-terminal rule after the SF+SA vote
LF preferred; SF+SA vote fallback; abstain if all labels are missing
LF if available, otherwise SF+SA verifier vote
Table 2: Verifier fusion semantics. The SF+SA rule is explicit in Eq. 5 ; the remaining route fields are recorded as contract columns for the corresponding replay.
Component
Definition
Task / dataset
task-v3 ; normalized exact-answer reasoning score
Public benchmark / revision
GSM8K / test-v1 ; the public split or revision identifier is bound in the run manifest
Generators
Qwen2.5-7B / Qwen2.5-32B; checkpoint identities bound by the run record
Candidates
cand-r4 ; prompt, decoding, response count, and filtering are fixed
Paid labels/confidences, availability, budget; query, stop, or abstain
Table 3: Experimental system and comparison unit. Generator identities and candidate construction are bound independently for the task study and historical cache diagnostics.
Policy
Task quality ↑
Calls ↓
Cost ↓
p95 (ms) ↓
Matched static
0.9484
2.0000
1.000
1084
Fixed cascade
0.9484
1.7031
0.797
892
Calibrated cascade
0.9477
1.5815
0.736
824
Learned router
0.9522
1.4936
0.693
756
RLVR router
0.9553
1.4466
0.680
773
Table 4: Matched task and serving comparison on 1,319 held-out requests. Quality is the reported full-set task score; calls are per request. Cost is normalized to matched static; p95 is live-serving latency. Best values within each metric are bold.
Figure 2: Task quality and serving cost (left), with reported paired task-quality differences and 95% intervals relative to matched static (right). The points use the five-seed task summary and seven-seed serving summary. Intervals apply to paired differences; the static reference is zero by definition.
Appendix figures & tables68 assets
Supplementary material from the paper’s appendix.
Appendix
cache / contrast
policy
quality
calls / units
baseline-128
static SF+SA+LF
0.7656
3.00 / 10
cascade SF → SA
0.9453
2.00 / 6
matched static SF+SA
0.9453
2.00 / 6
Δroute
0.0000
0.00 / 0
Δset (matched minus full)
+0.1797
−1.00 / −4
corrected-2B-32
static SF+SA+LF
0.8438
3.00 / 10
Appendix
Table 5: Matched decomposition on the raw-tail held-out caches. Values are proxy accuracy / mean calls / declared units per example. The route contrast compares cascade with static SF+SA; the set contrast subtracts full static SF+SA+LF from matched static SF+SA.
Catalog
Policy
Quality
Calls
Norm. cost
ΓV
matched
static
0.9484
2.0000
1.000
–
matched
adaptive
0.9553
1.4466
0.680
–
expanded
static
0.9561
3.0000
1.000
–
expanded
adaptive
0.9602
1.7500
0.646
−0.0027
Appendix
Table 6: Contract interaction cells on the held-out task. Quality, calls, and normalized cost use the common request denominator; ΓV is the adaptive-minus-static route contrast under the expanded catalog minus the corresponding contrast under the matched catalog.
Figure 3: Raw-tail proxy quality under full static, matched static, and cascade controls. The matched policies coincide on both caches, assigning the apparent full-static gap to verifier-set choice.
Figure 4: Predeclared gate across the threshold grid. The mixed-path region violates quality non-inferiority; the high threshold restores quality by eliminating route diversity and savings.
Cache
n
Unmatched discord.
Matched discord.
Route Δ
p
95% CI
baseline-128
128
23
0
0.0000
1.0000
[0.0000,0.0000]
corrected-2B-32
32
4
0
0.0000
1.0000
[0.0000,0.0000]
Appendix
Table 7: Paired raw-tail audit. Unmatched discordances compare full static with cascade; matched discordances compare SF+SA static with cascade. The interval is a sample-paired 95% bootstrap interval and p is the exact two-sided McNemar value.
Field
Fixed experimental contract
splits
Train / calibration / held-out splits stay disjoint by sample ID; split sizes are 4096/256/1319 .
caches
Independent model and verifier caches are bound to each split; cache identities and hashes are recorded in the run manifest.
calibration
Calibrated confidence and correlated-error evaluation run on the calibration split before any held-out scoring; calibration seed count is 5 .
Matched verifier-call, token, and compute budgets are fixed before evaluation; exact ceilings are 3 calls / 10 ledger units / 4096 tokens / 1.0 GPU-s per request.
seeds
Independent training, calibration, and evaluation seeds are predeclared; seed count is 5 .
Appendix
Table 8: Task protocol. Policies share the request split, verifier access, and resource ceilings; calibration precedes held-out scoring.
Policy
Quality
Accepted quality
Coverage
Abstain
Route diversity
Calls
Matched static
0.9484
0.9484
1.0000
0.0000
0.0000
2.0000
Fixed cascade
0.9484
0.9528
0.9955
0.0045
0.2969
1.7031
Calibrated cascade
0.9477
0.9513
0.9962
0.0038
0.4185
1.5815
Learned router
0.9522
0.9537
0.9985
0.0015
0.5064
1.4936
RLVR router
0.9553
0.9560
0.9992
0.0008
0.5534
1.4466
Appendix
Table 9: Task outcomes on 1,319 held-out requests (five-seed study). The full-coverage score equals the task-quality column for every policy in the source. Coverage and abstention are complementary fractions. Diversity is the reported one-call path fraction.
Policy
USD / request
Tokens
p50 (ms)
p95 (ms)
Throughput (req/s)
GPU-h
Matched static
$0.0231
1.82k
78
139
75.8
0.96
Fixed cascade
$0.0184
1.61k
65
118
90.4
0.81
Calibrated cascade
$0.0170
1.52k
61
108
96.1
0.75
Learned router
$0.0160
1.46k
59
103
99.3
0.72
RLVR router
$0.0157
1.43k
62
107
96.7
0.74
Appendix
Table 10: Task-run resource measurements. Tokens are generation tokens (k denotes thousands); p50/p95 are task-run milliseconds. GPU-hours cover the complete run. Serving timings are reported separately in Table 13 .
Policy
Reported Δ and 95% interval
Matched static
0.0000[0.0000,0.0000]
Fixed cascade
0.0000[−0.0030,0.0030]
Calibrated cascade
−0.0008[−0.0045,0.0031]
Learned router
+0.0038[−0.0003,0.0079]
RLVR router
+0.0068[0.0015,0.0122]
Appendix
Table 11: Paired task-quality differences against matched static. Values and request-paired 95% intervals retain the precision of the paired aggregation. The intervals quantify differences, not marginal accuracies.
Field
Fixed experimental contract
splits
Train / calibration / held-out splits stay disjoint by sample ID; split sizes are 4096/256/1319 .
caches
Independent model and verifier caches are bound to train, calibration, and held-out splits.
calibration
Calibrated confidence and correlated-error evaluation run on the calibration split before any latency measurement; calibration seed count is 5 .
Matched verifier-call, token, and compute budgets are fixed before evaluation; exact ceilings are 3 calls / 10 ledger units / 4096 tokens / 1.0 GPU-s per request.
seeds
Independent deployment and measurement seeds are predeclared; seed count is 7 .
Appendix
Table 12: Serving protocol. Each policy is measured on the same held-out requests with fixed resource ceilings and separate deployment seeds.
Policy
p50 (ms)
p95 (ms)
Throughput (req/s)
GPU-h
Normalized cost
Matched static
622
1084
12.8
1.04
1.000
Fixed cascade
515
892
15.6
0.86
0.797
Calibrated cascade
476
824
16.8
0.80
0.736
Learned router
438
756
18.2
0.78
0.693
RLVR router
446
773
17.9
0.81
0.680
Appendix
Table 13: Live-serving measurements on the matched request set (seven measurement seeds). Task quality, accepted quality, full-coverage quality, coverage, abstention, diversity, calls, money/tokens, and paired intervals coincide with Tables 9 – 11 .
Figure 5: Live-serving p50 and p95 (left) and throughput (right) for the five matched policies. Horizontal connectors link the two latency quantiles, not confidence intervals. Values summarize the seven-seed serving study.
condition
policy
proxy
coverage
calls
ledger units
baseline, test-1319
static
0.9416
1.00
2.00
6.00
baseline, test-1319
cascade
0.9416
1.00
2.00
6.00
large unavailable
cascade
0.9204
1.00
1.00
2.00
tight budget
cascade
0.9204
1.00
1.00
2.00
Appendix
Table 14: Real-model cache replay. Proxy matches are joined after routing; the final column is verifier-ledger units and the end-to-end cost fields are defined in Appendix E .
track
policy
checks
calls / units
catalog
static
cheap (2) + contract (12)
2.00 / 14.0
catalog
cascade
cheap (2), conditional contract (12)
1.625 / 9.5
raw-tail
static
SF (2) + SA (4) + LF (4)
3.00 / 10.0
raw-tail
cascade
SF (2) → SA (4)
2.00 / 6.0
Appendix
Table 15: Verifier-call contracts in the two evidence tracks. Calls and declared units are ledger quantities; the raw-tail labels name provider checks, not additional model-generation runs. The tracks have different verifier IDs, costs, and budgets.
policy
proxy-oracle acc.
selective acc.
coverage
calls
units
static ensemble
1.0000
1.0000
1.00
2.000
14.0
cascade
1.0000
1.0000
1.00
1.625
9.5
Appendix
Table 16: Historical mechanism screen on the 16-case frozen catalog. Quality columns are proxy-oracle joins; calls and declared verifier-cost units are per-request ledger measurements. Source mappings and hashes are recorded in the run manifest.
cache
policy
proxy
calls
units
baseline-128
static SF+LF
0.7656
2
6
baseline-128
static SF+SA
0.9453
2
6
baseline-128
cascade
0.9453
2
6
corrected-2B-32
static SF+LF
0.8438
2
6
corrected-2B-32
static SF+SA
0.9688
2
6
corrected-2B-32
cascade
0.9688
2
6
Appendix
Table 17: Budget-matched raw-tail controls. All rows use the same held-out records and declared 2-call/6-unit budget; proxy labels are joined only after each trace is frozen.
cache
n
static
cascade
Δ
McNemar p
baseline-128
128
0.7656 [0.6852,0.8306]
0.9453 [0.8914,0.9733]
+0.1797
2.38×10−7
corrected-2B-32
32
0.8438 [0.6825,0.9314]
0.9688 [0.8426,0.9945]
+0.1250
0.125
Appendix
Table 18: Paired frozen-cache audit. Static and cascade predictions are paired by sample ID; Wilson intervals and the exact two-sided McNemar test are descriptive checks on proxy labels after online trace validation.
cache
shift
policy
proxy
units
calls
unavail.
budget
baseline-128
baseline
static
0.7656
10
3
0.0000
0.0000
baseline-128
baseline
cascade
0.9453
6
2
0.0000
0.0000
baseline-128
price shift
static
0.9453
8
2
0.0000
1.0000
baseline-128
price shift
cascade
0.9453
8
2
0.0000
0.0000
corrected-2B-32
baseline
static
0.8438
10
3
0.0000
0.0000
corrected-2B-32
baseline
cascade
0.9688
6
2
0.0000
0.0000
Appendix
Table 19: Selected held-out raw-tail replay rows. Proxy accuracy is joined after the routing trace is frozen; declared units and calls are ledger quantities in the separate SF/SA/LF catalog. The price-shift rows are stress diagnostics; unavailable and tight-budget rows are retained in the evidence metrics but omitted here.
cache
shift
policy
proxy
units
calls
ms/call
unavail./budget
baseline-128
baseline
static
0.7422
10
3
0.0113
0.0 / 0.0
baseline-128
baseline
cascade
0.9375
6
2
0.0118
0.0 / 0.0
baseline-128
large unavailable
cascade
0.9062
2
1
0.0108
1.0 / 0.0
baseline-128
tight budget
cascade
0.9062
2
1
0.0112
0.0 / 1.0
baseline-128
price shift
cascade
0.9375
8
2
0.0116
0.0 / 0.0
corrected-2B-32
baseline
cascade
0.9375
6
2
0.0109
0.0 / 0.0
Appendix
Table 20: On-demand local-provider replay over validated model caches. Proxy accuracy is joined after routing; declared units are the raw-tail ledger, and mean call time is local CPU function wall-time. End-to-end network and service fields are defined separately in Appendix E .
Figure 6: On-demand local-provider diagnostics at the selected threshold on baseline-128. Panel (a) compares static and cascade proxy quality against declared units; panel (b) shows cascade quality under availability, budget, and price shifts. Points are the corresponding frozen-cache measurements in Table 20 .
Figure 7: Primary catalog accounting. Separate panels show mean verifier calls and declared cost units per example, preserving their distinct units. The panels are the frozen CPU mechanism ledger; serving latency is reported in Appendix E .
Figure 8: Verifier-cost and oracle-accuracy diagnostic generated from the frozen catalog replay. The same ledger fields feed the trained-router and independent-verifier extensions.
Figure 9: Held-out raw-tail replay across both validated caches. The left panel compares full-static and cascade proxy accuracy; the right panel compares declared verifier units. The matched SF+SA static control shares the cascade quality and cost in both caches (Table 17 ).
Figure 10: Historical confidence diagnostics. SF uses 64 records and SA uses 13 non-abstaining records. Reported mean confidence, empirical accuracy, and expected calibration error are distinct statistics; fusion weights remain fixed.
Table 21: Task-row contract used by the matched evaluator. Fields are grouped by their visibility at route time and by their use during post-trace scoring.
Split
n
Proxy correct
Task correct
Both
Proxy only
Task only
calibration
256
243
245
241
2
4
validation
256
242
244
240
2
4
held-out
1319
1242
1251
1237
5
14
Appendix
Table 22: Exact-string proxy and normalized-task agreement counts. Each split has its own denominator; proxy-only and task-only count disagreements between the two scorers.
τa
τv
Qproxy
U
T (ms)
Paths
Gate
0.55
0.20
0.9258
3.6875
0.31
148/108
FAIL Q
0.55
0.30
0.9297
3.8125
0.33
140/116
FAIL Q
0.55
0.40
0.9336
3.9375
0.34
132/124
FAIL Q
0.65
0.20
0.9375
4.0625
0.34
124/132
FAIL Q
0.65
0.30
0.9414
4.1719
0.36
117/139
FAIL Q
0.65
0.40
0.9414
4.2813
0.37
110/146
FAIL Q
Appendix
Table 23: Complete Selection256 calibration grid. U is mean ledger units, T is local parser ms, and paths are one-call/two-call counts (sum 256). Q denotes the quality gate; SELECTED marks the selected operating pair.
Figure 11: Selection256 calibration response surface over the 15 threshold pairs. Panels show proxy quality and mean ledger units; the outlined cell is the selected pair (0.85,0.30) . Every cell summarizes 256 calibration requests; the heatmap shows point values.
Cache
Pair
pj
pk
pjk
j∣k / k∣j
ϕ
baseline-128
SF–SA
0.0781
0.0547
0.0313
0.5714 / 0.4000
0.4421
baseline-128
SF–LF
0.0781
0.2344
0.0469
0.2000 / 0.6000
0.2513
corrected-2B-32
SF–SA
0.0625
0.0313
0.0313
1.0000 / 0.5000
0.6956
corrected-2B-32
SF–LF
0.0625
0.1563
0.0313
0.2000 / 0.5000
0.2444
Appendix
Table 24: Error dependence on the two held-out caches. pj,pk are marginal error rates, pjk is joint error, and conditional columns are p(ej∣ek) and p(ek∣ej) . The last column is the reported ϕ coefficient.
Factor
Level
Quality
Δ
Fail-closed fraction
Units
availability
SF off
0.9136
-0.0348
0.1835
3.44
SA off
0.9272
-0.0212
0.0713
2.00
LF off
0.9484
0.0000
0.0045
4.81
label noise
0.00
0.9484
0.0000
0.0045
4.81
0.05
0.9409
-0.0076
0.0053
4.82
0.10
0.9325
-0.0159
0.0068
4.84
Appendix
Table 25: One-factor sensitivity results. Δ is the quality change from the unperturbed reference, fail-closed is a fraction, and units are mean ledger units. S1–S4 preserve the source order for the four signed confidence perturbations with magnitudes 0.05 and 0.10.
Figure 12: Robustness under label noise (left) and call-budget ceilings (right). Points are the one-factor sweep values on the matched request set; lines connect tested levels. The curves show quality degradation under noise and the two-call budget plateau.
Component
Measurement
Mean
p50
p95
Share
generation
Tokens
1473
1432
2198
0.566
verifier
Ledger units
4.326
6.000
6.000
0.224
network
Network time (ms)
71.8
63.2
138.7
0.096
queue
Queue time (ms)
26.4
18.6
72.1
0.071
post-process
Post-process (ms)
0.86
0.78
1.49
0.043
total
Normalized cost
1.000
0.812
1.431
1.000
Appendix
Table 26: Component ledger from the serving result summary. Raw measurements retain their row-specific units. Shares are normalized cost shares; the normalized-total row is a separate cost statistic. Raw token, unit, and time rows are never summed as latency.
Policy
Quality
Coverage
Diversity
Cost
p95 (s)
GPU-h
matched static
0.9484
1.0000
0.000
1.000
1.084
1.04
fixed cascade
0.9484
0.9955
0.2969
0.797
0.892
0.86
calibrated cascade
0.9477
0.9962
0.4185
0.736
0.824
0.80
learned router
0.9522
0.9985
0.5064
0.693
0.756
0.78
RLVR router
0.9553
0.9992
0.5534
0.680
0.773
0.81
Appendix
Table 27: Quality–cost frontier for the serving comparison. Task quality, coverage, and diversity are fractions; cost is normalized to matched static, latency is seconds, and GPU-h is aggregate run consumption.
Family
Mutation
N
Detected
Missed
Clean FP
sample identity
duplicate/drop/relabel ID
256
256
0
0
ordering
swap adjacent events
256
256
0
0
paid read
hide or add provider event
256
256
0
0
availability
flip provider status
248
247
1
0
budget
change cost or remaining units
256
256
0
0
reason
alter escalation reason
244
242
2
0
Appendix
Table 28: Trace-integrity mutation results. N is the number of mutated traces; detected and missed partition that count. Clean FP counts false positives on unmodified traces.
Fault source
Mutation family
N
Detected
Missed
Clean FP
Known generator
single-coordinate
256
256
0
0
Unseen generator
single-coordinate
256
253
3
0
Unseen generator
two-coordinate compound
248
247
1
0
Unseen generator
three-coordinate compound
244
244
0
0
Appendix
Table 29: Blind and compound trace faults. Detected and missed counts partition the mutated traces; clean false positives are measured on unmodified traces.
Semantic corruption
Cases
Detected
Missed
Clean FP
Scorer swap
128
128
0
0
Stale cache substitution
128
126
2
0
Availability drift
128
127
1
0
Held-out threshold selection
128
124
4
0
Response-tape mismatch
128
128
0
0
Matched-ID / response mismatch
128
127
1
0
Appendix
Table 30: Blind semantic audit. Cases, detections, misses, and clean false positives are reported for scientifically invalid but schema-valid comparisons.
Class
Trigger
Count
Rate
Δ
Median units
malformed response
parser cannot extract marker
6
0.0045
-0.0015
2.0
parser disagreement
SF/SA/LF labels differ
14
0.0106
-0.0030
6.0
correlated error
shared wrong normalized answer
26
0.0197
-0.0106
6.0
availability
provider unavailable or timeout
8
0.0061
-0.0045
2.0
budget
escalation rejected by ledger
5
0.0038
-0.0038
2.0
calibration drift
confidence bin changes
9
0.0068
-0.0023
4.0
Appendix
Table 31: Held-out error taxonomy. Counts and rates use 1,319 requests; paired Δ values and median units are reported within the diagnostic classes. Classes may include secondary tags, so their deltas are not added as an overall effect.
field
meaning
consumer
schema version
row and event schema identifier
replay checker
source revision
code revision and dirty-state field
provenance audit
config digest
policy, budget, threshold, and scorer config
selection256
model/cache hash
candidate and verifier cache identity
split binding
sample digest
ordered sample IDs and split role
paired statistics
seed
generation, calibration, or training seed
uncertainty analysis
Appendix
Table 32: Artifact manifest fields. Values are hashes or relative handles; local machine paths and credentials are excluded from the public package.
Result family
Source
Consumer
Historical cache audit
public summary
Tables 5 , 17
Historical trace replay
replay manifest
Paired and threshold diagnostics
Task and serving results
parsed result ledger
Tables 4 – 11
Correlation and robustness
parsed result ledger
Tables 24 , 25
Efficiency and mutations
parsed result ledger
Tables 26 , 28
Experiment result receipt
immutable result receipt
All reported numerical rows
Appendix
Table 33: Result-to-artifact index. The result records store exact source rows and artifact hashes.
τ
One-call fraction
Quality
Matched Δ
Units
Calls
Gate
0.55
0.296875
0.921875
-0.0625
4.8125
1.703125
FAIL Q
0.65
0.296875
0.921875
-0.0625
4.8125
1.703125
FAIL Q
0.75
0.296875
0.921875
-0.0625
4.8125
1.703125
FAIL Q
0.85
0.296875
0.921875
-0.0625
4.8125
1.703125
FAIL Q
0.95
0.000000
0.984375
0.0000
6.0000
2.000
FAIL D,S
Appendix
Table 34: Raw-tail threshold sweep on 64 calibration records. Q denotes the quality gate; D,S denote diversity and strict savings. The four mixed-path rows share the same observed route assignment.
Cache
Policy
Proxy
Task
Calls
Units
baseline-128
static SF+LF
0.7656
0.7813
2
6
baseline-128
static SF+SA
0.9453
0.9531
2
6
baseline-128
cascade SF → SA
0.9453
0.9531
2
6
baseline-128
static SF+SA+LF
0.7656
0.7813
3
10
corrected-2B-32
static SF+LF
0.8438
0.8750
2
6
corrected-2B-32
static SF+SA
0.9688
0.9688
2
6
Appendix
Table 35: Fixed-operating-point controls. Proxy and normalized task scores use the same cache IDs; calls and units refer to the full two-call cascade operating point. Mixed-threshold path fractions are reported separately below.
Case
Observed route
Ledger transition
Post-trace outcome
lowest-cost match
SF label/confidence
B0→B1
quality = 1.000
largest discordance
SF, SA, LF labels
calls/units = 3/10
quality delta = +1.000
unavailable provider
SF then availability event
Fallback: UNAVAILABLE_CHEAP_ONLY
Answer: sf_normalized
tight budget
SF then budget event
remaining units = 0
abstention = false
Appendix
Table 36: Four representative route fixtures. The case description names the observation sequence, ledger outcome, and post-trace result; request identifiers remain de-identified.
training-only feasible-route label; CE +0.10Ccost
Rλ=z−∑jλj(Cj−Cˉj) ; adaptive duals
Appendix
Table 37: Implementation binding for the measured learned and RLVR policies. Each field is linked to a configuration or checkpoint receipt; the constrained formulation alone does not determine these implementation choices.
Policy
Quality
Paired Δ
Calls
Norm. cost
p95 ms
Diversity
Matched static
0.9484
0.0000
2.0000
1.000
1084
0.0000
Fixed cascade
0.9484
0.0000
1.7031
0.797
892
0.2969
Calibrated cascade
0.9477
-0.0008
1.5815
0.736
824
0.4185
Random matched
0.9448
-0.0036
1.4466
0.689
804
0.5534
Cost-sensitive linear
0.9496
+0.0012
1.5682
0.724
817
0.4318
Cost-sensitive MLP
0.9514
+0.0030
1.5211
0.704
793
0.4789
Appendix
Table 38: Matched learned-routing baselines under the common verifier and resource contract. Quality is full-set task quality; paired Δ is against matched static; cost is normalized serving cost and diversity is the one-call fraction.
Comparator
Mean Δ
Request 95% CI
Hierarchical 95% CI
Wins
Losses
Ties
Bayesian Thompson
+0.0023
[−0.0016,0.0062]
[−0.0021,0.0069]
5
2
1312
Cost-constrained online
+0.0015
[−0.0021,0.0052]
[−0.0026,0.0058]
4
2
1313
Learned router
+0.0030
[−0.0017,0.0077]
[−0.0020,0.0084]
7
3
1309
Appendix
Table 39: Direct paired comparison of RLVR with the strongest learned baselines. Differences are RLVR minus the named comparator; intervals are request-paired and hierarchical seed-by-request 95% intervals on the same held-out requests.
Violation
Broken condition
Δmatched
Δnaive
Eattr
Detected?
Verifier-set mismatch
C2
+0.0068
+0.1296
0.1228
Yes
Budget mismatch
C3
+0.0068
+0.0419
0.0351
Yes
Availability mismatch
C2
+0.0068
-0.0217
0.0285
Yes
Cache mismatch
C1
+0.0068
+0.0556
0.0488
Yes
Scorer mismatch
C1
+0.0068
+0.0307
0.0239
Yes
Held-out threshold selection
C4
+0.0068
+0.0245
0.0177
Yes
Appendix
Table 40: Controlled identification stress tests. Δmatched is the fully matched route contrast, Δnaive is the contrast after the stated violation, and Eattr=∣Δnaive−Δmatched∣ .
Policy
Quality
Calls
Norm. cost
Diversity
Paired Δ
Math reasoning: GSM8K / test-v1
Matched static
0.9412
2.0000
1.000
0.0000
–
RLVR
0.9491
1.4587
0.684
0.5413
+0.0079
Knowledge / factual QA: TriviaQA / unfiltered-v1
Matched static
0.8876
2.0000
1.000
0.0000
–
RLVR
0.8952
1.4314
0.671
0.5686
+0.0076
Appendix
Table 41: Multi-task validation under the shared observed-only routing contract. Each task family has its own benchmark and split record; quality, calls, cost, route diversity, and the paired RLVR-minus-static contrast use that family-specific held-out denominator.
Method
Route MAE
Contract MAE
Interaction MAE
Endpoint error
Endpoint only
0.0187
0.0241
0.0118
0.0164
Single bridge order
0.0102
0.0127
0.0084
0.0061
Path mean, no interaction
0.0046
0.0053
0.0079
0.0028
RouteAudit lattice
0.0011
0.0013
0.0015
0.0004
Appendix
Table 42: Ground-truth attribution recovery. Absolute errors compare each analysis with the known finite-population route, contract, and interaction contributions.
Figure 13: Controlled attribution recovery. Route mean absolute error and endpoint reconstruction error are shown for the four analysis methods on the common request population (Table 42 ); smaller values are better. The plotted point values are the reported aggregate errors, without additional uncertainty estimates.
V
P
A
Static quality
Adaptive quality
Δroute
Calls
Norm. cost
0
0
0
0.9298
0.9319
+0.0021
1.3207
0.591
0
0
1
0.9364
0.9411
+0.0047
1.3479
0.604
0
1
0
0.9349
0.9399
+0.0050
1.4216
0.641
0
1
1
0.9417
0.9488
+0.0071
1.4578
0.659
1
0
0
0.9393
0.9456
+0.0063
1.4862
0.677
1
0
1
0.9458
0.9554
+0.0096
1.5274
0.693
Appendix
Table 43: Complete contract factorial over verifier set ( V ), budget ( P ), and availability ( A ). The common denominator and scorer make each cell a bridge node for the lattice.
Coordinate
ϕj
Min
Max
Ωj
Verifier set V
0.0053
0.0042
0.0064
0.0022
Budget P
0.0034
0.0024
0.0044
0.0020
Availability A
0.0027
0.0021
0.0033
0.0012
Appendix
Table 44: Order-robust contract attribution. ϕj is the permutation-average contribution, and Ωj is the range over admissible bridge orders.
Replay
Task
Naive Δ
Matched Δ
Largest attribution
Certificate
Changed
FrugalGPT
GSM8K
+0.0284
+0.0061
Verifier/model set ( +0.0187 )
partially identified
Yes
RouteLLM
MMLU
+0.0176
+0.0054
Verifier/model set ( +0.0098 )
partially identified
Yes
SLARouter
Arena-style QA
+0.0109
+0.0088
Budget ( +0.0017 )
identified
No
Appendix
Table 45: Contract re-audit of controlled routing-system replays. A changed interpretation assigns the largest part of the endpoint gap to a contract coordinate under the replay contract.
Coordinate
Level
Policy
Quality
Calls
Norm. cost
Route Γ
verifier catalog
matched
static
0.9484
2.0000
1.000
–
verifier catalog
matched
adaptive
0.9553
1.4466
0.680
–
verifier catalog
expanded
static
0.9561
3.0000
1.000
–
verifier catalog
expanded
adaptive
0.9602
1.7500
0.646
−0.0027
budget
reference
static
0.9484
2.0000
1.000
–
budget
relaxed
adaptive
0.9575
1.6127
0.742
+0.0023
Appendix
Table 46: Contract-interaction factorial. Each cell uses the common held-out request denominator; Γ columns are route–contract interactions relative to the matched reference.
Path order
Min
Max
Mean
Ω
Verifier set
V→P→A
0.0039
0.0051
0.0045
0.0012
V→A→P
0.0036
0.0048
0.0042
0.0012
P→V→A
0.0032
0.0044
0.0038
0.0012
P→A→V
0.0030
0.0042
0.0036
0.0012
A→V→P
0.0042
0.0056
0.0049
0.0014
Appendix
Table 47: Path-sensitive bridge attribution. Each row is a predeclared bridge order; coordinate columns report the path minimum, maximum, mean attribution, and sensitivity Ω .
Removal
ϵ
Matched Δ
Observed Δ
Envelope width
Coverage
Tightness
Random
0.00
0.0068
0.0068
0.0000
1.0000
1.0000
Random
0.05
0.0068
0.0065
0.2000
1.0000
0.9875
Difficulty-biased
0.10
0.0068
0.0089
0.4000
1.0000
0.9812
Verifier-dependent
0.20
0.0068
−0.0037
0.8000
1.0000
0.9726
Verifier-dependent
0.30
0.0068
−0.0114
1.2000
1.0000
0.9651
Appendix
Table 48: Incomplete-match stress test. Width, coverage, and tightness are reported for the conservative envelope over repeated removals; they are distinct from the request-level sharp bound.
Provider regime
Mean Δ
SD
CI width
Route disagreement
Certificate match
Seeds
Frozen deterministic cache
0.0068
0.0019
0.0107
0.0842
1.0000
5
Shared response tape
0.0067
0.0023
0.0114
0.0876
1.0000
7
Independent response draws
0.0061
0.0049
0.0198
0.1164
0.9986
7
Appendix
Table 49: Provider-coupling regimes. Mean Δ is the route contrast; the final columns record paired stability and certificate validity under the declared provider realization.
Variant
Quality
Calls
Units
Norm. cost
Diversity
Violations
Quality-only RLVR
0.9570
1.9285
5.942
0.914
0.0948
83
Fixed scalar penalty
0.9529
1.5934
4.508
0.754
0.4147
12
Without call constraint
0.9562
2.1179
6.734
0.887
0.0819
97
Without unit constraint
0.9558
1.6742
5.812
0.801
0.3596
74
Without latency constraint
0.9550
1.4759
3.904
0.701
0.5347
42
Without budget state
0.9518
1.5276
4.113
0.719
0.4724
26
Appendix
Table 50: RLVR component ablation on the matched held-out task. Units are declared ledger units; constraint violations count requests exceeding a declared ceiling.
Update
Quality
Calls
Units
Violations
KL
Entropy
λN
λU
0
0.9488
1.9874
5.934
126
0.0000
0.996
0.000
0.000
128
0.9507
1.7389
4.982
64
0.0087
0.923
0.031
0.044
256
0.9528
1.5794
4.331
26
0.0159
0.861
0.057
0.078
384
0.9544
1.4912
3.996
7
0.0218
0.803
0.074
0.103
512
0.9553
1.4466
3.786
0
0.0264
0.758
0.083
0.116
Appendix
Table 51: RLVR training dynamics at predeclared update checkpoints. Entries report seed-aggregated means; quality is task reward, calls and units are per request, and violations count requests above the declared ceilings.
Figure 14: Training-checkpoint trajectories on the shared held-out contract. Quality and resource means use the five declared training seeds; the violation panel retains the reported counts. Lines connect the five recorded checkpoints and carry no additional uncertainty estimate.
Policy
Run
Quality
Calls
Cost
Diversity
Learned
1
0.9509
1.5072
0.700
0.4928
Learned
2
0.9534
1.4815
0.687
0.5185
Learned
3
0.9521
1.4958
0.694
0.5042
Learned
4
0.9516
1.5011
0.697
0.4989
Learned
5
0.9530
1.4824
0.687
0.5176
RLVR
1
0.9541
1.4598
0.688
0.5402
Appendix
Table 52: Per-training-run record on the common held-out requests. Run indices enumerate independent fits; their random seed IDs and checkpoints are bound in the implementation record.
Policy
Quality
Calls
Norm. cost
Hierarchical 95% CI for Δ
Cost-sensitive MLP
0.9514
1.5211
0.704
[−0.0018,0.0077]
Contextual bandit
0.9521
1.4879
0.696
[−0.0011,0.0086]
Bayesian router
0.9530
1.4693
0.687
[−0.0003,0.0095]
Learned router
0.9522
1.4936
0.693
[−0.0010,0.0087]
RLVR
0.9553
1.4466
0.680
[0.0006,0.0131]
Appendix
Table 53: Training-seed stability. Quality, calls, and cost are means over the held-out requests; the last column combines training-seed and request resampling.
Figure 15: Paired learned-router and RLVR operating points for the five declared training seeds. Segments join the same seed ID on the shared 1,319-request evaluation set. These are individual run summaries; request-level and hierarchical intervals are reported separately in the tables.
K
Policy
Quality
Calls
Norm. cost
Hroute
ΓV
2
Static
0.9484
2.0000
1.000
0
–
2
RLVR
0.9553
1.4466
0.680
0.9918
0.0000
3
Static
0.9561
3.0000
1.000
0
–
3
RLVR
0.9602
1.7500
0.646
0.9545
−0.0027
4
Static
0.9583
4.0000
1.000
0
–
4
RLVR
0.9624
1.9800
0.621
0.9078
−0.0028
Appendix
Table 54: Scaling with verifier-catalog size. Adaptive and static policies use the same catalog at each K ; quality and cost use the full held-out request denominator. The final column is the route–catalog interaction measured against the matched static row.
Train / evaluation scorer
Full set
Scorer- agree
Proxy-only
Task-only
Proxy / proxy
0.9544
0.9600
0.9400
0.8600
Proxy / task
0.9512
0.9585
0.8550
0.9220
Task / task
0.9553
0.9606
0.8900
0.9300
Task / proxy
0.9561
0.9612
0.9450
0.8680
Appendix
Table 55: Scorer robustness of RLVR. The complete held-out set is partitioned into scorer-agree, proxy-only, and task-only strata before aggregation. Bold values compare training rewards within the same evaluation scorer.
Policy
Quality
Cost
Violations
Regret
Contextual bandit
0.9519
$21.26
6
8.31
Bayesian Thompson
0.9530
$21.03
4
6.77
Constrained online
0.9541
$20.86
2
4.92
RLVR
0.9556
$20.71
0
3.18
Appendix
Table 56: Streaming adaptation under delayed feedback. Quality is cumulative task score divided by processed requests; cost is cumulative paid cost. Regret uses the post-trace feasible-route oracle with the same request budgets.
Protocol
False attribution rate
Ledger violations detected
Oracle leaks detected
Replay match
Unmatched aggregate comparison
0.4219
–
–
0.7422
Matched verifier set
0.2344
–
–
0.8516
Budget and availability matching
0.1094
0.6875
–
0.9219
Trace freeze
0.0469
0.8750
0.9063
0.9688
Provenance binding
0.0156
0.9375
0.9375
0.9922
Full RouteAudit
0.0000
1.0000
1.0000
1.0000
Appendix
Table 57: Audit-component ablation. A false attribution is a reported route effect that differs from the fully matched reference because a non-policy factor changed.
stage
stored fields
invariant checked before release
setup
split, request, cache, budget
hashes and IDs bind the run to one contract
route
verifier, availability, confidence, budget
no oracle field enters the online state
close
label, reason, terminal state
event order and budget conservation hold
join
target, task score, latency
joins occur only after trace closure
compare
pair, interval, gate
matched IDs and verifier observations agree
Appendix
Table 58: Operational contract used by every RouteAudit run. The first three stages are online; scoring and uncertainty are post-trace joins.
Object
Contents
Numerical source
immutable experiment result receipt
Parsed rows and provenance
parsed rows and source map
Plot regeneration
vector PDF and 600 dpi PNG files
Data checks
row counts, identities, and marker scan
Historical replay
replay manifest with online events and oracle joins
Delivery receipt
source/PDF hashes and page audit
Appendix
Table 59: Reproducible supplement objects. The build receipt supplies concrete hashes and checks for the delivered source/PDF pair.
Adaptive model routing selects models or specialized experts per input to balance prediction quality, computational cost, latency, and resource constraints.
Repeated verifier calls are useful only when they contribute conditional information. We introduce VStress, an auditable replay contract, and VStress-CA, a correlation-aware allocation policy that estimates the conditional marginal information of an unqueried verifier on a sealed calibration split, discounts uncertainty, normalizes by call cost, and stops or abstains when the next call is not informative. The controller freezes its decision and cost ledger before joining the clean oracle; a dependence-shift alarm disables channel preference and falls back to exact-stop. The controlled audit gives the mechanism boundary: at 35% symmetric corruption, majority-5 improves balanced accuracy from 0.6578 to 0.7739, whereas at 65% it loses 0.1226 points. In the matched fixed-budget comparison, breadth, redundancy, and adaptive allocation obtain balanced accuracies 0.6048, 0.6375, and 0.6538, with 3.4216 calls per item and an RLVR score of 0.6417 for VStress-CA. Dependence diagnostics also increase from same-model repeats to cross-family channels, with conditional marginal gains of 0.0126, 0.0462, and 0.0913. These measurements turn correlation from a post-hoc warning into an auditable allocation decision.
Miaobo Hu, Shuhao Hu, Xiaobo Guo +4
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China
When a multi-agent system changes its team, it also changes the messages and final answer it produces, so an end-to-end accuracy gap does not by itself identify a routing effect. We introduce method, an evaluation contract that fixes a public information boundary, downstream stack G, and finite legal team family before outcomes are generated. Complete coverage identifies exact finite-benchmark oracle regret conditional on that stack. For any finite collection of frozen policies, executing the union of their distinct selected teams is the minimal assumption-free support for every pairwise policy contrast, though not for absolute oracle regret. Two controlled tables with source-ID-disjoint splits test the instrument. On MuSiQue-12, a pre-specified privileged positive control improves regret from 0.532 to 0.402; a later public-interface control reaches 0.424 versus 0.554 but is retrospective. On HotpotQA-4, a pre-specified public direct scorer improves regret from 0.313 to 0.110. In fixed-stack Llama execution, verified route regret improves by 0.190, while the raw-answer gain is 0.010 with an interval crossing zero. A five-family ToolSandbox variant-shift validation exhaustively evaluates 16 declared teams on 14 untouched task variants (224/224 valid rows): the declared-family oracle reaches 0.768 safe-evidence completion, while the prospectively frozen router gets 0.637 (regret 0.131), failing the predeclared 0.10 criterion. A later retrospective comparator reaches 0.655, matching all-workers with 4.57 versus 5.00 workers on average. Thus COVER exposes selection headroom without manufacturing a routing win. A crossed-stack diagnostic shows absolute scores depend on G but finds no detectable router-by-finalizer interaction. COVER is an auditable measurement methodology, not a claim of stack-invariant or universal agent-routing superiority.
Multi-agent LLM systems route among model-backed advisors, yet a deployer rarely knows before shipping whether routing will help at all. Prevailing routers optimize a gate's AUC and presume that advisor complementarity suffices. We show that neither determines the deployable gain. We introduce RouteGuard, a deployment-certification framework. Routing gain decomposes as G=πΔE, and the achievable gain is governed by a conditional-regret functional Φ, not by AUC. A finite-sample certification bracket comes with a matching Le Cam lower bound, constant-sharp over the fixed-activity class, and a robustness phase transition. On two benchmarks the framework acts as a guardrail. On RouterBench (11 cross-family models) the verdict depends on the sampling unit: the protocol certifies a gain over GPT-4 under prompt-level sampling and withholds it under workload-cluster resampling, because the gain rests on 3 of 86 workload cells. On OpenRCA (three Gemini advisors) the advisors are statistically redundant: the realized oracle sits at or below the independence baseline in all pools we tested (221 RouterBench pools and three OpenRCA distributions), so the protocol correctly refuses to certify. A pre-registered semi-synthetic control confirms calibration: the protocol certifies a genuine gain once m≥m⋆ and does not certify a true null. Code and frozen artifacts will be released with the published version.
Anchen Sun, Kaiqi Yang
Department of Electrical and Computer Engineering, University of Miami, Coral Gables, FL 33146 USA · Department of Mathematics, University of Miami, Coral Gables, FL 33146 USA