Proxy Confidence: Auditing Black-Box LLM Agents with a Surrogate's Log-Probabilities

Yikai Zhao; Saurabh Pandey; Pradeep Kumar Misra

cs.AIOct 2, 2026

Proxy Confidence: Auditing Black-Box LLM Agents with a Surrogate's Log-Probabilities

Authors: Yikai Zhao, Saurabh Pandey, Pradeep Kumar Misra

Organizations: Amazon

Abstract

A deployed LLM agent emits tool calls, queries, and code that can be silently wrong -- by the time the error surfaces, the action has run. Frontier chat APIs hide the model's token probabilities; the agent's stated confidence barely beats chance on the mistakes that matter; and resampling does not help, since frontier models are highly repetitive, reproducing the same call across samples. We recover the missing signal from a low-cost open-weight surrogate run in parallel. It reads the same context, schema, and proposed action as the agent, then scores the call from its own log-probabilities through a family of complementary readouts: teacher forcing and request-PMI weigh the likelihood of each argument value, a discriminative verdict judges the call as a whole, and tool-choice competition tests the function against its siblings. One principle says which to trust: a generative likelihood localizes wrong argument values, while the verdict catches holistically wrong calls. When the error type is unknown, an ensemble is the low-regret default. The readout is training-free, needs no access to the agent's internals, and costs one prefill pass alongside the tool call. On difficult coding tasks it reaches AUROC 0.825 where the actor's stated confidence is near chance (0.598), and the generative readouts beat it by +0.07 to +0.28 across three further actors. Against self-consistency it gains +0.14 to +0.19 on near-deterministic actors, at 1/K the cost. The signal drives two deployment modes: a real-time gate escalating the least-trustworthy calls for review (+0.05 to +0.30 accepted-action accuracy at 50% coverage), and confidence feedback, returning the tool result with the score so the agent adapts its next step -- lifting task success on live-execution benchmarks (+0.119 and +0.137, p <= 1e-4) and beating a random-value control where step errors are silent (+0.078, p = 0.003).

Figures & tables

Figure 1: The proxy-confidence sidecar: a training-free plug-in hook. The sidecar (dashed) taps the agent’s tool-call hook, reads only

(\text{context},\text{schema},\text{action})

—no logits, weights, or fine-tuning—and emits a confidence

c

, scored in parallel with tool execution. Drawn here are the two in-loop modes: a real-time oversight gate (Mode A: high

c

commit, low

c

escalate to a human) and confidence feedback (Mode B: return the tool result together with

c

to the actor, which chooses its next move). Details in § 7 –§ 8 .

Signal	Readout	Targets
Verdict	$P(\text{YES})$ on “is this correct?”, renormalized over YES/NO	coarse tool/code correctness
Teacher forcing	per-token $\log P$ of $\mathcal{A}$ ’s exact call, localized to argument value tokens	wrong argument values
Request-PMI	$\log P(o\mid\text{full})$ ${-}\log P(o\mid\text{req.\ withheld})$ , on value tokens	groundedness of args in request
Tool-choice competition	does $\mathcal{S}$ prefer $\mathcal{A}$ ’s function over $k$ siblings?	wrong function choice

Table 1: The four proxy readouts and the failure mode each targets.

Actor (family; $n$ )	TF	PMI	verd.	self	PMI $-$ self
C-120b (family C; 775)	0.735	0.794	0.526	0.515	$+0.279$
C-20b (family C; 771)	0.743	0.793	0.552	0.534	$+0.255$
B (family B; 1261)	0.636	0.721	0.546	0.648	$+0.071$

Table 2: PMI improves on stated confidence for all three actors outside family A. Proxy signals scoring each actor’s xLAM calls through Qwen3-32B; best proxy in bold. Actors are de-identified (§ 3 ); the comparison of interest is proxy-vs-self within each row, not between rows.

Dataset	locus	verd.	TF	PMI	actor	fuse	$P^{\star}$
AppWorld	holistic	0.894	n/a	n/a	0.788	n/a	.996
BFCL base	holistic	0.733	0.633	n/a	0.703	0.720	.945
BFCL long	holistic	0.719	0.640	n/a	0.682	0.700	.968
BFCL v3	args	0.711	0.549	0.714	0.636	0.700	.950
parallel	args	0.661	0.653	0.662	0.577	0.695	.980
NESTFUL	args	0.594	0.643	0.554	0.620	0.655	$.840$

Table 3: The champion signal tracks the error locus. Per-signal AUROC (32B scorer) vs. the actor’s self-assessment, best single signal in bold.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 2: The gain is access, not scale. On one Qwen3-32B: the verdict token’s log-probability (blue) vs. the same model’s verbalized confidence (grey) and the actor’s own self-assessment (orange, A1), across six datasets. Capability is held constant between blue and grey.

Fixed policy	worst-case regret (where)	mean regret
always verdict	$+0.180$ (miss_param)	$0.046$
always teacher-forcing	$+0.165$ (BFCL v3)	$0.087$
oriented fusion	$\mathbf{+0.019}$ (BFCL long)	$\mathbf{0.007}$

Appendix

Table 4: Oriented fusion has no catastrophic regime. Worst-case regret over the six single-call regimes where verdict and TF are both defined; lower is more robust.

Dataset	$N$	base	AUROC	acc@50% ( $\Delta$ )	AURC $\downarrow$	rand
BFCL (verdict+TF)	2388	0.692	0.596	0.743 ( $+0.051$ )	0.241	0.308
xLAM (verdict)	412	0.367	0.713	0.524 ( $+0.158$ )	0.484	0.633
xLAM (needle)	725	0.542	0.890	0.843 ( $+0.300$ )	0.179	0.458
NESTFUL (needle)	238	0.613	0.788	0.832 ( $+0.218$ )	0.178	0.387
tau2 (trajectory)	149	0.772	0.679	0.867 ( $+0.095$ )	0.149	0.228

Appendix

Table 5: Ranking by proxy confidence beats random abstention everywhere. Accept the top-confidence fraction and abstain on the rest.

Figure 3: Self-consistency collapses as the actor becomes deterministic. SC AUROC vs. sample unanimity (xLAM, $K{=}5$ , execution-free; $r{=}{-}0.96$ ); proxy TF/PMI (shaded) is entropy-invariant.

Dataset	Actor	Qwen	Gemma	self	$P$ (Q/G)
AppWorld	A1	0.894	0.836	0.788	1.00/.66
AppWorld	A2	0.919	0.884	0.914	.57/.14
BFCL base	A1	0.733	0.728	0.703	.95/.90
BFCL base	A2	0.800	0.823	0.732	1.00/1.00
BFCL long	A1	0.719	0.723	0.682	.97/.98
BFCL long	A2	0.820	0.821	0.759	1.00/1.00

Appendix

Table 6: The verdict is not one actor

\times

scorer artifact. Verdict AUROC across actors and scorers.

P

=

paired-bootstrap exceedance vs. that actor’s self-assessment; italic self

=

actor not beaten (self-assesses well).

Dataset	verd.	TF/PMI	used by (actors; scorers)
AppWorld	✓	—	grid, hero, Mode B (A+B+C; Q,Gm)
BFCL base/long	✓	—	grid, hero, cap14 (A+B+C; Q,q,Gm)
BFCL parallel/v3	✓	✓	hero, cap14, battery (A; Q,q)
BFCL miss_func/param	✓	✓ †	hero, cap14 (A; Q,q)
xLAM	✓	✓	hero, cap14, selective, active, SC, § 6 (A+B+C; Q,q)
NESTFUL	✓	✓	hero, cap14, selective, active (A; Q,q)

Appendix

Table 7: Which datasets support which signal, and where each result is used.

Figure 4: Teacher forcing is size-invariant; the verdict is not. Each dumbbell joins 14B (orange) and 32B (blue) AUROC for one cell, same items and prefill. Top: teacher forcing. Middle: verdict. Bottom: AppWorld exception. Chance $=0.5$ (dotted).

Dataset	Signal	14B	32B	$\Delta$
Teacher forcing—access-bound ( $\|\Delta\|{\leq}0.015$ , TOST-equivalent)
NESTFUL/A1	TF ${}_{\text{val}}$	0.636	0.650	$-.014$
NESTFUL/A2	TF ${}_{\text{val}}$	0.604	0.611	$-.006$
xLAM/A1	TF ${}_{\text{val}}$	0.674	0.670	$+.004$
xLAM/A2	TF ${}_{\text{val}}$	0.702	0.690	$+.011$
parallel/A1	TF ${}_{\text{val}}$	0.628	0.633	$-.005$

Appendix

Table 8: 14B matches 32B on teacher forcing, trails on request-PMI and the verdict. Qwen3-14B vs. 32B, matched items per signal.

\Delta{=}

14B

-

32B; verdict uses matched YES/NO teacher-forcing. Full tally in § 6 .

Readout	$p50$	$p90$	$p99$
Teacher forcing	$159$ ms	$166$ ms	$179$ ms
Verdict	$123$ ms	$132$ ms	$137$ ms

Appendix

Table 9: A scored call is a single prefill of

{\approx}160

ms. Measured sidecar latency at realistic full-context length (

{\approx}1.5

k tokens; Qwen3-14B fp8, one L40S GPU).

	nl2bash		AppWorld test_challenge
Arm	success	vs. control	success	vs. control
control (no number)	$0.399$	—	$0.595$	—
vigilance (no number)	$0.389$	$-0.010$ n.s.	—	—
random $U(0,1)$	$0.440$	$+0.041$ n.s.	$0.727$	$+0.132$ ( $p{<}10^{-4}$ )
verdict only	$0.430$	$+0.031$ n.s.	—	—
request-PMI	$0.472$	$+0.073$ ( $p{=}.029$ )	—	—

Appendix

Table 10: Inline readout comparison, and why a random-value arm is required. A2 actor; each readout appended to the real tool result. Contrasts are paired against the unannotated control; nl2bash arms all cover the same 193 tasks, AppWorld all 417. Vigilance announces the audit but shows no number; random shows an independent

U(0,1)

draw. The two benchmarks diverge on the random arm: on nl2bash it is flat and fusion beats it, so the annotation’s content matters; on AppWorld it matches fusion, so there any scalar suffices. Against the unannotated control alone both benchmarks would have looked alike.

Dataset	Self-verif.?	blind	all	all vs. blind / placebo (Holm $p$ )
xLAM, A1 ( $n{=}196$ )	no (schema)	$0.066$	$\mathbf{0.148}$	$0.004$ / $0.013$
xLAM, A2 ( $n{=}196$ )	no (schema)	$0.097$	$0.138$	$0.46$ / $0.006$
NESTFUL ( $n{=}184$ )	yes (recomp.)	$0.844$	$0.853$	n.s. / n.s.
BFCL-v4 ( $n{=}1499$ )	yes (real res.)	$0.892$	$0.892$	n.s. (all $\approx$ )
Selective review at fixed budget $k{=}1$ call (signal- vs. random-gated):
xLAM ( $n{=}192$ )	—	sig $0.92$ / rand $0.42$		$p{<}10^{-4}$ (95/1)

Appendix

Table 11: Feedback helps only when the actor cannot self-verify. Signal-guided self-review vs. deranged-placebo, all -signal arm. Self-verif.

=

can the actor check the call at review time. Bottom: repair rate at a one-call audit budget, signal- vs. random-gated.

Figure 5: Failure-mode map (same data as Table 3 ). Signal $\times$ dataset AUROC; black box $=$ best proxy signal per dataset; grey $=$ signal undefined for that action type.

Figure 6: Teacher forcing tracks argument-value accuracy; the verdict does not. Spearman $\rho$ vs. deterministic per-parameter accuracy (no LLM judge) as the target sharpens to literal argument values (NESTFUL, A2). TF rises to $\rho{=}0.41$ ; verdict $0.02$ .

Signal	AUROC	type	note
Ensemble disagreement	0.70–0.80	epistemic	orthogonal, near-free
Self-consistency ( $k{=}5$ )	0.71–0.75	epistemic	good on easy calls
Semantic entropy	0.68–0.71	epistemic	no gain over above
Schema-satisfaction	0.50–0.51	determ.	null (frontier actors)
verdict variants (vs. plain verdict, same rows)
Debiased verdict	0.51–0.67	discrim.	no gain on tool calls

Appendix

Table 12: Extra readouts an open scorer enables, and verdict variants.

Explore similar work

Evaluating Tool-Using Language Agents: Judge Reliability, Propagation Cascades, and Runtime Mitigation in AgentProp-Bench
Apr 17, 2026Bhaskar GurramAI Agent Reliability Tool-Augmented Language Model Agents
Zasti Inc., Ashburn, VA, USA
Agents' Overreliance on Unreliable Tools
Sep 4, 2026Hoyeol Yang, Woojung Song, Taewon Kim +3LLM Tool Use LLM Agent Reliability
Seoul National University, Seoul, Republic of Korea
ToolFailBench: Diagnosing Tool-Use Failures in LLM Agents
Jul 6, 2026Harsh SoniAI Agent Reliability Tool-Use Evaluation
UC Berkeley

Apr 17, 2026J/K move · Enter open · S save

Evaluating Tool-Using Language Agents: Judge Reliability, Propagation Cascades, and Runtime Mitigation in AgentProp-Bench

Bhaskar Gurram

Zasti Inc., Ashburn, VA, USA

AI Agent Reliability Tool-Augmented Language Model Agents LLM-as-a-Judge LLM Agent Evaluation AI Agent Benchmarks LLM Reliability+2

Open paper AskPDF arXiv

Automated evaluation of tool-using large language model (LLM) agents is widely assumed to be reliable, but this assumption has rarely been validated against human annotation. We introduce AgentProp-Bench, a 2,000-task benchmark with 2,300 traces across four domains, nine production LLMs, and a 100-label human-validated subset. We quantify judge reliability, characterize error propagation, and evaluate a runtime mitigation. Substring-based judging agrees with human annotation at kappa=0.049 (chance-level); a three-LLM ensemble reaches kappa=0.432 (moderate) with a conservative bias. Under validated evaluation, a parameter-level injection propagates to a wrong final answer with human-calibrated probability approximately 0.62 (range 0.46-0.73 across models). Rejection (catching bad parameters) and recovery (correcting after acceptance) are independent model capabilities (Spearman rho=0.126, p=0.747). A tuned runtime interceptor reduces hallucination on GPT-4o-mini by 23.0 percentage points under a concurrent n=600 control, but shows no significant effect on Gemini-2.0-Flash, whose aggressive parameter rejection eliminates the target failure mode. All code, data, traces, and human labels are released at https://github.com/bhaskargurram-ai/agenthallu-bench.

Abstract

Explore similar work

Evaluating Tool-Using Language Agents: Judge Reliability, Propagation Cascades, and Runtime Mitigation in AgentProp-Bench

Agents' Overreliance on Unreliable Tools

ToolFailBench: Diagnosing Tool-Use Failures in LLM Agents