Judge, Retrieve, or Abstain: Uncertainty-Guarded LLM Judging with Provable Risk Guarantees

Sher Badshah; Ali Emami; Hassan Sajjad

cs.CLAug 18, 2026

Judge, Retrieve, or Abstain: Uncertainty-Guarded LLM Judging with Provable Risk Guarantees

Authors: Sher Badshah, Ali Emami, Hassan Sajjad

Organizations: Dalhousie University · New York University Abu Dhabi · Emory University

Abstract

Using LLMs as judges has become standard practice for evaluating model outputs at scale. This is particularly common for subjective, open-ended tasks such as assessing helpfulness or alignment, where no single reference answer exists. However, objective tasks introduce a distinct reliability challenge for reference-free LLM judging. In the absence of a reference answer, the judge evaluates factual correctness either through its parametric knowledge or through tool augmentation. Although the former enables efficient evaluation, the judge may hallucinate or lack sufficient evidence for its verdict. Conversely, tool augmentation can provide additional evidence but introduces extra computational cost and requires an appropriate mechanism to determine when and how that evidence should be used reliably. More importantly, neither approach alone provides formal control over the risk of accepted verdicts or guarantees their reliability at a specified level. We propose a risk-controlled framework that calibrates uncertainty thresholds on a held-out set so that the false discovery rate among accepted verdicts remains below a user-specified level~ $α$ with high probability, using finite-sample Clopper--Pearson intervals. When the parametric mode is not sufficiently confident, the instance is routed to a retrieval-augmented mode, where the judge gathers web evidence and re-evaluates the instance under a second calibrated threshold. The finite-sample guarantee carries over to this two-threshold routing without additional assumptions. Across open-domain QA benchmarks and judges of varying scales, the framework maintains the target error rate while achieving substantially higher coverage than single-mode baselines.

Figures & tables

Figure 1

Figure 1: Test-time FDR (mean

\pm

std over 100 splits) across all 32 configurations. Rows correspond to the two candidate models; columns to the four datasets. Each line represents a different judge (see legend). The dashed diagonal marks FDR

=\alpha

		TriviaQA					NQ-Open					PopQA					HotpotQA
Cand.	Judge	.05	.10	.15	.20	.25	.05	.10	.15	.20	.25	.05	.10	.15	.20	.25	.05	.10	.15	.20	.25
Qwen-8B	Qwen-4B	.03	.39	.66	.91	1.0	.00	.03	.08	.14	.28	.00	.01	.91	1.0	1.0	.00	.04	.09	.14	.30
	Qwen-8B	.12	.63	.91	1.0	1.0	.00	.02	.24	.82	1.0	.00	.87	1.0	1.0	1.0	.00	.04	.49	.74	.96
	Qwen-14B	.21	.78	.98	1.0	1.0	.00	.05	.56	.81	.99	.19	.89	1.0	1.0	1.0	.02	.26	.64	.86	1.0
	Llama-8B	.13	.61	.86	1.0	1.0	.00	.20	.61	.83	1.0	.02	.88	1.0	1.0	1.0	.01	.30	.65	.85	1.0
Llama-70B	Qwen-4B	.05	.37	.80	.98	1.0	.00	.05	.18	.32	.48	.00	.00	.11	.35	.92	.00	.09	.20	.34	.55

Table 1: Coverage of the joint two-mode framework (two-threshold routing with retrieval) at

\alpha\in\{.05,.10,.15,.20,.25\}

(mean over 100 splits) across all datasets.

Figure 2: Routing distribution across configurations. Within each group of four bars, judges are ordered left to right: Qwen3-4B, Qwen3-8B, Qwen3-14B, Llama-3.1-8B.

		TriviaQA		NQ-Open		PopQA		HotpotQA
Cand.	Judge	M1 Only	Joint	M1 Only	Joint	M1 Only	Joint	M1 Only	Joint
Qwen-8B	Qwen-4B	.47	.91	.01	.14	.04	1.0	.00	.14
	Qwen-8B	.59	1.0	.07	.82	.03	1.0	.05	.74
	Qwen-14B	.87	1.0	.14	.81	.15	1.0	.35	.86
	Llama-8B	.77	1.0	.22	.83	.21	1.0	.40	.85
Llama-70B	Qwen-4B	.67	.98	.07	.32	.05	.35	.09	.34

Table 2: Coverage at

\alpha\!=\!0.20

: Mode 1 Only vs. Joint (two-threshold routing with retrieval).

	TriviaQA						HotpotQA
	FDR			Coverage			FDR			Coverage
$\alpha$	$\tau_{0}$	$\tau_{1}$	$\Delta$	$\tau_{0}$	$\tau_{1}$	$\Delta$	$\tau_{0}$	$\tau_{1}$	$\Delta$	$\tau_{0}$	$\tau_{1}$	$\Delta$
.05	.03	.04	.00	.12	.17	$+$ .05	.00	.00	.00	.00	.00	.00
.10	.08	.09	.00	.63	.68	$+$ .06	.05	.03	$-$ .01	.04	.02	$-$ .02
.15	.14	.13	.00	.91	.94	$+$ .04	.13	.13	.00	.49	.49	.00
.20	.16	.15	$-$ .01	1.0	1.0	.00	.18	.18	.00	.74	.72	$-$ .02

Table 3: FDR and coverage under retrieval drift from the original (

\tau_{0}

) to the re-queried snapshot (

\tau_{1}

\sim

3 months later).

\Delta=\tau_{1}-\tau_{0}

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Description
$\mathbf{x},\;\hat{y},\;y^{*}$	Question, candidate answer, ground truth
$F,\;\mathcal{J}$	Examinee LLM, judge LLM
$v\in\{0,1\}$	Evaluation verdict
$e^{*}\in\{0,1\}$	Ground-truth evaluation label
$\mathcal{A}(v,e^{*})$	Admission function: $\mathbf{1}[v\!=\!e^{*}]$
$\mathcal{A}_{\mathrm{ref}},\;\lambda_{A}$	Relevance function and threshold

Appendix

Table 4: Notation reference.

		TriviaQA		NQ-Open		PopQA		HotpotQA
Cand.	Judge	M1	M2	M1	M2	M1	M2	M1	M2
Qwen-8B	Qwen-4B	68	79	52	71	47	84	54	69
	Qwen-8B	71	84	54	79	44	88	58	76
	Qwen-14B	78	86	61	77	65	88	67	78
	Llama-8B	77	82	63	78	68	88	68	78
Llama-70B	Qwen-4B	75	81	62	69	60	75	63	68

Appendix

Table 5: Judge verdict accuracy (%) before calibration. Mode 1 uses parametric knowledge only whereas Mode 2 uses retrieval-augmented evaluation (

k\!=\!3

). LLM evaluator as admission function.

TriviaQA
		$\alpha$
Judge	Metric	.05	.10	.15	.20	.25	.30	.35	.40	.45	.50	.55	.60
Qwen-4B	FDR	.03	.08	.13	.18	.21	.21	.21	.21	.21	.21	.21	.21
Qwen-4B	Cov.	.03	.39	.66	.91	1.0	1.0	1.0	1.0	1.0	1.0	1.0	1.0
Qwen-8B	FDR	.03	.08	.14	.16	.16	.16	.16	.16	.16	.16	.16	.16
Qwen-8B	Cov.	.12	.63	.91	1.0	1.0	1.0	1.0	1.0	1.0	1.0	1.0	1.0

Appendix

Table 6: Full results for the joint two-mode framework at all

\alpha

levels across all four datasets. Candidate: Qwen3-8B, LLM evaluator, PE,

k\!=\!3

Figure 3: Coverage (mean

\pm

std over 100 splits) across all 32 configurations. Rows: candidates; columns: datasets.

Figure 4: FDR control on TriviaQA.

Figure 5: FDR control on NQ-Open.

Figure 6: FDR control on PopQA.

Figure 7: FDR control on HotpotQA.

	Exact Match						Token F1						LLM Evaluator
	FDR			Coverage			FDR			Coverage			FDR			Coverage
Dataset	.15	.20	.25	.15	.20	.25	.15	.20	.25	.15	.20	.25	.15	.20	.25	.15	.20	.25
TriviaQA	.13	.14	.14	.97	1.0	1.0	.13	.18	.18	.72	.98	1.0	.13	.14	.14	.98	1.0	1.0
NQ-Open	.00	.05	.22	.00	.11	.69	.00	.00	.18	.00	.00	.45	.13	.18	.22	.56	.81	.99
PopQA	.12	.12	.12	1.0	1.0	1.0	.13	.13	.13	.99	1.0	1.0	.12	.12	.12	1.0	1.0	1.0
HotpotQA	.04	.18	.23	.03	.57	.86	.04	.17	.23	.04	.47	.83	.13	.18	.22	.64	.86	1.0

Appendix

Table 7: Sensitivity to admission function. Candidate: Qwen3-8B, Judge: Qwen3-14B, UQ: PE (white-box). FDR and coverage at

\alpha\in\{.15,.20,.25\}

	$k=3$		$k=6$		$k=9$
Dataset	FDR	Cov.	FDR	Cov.	FDR	Cov.
TriviaQA	.18	.99	.17	1.0	.17	.99
NQ-Open	.18	.47	.18	.46	.18	.46
PopQA	.18	.48	.18	.41	.17	.36
HotpotQA	.18	.47	.18	.46	.18	.46

Appendix

Table 8: Retrieval depth ablation at

\alpha\!=\!0.20

. Candidate: Llama-70B, Judge: Qwen3-8B, LLM evaluator.

	TriviaQA					HotpotQA
$\delta~\backslash~\alpha$	.05	.10	.15	.20	.25	.05	.10	.15	.20	.25
0.05	.12	.63	.91	1.0	1.0	.00	.04	.49	.74	.96
0.01	.04	.53	.87	1.0	1.0	.00	.01	.37	.69	.93

Appendix

Table 9: Sensitivity to the confidence level

\delta

(coverage). Candidate: Qwen3-8B, Judge: Qwen3-8B, LLM evaluator,

k\!=\!3

, PE. FDR is controlled at the target

\alpha

under both settings.

	TriviaQA					HotpotQA
$n_{\mathrm{cal}}/n~\backslash~\alpha$	.05	.10	.15	.20	.25	.05	.10	.15	.20	.25
0.10	.02	.38	.81	.96	.99	.00	.03	.25	.55	.83
0.25	.08	.54	.89	.99	1.0	.00	.04	.41	.70	.92
0.50	.12	.63	.91	1.0	1.0	.00	.04	.49	.74	.96
0.75	.15	.66	.92	1.0	1.0	.00	.04	.53	.77	.98

Appendix

Table 10: Sensitivity to calibration-set size (coverage).

n_{\mathrm{cal}}/n

is the fraction of the 2,000 items used for calibration. Candidate: Qwen3-8B, Judge: Qwen3-8B, LLM evaluator,

k\!=\!3

, PE. FDR is controlled at the target

\alpha

for every size.

	TriviaQA			NQ-Open			PopQA			HotpotQA
$\alpha$	PE	Probe	LoRA	PE	Probe	LoRA	PE	Probe	LoRA	PE	Probe	LoRA
.05	.12	.00	.00	.00	.00	.02	.00	.31	.71	.00	.00	.02
.10	.63	.14	.70	.02	.07	.28	.87	.82	.94	.04	.02	.44
.15	.91	.90	.94	.24	.42	.64	1.0	1.0	1.0	.49	.47	.76
.20	1.0	1.0	1.0	.82	.79	.86	1.0	1.0	1.0	.74	.78	.95
.25	1.0	1.0	1.0	1.0	1.0	1.0	1.0	1.0	1.0	.96	.99	1.0

Appendix

Table 11: Coverage under three uncertainty scores: the white-box predictive entropy (PE) used in the main paper and two supervised scores, a Probe and a LoRA+Prompt adapter on the judge’s hidden states. Candidate: Qwen3-8B, Judge: Qwen3-8B, LLM evaluator,

k\!=\!3

. FDR is controlled at the target

\alpha

for all three.

	TriviaQA					HotpotQA
Variant $\backslash$ $\alpha$	.05	.10	.15	.20	.25	.05	.10	.15	.20	.25
Direct	.03	.28	.47	.59	.74	.00	.00	.01	.05	.17
Self-eval	.02	.32	.49	.60	.75	.00	.00	.02	.07	.19
Empty	.04	.38	.63	.83	.99	.00	.09	.32	.57	.84
Joint (ours)	.12	.63	.91	1.0	1.0	.00	.04	.49	.74	.96

Appendix

Table 12: Baseline comparison (coverage). Candidate: Qwen3-8B, Judge: Qwen3-8B, LLM evaluator,

k\!=\!3

, PE. FDR is controlled at the target

\alpha

for every variant.

	TriviaQA			NQ-Open			PopQA			HotpotQA
$\alpha$	Point.	Cons.	$\Delta$	Point.	Cons.	$\Delta$	Point.	Cons.	$\Delta$	Point.	Cons.	$\Delta$
.05	.12	.00	$-$ .12	.00	.00	.00	.00	.00	.00	.00	.00	.00
.10	.63	.01	$-$ .62	.02	.00	$-$ .02	.87	.00	$-$ .87	.04	.00	$-$ .04
.15	.91	.64	$-$ .27	.24	.00	$-$ .24	1.0	.93	$-$ .07	.49	.00	$-$ .49
.20	1.0	.93	$-$ .07	.82	.05	$-$ .76	1.0	1.0	.00	.74	.30	$-$ .45
.25	1.0	1.0	.00	1.0	.80	$-$ .20	1.0	1.0	.00	.96	.72	$-$ .24

Appendix

Table 13: Conservative joint calibration (coverage). Per dataset, Point. and Cons. denote the pointwise and conservative coverage, and

\Delta=\text{Cons.}-\text{Point.}

(computed on unrounded values). Candidate: Qwen3-8B, Judge: Qwen3-8B, LLM evaluator,

k\!=\!3

, PE. FDR is controlled at the target

\alpha

under both variants.

	TriviaQA	NQ-Open	PopQA	HotpotQA
% raised ( $U_{2}\!>\!U_{1}$ )	20.1	27.9	14.3	29.1
mean $\Delta U$	$+0.10$	$+0.11$	$+0.10$	$+0.12$
% lowered ( $U_{2}\!<\!U_{1}$ )	68.0	68.8	84.6	69.7
mean $\Delta U$	$-0.07$	$-0.09$	$-0.10$	$-0.12$

Appendix

Table 14: Effect of retrieval on judge uncertainty. Results are obtained with Qwen3-8B as both candidate and judge, the LLM evaluator, and

k\!=\!3

retrieval using predictive entropy (PE). The “raised” and “lowered” columns report the fraction of instances where retrieval increases or decreases uncertainty, respectively. Mean

\Delta U

is averaged within each group.

Question	Candidate	Gold	Mode 1 (Wrong)	Mode 2 (Corrected)
Who voices Jack Kahuna Laguna in SpongeBob SquarePants?	Steve Carell	Johnny Depp	“Steve Carell provided the voice…This is a well-documented fact.” Verdict: True	“The web search results indicate that Johnny Depp, not Steve Carell, provided the voice…” Verdict: False
Which North African country’s flag is just the colour green?	Chad	Libya	“The flag of Chad is indeed a single solid green rectangle…” Verdict: True	“The search results indicate that Libya’s flag was a single green field…” Verdict: False
‘The Great American Chocolate Bar’ is better known as?	Milky Way	Hershey	“The Great American Chocolate Bar is indeed known as the Milky Way bar…” Verdict: True	“The search results indicate that it is a nickname for the Hershey Milk Chocolate Bar…” Verdict: False

Appendix

Table 15: Examples where retrieval corrects the judge’s verdict. Candidate: Qwen3-8B, Judge: Qwen3-14B.

Question	#	Retrieved Snippet
Who voices Jack Kahuna Laguna in SpongeBob?	1	“…American actor and musician Johnny Depp guest starred in the episode as the voice of Jack Kahuna Laguna, a surf guru…” — Wikipedia
	2	“Jack Kahuna Laguna has a striking resemblance to his voice actor, Johnny Depp.” — Fandom
	3	“…it guest starred Johnny Depp as the legendary surfer, Jack Kahuna Laguna.” — IMDb
Which N. African country’s flag is just green?	1	“The flag of the Libyan Arab Jamahiriya…consisted of a simple green field. It was the only national flag…with only one colour.” — Wikipedia
	2	“Republic of Ireland and the Ivory Coast the flags are the same…” — Facebook
	3	“Green is an important colour in Islam…” — Reddit

Appendix

Table 16: Retrieved evidence (

k\!=\!3

) for the examples in Table 15 . The first result in each case directly contains the correct answer.

Explore similar work

Judge Reliability Harness: Stress Testing the Reliability of LLM Judges
Mar 5, 2026Sunishchal Dev, Andrew Sloan, Joshua Kavner +2LLM-as-a-Judge LLM Evaluation
RAND Corporation, Santa Monica, CA, USA
JEV-as-a-Judge: Accept When Confident, Escalate When Unsure
Sep 22, 2026Yubo Li, Yidi Miao, Ramayya Krishnan +1LLM-as-a-Judge Language Model Generation Evaluation
Carnegie Mellon University
Towards Provably Unbiased LLM Judges via Bias-Bounded Evaluation
Mar 5, 2026Benjamin Feuer, Lucas Rosenblatt, Oussama ElachqarBias in LLM-as-a-Judge LLM-as-a-Judge
1Stanford University · 3New York University

Mar 5, 2026J/K move · Enter open · S save

Judge Reliability Harness: Stress Testing the Reliability of LLM Judges

Sunishchal Dev, Andrew Sloan, Joshua Kavner, Nicholas Kong+1

RAND Corporation, Santa Monica, CA, USA

LLM-as-a-Judge LLM Evaluation Language Model Robustness

Open paper AskPDF arXiv

We present the Judge Reliability Harness, an open source library for constructing validation suites that test the reliability of LLM judges. As LLM based scoring is widely deployed in AI benchmarks, more tooling is needed to efficiently assess the reliability of these methods. Given a benchmark dataset and an LLM judge configuration, the harness generates reliability tests that evaluate both binary judgment accuracy and ordinal grading performance for free-response and agentic task formats. We evaluate four state-of-the-art judges across four benchmarks spanning safety, persuasion, misuse, and agentic behavior, and find meaningful variation in performance across models and perturbation types, highlighting opportunities to improve the robustness of LLM judges. No judge that we evaluated is uniformly reliable across benchmarks using our harness. For example, our preliminary experiments on judges revealed consistency issues as measured by accuracy in judging another LLM's ability to complete a task due to simple text formatting changes, paraphrasing, changes in verbosity, and flipping the ground truth label in LLM-produced responses. The code for this tool is available at: https://github.com/RANDCorporation/judge-reliability-harness

Abstract

Explore similar work

Judge Reliability Harness: Stress Testing the Reliability of LLM Judges

JEV-as-a-Judge: Accept When Confident, Escalate When Unsure

Towards Provably Unbiased LLM Judges via Bias-Bounded Evaluation