VERA: Verdict-Conditioned Reliability for Adaptive LLM Judges

Qiushui Xu; Syamil Mohd Razak; Tao Yuan; Piotr Habas

cs.LGOct 4, 2026

VERA: Verdict-Conditioned Reliability for Adaptive LLM Judges

Authors: Qiushui Xu, Syamil Mohd Razak, Tao Yuan, Piotr Habas

Organizations: Penn State University · Amazon

Abstract

Accurately estimating judgment reliability is a central challenge in adapting LLM judges to newly verified feedback while preserving previously learned behavior. However, existing approaches often rely on output-level confidence, which can be overconfident and poorly aligned with judgment correctness. We propose VERA, a VErdict-conditioned Reliability Axis that estimates reliability from hidden activations by distinguishing correct from incorrect judgments within each predicted-verdict group. Using VERA as a control signal, we develop a VERA-guided periodic adaptation framework that integrates reliability-ranked corrective updates, reliability-residual replay, and periodic refresh of the reliability directions. After VERA-guided adaptation on Chatbot Arena, 8B- and 14B-parameter judges outperform the strongest baseline on each of four held-out public benchmarks, with relative gains of up to 23.01%. The framework also improves focal-class recall by up to 16.1% relative to the strongest adaptive baselines on a separate proprietary temporal auditing task.

Figures & tables

Figure 1: Reliability signals at frozen checkpoints. AUROC for distinguishing correct from incorrect judgments using verbalized confidence, model probability, and VERA , evaluated on the proprietary auditing task and MT-Bench-HJ with Qwen3-8B and Qwen3-14B. Values and evaluation conventions follow Table 3 . The dashed line denotes chance-level discrimination.

Figure 2: Overview of VERA -guided periodic adaptation. Pre-update reliability guides corrective training; the updated judge and current reliability estimator guide replay selection for later rounds.

Model	Method	MT-Bench-HJ (%)	Auto-J(%)	JudgeBench(%)	LLMBar(%)
Qwen3-8B	IFT	$64.42\pm 0.13$	$58.93\pm 0.61$	$56.48\pm 0.59$	$56.06\pm 0.37$
	DPO	$64.57\pm 0.19$	$58.93\pm 0.49$	$57.06\pm 0.32$	$59.63\pm 0.12$
	VERA	66.69 $\pm$ 0.36	59.85 $\pm$ 1.01	57.22 $\pm$ 1.86	64.95 $\pm$ 2.40
Relative gain over best baseline		$+3.28$	$+1.56$	$+0.28$	$+8.92$
Qwen3-14B	IFT	$64.10\pm 0.21$	$60.69\pm 0.17$	$60.20\pm 1.17$	$54.95\pm 0.58$
	DPO	$64.50\pm 0.13$	$60.85\pm 0.25$	$60.46\pm 1.31$	$57.05\pm 1.25$

Table 1: Performance on four held-out public judging benchmarks (%). MT-Bench-HJ reports row accuracy; Auto-J reports two-order agreement; JudgeBench reports two-order accuracy on its

350

-item GPT-4o subset; LLMBar reports the macro-average over four adversarial subsets. Entries report means and sample standard deviations over three seeds. Gains are relative improvements (%) over the stronger of IFT and DPO within each model–benchmark comparison. Bold marks the best score within each model scale. Scores should be compared within columns rather than averaged across benchmarks with different evaluation metrics.

Model	Method	Earlier anchor		Forward set
		Recall (%)	F1 (%)	Recall (%)	F1 (%)
Qwen3-8B	Frozen initial judge	85.8	78.8	65.7	58.8
	Periodic RAG	82.0	79.0	61.6	61.6
	Periodic IFT	72.5	77.4	57.6	64.0
	Periodic DPO	84.1	78.7	62.6	57.9
	VERA	86.0	79.1	72.7	66.1

Table 2: Periodic adaptation on the proprietary auditing task (%). Recall and F1 refer to the Critical verdict at the final update. VERA uses replay fractions

f=0.010

and

f=0.015

at 8B and 14B, respectively. The frozen initial judge is task-trained but receives no subsequent feedback updates. Gains are relative improvements (%) over the strongest adaptive baseline among periodic IFT, DPO, and RAG, determined separately for each model scale and metric. Bold marks the best reported score within each model scale and column.

Setting	Model	Stage	Verbalized	Softmax	VERA
Proprietary	8B	Frozen	0.649	0.593	0.812
		IFT	0.497	0.857	0.867
		DPO	0.513	0.852	0.866
	14B	Frozen	0.618	0.628	0.831
		IFT	0.747	0.866	0.869
		DPO	0.779	0.870	0.872

Table 3: AUROC for distinguishing correct from incorrect judgments. Frozen denotes the pre-IFT checkpoint. MT-Bench-HJ verbalized confidence uses a separate generation pass, with correctness determined by that pass’s verdict. Bold marks the largest AUROC or mean in each row.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	IFT	DPO	VERA -guided periodic update
Optimizer	AdamW	AdamW	AdamW
Maximum sequence length (tokens)	$4{,}096$	$2{,}048$	$2{,}048$
Peak learning rate	$1\times 10^{-4}$	$5\times 10^{-6}$	$5\times 10^{-5}$
Training epochs	$2$	$1$	$1$ per feedback batch
Effective global batch size	$256$	$128$	$128$
Warmup ratio	$0.03$	$0.05$	$0.05$

Appendix

Table 4: Training hyperparameters shared across tasks and model scales. IFT initializes the judge; DPO and VERA -guided adaptation are subsequent training alternatives. Effective global batch sizes include gradient accumulation across eight data-parallel workers. For periodic adaptation, the epoch count applies to each feedback-batch update.

Hyperparameter	8B	14B	8B	14B
	Proprietary		Public
Feedback batches $T$	$4$	$4$	$5$	$5$
Selected layer $\ell^{\star}$	$20$	$38$	$33$	$32$
Initial selection fraction (%)	$1.00$	$1.50$	$0.75$	$1.50$
Feedback selection fraction (%)	$1.00$	$1.50$	$3.83$	$7.66$
Final replay examples	$747$	$1{,}120$	$3{,}880$	$7{,}760$

Appendix

Table 5: Task- and scale-specific adaptation settings. Layer indices are zero-indexed decoder-block indices. Proprietary selection uses fraction

f

for both the initial pool and feedback batches. Public fractions are rounded percentages of

98{,}400

unique source items and

6{,}268

unique items per feedback batch, respectively; the listed item counts specify the exact budgets. Each public item contributes two response orders. Final public replay counts therefore equal

2(740+5\times 240)=3{,}880

at 8B and

2(1{,}480+5\times 480)=7{,}760

at 14B.

Setting	Model	Stage	Global	Per-class ( VERA )	LR probe
Proprietary	Qwen3-8B	Frozen	0.671	0.812	0.761
Proprietary	Qwen3-8B	IFT	0.848	0.867	0.862
MT-Bench-HJ	Qwen3-8B	Frozen	$0.534$	$\mathbf{0.712}$	$0.669$
MT-Bench-HJ	Qwen3-8B	IFT	$\mathbf{0.730\pm 0.011}$	$0.719\pm 0.009$	$0.725\pm 0.006$

Appendix

Table 6: Correctness AUROC for three hidden-state estimators at Qwen3-8B. Frozen denotes the pre-IFT checkpoint. Each estimator uses its own validation-selected layer. Proprietary rows use the proprietary training, validation, and test splits; MT-Bench-HJ rows use JudgeLM for construction and layer selection and MT-Bench-HJ for transfer evaluation. MT-Bench-HJ ties are valid verdicts and are retained. Frozen proprietary abstentions are excluded consistently across estimators (

n=1{,}983

retained judgments); the proprietary IFT judge has no abstentions. MT-Bench-HJ IFT entries report means and sample standard deviations over three seeds; Frozen is a single evaluation. MT-Bench-HJ results use regenerated predictions, as in Table 7 ; Table 3 retains the original reference evaluations. Bold marks the highest AUROC or mean in each row.

Figure 3: Correctness-relevant geometry on the proprietary test set. Each panel contains the same $2{,}634$ judgments at the fixed layer used for 8B periodic adaptation. After task-specific training, the score varies systematically within each predicted verdict, and the examples that remain incorrect at all three checkpoints occur predominantly in lower-score regions. This figure visualizes the reliability signal available to VERA ; it is not an endpoint comparison of periodic-adaptation methods.

Panel of Figure 4 — Figure 4: Correctness-relevant geometry on the public source and target distributions. The top row shows $1{,}247$ JudgeLM validation judgments, the source distribution for IFT; the bottom row shows $6{,}710$ MT-Bench-HJ judgments, which are not used for judge training or layer selection. JudgeLM IFT leaves few source errors, so the top-row error geometry should be interpreted qualitatively. On MT-Bench-HJ, lower-score regions continue to contain many of the judgments that remain incorrect across checkpoints, showing that correctness-relevant structure remains present at the JudgeLM-selected layer under the target-distribution shift.

Setting	Model	Stage	Acc. (%)	Macro-F1 (%)	Softmax AUROC
Proprietary	8B	Frozen	49.5	56.5	0.543
		IFT	$\mathbf{86.3\pm 0.2}$	$\mathbf{84.3\pm 0.1}$	$\mathbf{0.861\pm 0.005}$
		DPO	$\mathbf{86.3\pm 0.2}$	$\mathbf{84.3\pm 0.2}$	$0.859\pm 0.007$
	14B	Frozen	30.4	24.4	0.630
		IFT	87.1	84.8	0.866
		DPO	86.9	84.6	0.870

Appendix

Table 7: Judge quality and correctness ranking across post-training stages. Accuracy and macro-F1 are reported as percentages; AUROC uses softmax confidence. Frozen denotes the pre-IFT checkpoint. For MT-Bench-HJ, IFT and DPO correspond to JudgeLM instruction fine-tuning and subsequent Arena preference optimization, respectively. Entries with

\pm

report means and sample standard deviations over three seeds. Entries without

\pm

report single evaluations. MT-Bench-HJ metrics here use the predictions regenerated for reliability evaluation; Table 1 uses the original benchmark predictions. Proprietary 8B Frozen metrics retain abstentions as incorrect, whereas Table 3 excludes them for that comparison. Bold marks the highest reported score or mean within each setting–model combination, including ties at the displayed precision.

Panel of Figure 5 — Figure 5: Validation raw-score AUROC by layer on the proprietary task. Directions are estimated on the training split and evaluated on the disjoint validation split. Open circles mark the IFT-selected layers used throughout periodic adaptation; Frozen curves are diagnostic. The dashed line denotes chance-level AUROC.

Panel of Figure 6 — Figure 6: Validation rank-normalized AUROC by layer on JudgeLM. Directions are estimated on training judgments; validation scores are ranked within each predicted-verdict group before computing AUROC. Open circles mark the IFT layers selected by the correct–incorrect mean-rank gap, which also maximize rank-normalized AUROC in these sweeps. Frozen curves are diagnostic, and the dashed line denotes chance-level AUROC.

Signal	MT-Bench-HJ	Auto-J	JudgeBench	LLMBar	RewardBench 2	Avg.
Source VERA	0.732	0.711	0.616	0.565	0.760	0.677
In-domain VERA (ours)	0.732	0.721	0.633	0.745	0.814	0.729
Softmax	0.732	0.719	0.623	0.576	0.796	0.689
Verbalized	0.535	0.568	0.556	0.466	0.545	0.534

Appendix

Table 8: Source and in-domain reliability directions at Qwen3-8B. All entries report correctness-ranking AUROC for the same JudgeLM-IFT checkpoint. Source VERA uses JudgeLM for direction construction and layer selection. In-domain VERA uses grouped five-fold cross-validation on each target: examples sharing a uid remain in the same fold, directions are estimated on four folds, and scores are computed on the remaining fold. The reported in-domain result is selected by the highest pooled cross-validation AUROC across layers. On RewardBench 2, Source VERA uses a global direction because JudgeLM lacks a fourth response option; in-domain VERA constructs directions for all four options. Avg. is the unweighted mean across the five benchmarks. Bold marks the highest value in each column, including ties at the displayed precision.

Model	Selection rule	Anchor Rec. (%)	Forward Rec. (%)	Forward F1 (%)
Qwen3-8B	No replay	78.7	61.6	63.9
	Random	85.0	69.7	63.0
	Softmax residual	85.5	69.7	63.0
	VERA residual	86.0	72.7	66.1

Appendix

Table 9: Replay-selection ablation on the proprietary task (%). Recall and F1 refer to the Critical verdict. All configurations use Qwen3-8B; replay selectors use matched budgets at

f=0.010

. Bold marks the best result in each column.

Model	Fraction $f$	Earlier anchor		Forward set
		Recall (%)	F1 (%)	Recall (%)	F1 (%)
Qwen3-8B	$0.010$	86.0	79.1	72.7	66.1
	$0.015$	85.3	79.5	69.7	65.1
	$0.020$	89.0	78.1	75.8	62.8
Qwen3-14B	$0.010$	84.2	80.0	72.7	68.6
	$0.015$	85.3	79.1	75.8	69.1

Appendix

Table 10: Sensitivity to the reliability-residual replay budget (%). Metrics refer to the Critical verdict after four updates. The anchor and forward sets contain

2{,}634

and

894

examples, respectively. Each configuration uses one training seed. Bold marks the best result within each model scale and column.

Configuration	Earlier anchor			Forward set
	Bal. Acc.	Recall	F1	Bal. Acc.	Recall	F1
Complete VERA	86.5	86.0	79.1	83.4	72.7	66.1
$-$ replay	85.2	78.7	79.2	78.8	61.6	63.9
$-$ reliability refresh	86.9	84.9	80.3	81.2	67.7	64.7
$-$ rank-UL	85.4	79.1	79.4	77.6	58.6	63.0

Appendix

Table 11: Component ablation on the proprietary task (%). All configurations use Qwen3-8B,

f=0.010

, and one training seed. Recall and F1 refer to the Critical verdict; balanced accuracy averages recall across both verdicts. Each ablation removes one component from the complete framework. Bold marks the highest score in each column.

Model	Selector	Target-failure recall (%)	Enrichment
Qwen3-8B	Random	28.1	$1.05\times$
Qwen3-8B	VERA residual	68.5	$\mathbf{2.56\times}$
Qwen3-14B	Random	26.3	$0.98\times$
Qwen3-14B	VERA residual	81.6	$\mathbf{3.04\times}$

Appendix

Table 12: Selection of RewardBench 2 Safety failures. Each selector retains

300

of the same

1{,}119

selection items. Recall is the fraction of target failures selected; enrichment is their prevalence among selected items divided by their prevalence in the full selection pool (

8.0\%

at 8B and

6.8\%

at 14B). These metrics are computed on the selection split before the repair update. Bold marks the larger value within each model scale and column.

Model	Intervention	Safety Acc. (%)	Safety gain (%)	MT-Bench-HJ Acc. (%)
Qwen3-8B	Before repair	49.6	–	66.4
	Random replay	80.6	$+31.0$	65.7
	VERA replay	86.9	$\mathbf{+37.3}$	65.2
Qwen3-14B	Before repair	54.0	–	68.0
	Random replay	72.1	$+18.1$	65.9
	VERA replay	78.6	$\mathbf{+24.6}$	66.0

Appendix

Table 13: Held-out effect of Safety repair. Both replay interventions use

900

pairwise examples and

24

optimizer steps. Safety gain is the absolute change (%) from the unrepaired checkpoint at the same model scale. Selection and evaluation items are disjoint, with all rotations grouped by item. Bold marks the best value within each model scale and column.

Model	Harness	Acc. (%)	Verbalized AUROC
Qwen3-8B	0-judge	63.7	0.665
	3-subjudge	65.3	0.634
	3-subjudge + RAG	66.0	0.658
Qwen3-14B	0-judge	65.6	0.636
	3-subjudge	68.9	0.628
	3-subjudge + RAG	70.4	0.603

Appendix

Table 14: Training-free harness comparison on the proprietary task. Model parameters are fixed within each scale, and all harnesses may output Unsure . Accuracy is computed over all

2{,}634

examples, counting Unsure as incorrect. Verbalized AUROC is computed only on non- Unsure judgments; its evaluation subset may therefore differ across harnesses. Bold marks the highest accuracy within each model scale.

Model	Configuration	Time (s/example)
Qwen3-8B	0-judge	14.04
	3-subjudge	36.21
	3-subjudge + RAG	37.18
	Post-trained judge †	0.24
Qwen3-14B	0-judge	12.18
	3-subjudge	43.64

Appendix

Table 15: Reported inference times on the proprietary task. Harness rows report mean per-example time under the sequential local implementation, excluding model loading, file I/O, retrieval-service time, and idle time after a shard finishes. † Post-trained-judge times are measured end to end over all

2{,}634

examples on one A100-40GB GPU with batch size four, including model loading. The timing conventions differ across the two groups.

Model	Harness	Acc. (%)	Verbalized AUROC
Qwen3-8B	0-judge	62.0	0.533
	3-subjudge †	60.6	0.551
	3-subjudge + RAG †	59.6	0.565
Qwen3-14B	0-judge	62.8	0.557
	3-subjudge †	64.2	0.566
	3-subjudge + RAG †	64.4	0.558

Appendix

Table 16: Training-free harness comparison on MT-Bench-HJ. All metrics use all

6{,}710

examples; ties are valid verdicts and are retained. Verbalized AUROC uses the explicit high, medium, low, or unsure confidence label. For otherwise valid truncated outputs, confidence is recovered only from its named field. The 0-judge rows use temperature zero; † stochastic panel rows are single-run results. Bold marks the highest accuracy or AUROC within each model scale.

Published Auto-J results
Evaluator	Agreement (%)	Consistency (%)
LLaMA-2-13B-Chat	29.81	48.56
WizardLM-13B-v1.2	36.35	57.69
Vicuna-13B-v1.5	39.22	62.07
PandaLM	39.44	66.88
Claude-2	42.60	63.43

Appendix

Table 17: Auto-J Eval-P results. Published references are taken from the Auto-J comparison. IFT, DPO, and VERA denote JudgeLM instruction fine-tuning, subsequent Arena DPO, and VERA -guided periodic adaptation, respectively. Entries report means and sample standard deviations over three seeds. Gains are relative improvements (%) computed from displayed means, comparing VERA with the stronger of IFT and DPO for each metric and model scale. Bold marks the best mean of these three methods.

Published RewardBench 2 results
Evaluator	Avg. †	Fact.	Prec. IF	Math	Safety	Focus	Ties †
LMUnit-Qwen2.5-72B	82.1	87.2	54.4	72.7	91.3	96.8	90.1
LMUnit-Llama3.1-70B	80.5	84.6	48.8	71.6	90.7	97.0	90.6
Gemini-2.5-Pro	79.5	75.5	61.9	89.8	88.1	80.5	81.1
Gemini-2.5-Flash	77.7	67.4	57.5	85.2	90.9	84.1	80.9
Claude Opus 4	76.5	82.7	41.9	74.9	89.5	86.2	83.7

Appendix

Table 18: RewardBench 2 results (%). Our judges use position-averaged discrete best-of-four accuracy over four cyclic rotations. Entries report means and sample standard deviations over three seeds. Bold marks the best mean of IFT, DPO, and VERA within each model scale. † The published Ties score incorporates score margins, whereas our Ties evaluation uses discrete selection. Consequently, neither the Ties column nor an average incorporating it is directly comparable between published-reference and Qwen3 blocks.

Published JudgeBench results
Evaluator	Accuracy (%)
o3-mini (high), Arena-Hard	80.86
Claude-3.5-Sonnet, Arena-Hard	64.29
GPT-4o, Arena-Hard	56.57
Skywork-LLaMA-3.1-8B judge	53.43
VertexAI Gemini-1.5-Pro	44.57

Appendix

Table 19: JudgeBench objective-correctness results. Scores use the

350

-item GPT-4o subset with two-order aggregation. Published references provide contextual comparisons with prompted and fine-tuned LLM judges. Entries report means and sample standard deviations over three seeds. Gains are relative improvements (%) over the stronger of IFT and DPO, calculated from the displayed two-decimal means. Bold marks the best mean of our three methods within each model scale.

Published LLMBar results
Evaluator	Adversarial Acc. (%)
ChatGPT, vanilla	32.0
ChatGPT, enhanced prompt	38.9
LLaMA-2-70B, vanilla	32.6
LLaMA-2-70B, enhanced prompt	43.4
PaLM2, vanilla	60.7

Appendix

Table 20: LLMBar adversarial instruction-following results. Accuracy is macro-averaged across four adversarial subsets using position-averaged evaluation over both response orders. Published references retain multiple prompting configurations because LLMBar explicitly compares these protocols. Entries report means and sample standard deviations over three seeds. Gains are relative improvements (%) over the stronger of IFT and DPO; bold marks the best mean of our three methods within each model scale.

Explore similar work

Feb 6, 2026cs.CL

FairJudge: An Adaptive, Debiased, and Consistent LLM-as-a-Judge

Existing LLM-as-a-Judge systems suffer from three fundamental limitations: limited adaptivity to task- and domain-specific evaluation criteria, systematic biases driven by non-semantic cues such as position, length, format, and model provenance, and evaluation inconsistency that leads to contradictory judgments across different evaluation modes (e.g., pointwise versus pairwise). To address these issues, we propose FairJudge, an adaptive, debiased, and consistent LLM-as-a-Judge. Unlike prior approaches that treat the judge as a static evaluator, FairJudge models judging behavior itself as a learnable and regularized policy. From a data-centric perspective, we construct a high-information-density judging dataset that explicitly injects supervision signals aligned with evaluation behavior. Building on this dataset, we adopt a curriculum-style SFT-DPO-GRPO training paradigm that progressively aligns rubric adherence, bias mitigation, and cross-mode consistency, while avoiding catastrophic forgetting. Experimental results on multiple internal and public benchmarks show that FairJudge consistently improves agreement and F1, reduces non-semantic biases, and outperforms substantially larger instruction-tuned LLMs. All resources will be publicly released after acceptance to facilitate future research.

Bo Yang, Lanfei Feng, Yunkui Chen +3

College of Computer Science, Zhejiang University, Hangzhou, China · School of Software Technology, Zhejiang University, Hangzhou, China

Llm-As-A-Judge Raw Judge Outputs Novel Modular Learning Framework

Ask PDF

Jun 21, 2026cs.CL

BabelJudge: Measuring LLM-as-a-Judge Reliability Across Languages and Agent Trajectories

LLM-as-a-judge has become the dominant approach to scalable evaluation in NLP pipelines, yet judges themselves carry systematic biases that raw accuracy hides: they favor responses placed in slot A (position bias), they prefer longer responses regardless of quality (verbosity bias), and their reliability degrades sharply in lower-resource languages. We introduce BabelJudge, an open-source benchmark and reliability audit framework that measures all four failure modes -- position bias, verbosity bias, order inconsistency, and cross-lingual degradation -- on any judge model, without requiring human preference labels. The key insight is gold-labelling by degradation: starting from a high-quality reference response and applying a controlled perturbation yields a pairwise item whose gold label is known by construction, eliminating annotation cost. We evaluate Qwen2.5-7B-Instruct-4bit across English, Hindi, Arabic, and Swahili and find that our composite bias-penalised reliability score drops from 0.714 in Hindi to 0.550 in Swahili, a gap that raw accuracy (0.835 vs. 0.660) understates. Swahili order consistency collapses to 0.480, meaning judge verdicts are near-random under slot-order swaps -- a failure mode invisible to accuracy alone. We further extend the framework to agentic evaluation via nine trajectory-level perturbations (argument corruption, tool swaps, hallucinated calls, missing steps) and three new metrics: tool accuracy, hallucination detection rate, and trajectory-length bias. BabelJudge is released as a Python package supporting 11 judge backends. Code: https://github.com/Shreyaskc/BabelJudge

Shreyas KC

Llm-As-A-Judge Raw Judge Outputs Natural Language Processing

Ask PDF

Jun 12, 2026cs.AI

Metric Match: A Subset Selection Approach to Evaluating LLM Judge Reliability

LLM judges are used to reduce the need for costly human labor in evaluating open-ended text generation. However, the reliability of these judges depends critically on their alignment with human raters -- a property that itself depends on costly human annotations. In this work, we develop a method (Metric Match) for estimating correlation-based reliability metrics of LLM judges from limited annotations. Metric Match selects a subset of samples for human annotation such that the subset matches the population reliability metric with respect to acquired synthetic labels. We empirically show that Metric Match achieves a win-rate of 0.838 against random subset selection across four different correlation metrics and 15 datasets, with an 18.7% decrease in average estimation error and reduces annotation needs by 32.5%. We provide a cost model and highlight a medical case study where our method saves $1,041.67 compared to random selection for expert annotation. Further, we shift our task from reliability estimation to reliability classification of whether a given judge is above a deployment threshold, outperforming random selection with Metric Match. All project code is publicly available, and we additionally provide an installable package for ease of use.

Alyssa Unell, Natalie Dullerud, Naomi Boneh +4

Department of Computer Science Stanford University · Department of Medicine Stanford University

Large Language Model Judges Judges Human Annotations

Ask PDF