When Can We Trust LLM Graders? Calibrating Confidence for Automated Assessment

Robinson Ferrer; Damla Turgut; Zhongzhou Chen; Shashank Sonkar

cs.CLMar 31, 2026

When Can We Trust LLM Graders? Calibrating Confidence for Automated Assessment

Authors: Robinson Ferrer, Damla Turgut, Zhongzhou Chen, Shashank Sonkar

Organizations: University of Central Florida

Abstract

Large Language Models (LLMs) show promise for automated grading, but their outputs can be unreliable. Rather than improving grading accuracy directly, we address a complementary problem: predicting when an LLM grader is likely to be correct. This enables selective automation where high-confidence predictions are processed automatically while uncertain cases are flagged for human review. We compare three confidence estimation methods (self-reported confidence, self-consistency voting, and token probability) across seven LLMs of varying scale (4B to 120B parameters) on three educational datasets: RiceChem (long-answer chemistry), SciEntsBank, and Beetle (short-answer science). Our experiments reveal that self-reported confidence consistently achieves the best calibration across all conditions (avg ECE 0.166 vs 0.229 for self-consistency). Surprisingly, self-consistency remains 38% worse despite requiring 5 $\times$ the inference cost. Larger models exhibit substantially better calibration though gains vary by dataset and method (e.g., a 28% ECE reduction for self-reported), with GPT-OSS-120B achieving the best calibration (avg ECE 0.100) and strong discrimination (avg AUC 0.668). We also observe that confidence is strongly top-skewed across methods, creating a ``confidence floor'' that practitioners must account for when setting thresholds. These findings suggest that simply asking LLMs to report their confidence provides a practical approach for identifying reliable grading predictions. Code is available here.

Figures & tables

Dataset	Method	Acc	ECE	Brier	AUC	Cov@10%
RiceChem	self-reported	72.4	0.162	0.229	0.593	0.005
RiceChem	self-consistency	71.9	0.229	0.251	0.575	0.000
RiceChem	token-probability	72.4	0.212	0.243	0.565	0.006
SciEntsBank	self-reported	74.9	0.208	0.255	0.588	0.094
SciEntsBank	self-consistency	74.4	0.227	0.238	0.561	0.003
SciEntsBank	token-probability	75.4	0.215	0.227	0.577	0.087

Table 1: Calibration metrics averaged across small models (Llama-8B, Qwen-4B, GPT-OSS-20B). Self-reported confidence achieves the lowest ECE across all datasets. Lower ECE and Brier scores indicate better calibration; higher AUC and Coverage indicate better discrimination.

Dataset	Method	Acc	ECE	Brier	AUC	Cov@10%
RiceChem	self-reported	74.2	0.150	0.209	0.612	0.038
RiceChem	self-consistency	75.4	0.218	0.228	0.563	0.000
RiceChem	token-probability	74.4	0.250	0.252	0.644	0.000
SciEntsBank	self-reported	78.3	0.121	0.183	0.639	0.310
SciEntsBank	self-consistency	78.3	0.198	0.204	0.550	0.004
SciEntsBank	token-probability	78.1	0.215	0.216	0.657	0.198

Table 2: Calibration metrics averaged across large models (GPT-OSS-120B, Llama-70B, Qwen-80B variants). Self-reported confidence consistently achieves the lowest ECE, with substantial improvement over small models.

Dataset	Metric	Comparator	$\Delta$	CI	B/W/T; Sig
RiceChem	ECE	self-consistency	$-0.0860$	$[-0.0886,-0.0703]$	7/0/0 7/7
SciEntsBank	ECE	self-consistency	$-0.0753$	$[-0.0820,-0.0684]$	6/1/0 7/7
Beetle	ECE	token-probability	$-0.1000$	$[-0.1078,-0.0910]$	6/1/0 7/7
RiceChem	Brier	token-probability	$-0.0380$	$[-0.0399,-0.0357]$	6/1/0 7/7
SciEntsBank	Brier	self-consistency	$-0.0213$	$[-0.0258,-0.0190]$	6/1/0 7/7
Beetle	Brier	token-probability	$-0.0459$	$[-0.0503,-0.0401]$	6/1/0 7/7

Table 3: Paired dominance summaries from Step 6 (model-level paired bootstrap on identical responses; within-dataset FDR).

\Delta

=

self-reported

-

comparator (negative favors self-reported). CI is the median model-level paired bootstrap interval across models. B/W/T

=

number of models where self-reported is better/worse/tied; Sig

=

number significant.

Dataset	Method	med $\|\beta\|$	IQR [Q1–Q3]	% sig
RiceChem	self-reported	0.246	0.134–0.396	43.9
	self-consistency	0.047	0.036–0.068	27.1
	token-probability	0.075	0.056–0.092	12.4
SciEntsBank	self-reported	0.162	0.056–0.320	35.3
	self-consistency	0.075	0.044–0.108	35.3
	token-probability	0.059	0.043–0.109	14.9

Table 4: Calibration-regression interactions (confidence

\times

feature). Rates are significant interactions (FDR within dataset). Median

|\beta|

indicates effect size; larger values indicate stronger shifts in the confidence–accuracy slope.

Panel of Figure 1 — Figure 1: Calibration curves and confidence distributions for GPT-OSS-120B with self-reported confidence. Left column: calibration curves where the diagonal indicates perfect calibration and points below the diagonal indicate overconfidence. Right column: histograms showing that confidence scores cluster in the 0.7–0.9 range, with fewer than 1% of predictions below 0.6.

Model	RiceChem				SciEntsBank				Beetle
	Acc	ECE	Brier	AUC	Acc	ECE	Brier	AUC	Acc	ECE	Brier	AUC
GPT-OSS-120B	75.1	.093	.182	.690	79.1	.081	.162	.656	74.2	.125	.196	.660
Llama-3.3-70B	72.4	.166	.226	.527	79.0	.117	.173	.602	73.3	.169	.218	.620
Qwen3-80B-Think	77.5	.142	.187	.636	76.4	.161	.198	.645	72.1	.198	.233	.618
Qwen3-80B-Inst	71.7	.197	.240	.594	78.7	.126	.198	.655	74.5	.132	.205	.641

Table 5: Individual large model results with self-reported confidence. ECE = Expected Calibration Error, Brier = Brier Score, AUC = AUC-ROC for correctness prediction. Best values per dataset shown in bold.

Reasoning	RiceChem				SciEntsBank				Beetle
	Acc	ECE	Brier	AUC	Acc	ECE	Brier	AUC	Acc	ECE	Brier	AUC
High	74.5	.099	.186	.690	79.1	.085	.162	.663	75.4	.116	.186	.674
Medium	75.1	.093	.182	.690	79.1	.081	.162	.656	74.2	.125	.196	.660
Low	74.3	.101	.186	.698	79.1	.085	.162	.660	75.0	.119	.189	.671

Table 6: GPT-OSS-120B performance across reasoning levels with self-reported confidence. Medium is the default setting used in main experiments. Best values per dataset shown in bold.

Confidence Estimation in Language Models

Confidence estimation in language models measures whether a model’s uncertainty signals predict answer correctness and support reliable decisions.

See all 229 papers

Mar 12, 2026

CHiL(L)Grader: Calibrated Human-in-the-Loop Short-Answer Grading

Scaling educational assessment with large language models requires not just accuracy, but the ability to recognize when predictions are trustworthy. Instruction-tuned models tend to be overconfident, and their reliability deteriorates as curricula evolve, making fully autonomous deployment unsafe in high-stakes settings. We introduce CHiL(L)Grader, the first automated grading framework that incorporates calibrated confidence estimation into a human-in-the-loop workflow. Using post-hoc temperature scaling, confidence-based selective prediction, and continual learning, CHiL(L)Grader automates only high-confidence predictions while routing uncertain cases to human graders, and adapts to evolving rubrics and unseen questions. Across three short-answer grading datasets, CHiL(L)Grader automatically scores 35-65% of responses at expert-level quality (QWK >= 0.80). A QWK gap of 0.347 between accepted and rejected predictions confirms the effectiveness of the confidence-based routing. Each correction cycle strengthens the model's grading capability as it learns from teacher feedback. These results show that uncertainty quantification is key for reliable AI-assisted grading.

Pranav Raikote, Korbinian Randl, Ioanna Miliou +2

Department of Computer and Systems Sciences, Stockholm University, 164 25 Kista, Sweden

Automated Grading AI in Education Human-in-the-Loop Evaluation Language Model Calibration

Ask PDF

Apr 30, 2026

Confidence Estimation in Automatic Short Answer Grading with LLMs

Automatic Short Answer Grading (ASAG) with generative large language models (LLMs) has recently demonstrated strong performance without task-specific fine-tuning, while also enabling the generation of synthetic feedback for educational assessment. Despite these advances, LLM-based grading remains imperfect, making reliable confidence estimates essential for safe and effective human-AI collaboration in educational decision-making. In this work, we investigate confidence estimation for ASAG with LLMs by jointly considering model-based confidence signals and dataset-derived uncertainty. We systematically compare three model-based confidence estimation strategies, namely verbalizing, latent, and consistency-based confidence estimation, and show that model-based confidence alone is insufficient to reliably capture uncertainty in ASAG. To address this limitation, we propose a hybrid confidence framework that integrates model-based confidence signals with an explicit estimate of dataset-derived aleatoric uncertainty. Aleatoric uncertainty is operationalized by clustering semantically embedded student responses and quantifying within-cluster heterogeneity. Our results demonstrate that the proposed hybrid confidence measure yields more reliable confidence estimates and improves selective grading performance compared to single-source approaches. Overall, this work advances confidence-aware LLM-based grading for human-in-the-loop assessment, supporting more trustworthy AI-assisted educational assessment systems.

Longwei Cong, Sonja Hahn, Sebastian Gombert +3

DIPF | Leibniz Institute for Research and Information in Education, 60323 Frankfurt am Main, Germany · Faculty of Computer Science, Goethe University Frankfurt, 60629 Frankfurt am Main, Germany · Chemnitz University of Technology, 09111 Chemnitz, Germany

Uncertainty Quantification Selective Prediction Generative AI in Education LLM Uncertainty Estimation Aleatoric Uncertainty Automated Grading

Ask PDF

Jun 20, 2026

Latent Confidence Alignment for LLM Self-Assessment

Confidence calibration in large language models (LLMs) is commonly evaluated by comparing predicted confidence with observed accuracy. However, such approaches do not model item difficulty, making it difficult to interpret discrepancies and to determine whether model confidence reflects genuine self-assessment or is merely a byproduct of the response generation process. To address this, we adopt a Rasch model-based latent ability framework and a metacognitive perspective, and propose Latent Confidence Alignment Error (LCAE) to measure the consistency between model self-assessment and the latent error probability implied by model ability and item difficulty. We further incorporate item difficulty as an external signal with a reasoning mechanism. Experiments on a medical-domain dataset with 20 models show that the proposed approach improves self-assessment quality without affecting model ability, and reveals an association between reliability and inference cost.

Ting-Yu Chen, Tingting Yu, Pei-Cing Huang +3

Department of Information Management National Sun Yat-Sen University Kaohsiung, Taiwan · Kaohsiung Medical University Hospital Kaohsiung Medical University Kaohsiung, Taiwan

LLM Evaluation Confidence Estimation in Language Models Language Model Self-Assessment Language Model Calibration Metacognition in Language Models

Ask PDF

Abstract

Confidence Estimation in Language Models

Explore similar work

CHiL(L)Grader: Calibrated Human-in-the-Loop Short-Answer Grading

Confidence Estimation in Automatic Short Answer Grading with LLMs

Latent Confidence Alignment for LLM Self-Assessment