Self-Aware Knowledge Probing: Evaluating Language Models' Relational Knowledge through Confidence Calibration

Christopher Kissling; Elena Merdjanovska; Alan Akbik

cs.CLJan 26, 2026

Self-Aware Knowledge Probing: Evaluating Language Models' Relational Knowledge through Confidence Calibration

Authors: Christopher Kissling, Elena Merdjanovska, Alan Akbik

Organizations: Humboldt-Universität zu Berlin · Science of Intelligence

Abstract

Knowledge probing quantifies how much relational knowledge a language model (LM) has acquired during pre-training. Existing knowledge probes evaluate model capabilities through metrics like prediction accuracy and precision. Such evaluations fail to account for the model's reliability, reflected in the calibration of its confidence scores. In this paper, we establish an empirical framework to evaluate the calibration of relational knowledge in a closed answer-set setting. We cover three modalities of model confidence: (1) intrinsic confidence, (2) paraphrase consistency and (3) linguistic confidence. Our extensive analysis of 21 causal and 8 masked language models reveals that most models, but especially those pre-trained with the masking objective, are overconfident. The best-calibrated scores come from confidence estimates that account for inconsistencies due to statement rephrasing. Moreover, calibration improves when increasing the number of answer options.

Figures & tables

Figure 1: Using knowledge facts (like the place of death of a particular person), we measure an LM’s awareness of its relational knowledge in three modalities: (1) intrinsic confidence, (2) paraphrase consistency of predictions across multiple rephrasings of the same statement, and (3) linguistic confidence using epistemic markers.

Figure 2: Calibration curves of our continuous confidence estimates. ACE in parentheses.

Figure 3: Average

C_{\text{Base}}

confidence, accuracy and ACE as a function of the number of answer options (including the correct answer). Each point represents the mean over three repeated samplings.

Figure 4:

\emph{Accuracy}

against

\emph{ACE}

C_{\text{Base}}

for the 1:1 relations of BEAR and the Wiki-FACTOR dataset. For BEAR, we sampled the number of answer options to five per instance. On FACTOR, each question consists of four answer options.

Model	Acc.	$C_{\text{Margin}}$	$C_{\text{Average}}$
opt-125m (125M)	15.6	0.186	0.130
roberta-base (125M)	16.0	0.291	0.162
gpt2-medium (355M)	18.4	0.179	0.110
bert-base-cased (109M)	18.1	0.276	0.205
gpt2-large (812M)	21.5	0.163	0.116
roberta-large (355M)	21.1	0.278	0.158

Table 2: Brier Scores (

\downarrow

) for matched CLM/MLM pairs. Accuracy (in %) is the average over all templates.

Figure 5: Difference in ACE between the non-injected and injected templates. The x-axis shows the applied injection. Negative values (green) show an improvement.

Figure 6: Calibration curves of

C_{\text{Base}}

derived from the original and verbal injected templates.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Model	# Param.	Objective	Accuracy
gemma-2b	2.0b	CLM	50.0
gemma-7b	7.0b	CLM	62.4
gpt2	137M	CLM	12.8
gpt2-medium	355M	CLM	18.4
gpt2-large	812M	CLM	21.5
gpt2-xl	1.6b	CLM	25.3

Appendix

Table 3: Models considered in this work. (IT) marks an instruction-tuned variant.

Figure 7: Performance of different token log-likelihood reduction strategies, measured by Accuracy, ACE and harmonic mean. Higher harmonic mean is better. Scores are averaged across CLMs and MLMs.

Figure 8: Distributions of

C_{\text{Base}}

for opt-1.3b and the different reduction strategies.

Figure 9: ACE -accuracy scatter plot, demonstrating negative correlation between the two metrics. The first row shows the bert , gemma , gpt2 , opt , roberta and xlm-roberta model families. The second row shows the gemma-3 , llama3 , modern-bert and qwen3.5 model families.

Figure 10: Accuracy Rejection Curves Nadeem et al. (2008) (fraction of rejected answers versus accuracy among the non-rejected answers) for confidence thresholds of

0.1,\dots,0.9

. For a given threshold, curves closer to the upper left indicate better performance.

		ACE ( $\downarrow$ )		Brier Score ( $\downarrow$ )
Model	$Accuracy$	$C_{\text{Base}}$	$C_{\text{Margin}}$	$C_{\text{Base}}$	$C_{\text{Margin}}$
gemma-2b	40.21	0.433	0.387	0.457	0.432
gemma-7b	16.50	0.740	0.690	0.753	0.720
gpt2	26.95	0.528	0.482	0.545	0.513
gpt2-medium	31.93	0.481	0.438	0.504	0.477
gpt2-large	35.74	0.448	0.433	0.484	0.468

Appendix

Table 4: ACE and Brier Score for

C_{\text{Base}}

and

C_{\text{Margin}}

, as well as accuracy on the Wiki-FACTOR Muhlgay et al. (2024) dataset. Bold values indicate the best-performing estimate per model. ACE: • [0,0.05), • [0.05,0.10), • [0.10,0.20), •

\geq 0.20

; Brier Score: • [0–0.10), • [0.10–0.20), • [0.20–0.25), •

\geq 0.25

. All gemma-3 and Qwen3.5 models are the instruction-tuned variants.

Figure 11: Difference in Accuracy between the non-injected and injected templates. Positive values show an improvement over the non-injected version.

Language Model Probing

Language model probing measures what information and capabilities models encode and how those signals relate to their behavior.

See all 265 papers

Measurement Validity and Representation Dynamics of Confidence Signals in Large Language Models
Oct 19, 2026Debojeet Ghosh, Shreyansh Kumar, Ayushman GargConfidence Estimation in Language Models Language Model Calibration
CALIBER: Calibrating Confidence Before and After Reasoning in Language Models
Jun 23, 2026Conor Finlay, Joshua Kurien, Saurabh Dash +2Confidence Estimation in Language Models Language Model Calibration
1Cohere Labs
Identifying Influential N-grams in Confidence Calibration via Regression Analysis
Apr 7, 2026Shintaro Ozaki, Wataru Hashimoto, Hidetaka Kamigaito +2Language Model Calibration Confidence Estimation in Language Models
Nara Institute of Science and Technology (NAIST) · The University of Tokyo

Oct 19, 2026J/K move · Enter open · S save

Measurement Validity and Representation Dynamics of Confidence Signals in Large Language Models

Debojeet Ghosh, Shreyansh Kumar, Ayushman Garg

Confidence Estimation in Language Models Language Model Calibration Verbalized Confidence Estimation Language Model Probing Semantic Entropy

Open paper AskPDF arXiv

Whether a language model "knows what it knows" matters for trustworthy deployment and for deciding when it should abstain. Prior work usually measures this through one channel: verbalized confidence, a linear probe on hidden states, or agreement among sampled answers (semantic entropy), and often treats these as interchangeable. We compare all three in a pre-registered audit of five Qwen2.5 models (0.5B-7B, including a base/instruct pair at 7B) on factual retrieval (PopQA, SimpleQA) and multi-step reasoning (GSM8K, MATH). First, probes trained on prefill states to predict whether sampled answers will disagree become informative earlier for retrieval (about 7% of layer depth on average) than for reasoning (about 28%). The pre-registered five-layer test supported this only marginally; a finer 21-layer follow-up, designed after those results, supports it more clearly. In most retrieval cells, however, a prompt-only TF-IDF classifier matches the probe, so the early signal may reflect surface features of the question. Second, verbalized confidence is unstable: slightly different prompt templates yield scores that barely correlate. Third, Expected Calibration Error is misleading: verbal confidence looks well calibrated because it clusters near the base rate while discriminating poorly, whereas sampling agreement has worse ECE but about twice the resolution. Of five pre-registered hypotheses, two were falsified, one was partly supported, one was not supported, and one could not be tested because the base model's verbal confidence was degenerate.

Abstract

Language Model Probing

Explore similar work

Measurement Validity and Representation Dynamics of Confidence Signals in Large Language Models

CALIBER: Calibrating Confidence Before and After Reasoning in Language Models

Identifying Influential N-grams in Confidence Calibration via Regression Analysis