Self-Anchoring Calibration Drift in Large Language Models: How Multi-Turn Conversations Reshape Model Confidence

Harshavardhan

cs.CLMar 1, 2026

Self-Anchoring Calibration Drift in Large Language Models: How Multi-Turn Conversations Reshape Model Confidence

Authors: Harshavardhan

Organizations: Independent Researcher

Abstract

We introduce Self-Anchoring Calibration Drift (SACD), a hypothesized tendency for large language models (LLMs) to show systematic changes in expressed confidence when building iteratively on their own prior outputs across multi-turn conversations. We report an empirical study comparing three frontier models -- Claude Sonnet 4.6, Gemini 3.1 Pro, and GPT-5.2 -- across 150 questions spanning factual, technical, and open-ended domains, using three conditions: single-turn baseline (A), multi-turn self-anchoring (B), and independent repetition control (C). Results reveal a complex, model-heterogeneous pattern that partially diverges from pre-registered hypotheses. Claude Sonnet 4.6 exhibited significant decreasing confidence under self-anchoring (mean CDS = -0.032, t(14) = -2.43, p = .029, d = -0.627), while also showing significant calibration error drift (F(4,56) = 22.77, p < .001, eta^2 = .791). GPT-5.2 showed the opposite pattern in open-ended domains (mean CDS = +0.026) with significant ECE escalation by Turn 5. Gemini 3.1 Pro showed no significant CDS (t(14) = 0.38, p = .710), but its Condition C data reveals a striking ECE pattern: without self-anchoring, Gemini's calibration error drops from .327 to near zero across repetitions, whereas self-anchoring holds ECE flat at approximately .333 -- indicating that SACD can manifest as suppression of natural calibration improvement rather than ac

Figures & tables

Model	Cond.	n	Mean CDS	SD	ECE T1	ECE T5	$\Delta$ ECE	CDR/turn
Claude 4.6	A	15	—	—	.352	—	—	—
Claude 4.6	B	15	$-$ .032 ∗	.051	.352	.320	$-$ .032	$-$ .010
Claude 4.6	C	15	$-$ .004	.015	.292	.112	$-$ .180	$-$ .032
Gemini 3.1	A	15	—	—	.331	—	—	—
Gemini 3.1	B	15	$+$ .001	.004	.331	.333	$+$ .001	$-$ .013
Gemini 3.1	C	15	$+$ .002	.020	.327	.005	$-$ .322	$+$ .001

Table 1: Main results by model and condition. CDS = Confidence Drift Score (T5

-

T1).

{}^{*}p<.05

Figure 1: Mean expressed confidence across five turns under Condition B (Self-Anchoring) for each model, by domain. Dashed horizontal lines indicate single-turn baseline means from Condition A. Shaded bands show 95% CIs.

Figure 2: Expected Calibration Error (ECE) across turns by model and domain. Solid lines = Condition B (Self-Anchoring); dashed lines = Condition C (Independent Repetition). Note the Gemini 3.1 Pro panel: the dashed orange line (C) drops from .327 at Turn 1 to near zero by Turn 2, showing rapid natural calibration improvement — an improvement entirely absent in Condition B (solid), where ECE stays flat at $\approx.333$ .

Figure 3: Reliability diagrams for Claude Sonnet 4.6 (Condition B), comparing Turn 1 (blue) and Turn 5 (pink) across domains. The dashed diagonal represents perfect calibration. Points above the diagonal indicate overconfidence.

Figure 4: Mean Confidence Drift Score (CDS $=T5-T1$ ) for Conditions B and C by model. Error bars show 95% CIs. Positive values indicate confidence escalation; negative values indicate suppression.

Model	Factual	Technical	Open-ended
Claude Sonnet 4.6	.000	$-$ .002	$-$ .094
Gemini 3.1 Pro	.000	.000	$+$ .004
GPT-5.2	.000	$-$ .004	$+$ .026

Table 2: Mean Confidence Drift Score (CDS) by model and domain in Condition B.

Figure 5: Calibration Drift Score (CDS $=T5-T1$ ) by model and domain (Condition B). Blue indicates confidence suppression; red indicates escalation. All models show CDS $\approx 0$ for factual questions.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Hypothesis	Model	Test	Statistic	$p$	Effect Size	Result
H1 (CDS)	Claude	One-sample $t$	$t(14)=-2.43$	.029 ∗	$d=-0.627$	Partial †
H1 (CDS)	Gemini	One-sample $t$	$t(14)=1.47$	.164	$d=0.379$	No
H1 (CDS)	GPT-5.2	One-sample $t$	$t(14)=1.41$	.181	$d=0.364$	No
H2 (B vs C)	Claude	Mann-Whitney	$U=74.5$	.036 ∗	$r=.338$	Yes
H2 (B vs C)	Gemini	Mann-Whitney	$U=135.0$	.239	$r=-.200$	No ††
H2 (B vs C)	GPT-5.2	Mann-Whitney	$U=140.0$	.230	$r=-.244$	No

Appendix

Table 3: Complete statistical test results for pre-registered hypotheses.

{}^{*}p<.05

;

{}^{***}p<.001

. BH correction applied per model. † H1 supported for Claude but direction opposite to prediction. †† Gemini H2: CDS non-significant but ECE B vs. C contrast is large (Cond. C ECE: .327

\to

.005; Cond. B:

\approx

.333 flat).

Model	Condition	Turn 1	Turn 2	Turn 3	Turn 4	Turn 5
Claude 4.6	A (baseline)	.352	—	—	—	—
Claude 4.6	B (self-anchor)	.352	.115	.344	.076	.320
Claude 4.6	C (ind. repeat)	.292	.055	.059	.095	.112
Gemini 3.1	A (baseline)	.331	—	—	—	—
Gemini 3.1	B (self-anchor)	.331	.199	.332	.066	.333
Gemini 3.1	C (ind. repeat)	.327	.005	.006	.006	.005

Appendix

Table 4: Expected Calibration Error (ECE) by model, condition, and turn. Lower values indicate better calibration.

Confidence Estimation in Language Models

Confidence estimation in language models measures whether a model’s uncertainty signals predict answer correctness and support reliable decisions.

See all 229 papers

Closing the Confidence-Faithfulness Gap in Large Language Models
Mar 26, 2026Miranda Muqing Miao, Lyle UngarConfidence Estimation in Language Models Verbalized Confidence Estimation
University of Pennsylvania · Department of Computer and Information Science
Large Language Models Are Overconfident in Their Own Responses
Jun 2, 2026Mario Sanz-Guerrero, Manuel Mager, Katharina von der WenseConfidence Estimation in Language Models LLM Prompting
∇Johannes Gutenberg University Mainz, Germany · ♠University of Colorado Boulder, USA
Confidence Should Be Calibrated More Than One Turn Deep
Apr 7, 2026Zhaohan Zhang, Chengzhengxu Li, Xiaoming Liu +3Confidence Estimation in Language Models Language Model Calibration
Queen Mary University of London · Xi’an Jiaotong University

Mar 26, 2026J/K move · Enter open · S save

Closing the Confidence-Faithfulness Gap in Large Language Models

Miranda Muqing Miao, Lyle Ungar

University of Pennsylvania · Department of Computer and Information Science

Confidence Estimation in Language Models Verbalized Confidence Estimation Language Model Calibration LLM Interpretability Activation Steering

Open paper AskPDF arXiv

Large language models (LLMs) tend to verbalize confidence scores that are largely detached from their actual accuracy, yet the geometric relationship governing this behavior remain poorly understood. In this work, we present a mechanistic interpretability analysis of verbalized confidence, using linear probes and contrastive activation addition (CAA) steering to show that calibration and verbalized confidence signals are encoded linearly but are orthogonal to one another -- a finding consistent across three open-weight models and four datasets. Interestingly, when models are prompted to simultaneously reason through a problem and verbalize a confidence score, the reasoning process disrupts the verbalized confidence direction, exacerbating miscalibration. We term this the "Reasoning Contamination Effect." Leveraging this insight, we introduce a two-stage adaptive steering pipeline that reads the model's internal accuracy estimate and steers verbalized output to match it, substantially improving calibration alignment across all evaluated models.

Abstract

Confidence Estimation in Language Models

Explore similar work

Closing the Confidence-Faithfulness Gap in Large Language Models

Large Language Models Are Overconfident in Their Own Responses

Confidence Should Be Calibrated More Than One Turn Deep