cs.LGMar 23, 2026

Causal Evidence that Language Models use Confidence to Drive Behavior

Authors: Dharshan Kumaran, Nathaniel Daw, Simon Osindero, Petar Veličković, Viorica Patraucean

Organizations: Google DeepMind · Princeton University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Metacognition -- assessing the quality of one's own cognitive performance -- guides adaptive behavior across species. Substantial research demonstrates that confidence signals can be extracted from language model outputs, yet a fundamental question remains: do models actually use these signals to control behavior, such as deciding whether to answer or abstain? To investigate, we developed a four-phase paradigm. Phase1 elicited baseline confidence estimates without an abstention option. Phase2 revealed that LLMs apply an implicit threshold to internal confidence when deciding to abstain, with confidence effect sizes approximately an order of magnitude larger than alternative mechanisms. Phase3 provided direct causal evidence through activation steering: boosting or suppressing confidence signals correspondingly decreased or increased abstention rates. Phase4 extended this by systematically varying instructed thresholds, demonstrating that LLMs actively deploy confidence signals to implement abstention policies. Critically, beyond calibrated log-probability based confidence derived from the output distribution, verbal confidence independently predicted abstention across all models, despite being objectively less discriminatory of answer correctness. Activation decoding at the last pre-answer token further showed that both observable measures are lossy readouts of a richer internal representation. Together, these results suggest that abstention is not fully captured by the strength of evidence in the output distribution alone, but is better explained by the joint operation of a multidimensional internal confidence representation and threshold-based policies -- consistent with structured metacognitive control in LLMs, a capacity of growing importance as models transition to autonomous agents that must recognize their own uncertainty.

Figures & tables

Figure 1: Overview of the 4 Phases of the Abstention Paradigm, and Confidence-Decision Pathway. A) In Phase 1, the model had to choose between 4 real options, with no abstention option. The inclusion of Phase 1 allowed us to obtain log-probability based and verbal confidence ratings from the model uncontaminated by the presence of an abstention option/decision. Phase 1 Chosen Confidence was used to model behavior in Phases 2 and 4. In Phases 2 and 3, the same questions included an abstention option, with models instructed to abstain when no clearly correct answer was apparent—with activation steering applied in the latter. In Phase 4 the model was given specific instructions to first choose an answer, then rate its confidence in its answer – and finally output its answer if its confidence was greater than a threshold, T. If its confidence was lower than T, it was instructed to abstain. Note these are illustrative prompts, the full prompts used are in the Methods. B) In stage 1, the model generates an answer and associated confidence signals ( CC ). In our abstention paradigm, confidence signals drive behavior by being compared to a threshold ( TT ) with the decision process further parametrized by a policy temperature (( τ\tau ))(Stage 2). Although we separate answer generation from confidence signals in our schematic, confidence signals may be generated as a direct consequence of answer generation – as is the case in first-order accounts of confidence [ 7 , 42 ] .
Figure 2: GPT-4o calibrated confidence predicts error rate (Phase 1) and abstention behavior (Phase 2). (A) Calibrated confidence distributions for correct (green) and incorrect (red) Phase 1 responses. Dashed vertical lines denote the mean confidence for each group. (B) Relationship between binned confidence and error rate in Phase 1. Each point represents a confidence bin (width = 0.05); color encodes the number of trials. A strong negative correlation was observed between confidence and error rate ( r=−0.97r=-0.97 , p<0.001p<0.001 ). Model was calibrated on a separate set of 1000 trials (see Methods). (C) Model comparison showing calibrated confidence as the dominant predictor of abstention. Left panel: AIC values for single-predictor models, where lower values indicate better fit. Right panel: Standardized effect sizes from the full model including all predictors. Confidence exhibits an effect size ( ∣βstd∣=0.99|\beta_{\text{std}}|=0.99 ) approximately 10 times larger than alternative predictors, demonstrating that abstention is primarily predicted by confidence rather than question difficulty, knowledge accessibility, or surface-level semantic features. (D) Logistic regression model of natural abstention probability as a function of calibrated confidence for n=1000n=1000 trials. The grey curve indicates the model prediction from the confidence + difficulty model, holding difficulty at its mean value. The red dashed line marks the implicit decision threshold at 77%, where P(abstain)=0.5P(\mathrm{abstain})=0.5 . The red shaded region denotes the confidence interval spanning approximately ±10 percentage points around the threshold, illustrating the gradual transition of the decision boundary. Black circles represent observed abstention rates (mean ±\pm 95% CI) across confidence quintiles. Together, these panels reveal GPT-4o’s intrinsic abstention policy: higher calibrated confidence corresponds to a lower likelihood of abstaining, even without explicit threshold instructions.
Figure 3: Effects of Activation Steering on Abstention Behavior and Calibrated Confidence in Gemma 3 27B. (A) Change from baseline rate of abstention as a result of high and low confidence activation steering. Errors bars reflect SEM. Baseline level of abstention in no steering condition is 28.2%. (B) Response distribution averaged across layers 30-40 as a function of steering strength (from low confidence (negative), baseline (0), high confidence (positive)(C) Change from baseline calibrated confidence – max of the real options (regardless of whether a real option or abstention was the outcome), and (D) Change from baseline calibrated confidence in the abstention option (regardless of outcome), as a result of high and low confidence steering. Baseline max real confidence was 0.37, and abstention confidence was 0.25.
Figure 4: Results of Gemma 3 27B parallel mediation analysis: confidence redistribution is the dominant mechanism underlying steering effects on abstention. Path diagram showing how activation steering influences abstention rates through two parallel indirect pathways (confidence redistribution and policy changes) and a direct pathway. The confidence shift mediator ( M1M_{1} ) represents the change in maximum real calibrated confidence minus the change in calibrated abstention confidence induced by steering. The policy shift mediator ( M2M_{2} ) captures changes in the calibration curve mapping confidence to abstention probability. There was only a weak correlation between these mediators (r = -0.24, p<0.001p<0.001 ). Activation steering significantly increased net confidence shift toward real options ( a1=0.11a_{1}=0.11 , p<0.001p<0.001 ), which in turn strongly reduced abstention odds ( b1=−5.15b_{1}=-5.15 , p<0.001p<0.001 ), yielding an indirect effect of a1×b1=−0.55a_{1}\times b_{1}=-0.55 (95% CI [-0.65, -0.47]). Steering had a small but significant effect on the decision policy ( a2=−0.0038a_{2}=-0.0038 , p<0.001p<0.001 ), and policy shifts exerted a strong effect on abstention when they occurred ( b2=56.6b_{2}=56.6 , p<0.001p<0.001 ), yielding an indirect effect of a2×b2=−0.22a_{2}\times b_{2}=-0.22 (95% CI [-0.24, -0.19]). A direct effect persisted after controlling for both mediators ( c′=−0.32c^{\prime}=-0.32 , p<0.001p<0.001 ), relative to a total effect of c=−0.82c=-0.82 ( p<0.001p<0.001 ). Due to the nonlinear logit link function, the sum of the direct and indirect effects do not sum to the total effect, though the relative contributions remain interpretable. Confidence redistribution accounted for 67.1% of the total effect, while policy shifts accounted for 26.2%. All paths significant at p<0.001p<0.001 (***).
Figure 5: Phase 4: GPT-4o abstention behavior under explicit threshold instructions. (A) Profile of model responses across instructed confidence thresholds. Numbers inside bars reflect percentages. As threshold increases, abstention rate (blue) increases. (B) Abstention rate as a function of instructed threshold (x-axis) and model calibrated confidence (y-axis). When plotted against Phase 1 calibrated chosen confidence, abstention exhibits a diagonal gradient (bandness index =0.042=0.042 ), consistent with Phase 1 confidence being pre-decisional—formed independently and then compared against threshold as separate inputs to the decision rule (see Supplemental text for details). (C) Abstention rate as a function of instructed threshold (x-axis) and model calibrated confidence (y-axis). When plotted against Phase 4 calibrated confidence (maximum value among real options), abstention collapses into horizontal bands (bandness index =0.78=0.78 ) determined almost entirely by confidence, with threshold having little independent effect. This pattern demonstrates that Phase 4 confidence is post-decisional—it reflects the output of the threshold-conditioned abstention policy rather than the upstream belief signal (see Supplemental text for details). (D) Model comparison showing calibrated confidence as the dominant predictor of abstention. Left panel: AIC values for models with threshold plus single additional predictor, where lower values indicate better fit. Confidence substantially outperforms difficulty, RAG score, and sentence embeddings. Right panel: Standardized effect sizes from the full model including all predictors simultaneously. Confidence and threshold exhibit comparable effect sizes, both substantially larger than alternative predictors: RAG, difficulty, and top embedding component. (E)Computational model of the decision rule governing abstention. Behavior of a perfectly calibrated model is shown along the diagonal (i.e., Confidence = Threshold). In comparison, the fitted model (calibrated confidence + threshold + difficulty) exhibits conservatism driven by a large negative baseline bias (shift = −97.6-97.6 %), but this is partially offset by substantial overweighting of internal confidence (scale = 1.80). For example, at 80% confidence, the model’s indifference point occurs at threshold ≈\approx 46%, meaning it abstains 50% of the time despite its confidence substantially exceeding the threshold. The steep slope of the contours (e.g., T50T_{50} ) reflects this overweighting: a 1% increase in confidence offsets approximately 1.80% of instructed threshold. Contours T20T_{20} , T50T_{50} , T80T_{80} mark thresholds at which the model abstains with 20%, 50%, 80% probability. The relatively wide gap between T20T_{20} and T80T_{80} reflects the soft transition (policy temperature = 31.0 in percent units) of the decision boundary. (F) Illustration of abstention model fit. Observed versus predicted abstention rates in 12 bins. Error bars: 95% Wilson confidence intervals. Color indicates bin size. Dashed line: perfect calibration.
Figure 6: Proposed architecture for confidence-guided abstention. In Stage 1, the model processes the question and answer options, forming an internal confidence representation at the last pre-answer token (answer colon; AC). This multidimensional internal confidence representation is proposed to comprise two partially dissociable confidence-related components: a generative component ( XactX_{\text{act}} ), more closely linked to the answer-generation process, and an evaluative component ( XevalX_{\text{eval}} ), more closely linked to explicit verbal self-report. In Stage 2, these confidence-related signals are incorporated into a threshold policy, σ((T−C)/τ)\sigma((T-C)/\tau) , which determines whether the model answers or abstains. The partial dissociation between the two observable channels ( r≈0.3r\approx 0.3 – 0.40.4 ), together with the AC decoding results, suggests that abstention is not driven solely by a single first-order confidence signal, but instead depends on a richer internal confidence representation. This pattern is consistent with, though does not by itself uniquely establish, a second-order interpretation in the sense of Fleming and Daw [7] . Note that this figure represents a computational-level schematic of the functional relationships between confidence signals and abstention behaviour, not a mechanistic claim about information flow within a single forward pass. All confidence measures and AC activations were collected during Phase 1 (without an abstention option) and used to predict Phase 2 abstention behaviour.
Supplementary Figure 1: GPT4o: Performance on Phases 1 and 2. When the model makes errors it tends to pick similar foils, dissimilar and unrelated foils in decreasing order
PredictorModel A: Confidence OnlyModel B: Full Model
Coef.SEzzCoef.SEzz
Intercept2.746***0.26210.493.923***0.32412.10
Confidence−4.871-4.871 ***0.470−10.37-10.37−5.263-5.263 ***0.498−10.56-10.56
Difficulty–––0.3420.2651.29
RAG Score–––−0.412-0.4120.305−1.35-1.35
Embedding PC6–––0.106***0.0264.04
Supplementary Table 1: GPT-4o: Logistic regression predicting abstention in Phase 2
 rr (P1, P2)Fisher zzP1 wins (%)Mean KLWelch’s tt
ModelAnsAbsppAnsAbsAnsAbspp
Gemma 3 27B0.5790.4041.2 ⁣× ⁣10−31.2\!\times\!10^{-3}77.566.90.0770.1161.2 ⁣× ⁣10−111.2\!\times\!10^{-11}
GPT-4o0.8600.7483.7 ⁣× ⁣10−73.7\!\times\!10^{-7}87.677.70.0530.1333.6 ⁣× ⁣10−73.6\!\times\!10^{-7}
DeepSeek 671B0.5210.2991.2 ⁣× ⁣10−31.2\!\times\!10^{-3}91.767.40.2770.7172.7 ⁣× ⁣10−62.7\!\times\!10^{-6}
Qwen 80B0.7790.6059.6 ⁣× ⁣10−89.6\!\times\!10^{-8}90.476.70.0690.1966.5 ⁣× ⁣10−226.5\!\times\!10^{-22}
Supplementary Table 2: Cross-phase stability of real-option confidence distributions in Phase 2. For both the maximum-confidence comparison and the KL-divergence analysis, Phase 2 confidence was renormalised over the four real options only, excluding the abstention option. Higher correlation, a higher probability that the Phase 1 maximum-confidence option remains the winner, and lower KL divergence all indicate greater similarity to the Phase 1 real-option confidence distribution. Answered (‘Ans’) trials generally exhibited greater cross-phase stability than abstention (’Abs’) trials, indicating stronger reconfiguration of the real-option distribution when abstention was selected.
Supplementary Figure 2: Gemma 3 27B: Change from baseline abstention rate as a result of high- and low-confidence activation steering across layers 30–40. Analyses included n=500n=500 independent held-out questions. Points show the mean change in abstention rate relative to the no-steering baseline; error bars denote the standard error of the mean (SEM) across questions. Baseline abstention rate in the no-steering condition was 28.8%.
Supplementary Figure 3: GPT4o Phase 4: Response breakdown among answered questions. Increasing threshold increases proportion of correct responses, whilst reducing frequency of similar and dissimilar errors.
PredictorCoefficient ( β\beta )Std. ErrorzzPP
Intercept3.4170.10931.43<2.2×10−16<2.2\times 10^{-16}
Threshold (T)0.03270.0008538.44<2.2×10−16<2.2\times 10^{-16}
Confidence (%)−0.0582-0.05820.00171−34.11-34.115.996×10−2555.996\times 10^{-255}
Difficulty0.1590.0881.817.081×10−27.081\times 10^{-2}
RAG Score−0.875-0.8750.103−8.47-8.472.423×10−172.423\times 10^{-17}
Embedding PC10−0.043-0.0430.010−4.47-4.477.713×10−67.713\times 10^{-6}
Supplementary Table 3: Phase 4: Logistic regression model predicting abstention, GPT-4o.
ModelAIC ( ↓\downarrow )pseudo- R2R^{2} ( ↑\uparrow )Key likelihood-ratio tests
Threshold only (T)12874.30.105–
T + Difficulty12360.70.141–
T + RAG12765.80.113–
T + Embeddings (10 PCs)12778.10.114–
T + Confidence10921.00.241vs. T: χ2(1)=1955.3,  P<2.2×10−16\chi^{2}(1)=1955.3,\;P<2.2\times 10^{-16}
   vs. T+RAG: χ2(1)=1897.9,  P<2.2×10−16\chi^{2}(1)=1897.9,\;P<2.2\times 10^{-16}
Supplementary Table 4: Phase 4: Model comparison for abstention predictions, GPT-4o.
Supplementary Figure 4: Heatmap of GPT-4o’s abstention confidence in Phase 4, shown as a function of instructed threshold and Phase 1 chosen confidence. Warmer colors indicate stronger abstention confidence, also specified by numbers inside cells. The continuous abstention confidence reveals a linear additive structure: abstention confidence increases steadily with instructed threshold and decreases steadily with chosen confidence, with difficulty having only a minor influence. See Main text for details
PredictorModel A: Difficulty OnlyModel B: Difficulty + Confidence
Coef.SEzzCoef.SEzz
Intercept−0.234-0.2340.149−1.56-1.562.6922.6920.3996.756.75
Difficulty−1.215-1.215 ***0.221−5.49-5.49−0.837-0.837 ***0.236−3.55-3.55
Confidence–––−5.575-5.575 ***0.702−7.94-7.94
AIC1143.81077.9
Pseudo- R2R^{2}0.0260.084
Supplementary Table 5: Gemma 27B: Logistic regression predicting abstention in Phase 2
Supplementary Figure 5: Gemma 3 27B results across phases 1,2 and 4. (A) (A) Gemma 3 27B confidence predicts error rate (Phase 1). Relationship between binned confidence and error rate in Phase 1. Each point represents one confidence bin (bin width = 0.05; n=14n=14 confidence bins); colour indicates the number of independent trials contributing to each bin. A strong negative relationship was observed between confidence and error rate (two-sided Pearson correlation, r(12)=−0.966r(12)=-0.966 , P=9.02×10−8P=9.02\times 10^{-8} ). The model was calibrated on a separate dataset of 1,000 independent trials (see Methods). (B) Gemma 27B confidence predicts abstention behavior (phase 2). Logistic regression model of natural abstention probability as a function of confidence for n=1000n=1000 trials. The grey curve indicates the model prediction, fitted to individual binary abstention decisions. The red dashed line marks the implicit decision threshold at 38.6%, where P(abstain)=0.5P(\mathrm{abstain})=0.5 (holding difficulty constant at sample mean). The red shaded region denotes the confidence interval spanning approximately ±9\pm 9 percentage points around the threshold, illustrating the sharpness/softness of the decision boundary (policy temperature = 0.18 units, or 18 percentage points). Black circles represent observed abstention rates (mean ±\pm 95% CI) across confidence quintiles. The low threshold (38.6%) indicates that Gemma 27B continues answering even at relatively low confidence levels, only choosing to abstain when confidence drops below this point, revealing the model’s intrinsic abstention boundary even without explicit threshold instructions. (C) Performance of Gemma 27B in Phase 4 as a function of instructed threshold. (D) Profile of abstention behavior shown by Gemma in phase 4, as a function of Phase 1 confidence and instructed threshold. Abstention rate (0-1) increases from left to right (instructed threshold), and from bottom to top (binned confidence). Confidence: calibrated chosen confidence from phase 1, binned into fixed bins of size 0.1 (in the range 0.3-1.0). (E) Decision rule guiding abstention behavior shown by fitted model. Behavior of perfectly calibrated model shown along the diagonal (i.e. Conf = Threshold). In comparison, the fitted model is moderately conservative due to under-weighting its own confidence relative to the instructed threshold (scale = 0.66), such that a 1% increase in confidence offsets only approximately 0.66% of threshold. The baseline bias (shift = +1.0%) is negligible. Contours T20T_{20} , T50T_{50} , T80T_{80} mark thresholds at which the model abstains with 20%, 50%, 80% probability. Points to the right/below each line correspond to lower abstention rates, while points to the left/above correspond to higher abstention rates. The moderate gap between T20T_{20} and T80T_{80} reflects a moderately soft transition (policy temperature = 16.6) of the decision boundary.
Supplementary Figure 6: Performance of Gemma 27B on Phase 4 averaged across 20 prompts. (A) Performance of Gemma 3 27B averaged across 20 prompts (see Methods for details). (B) Abstention rate as a function of instructed threshold, averaged across 20 prompts.
PredictorCoefficient ( β\beta )Std. ErrorzzPP
Intercept−0.058-0.0580.138−0.42-0.424.991×10−14.991\times 10^{-1}
Threshold (T)0.0600.001155.35<2.2×10−16<2.2\times 10^{-16}
Confidence (%)−0.040-0.0400.0024−16.42-16.427.508×10−627.508\times 10^{-62}
Difficulty−0.778-0.7780.083−9.40-9.401.019×10−101.019\times 10^{-10}
Supplementary Table 6: Gemma 3 27B: Logistic regression predicting abstention (Phase 4), full model
ModelAIC ( ↓\downarrow )pseudo- R2R^{2} ( ↑\uparrow )Key likelihood-ratio tests
Threshold only (T)10021.80.342–
Threshold + Confidence9632.00.368vs. T: χ2(1)=391.82,  P=3.322×10−87\chi^{2}(1)=391.82,\;P=3.322\times 10^{-87}
Threshold + Difficulty9824.50.355–
Full (T + Confidence + Difficulty)9544.40.374vs. T+Conf: χ2(1)=89.62,  P=2.890×10−21\chi^{2}(1)=89.62,\;P=2.890\times 10^{-21}
Supplementary Table 7: Gemma 3 27B: Model comparison for abstention predictions (Phase 4)
PredictorModel A: Difficulty OnlyModel B: Difficulty + Confidence
Coef.SEzzCoef.SEzz
Intercept2.2382.238 ***0.21210.5510.554.3644.364 ***0.31313.9613.96
Difficulty−1.060-1.060 ***0.273−3.88-3.88−0.481-0.4810.290−1.66-1.66
Confidence–––−5.461-5.461 ***0.531−10.28-10.28
AIC930.9817.7
Pseudo- R2R^{2}0.0170.139
Supplementary Table 8: DeepSeek 671B: Logistic regression predicting abstention in Phase 2
Supplementary Figure 7: DeepSeek 671B results across phases 1,2 and 4. (A) DeepSeek 671B confidence predicts error rate (Phase 1). Relationship between binned confidence and error rate in Phase 1. Each point represents one confidence bin (bin width = 0.05; n=14n=14 confidence bins); colour indicates the number of independent trials contributing to each bin. A strong negative relationship was observed between confidence and error rate (two-sided Pearson correlation, r(12)=−0.930r(12)=-0.930 , P=1.49×10−6P=1.49\times 10^{-6} ). The model was calibrated on a separate dataset of 1,000 independent trials (see Methods). (B) DeepSeek 671B confidence predicts abstention behavior (phase 2). Logistic regression model of natural abstention probability as a function of confidence for n=1000n=1000 trials. The grey curve indicates the model prediction, fitted to individual binary abstention decisions. The red dashed line marks the implicit decision threshold at 74.2%, where P(abstain)=0.5P(\mathrm{abstain})=0.5 (holding difficulty constant at sample mean). The red shaded region denotes the confidence interval spanning approximately ±9\pm 9 percentage points around the threshold, illustrating the sharpness/softness of the decision boundary (policy temperature = 0.18 units, or 18 percentage points). Black circles represent observed abstention rates (mean ±\pm 95% CI) across confidence quintiles. The high threshold (74.2%) indicates that DeepSeek requires relatively high confidence before choosing to answer, readily abstaining when confidence drops below this point, revealing the model’s intrinsic abstention boundary even without explicit threshold instructions. (C) Performance of DeepSeek in Phase 4 as a function of instructed threshold. (D) Profile of abstention behavior shown by DeepSeek in phase 4, as a function of confidence and instructed threshold. Confidence: calibrated chosen confidence from phase 1, binned into fixed bins of size 0.1 (in the range 0.3- 0.9). (E) Decision rule guiding abstention behavior shown by fitted model. Behavior of perfectly calibrated model shown along the diagonal (i.e. Conf = Threshold). The fitted model exhibits near-perfect calibration with essentially equal weighting of confidence and threshold (scale = 1.05), such that a 1% increase in confidence offsets approximately 1.05% of threshold. The baseline bias is minimal (shift = −8.5-8.5 %). Contours T20T_{20} , T50T_{50} , T80T_{80} mark thresholds at which the model abstains with 20%, 50%, 80% probability. Points to the right/below each line correspond to lower abstention rates, while points to the left/above correspond to higher abstention rates. The wide gap between T20T_{20} and T80T_{80} reflects a relatively soft transition (policy temperature = 39.3) of the decision boundary, indicating gradual rather than sharp changes in abstention probability across the confidence-threshold space.
PredictorCoefficient ( β\beta )Std. ErrorzzPP
Intercept0.2160.2160.0772.812.814.95×10−34.95\times 10^{-3}
Threshold (T)0.0250.0007135.93<2.2×10−16<2.2\times 10^{-16}
Confidence (%)−0.027-0.0270.0015−17.64-17.642.71×10−702.71\times 10^{-70}
Difficulty−0.810-0.8100.066−12.19-12.192.95×10−342.95\times 10^{-34}
Supplementary Table 9: DeepSeek 671B: Logistic regression predicting abstention (Phase 4), full model
ModelAIC ( ↓\downarrow )pseudo- R2R^{2} ( ↑\uparrow )Key likelihood-ratio tests
Threshold only (T)13798.30.092–
Threshold + Confidence13254.00.128vs. T: χ2(1)=546.26,  P=8.206×10−121\chi^{2}(1)=546.26,\;P=8.206\times 10^{-121}
Threshold + Difficulty13540.50.109–
Full (T + Confidence + Difficulty)13136.40.136vs. T+Diff: χ2(1)=406.12,  P=2.43×10−90\chi^{2}(1)=406.12,\;P=2.43\times 10^{-90}
Supplementary Table 10: DeepSeek 671B: Model comparison for abstention predictions (Phase 4)
PredictorModel A: Difficulty OnlyModel B: Difficulty + Confidence
Coef.SEzzCoef.SEzz
Intercept0.2410.2410.1421.701.703.0173.017 ***0.3239.349.34
Difficulty−0.765-0.765 ***0.199−3.84-3.840.0410.0410.2270.180.18
Confidence–––−4.510-4.510 ***0.463−9.73-9.73
AIC1360.01256.6
Pseudo- R2R^{2}0.0110.088
Supplementary Table 11: Qwen 80B: Logistic regression predicting abstention in Phase 2
Supplementary Figure 8: Qwen 80B results across phases 1,2 and 4. (A) Qwen 80B confidence predicts error rate (Phase 1). Relationship between binned confidence and error rate in Phase 1. Each point represents one confidence bin (bin width = 0.05; n=14n=14 confidence bins); colour indicates the number of independent trials contributing to each bin. A strong negative relationship was observed between confidence and error rate (two-sided Pearson correlation, r(12)=−0.833r(12)=-0.833 , P=1.15×10−4P=1.15\times 10^{-4} ). The model was calibrated on a separate dataset of 1,000 independent trials (see Methods). (B) Qwen 80B confidence predicts abstention behavior (phase 2). Logistic regression model of natural abstention probability as a function of confidence for n=1000n=1000 trials. The grey curve indicates the model prediction, fitted to individual binary abstention decisions. The red dashed line marks the implicit decision threshold at 67.5%, where P(abstain)=0.5P(\mathrm{abstain})=0.5 (holding difficulty constant at sample mean of 0.646). The red shaded region denotes the confidence interval spanning approximately ±11\pm 11 percentage points around the threshold, illustrating the sharpness/softness of the decision boundary (policy temperature = 0.22 units, or 22 percentage points). Black circles represent observed abstention rates (mean ±\pm 95% CI) across confidence quintiles. The moderately high threshold (67.5%) indicates that Qwen 80B requires relatively high confidence before choosing to answer, readily abstaining when confidence drops below this point, revealing the model’s intrinsic abstention boundary even without explicit threshold instructions.(C) Performance of Qwen 80B in Phase 4 as a function of instructed threshold. (D) Profile of abstention behavior shown by Qwen 80B in phase 4, as a function of confidence and instructed threshold. Confidence: calibrated chosen confidence from phase 1, binned into fixed bins of size 0.1 (in the range 0.3- 1.0). (E) Decision rule guiding abstention behavior shown by fitted model. Behavior of perfectly calibrated model shown along the diagonal (i.e. Conf = Threshold). The fitted model substantially over-weights internal confidence relative to instructed threshold (scale = 1.49), such that a 1% increase in confidence offsets approximately 1.49% of threshold. The large negative shift ( −52.3-52.3 %) means that at very low confidence levels the model is overly cautious, but the high scale parameter causes the decision boundary to tilt steeply, such that at higher confidence levels (above 50%) the model becomes increasingly over-confident and answers more readily than a perfectly calibrated model would. Contours T20T_{20} , T50T_{50} , T80T_{80} mark thresholds at which the model abstains with 20%, 50%, 80% probability. Points to the right/below each line correspond to lower abstention rates, while points to the left/above correspond to higher abstention rates. The relatively wide gap between T20T_{20} and T80T_{80} reflects a moderately soft transition (policy temperature = 29.3 in percent units) of the decision boundary.
PredictorCoefficient ( β\beta )Std. ErrorzzPP
Intercept1.7851.7850.11315.8315.831.93×10−561.93\times 10^{-56}
Threshold (T)0.0340.000937.92<2.2×10−16<2.2\times 10^{-16}
Confidence (%)−0.051-0.0510.0016−31.02-31.022.90×10−2112.90\times 10^{-211}
Difficulty−0.148-0.1480.077−1.91-1.915.61×10−25.61\times 10^{-2}
Supplementary Table 12: Qwen 80B: Logistic regression predicting abstention (Phase 4), full model
ModelAIC ( ↓\downarrow )pseudo- R2R^{2} ( ↑\uparrow )Key likelihood-ratio tests
Threshold only (T)12284.50.111–
Threshold + Confidence10962.70.207vs. T: χ2(1)=1323.80,  P=7.622×10−290\chi^{2}(1)=1323.80,\;P=7.622\times 10^{-290}
Threshold + Difficulty12062.40.127–
Full (T + Confidence + Difficulty)10961.10.207vs. T+Conf: χ2(1)=3.64,  P=5.639×10−2\chi^{2}(1)=3.64,\;P=5.639\times 10^{-2}
Supplementary Table 13: Qwen 80B: Model comparison for abstention predictions (Phase 4)
Supplementary Figure 9: Distribution of verbal confidence class ratings observed in all 4 models.
 ECE ↓\downarrowAUROC ↑\uparrowPearson rr
ModelVerbalV isoCalUncalVerbalCalV ↔\leftrightarrow CV ↔\leftrightarrow U
Gemma 27B.241.018.033.388.533.631.373.145
GPT-4o.266.001.037.257.546.746.316.174
DeepSeek.159.032.107.242.559.681.410.203
Qwen 80B.187.005.079.290.604.703.434.226
Supplementary Table 14: Properties of three confidence measures: calibration (ECE), discrimination (AUROC), and pairwise correlations. All metrics computed on Phase 1 data ( N=1,000N=1{,}000 per model).
 Pseudo- R2R^{2}+Verbal+Calibrated
ModelAuxCal+AUncal+AVerb+AC+V+AΔ\Delta AICΔR2\Delta R^{2}Δ\Delta AICΔR2\Delta R^{2}
Gemma 27B.084.143.091.175.207+73+.064+35+.031
      P=4.602×10−18P=4.602\times 10^{-18}P=1.303×10−9P=1.303\times 10^{-9}
GPT-4o.044.153.063.148.215+82+.061+90+.067
      P=5.271×10−20P=5.271\times 10^{-20}P=1.009×10−21P=1.009\times 10^{-21}
DeepSeek.034.191.043.147.233+38+.043+80+.087
Supplementary Table 15: Three confidence measures predicting free abstention (Phase 2). All models include auxiliary controls (difficulty, RAG score, sentence embeddings).
 Pseudo- R2R^{2}+Verbal+Calibrated
ModelAuxCal+AUncal+AVerb+AC+V+AΔ\Delta AICΔR2\Delta R^{2}Δ\Delta AICΔR2\Delta R^{2}
Gemma 27B.376.395.376.425.430+533+.035+71+.005
      P=2.447×10−118P=2.447\times 10^{-118}P=1.087×10−17P=1.087\times 10^{-17}
GPT-4o.156.262.181.224.296+490+.034+1037+.072
      P=6.039×10−109P=6.039\times 10^{-109}P=5.879×10−228P=5.879\times 10^{-228}
DeepSeek.125.151.126.135.152+26+.002+264+.018
Supplementary Table 16: Three confidence measures predicting threshold-dependent abstention (Phase 4). All models include threshold ( TT ) and auxiliary controls.
   +Raw Verbal+Isotonic Verbal
ModelPhaseNNΔ\Delta AICΔR2\Delta R^{2}PPΔ\Delta AICΔR2\Delta R^{2}PP
Gemma 27B21,000+73+.0644.60×10−184.60\times 10^{-18}+21+.0191.93×10−61.93\times 10^{-6}
Gemma 27B411,000+533+.0352.45×10−1182.45\times 10^{-118}+225+.0152.50×10−512.50\times 10^{-51}
GPT-4o21,000+82+.0615.27×10−205.27\times 10^{-20}+94+.0701.48×10−221.48\times 10^{-22}
GPT-4o411,000+490+.0346.04×10−1096.04\times 10^{-109}+466+.0339.61×10−1049.61\times 10^{-104}
DeepSeek21,000+38+.0432.12×10−102.12\times 10^{-10}+9+.0111.05×10−31.05\times 10^{-3}
Supplementary Table 17: Robustness check: isotonic calibration of verbal confidence.
Supplementary Figure 10: Linear decoding across layers in Gemma 3 27B using activations collected from last pre-answer token (answer-colon; AC) before the model’s answer – using the Phase 1 prompt. a) Variance in verbal confidence obtained by cross validated regularized ridge regression – above that accounted for by calibrated confidence. Near zero variance explained by activation at the last question token (not shown). b) Variance in calibrated confidence by cross validated regularized ridge regression, over and above that accounted for by verbal confidence. Near zero variance explained by activation at the last question token (not shown). c) Discrimination of binary correctness (AUROC) by AC activations, and control (i.e. last question token) activations. Dashed lines show scalar baselines: calibrated confidence and verbal confidence.
More in this topic

Metacognition in Language Models

Metacognition in language models studies how models assess their knowledge and reasoning, express uncertainty, and regulate their own actions.

See all 127 papers

Explore similar work

CardsList
Jun 28, 2026

Reported Confidence in LLMs Tracks Commitment More Than Correctness

Confidence is an estimate of the probability that a chosen answer is correct. Verbal confidence reports are widely used as uncertainty measures in large language models, but whether they are best understood as estimates of correctness is unclear. We test this with a two-stage abstention paradigm from the neuroscience of perceptual decision making: a model first answers and reports its confidence, then decides whether to commit it to a user or abstain. Across four non-reasoning models, prompt framings, and confidence formats, verbal confidence predicted the commit/abstain decision substantially better than whether the answer was correct. Calibrated token log-probabilities showed the opposite profile, with abstention-prediction coupled to correctness discrimination, the signature of an answer-evidence signal. After removing the variance verbal confidence shared with log-probabilities, the residual stayed aligned with commitment while its link to correctness fell to near chance. The dissociation generalised to four reasoning models across four benchmarks of varying difficulty, from hard multiple-choice to frontier-level freeform questions. Mechanistic analyses in Gemma 3 and 4 were convergent: a post-answer state known to causally support verbal-confidence generation already encoded the future abstention decision before the abstention prompt, organised mainly by that decision rather than by correctness, the two lying in approximately orthogonal directions in activation space. Steering along a verbal-confidence-specific direction causally shifted abstention. Verbal and log-probability confidence are thus not interchangeable: log-probabilities track answer evidence and correctness, whereas verbal confidence is better understood as a behaviour-facing readout of an internal commit-readiness state, challenging the practice of treating verbal reports as proxies for reliability.

Dharshan Kumaran

Google DeepMind

LLM Uncertainty EstimationLLM InterpretabilityMetacognition in Language ModelsLLM AbstentionConfidence Estimation in Language ModelsVerbalized Confidence Estimation+1
AskPDF
Sep 26, 2026

The Commit-Abstain Circuit: Why Language Models Hallucinate Instead of Abstaining

Language models (LMs) often hallucinate by committing to confident answers rather than abstaining, even when they do not have enough information to answer reliably. A large body of existing work mitigates hallucination through detection or abstention mechanisms, but leaves open how models internally arrive at the decision to commit or abstain in the first place. We study this decision through mechanistic analysis, framing hallucination as unsupported commitment: the model commits despite exhibiting signals of unanswerability. Using causal gating, we identify a Commit-Abstain Circuit (CAC), a sparse, causally localised subset of attention heads and MLP sublayers underlying this decision. Across ten LMs (3B-14B) from five families and three benchmarks, the CAC exhibits a recurring accumulate-yet-undercorrect pattern: commitment-promoting components build up commitment in earlier layers, while abstention-promoting components act later as corrective signals that are often insufficient to overturn the accumulated commitment. Building on this finding, a lightweight policy trained on CAC activations improves decision accuracy by 12.2 points over the model's intrinsic commit-abstain margin, reduces false abstentions by 2.5 times, transfers to unseen benchmarks, and extends to larger models (27B-35B). The CAC is both diagnostic, clarifying how models overcommit, and practical, enabling improved abstention decisions.

Vy Nguyen, Ziqi Xu, Jeffrey Chan +5

RMIT University · Jilin University · Nanyang Technological University

Hallucination in Language ModelsCircuits in Language ModelsLLM Abstention
AskPDF
Apr 7, 2026

Identifying Influential N-grams in Confidence Calibration via Regression Analysis

While large language models (LLMs) improve performance by explicit reasoning, their responses are often overconfident, even though they include linguistic expressions demonstrating uncertainty. In this work, we identify what linguistic expressions are related to confidence by applying the regression method. Specifically, we predict confidence of those linguistic expressions in the reasoning parts of LLMs as the dependent variables and analyze the relationship between a specific nn-gram and confidence. Across multiple models and QA benchmarks, we show that LLMs remain overconfident when reasoning is involved and attribute this behavior to specific linguistic information. Interestingly, several of the extracted expressions coincide with cue phrases intentionally inserted on test-time scaling to improve reasoning performance. Through our test on causality and verification that the extracted linguistic information truly affects confidence, we reveal that confidence calibration is possible by simply suppressing those overconfident expressions without drops in performance.

Shintaro Ozaki, Wataru Hashimoto, Hidetaka Kamigaito +2

Nara Institute of Science and Technology (NAIST) · The University of Tokyo

Language Model CalibrationConfidence Estimation in Language ModelsLLM ReliabilityConfidence EstimationCausal Interventions in Language Models
AskPDF