Fine-grained Approaches for Confidence Calibration of LLMs in Automated Code Revision

Hong Yi Lin; Chunhua Liu; Haoyu Gao; Patanamon Thongtanunam; Christoph Treude

cs.SEApr 8, 2026

Fine-grained Approaches for Confidence Calibration of LLMs in Automated Code Revision

Authors: Hong Yi Lin, Chunhua Liu, Haoyu Gao, Patanamon Thongtanunam, Christoph Treude

Abstract

In today's AI-assisted software engineering landscape, developers increasingly depend on LLMs that are highly capable, yet inherently imperfect. The tendency of these models to produce incorrect outputs can reduce developer productivity. To this end, a canonical mitigation method is to provide calibrated confidence scores that faithfully reflect their likelihood of correctness at the instance-level. Such information allows users to make immediate decisions regarding output acceptance, abstain error-prone outputs, and better align their expectations with the model's capabilities. Since post-trained LLMs do not inherently produce well-calibrated confidence scores, researchers have developed post-hoc calibration methods, with global Platt-scaling of sequence-level confidence scores proving effective in many generative software engineering tasks but remaining unreliable or unexplored for automated code revision (ACR) tasks such as program repair, vulnerability repair, and code refinement. We hypothesise that the coarse-grained nature of this conventional method makes it ill-suited for ACR tasks, where correctness is often determined by local edit decisions and miscalibration can be sample-dependent, thereby motivating fine-grained confidence calibration. To address this, our study proposes local Platt-scaling applied separately to three different fine-grained confidence scores. Through experiments across 3 separate tasks and correctness metrics, as well as 14 different models of various sizes, we find that fine-grained confidence scores consistently achieve lower calibration error across a broader range of probability intervals, and this effect is further amplified when global Platt-scaling is applied. Our proposed approaches offer a practical solution to eliciting well-calibrated confidence scores, enabling more trustworthy and streamlined usage of imperfect models in ACR tasks.

Figures & tables

Study	Task	Single-Inference Confidence Score	Post-hoc Scaler
Spiess et al. [ 21 ]	Function Synthesis Line-level Completion Automated Program Repair	Average Token Probability Sequence Likelihood	Global Platt-Scaling
Virk et al. [ 22 ]	Code Summarisation	Average Token Probability Normalised Sequence Likelihood	Global Platt-Scaling
(Ours)	Automated Program Repair Vulnerability Repair Automated Code Refinement	Average Token Probability Normalised Sequence Likelihood Minimum Token Probability Lowest- $K$ Token Probability Attention-Weighted Uncertainty	Global Platt-Scaling Local Platt-Scaling
Bold: Contribution from this Study

TABLE I: Comparison with Past Work on Confidence Calibration for Code Generation Tasks in Software Engineering

Fig. 1: Miscalibrated (Left) and Well-Calibrated (Right) Confidence Scores for Code Revisions Generated by CodeLlama-70B in Automated Code Refinement

Panel of Figure 3 — Fig. 2: Zero-Shot Prompt Templates for ACR Tasks

		DCF-Bug		DCF-Vul			CR-Trans
		n=777		n=1,041			n=900
Model	Size	EM	EP	EM	EP	CP	EM	EP
	8B	32.9	59.3	7.7	29.2	59.6	36.3	59.3
Llama-3.1	70B	34.5	58.0	9.7	31.2	72.9	45.9	67.1
	7B	32.3	55.6	8.8	26.1	40.2	40.1	60.4
CodeLlama	70B	40.5	66.8	12.0	29.0	44.9	51.8	72.4

TABLE II: Correctness (%) Achieved by LLMs in ACR Tasks

Fig. 3: Median Skewness of Token-Level Softmax Probabilities for LLM-Generated Code Revisions ( $\tilde{\gamma}_{1}$ )

Fig. 4: DCF-Bug (Wasserstein Distance $W_{1}$ )

Fig. 5: DCF-Bug (Kendall’s $\tau_{b}$ Coefficient)

Fig. 6: DCF-Vul (Wasserstein Distance $W_{1}$ )

Fig. 7: DCF-Vul (Kendall’s $\tau_{b}$ Coefficient)

Fig. 8: CR-Trans (Wasserstein Distance $W_{1}$ )

Fig. 9: CR-Trans (Kendall’s $\tau_{b}$ Coefficient)

		EM															EP
		sl_norm			avg			min			low-K			attn- $\omega$			sl_norm			avg			min			low-K			attn- $\omega$
Model	Size	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC
	8B	0.11	0.21	3	0.03	0.21	2	0.05	0.16	9	0.06	0.18	7	0.04	0.18	7	0.07	0.23	3	0.06	0.23	2	0.08	0.21	8	0.08	0.21	7	0.08	0.22	7
Llama-3.1	70B	0.11	0.21	3	0.12	0.22	2	0.04	0.18	8	0.04	0.18	7	0.04	0.18	7	0.12	0.22	2	0.10	0.23	2	0.04	0.20	7	0.04	0.21	6	0.04	0.21	6
	7B	-	-	1	-	-	1	0.04	0.19	8	0.06	0.21	5	0.04	0.20	6	0.05	0.24	2	-	-	1	0.04	0.22	7	0.03	0.23	6	0.03	0.23	6
CodeLlama	70B	0.03	0.24	2	-	-	1	0.04	0.20	8	0.08	0.21	7	0.08	0.21	7	-	-	1	-	-	1	0.05	0.18	6	0.06	0.19	5	0.06	0.19	5

TABLE III: Global Platt-Scaling Results for DCF-Bug in Terms of Exact Match and Edit Progress

		EP															CP
		sl_norm			avg			min			low-K			attn- $\omega$			sl_norm			avg			min			low-K			attn- $\omega$
Model	Size	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC
	8B	0.04	0.20	2	0.02	0.20	2	0.02	0.19	7	0.02	0.20	4	0.02	0.20	4	0.04	0.24	2	-	-	1	0.06	0.23	5	0.04	0.24	4	-	-	1
Llama-3.1	70B	0.03	0.21	2	0.03	0.21	2	0.05	0.19	6	0.05	0.20	5	0.05	0.20	5	-	-	1	-	-	1	0.05	0.19	3	-	-	1	-	-	1
	7B	0.02	0.19	2	-	-	1	0.03	0.19	3	0.02	0.19	3	0.01	0.19	3	0.03	0.24	2	0.03	0.24	2	0.04	0.24	3	0.02	0.24	4	0.03	0.24	2
CodeLlama	70B	0.03	0.21	2	-	-	1	0.03	0.20	2	0.03	0.20	3	0.03	0.20	3	-	-	1	-	-	1	-	-	1	0.08	0.25	2	0.08	0.25	2

TABLE IV: Global Platt-Scaling Results for DCF-Vul in Terms of Edit Progress and Checks Passed

		EM															EP
		sl_norm			avg			min			low-K			attn- $\omega$			sl_norm			avg			min			low-K			attn- $\omega$
Model	Size	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC	ECE	$\boldsymbol{\mathcal{B}}$	BC
	8B	0.22	0.27	2	0.23	0.27	2	0.16	0.20	6	0.17	0.22	4	0.17	0.23	4	0.23	0.27	3	0.24	0.28	2	0.17	0.23	7	0.18	0.24	6	0.18	0.24	5
Llama-3.1	70B	0.25	0.28	2	0.25	0.29	2	0.17	0.22	7	0.19	0.25	5	0.19	0.25	5	0.21	0.24	3	0.21	0.25	3	0.14	0.21	7	0.16	0.23	5	0.16	0.23	5
	7B	-	-	1	-	-	1	0.19	0.23	6	0.21	0.25	4	0.21	0.25	4	0.29	0.31	2	0.29	0.31	2	0.25	0.27	5	0.26	0.29	4	0.26	0.29	4
CodeLlama	70B	0.31	0.33	2	0.31	0.34	2	0.21	0.25	7	0.25	0.29	5	0.25	0.29	5	0.31	0.29	2	0.32	0.30	2	0.26	0.25	5	0.28	0.27	4	0.28	0.27	4

TABLE V: Global Platt-Scaling Results for CR-Trans in Terms of Exact Match and Edit Progress

min

low-K

attn-

\omega

min

low-K

attn-

\omega

Model

Size

ECE

\boldsymbol{\mathcal{B}}

ECE

\boldsymbol{\mathcal{B}}

ECE

\boldsymbol{\mathcal{B}}

ECE

\boldsymbol{\mathcal{B}}

ECE

\boldsymbol{\mathcal{B}}

ECE

\boldsymbol{\mathcal{B}}

0.03 (-0.02)

0.15 (-0.01)

9 (=0)

0.03 (-0.03)

0.18 (=0)

7 (=0)

0.03 (-0.01)

0.18 (=0)

7 (=0)

0.06 (-0.02)

0.20 (-0.01)

9 (+1)

0.05 (-0.03)

0.21 (=0)

8 (+1)

0.06 (-0.02)

0.22 (=0)

9 (+2)

Llama-3.1

70B

0.03 (-0.01)

0.17 (-0.01)

8 (=0)

0.04 (=0)

0.18 (=0)

7 (=0)

0.04 (=0)

0.18 (=0)

7 (=0)

0.03 (-0.01)

0.20 (=0)

8 (+1)

0.04 (=0)

0.21 (=0)

6 (=0)

0.04 (=0)

0.21 (=0)

6 (=0)

0.02 (-0.02)

0.19 (=0)

8 (=0)

0.03 (-0.03)

0.21 (=0)

5 (=0)

0.02 (-0.02)

0.20 (=0)

6 (=0)

0.03 (-0.01)

0.22 (=0)

7 (=0)

0.03 (=0)

0.23 (=0)

6 (=0)

0.03 (=0)

0.23 (=0)

6 (=0)

CodeLlama

70B

0.04 (=0)

0.20 (=0)

8 (=0)

0.07 (-0.01)

0.20 (-0.01)

7 (=0)

0.07 (-0.01)

0.20 (-0.01)

7 (=0)

0.04 (-0.01)

0.18 (=0)

7 (+1)

0.06 (=0)

0.19 (=0)

5 (=0)

0.06 (=0)

0.19 (=0)

5 (=0)

TABLE VI: Local Platt-Scaling Results for DCF-Bug in Terms of Exact Match and Edit Progress

min

low-K

attn-

\omega

min

low-K

attn-

\omega

Model

Size

ECE

\boldsymbol{\mathcal{B}}

ECE

\boldsymbol{\mathcal{B}}

ECE

\boldsymbol{\mathcal{B}}

ECE

\boldsymbol{\mathcal{B}}

ECE

\boldsymbol{\mathcal{B}}

ECE

\boldsymbol{\mathcal{B}}

0.02 (=0)

0.19 (=0)

7 (=0)

0.02 (=0)

0.20 (=0)

4 (=0)

0.02 (=0)

0.20 (=0)

5 (+1)

0.03 (-0.03)

0.23 (=0)

5 (=0)

0.03 (-0.01)

0.24 (=0)

4 (=0)

0.02

0.24

3 (+2)

Llama-3.1

70B

0.05 (=0)

0.19 (=0)

6 (=0)

0.05 (=0)

0.20 (=0)

5 (=0)

0.05 (=0)

0.20 (=0)

5 (=0)

0.04 (-0.01)

0.19 (=0)

3 (=0)

0.05

0.20

3 (+2)

0.05

0.20

3 (+2)

0.03 (=0)

0.19 (=0)

3 (=0)

0.03 (=0)

0.19 (=0)

3 (=0)

0.01 (=0)

0.19 (=0)

3 (=0)

0.03 (-0.01)

0.24 (=0)

3 (=0)

0.02 (=0)

0.24 (=0)

4 (=0)

0.03 (=0)

0.24 (=0)

2 (=0)

CodeLlama

70B

0.02 (-0.01)

0.20 (=0)

3 (+1)

0.03 (=0)

0.20 (=0)

4 (+1)

0.03 (=0)

0.20 (=0)

4 (+1)

0.05

0.23

6 (+5)

0.05 (-0.03)

0.24 (-0.01)

2 (=0)

0.05 (-0.03)

0.24 (-0.01)

2 (=0)

TABLE VII: Local Platt-Scaling Results for DCF-Vul in Terms of Edit Progress and Checks Passed

min

low-K

attn-

\omega

min

low-K

attn-

\omega

Model

Size

ECE

\boldsymbol{\mathcal{B}}

ECE

\boldsymbol{\mathcal{B}}

ECE

\boldsymbol{\mathcal{B}}

ECE

\boldsymbol{\mathcal{B}}

ECE

\boldsymbol{\mathcal{B}}

ECE

\boldsymbol{\mathcal{B}}

0.13 (-0.03)

0.20 (=0)

9 (+3)

0.18 (-0.01)

0.22 (=0)

4 (=0)

0.17 (=0)

0.23 (=0)

4 (=0)

0.07 (-0.10)

0.22 (-0.01)

9 (+2)

0.11 (-0.07)

0.23 (-0.01)

8 (+2)

0.18 (=0)

0.24 (=0)

5 (=0)

Llama-3.1

70B

0.08 (-0.09)

0.22 (=0)

9 (+2)

0.19 (=0)

0.25 (=0)

5 (=0)

0.19 (=0)

0.25 (=0)

5 (=0)

0.09 (-0.05)

0.20 (-0.01)

8 (+1)

0.08 (-0.08)

0.21 (-0.02)

7 (+2)

0.16 (=0)

0.23 (=0)

5 (=0)

0.12 (-0.07)

0.22 (-0.01)

8 (+2)

0.21 (=0)

0.25 (=0)

4 (=0)

0.21 (=0)

0.25 (=0)

4 (=0)

0.10 (-0.15)

0.22 (-0.05)

7 (+2)

0.18 (-0.08)

0.25 (-0.04)

7 (+3)

0.26 (=0)

0.29 (=0)

4 (=0)

CodeLlama

70B

0.10 (-0.11)

0.22 (-0.03)

8 (+1)

0.23 (+0.02)

0.28 (-0.01)

8 (+3)

0.25 (=0)

0.29 (=0)

5 (=0)

0.18 (-0.08)

0.22 (-0.03)

7 (+2)

0.12 (-0.16)

0.21 (-0.06)

8 (+4)

0.28 (=0)

0.27 (=0)

4 (=0)

TABLE VIII: Local Platt-Scaling Results for CR-Trans in Terms of Exact Match and Edit Progress

		EM			EP
Model	Scaler	ECE	$\mathcal{B}$	BC	ECE	$\mathcal{B}$	BC
	Platt-scaling	0.04 (0.04)	0.20 (0.20)	8 (8)	0.05 (0.04)	0.18 (0.18)	6 (7)
	Temperature-scaling	0.18 (0.18)	0.24 (0.24)	4 (4)	0.02 (0.02)	0.18 (0.18)	4 (6)
	Histogram Binning	0.04 (0.05)	0.20 (0.20)	5 (7)	0.05 (0.05)	0.18 (0.18)	6 (6)
CodeLlama-70b	Isotonic Regression	0.05 (0.04)	0.20 (0.20)	8 (6)	0.05 (0.05)	0.18 (0.19)	5 (6)
	Platt-scaling	0.04 (0.04)	0.18 (0.17)	8 (8)	0.05 (0.03)	0.18 (0.18)	7 (8)

TABLE IX: Comparing Scalers for Fine-Grained Confidence Calibration using Minimum Token Probability on DCF-Bug

		EP			CP
Model	Scaler	ECE	$\mathcal{B}$	BC	ECE	$\mathcal{B}$	BC
	Platt-scaling	0.05 (0.05)	0.19 (0.19)	6 (6)	0.05 (0.04)	0.19 (0.19)	3 (3)
	Temperature-scaling	0.18 (0.17)	0.22 (0.21)	7 (10)	0.23 (0.23)	0.25 (0.25)	2 (2)
	Histogram Binning	0.04 (0.04)	0.19 (0.19)	5 (5)	0.05 (0.04)	0.19 (0.19)	2 (3)
Llama-3.1-70B	Isotonic Regression	0.04 (0.04)	0.19 (0.19)	5 (5)	0.05 (0.04)	0.19 (0.19)	2 (3)
	Platt-scaling	0.06 (0.05)	0.19 (0.19)	6 (6)	0.03 (0.03)	0.23 (0.22)	3 (5)

TABLE X: Comparing Scalers for Fine-Grained Confidence Calibration using Minimum Token Probability on DCF-Vul

		EM			EP
Model	Scaler	ECE	$\mathcal{B}$	BC	ECE	$\mathcal{B}$	BC
	Platt-scaling	0.21 (0.10)	0.25 (0.22)	7 (8)	0.26 (0.18)	0.25 (0.22)	5 (7)
	Temperature-scaling	0.09 (0.08)	0.21 (0.21)	4 (5)	0.21 (0.13)	0.23 (0.20)	3 (5)
	Histogram Binning	0.20 (0.08)	0.24 (0.21)	6 (7)	0.23 (0.16)	0.24 (0.21)	5 (6)
CodeLlama-70b	Isotonic Regression	0.20 (0.07)	0.24 (0.21)	7 (7)	0.23 (0.16)	0.24 (0.21)	5 (7)
	Platt-scaling	0.19 (0.12)	0.21 (0.21)	6 (10)	0.20 (0.08)	0.20 (0.17)	7 (8)

TABLE XI: Comparing Scalers for Fine-Grained Confidence Calibration using Minimum Token Probability on CR-Trans

Fig. 10: UMAP Visualisations of Qwen3-Embedding-8B Representations of Inputs w/ CodeLlama-70B Outputs

Fig. 11: Optimal Hyperparameters for Local Platt-Scaling w/ Minimum Token Probability

Fig. 12: Hyperparameter Search for Local Platt-Scaling w/ Minimum Token Probability using CR-Trans Test Samples

Fig. 13: Precision at Abstention Threshold for CodeLlama-70B in DCF-Bug

Fig. 14: Precision at Abstention Threshold for Llama-3.1-70B in DCF-Vul

Fig. 15: Precision at Abstention Threshold for CodeLlama-70B in CR-Trans

Confidence Estimation in Language Models

Confidence estimation in language models measures whether a model’s uncertainty signals predict answer correctness and support reliable decisions.

See all 246 papers

Mar 31, 2026

When Can We Trust LLM Graders? Calibrating Confidence for Automated Assessment

Large Language Models (LLMs) show promise for automated grading, but their outputs can be unreliable. Rather than improving grading accuracy directly, we address a complementary problem: predicting when an LLM grader is likely to be correct. This enables selective automation where high-confidence predictions are processed automatically while uncertain cases are flagged for human review. We compare three confidence estimation methods (self-reported confidence, self-consistency voting, and token probability) across seven LLMs of varying scale (4B to 120B parameters) on three educational datasets: RiceChem (long-answer chemistry), SciEntsBank, and Beetle (short-answer science). Our experiments reveal that self-reported confidence consistently achieves the best calibration across all conditions (avg ECE 0.166 vs 0.229 for self-consistency). Surprisingly, self-consistency remains 38% worse despite requiring 5

\times

the inference cost. Larger models exhibit substantially better calibration though gains vary by dataset and method (e.g., a 28% ECE reduction for self-reported), with GPT-OSS-120B achieving the best calibration (avg ECE 0.100) and strong discrimination (avg AUC 0.668). We also observe that confidence is strongly top-skewed across methods, creating a ``confidence floor'' that practitioners must account for when setting thresholds. These findings suggest that simply asking LLMs to report their confidence provides a practical approach for identifying reliable grading predictions. Code is available here.

Robinson Ferrer, Damla Turgut, Zhongzhou Chen +1

University of Central Florida

Confidence Estimation in Language Models Language Model Calibration Automated Grading Selective Prediction LLM-as-a-Judge

Ask PDF

Jun 30, 2026

An Empirical Study of Security Calibration in Large Language Models for Code

Large Language Models (LLMs) are rapidly transforming software development, yet their use in security-critical contexts raises a key question: do models know when their generated code is insecure? This property, known as calibration, measures whether a model's confidence aligns with the true correctness of its outputs. We present the first large-scale empirical study of security calibration in LLM-generated code. We evaluate GPT-4o-mini, Gemini-2.0-Flash, and Qwen3-Coder-Next across multiple temperature settings on two complementary benchmarks: self-contained security tasks and multi-language repository-level contexts. Our results suggest that overconfidence is prevalent across the evaluated LLMs. Functional calibration is consistently worse than security calibration, suggesting that models estimate security outcomes more reliably than functional correctness, potentially because functional correctness depends on complex execution behavior. We also examine whether calibration-guided automated repair can help remediate vulnerabilities in LLM-generated code, finding only limited improvements while frequently introducing functional regressions. Moreover, we study different mitigation strategies for reducing False Trust, where models assign high confidence to vulnerable code. The results show that although architectural gating improves calibration on controlled benchmarks, calibration deteriorates in realistic repository-level settings, increasing the risk of high-confidence vulnerable outputs.

Mohammed Latif Siddiq, Md. Nafiu Rahman, Joanna C. S. Santos

University of Notre Dame Notre Dame, IN, USA · Brac University Dhaka, Bangladesh

LLM Security Language Model Calibration Secure Code Generation LLM Code Generation

Ask PDF

Oct 15, 2026

Are You Sure Your LLM Is Sure? Repairing and Auditing Confidence

Confidence scores determine whether an LLM answer is accepted, reviewed, or regenerated, yet scalar recalibration can only correct errors visible through the score itself. We introduce RADAR (Repair, Adopt, and Diagnose After Recalibration), a reliability framework that separates confidence repair from deciding whether to deploy the repair and auditing the errors that remain. Under squared loss, we decompose prediction risk into outcome noise, information hidden by the score, and mapping error; restricting repair to order-preserving maps additionally introduces a pooling gap. For repair, we propose Probe-Anchored Isotonic Calibration (PAIC), which learns a monotone map from correctness-probe predictions on unlabeled target prompts. PAIC's excess risk depends on the probe's score-conditional bias and fitting error rather than its overall accuracy. For adoption, we derive a paired procedure with finite-sample control of false improvements. For auditing, we develop a prompt-clustered kernel test that detects feature-aligned residual errors while accounting for dependent answers from the same prompt. Across multiple LLM checkpoints, benchmarks, and distribution shifts, richer predictors can improve confidence estimation, but projecting their information back to a scalar score can lose these gains, and lower prediction risk need not yield better review decisions. Across 47,400 replayed raw-score allocations, paired betting adopts candidates with lower held-out test risk in 42.95% of allocations, versus 15.09% for one-sided empirical Bernstein. RADAR thus provides a principled workflow to repair what a scalar score can express, adopt a repair only when supported by evidence, and diagnose systematic errors that recalibration leaves behind. Code and data will be released upon acceptance.

Arijit Dey, Bhartendu Kumar, Ramya Hebbalaguppe +1

MBZUAI · TCS Research Labs, India · IIT Delhi

Confidence Estimation in Language Models Language Model Calibration Confidence Estimation Probability Calibration LLM Auditing Isotonic Regression

Ask PDF

Abstract

Confidence Estimation in Language Models

Explore similar work

When Can We Trust LLM Graders? Calibrating Confidence for Automated Assessment

An Empirical Study of Security Calibration in Large Language Models for Code

Are You Sure Your LLM Is Sure? Repairing and Auditing Confidence