Believe Your Model: Distribution-Guided Confidence Calibration

Xizhong Yang; Haotian Zhang; Huiming Wang; Mofei Song

cs.LGMar 4, 2026

Believe Your Model: Distribution-Guided Confidence Calibration

Authors: Xizhong Yang, Haotian Zhang, Huiming Wang, Mofei Song

Organizations: Southeast University · Kuaishou Technology

Abstract

Large Reasoning Models have demonstrated remarkable performance with the advancement of test-time scaling techniques, which enhances prediction accuracy by generating multiple candidate responses and selecting the most reliable answer. While prior work has analyzed that internal model signals like confidence scores can partly indicate response correctness and exhibit a distributional correlation with accuracy, such distributional information has not been fully utilized to guide answer selection. Motivated by this, we propose DistriVoting, which incorporates distributional priors as another signal alongside confidence during voting. Specifically, our method (1) first decomposes the mixed confidence distribution into positive and negative components using Gaussian Mixture Models, (2) then applies a reject filter based on positive/negative samples from them to mitigate overlap between the two distributions. Besides, to further alleviate the overlap from the perspective of distribution itself, we propose SelfStepConf, which uses step-level confidence to dynamically adjust inference process, increasing the separation between the two distributions to improve the reliability of confidences in voting. Experiments across 16 models and 5 benchmarks demonstrate that our method significantly outperforms state-of-the-art approaches.

Figures & tables

Figure 1: Overview of DistriVoting . First model the original distribution using GMM, then filter out negative samples, finally reject false positives from the positive distribution.

Model	Method	HMMT2025	GPQA-D	AIME2024	AIME2025	BRUMO2025	Avg.
DeepSeek-R1-8B	SC	69.11 $\pm$ 0.23	67.50 $\pm$ 0.07	86.67 $\pm$ 0.00	80.36 $\pm$ 0.13	93.07 $\pm$ 0.13	73.09 $\pm$ 0.06
	WSC	69.69 $\pm$ 0.18	67.65 $\pm$ 0.04	86.67 $\pm$ 0.00	80.78 $\pm$ 0.13	93.33 $\pm$ 0.00	73.30 $\pm$ 0.03
	BoN	70.05 $\pm$ 0.34	67.91 $\pm$ 0.17	90.52 $\pm$ 0.11	77.34 $\pm$ 0.30	92.24 $\pm$ 0.19	73.43 $\pm$ 0.13
	MoB-Adaptive	76.98 $\pm$ 0.76	68.70 $\pm$ 0.23	89.95 $\pm$ 0.84	84.53 $\pm$ 0.37	93.33 $\pm$ 0.39	75.30 $\pm$ 0.33
	WSC-Top50	73.80 $\pm$ 0.44	68.58 $\pm$ 0.04	90.00 $\pm$ 0.18	82.55 $\pm$ 0.34	93.33 $\pm$ 0.00	74.75 $\pm$ 0.07
	WSC-GMM	82.50 $\pm$ 0.49	69.58 $\pm$ 0.13	93.13 $\pm$ 0.15	83.91 $\pm$ 0.99	93.59 $\pm$ 0.08	76.64 $\pm$ 0.11

Table 1: Main results of SelfStepConf (SSC) and DistriVoting across benchmarks. Budget is set to 128 with 64 repetitions. SC denotes Self-Consistency (Majority Voting), WSC represents Weighted Self-Consistency (Weighted MajVoting), BoN indicates Best of N. The * marks answers generated using SSC, bold and underline respectively indicate the optimal and suboptimal results, and

\pm

represents the variation range of multiple repetitions. Additional experiments and ablation provided in Table 8 , Table 9 , Table 10 and Table 11 in §G.1 .

Method	Top50	K-Means	MeanShift	GMM
Acc (%)	74.47	75.29	76.55	77.60
WAcc (%)	74.61	75.32	76.57	77.68
AUROC ( $\uparrow$ )	0.5117	0.5204	0.5158	0.5831
Voting Acc (%)	75.10	75.19	75.50	76.95
Predict Acc (%)	53.64	56.19	59.34	60.46
Predict Time (ms/it)	0.0935	0.6014	1.8492	0.3369

Table 2: Ablation of clustering methods using DeepSeek-R1-8B, sampling 128 trajectories/query via DistriVoting (64 repeats). Complete results provided in Table 12 of §G.2 .

Method	8	16	32	64	128	256
WSC-Top50	73.17	73.86	74.56	74.62	74.75	74.79
WSC-GMM	73.18	74.53	75.83	76.30	76.64	77.04
WSC-GMM*	73.22	74.68	75.84	76.78	77.24	77.56
DIS-Top50	72.95	73.72	74.64	75.71	76.32	76.75
DIS-GMM	73.12	74.73	75.71	76.34	76.95	77.53
DIS-GMM*	73.18	74.74	75.90	76.87	77.84	78.18

Table 3: Ablation results for varying Budget using DeepSeek-R1-8B, sampling B trajectories/question (64 repeats). * denotes SSC -generated. Complete results provided in Table 13 of §G.3 .

Metric	Benchmark	Stage I	Stage II	Stage III
Acc	HMMT2025	60.36	76.71	77.43
	GPQA-D	64.54	71.69	75.86
	AIME2024	86.83	93.90	94.08
	AIME2025	79.92	87.79	88.75
	BRUMO2025	81.22	91.06	91.34
	Avg.	69.27	77.60	80.41

Table 4: Effectiveness Analysis of DistriVoting using DeepSeek-R1-8B, sampling 128 trajectories/question (64 repeats). Complete results are provided in Table 14 of §G.4 .

Figure 2: Confidence distribution (HMMT2025) via DeepSeek-R1-8B, sampling 512 trajectories/query. Up: Confidence frequency histogram; Down: Accuracy curves for positive/negative intervals. Complete results are provided in Table 15 of §G.5 .

Figure 3: Optimal top-threshold traversal results using Qwen3-8B, sampling 256 responses/question (64 repeats). Complete results are provided in Figure 14 of §G.6 .

Figure 4: Visualizing answer distribution as Gaussian components in GMM using DeepSeek-R1-8B, sampling 512 responses/query on HMMT2025. Complete results provided in Figure 15 of §G.7 .

Figure 5: Comparing trajectory-level confidence between SSC and BasicInference using Qwen3-14B-NonThinking, sampling 512 responses/query. Complete results provided in Figure 16 of §G.8 .

Figure 6: Pass@K comparison between SSC and BasicInference using different models, sampling 256 responses/query on GPQA-D (64 repeats). Complete results provided in Figure 17 of §G.9 .

Figure 7: Pass@1 comparison between SSC and BasicInference using different models, sampling 1 response/query (64 repeats). Complete results are provided in Table 17 of §G.10 .

Figure 8: Confidence and token count trends of the SSC and BasicInference. The trajectory is generated by DeepSeek-R1-8B on the 188th question of GPQA-D at temperature=0 .

Benchmark	BasicInference			SelfStepConf
Benchmark	Step	Token	Confs.	Step	Token	Confs.
HMMT2025	154.00	28266.73	17.03	154.40	28604.80	17.20
GPQA-D	30.80	9560.65	13.31	29.27	9411.71	13.33
AIME2024	88.63	21239.20	17.96	83.33	20733.97	17.92
AIME2025	123.23	26673.87	17.87	128.10	26280.73	17.91
BRUMO2025	135.40	23137.50	17.44	124.50	22291.60	17.45

Table 5: Response length changes between SSC and BasicInference using DeepSeek-R1-8B at temperature=0 , sampling 1 response/query. Complete results provided in Table 18 of §G.11 .

Appendix figures & tables29 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 9 — Figure 9: Parameter sensitivity analysis of $\alpha$ under different metrics. Using DeepSeek-R1-8B to generate 128 trajectories per query, with experiments repeated 64 times. Complete results are provided in Table 19 of §G.12 .

Panel of Figure 10 — Figure 10: Parameter sensitivity analysis of $\delta$ under different metrics. Using DeepSeek-R1-8B to generate 128 trajectories per query, with experiments repeated 64 times. Complete results are provided in Table 20 of §G.13 .

Figure 11: Parameter sensitivity analysis of $\alpha\times\delta$ under different metrics. Using DeepSeek-R1-8B to generate 128 trajectories per query of HMMT2025, with experiments repeated 64 times. Complete results are provided in Table 21 and Table 22 of §G.14 .

Panel of Figure 12 — Figure 12: Parameter sensitivity analysis of $N_{\mathcal{C}}$ under different voting methods. Using DeepSeek-R1-8B to generate 128 trajectories per query for voting, with experiments repeated 64 times. Complete results are provided in Table 16 of §G.6 and Table 23 , Table 24 of §G.15 .

Benchmark	“wait”	“Wait”	“Hmm”	“Alternatively”
HMMT2025	59.22	58.91	59.53	58.75
GPQA-D	62.71	62.34	62.46	62.98
AIME2024	82.86	85.45	84.85	83.02
AIME2025	75.07	73.80	73.00	72.97
BRUMO2025	79.84	79.27	78.23	78.18
Avg.	67.06	66.87	66.78	66.85

Appendix

Table 6: Ablation study results of different reflection tokens. Using DeepSeek-R1-8B to generate 64 trajectories for each question across 5 benchmarks, computing the average pass@1.

Benchmark	“\n\n”	“\n”	HET	Fixed Window
Benchmark	“\n\n”	“\n”	HET	256	512	1024	2048
HMMT2025	59.22	58.85	58.91	58.39	58.75	58.06	57.76
GPQA-D	62.71	61.43	57.37	56.69	56.32	60.11	57.07
AIME2024	82.86	82.76	81.61	82.24	82.97	82.76	81.41
AIME2025	75.07	73.23	70.68	73.70	73.54	73.02	70.83
BRUMO2025	79.84	78.75	78.59	79.01	77.76	78.28	77.97

Appendix

Table 7: Ablation study results of different step splitting methods. Using DeepSeek-R1-8B to generate 64 trajectories for each question across 5 benchmarks, computing the average pass@1 . HET denotes High-Entropy Token.

Figure 13: Ablation study results of different step splitting methods. Using DeepSeek-R1-8B to generate 64 trajectories for each question across 5 benchmarks. HET denotes High-Entropy Token. Complete results are provided in Table 25 of §G.16 .

Model	Benchmark	Pass@1	SC	WSC	BoN	MoB	WSC			DIS
Model	Benchmark	Pass@1	SC	WSC	BoN	MoB	Top50	GMM	GMM*	Top50	GMM	GMM*
Qwen3-0.6B	HMMT2025	8.89	18.97	20.37	17.24	17.22	17.54	14.51	16.65	17.98	20.74	14.21
	GPQA-D	21.31	26.62	26.61	28.78	27.72	24.09	28.52	27.83	26.73	27.54	29.21
	AIME2024	10.73	17.08	17.55	13.44	23.95	29.58	23.84	29.74	26.20	28.45	27.22
	AIME2025	16.78	30.00	30.00	29.95	33.45	33.39	33.47	33.33	30.24	33.49	35.21
	BRUMO2025	16.93	23.13	23.33	23.39	30.25	29.79	28.83	28.96	30.45	29.66	27.11

Appendix

Table 8: All main results of SelfStepConf (SSC) and DistriVoting on various benchmarks using Thinking Mode of Qwen3-Series models. Set Budget to 128 and repeat 64 times. SC denotes Self-Consistency (i.e., Majority Voting), WSC represents Weighted Self-Consistency (i.e., Weighted Majority Voting), BoN represents Best of N, and * indicates answers generated using the SSC approach .

Model	Benchmark	Pass@1	SC	WSC	BoN	MoB	WSC			DIS
Model	Benchmark	Pass@1	SC	WSC	BoN	MoB	Top50	GMM	GMM*	Top50	GMM	GMM*
Qwen3-0.6B	HMMT2025	0.97	3.34	3.83	3.50	6.91	6.95	4.94	6.94	5.27	6.90	5.70
	GPQA-D	23.14	23.39	23.17	23.79	22.12	20.45	23.92	24.09	24.24	24.32	24.94
	AIME2024	2.66	3.33	3.33	3.39	3.34	3.41	4.67	6.77	6.62	3.33	5.36
	AIME2025	2.45	6.67	6.67	3.44	7.83	6.61	7.51	9.96	10.03	9.69	8.40
	BRUMO2025	8.12	16.51	16.67	16.46	16.59	17.32	20.53	21.34	16.96	19.74	20.06

Appendix

Table 9: All main results of SelfStepConf (SSC) and DistriVoting on various benchmarks using NonThinking Mode of Qwen3-Series models. Set Budget to 128 and repeat 64 times. SC denotes Self-Consistency (i.e., Majority Voting), WSC represents Weighted SC (i.e., Weighted Majority Voting), BoN represents Best of N, and * indicates answers generated using the SSC approach .

SelfStepConf	DistriVoting			Voting Acc (%)
SelfStepConf	GMM Filter	Reject Filter	HierVoting	HMMT2025	GPQA-D	AIME2024	AIME2025	BRUMO2025	Avg.
✗	✗	✗	✗	73.80 $\pm$ 0.44	68.58 $\pm$ 0.04	90.00 $\pm$ 0.18	82.55 $\pm$ 0.34	93.33 $\pm$ 0.00	74.75 $\pm$ 0.07
	✓	✗	✗	82.50 $\pm$ 0.49	69.58 $\pm$ 0.13	93.13 $\pm$ 0.15	83.91 $\pm$ 0.99	93.59 $\pm$ 0.08	76.64 $\pm$ 0.11
	✗	✓	✗	73.91 $\pm$ 0.39	68.47 $\pm$ 0.15	89.90 $\pm$ 0.08	82.86 $\pm$ 0.31	93.33 $\pm$ 0.00	74.71 $\pm$ 0.11
	✗	✗	✓	79.95 $\pm$ 0.23	69.36 $\pm$ 0.07	93.07 $\pm$ 0.05	84.38 $\pm$ 0.16	93.44 $\pm$ 0.03	76.28 $\pm$ 0.08
	✓	✓	✗	83.07 $\pm$ 0.26	69.31 $\pm$ 0.23	93.28 $\pm$ 0.05	86.77 $\pm$ 0.29	93.75 $\pm$ 0.13	76.82 $\pm$ 0.14

Appendix

Table 10: Complete ablation experiments on the method components in the main experiments using DeepSeek-R1-8B , sampling 128 trajectories/query (64 repeats). The naive version of the GMM Filter is the Top50 Filter, the naive version of HierVoting is Weighted-SC, and the naive version of the Reject Filter is no filtering. – indicates no filtering at all, i.e., directly using all trajectories for voting.

SelfStepConf	DistriVoting			Voting Acc (%)
SelfStepConf	GMM Filter	Reject Filter	HierVoting	HMMT2025	GPQA-D	AIME2024	AIME2025	BRUMO2025	Avg.
✗	✗	✗	✗	63.96 $\pm$ 0.29	72.03 $\pm$ 0.17	88.28 $\pm$ 0.31	76.93 $\pm$ 0.26	92.71 $\pm$ 0.00	75.22 $\pm$ 0.12
	✓	✗	✗	64.84 $\pm$ 0.23	72.42 $\pm$ 0.11	88.65 $\pm$ 0.31	79.22 $\pm$ 0.34	92.71 $\pm$ 0.03	75.79 $\pm$ 0.10
	✗	✓	✗	63.85 $\pm$ 0.23	72.01 $\pm$ 0.15	88.28 $\pm$ 0.36	76.98 $\pm$ 0.29	93.33 $\pm$ 0.00	75.26 $\pm$ 0.10
	✗	✗	✓	64.79 $\pm$ 0.36	72.29 $\pm$ 0.25	88.70 $\pm$ 0.49	79.11 $\pm$ 0.31	93.28 $\pm$ 0.05	75.76 $\pm$ 0.18
	✓	✓	✗	65.00 $\pm$ 0.21	72.51 $\pm$ 0.16	88.54 $\pm$ 0.44	78.96 $\pm$ 0.55	93.33 $\pm$ 0.00	75.88 $\pm$ 0.09

Appendix

Table 11: Complete ablation experiments on the method components in the main experiments using Qwen3-32B , sampling 128 trajectories/query (64 repeats). The naive version of the GMM Filter is the Top50 Filter, the naive version of HierVoting is Weighted-SC, and the naive version of the Reject Filter is no filtering. – indicates no filtering at all, i.e., directly using all trajectories for voting.

Method	Metric	HMMT2025	GPQA-D	AIME2024	AIME2025	BRUMO2025	Avg.
Top50	Acc (%)	71.94	68.65	91.24	84.40	88.63	74.47
	WAcc (%)	72.39	68.71	91.41	84.57	89.02	74.61
	AUROC ( $\uparrow$ )	0.5234	0.4921	0.5358	0.5129	0.6037	0.5117
	Voting Acc (%)	78.28	67.85	93.18	83.44	93.33	75.10
	Predict Acc (%)	56.77	52.66	53.62	53.98	56.68	53.64
	Predict Time (ms/it)	0.0300	0.1110	0.0961	0.0884	0.0443	0.0935

Appendix

Table 12: Complete ablation study results of different correctness prediction methods. Using DeepSeek-R1-8B to generate 128 trajectories for each question across 5 benchmarks with DistriVoting , repeated 64 times.

Budget	Method	HMMT2025	GPQA-D	AIME2024	AIME2025	BRUMO2025	Avg.
8	WSC-Top50	71.09 $\pm$ 1.30	67.37 $\pm$ 0.87	89.06 $\pm$ 0.34	80.52 $\pm$ 0.16	90.31 $\pm$ 0.34	73.17 $\pm$ 0.51
	WSC-GMM	72.71 $\pm$ 1.98	66.99 $\pm$ 0.26	89.53 $\pm$ 0.52	81.25 $\pm$ 0.49	90.10 $\pm$ 0.52	73.18 $\pm$ 0.25
	WSC-GMM*	73.07 $\pm$ 0.65	67.18 $\pm$ 0.89	89.38 $\pm$ 0.42	80.31 $\pm$ 0.42	90.00 $\pm$ 0.49	73.22 $\pm$ 0.56
	DIS-Top50	70.78 $\pm$ 1.30	67.14 $\pm$ 0.69	88.75 $\pm$ 0.39	80.47 $\pm$ 0.00	90.16 $\pm$ 0.34	72.95 $\pm$ 0.31
	DIS-GMM	72.71 $\pm$ 2.11	66.94 $\pm$ 0.41	89.27 $\pm$ 0.57	81.20 $\pm$ 0.37	90.05 $\pm$ 0.42	73.12 $\pm$ 0.24
	DIS-GMM*	72.97 $\pm$ 0.60	67.16 $\pm$ 1.22	89.32 $\pm$ 0.44	80.21 $\pm$ 0.34	89.90 $\pm$ 0.50	73.18 $\pm$ 0.78

Appendix

Table 13: Complete ablation study results of different Budget. Using DeepSeek-R1-8B to generate Budget trajectories for each question across 5 benchmarks with DistriVoting , repeated 64 times. * indicates answers generated using the SelfStepConf approach.

Model	Metric	Stage	HMMT2025	GPQA-D	AIME2024	AIME2025	BRUMO2025	Avg.
DeepSeek-R1-8B	Acc	I	60.36	64.54	86.83	79.92	81.22	69.27
		II	76.71	71.69	93.90	87.79	91.06	77.60
		III	77.43	75.86	94.08	88.75	91.34	80.41
	WAcc	I	61.59	64.73	87.40	80.70	82.31	69.74
		II	76.92	71.72	94.00	87.86	91.24	77.68
		III	77.63	75.89	94.17	88.83	91.52	80.48

Appendix

Table 14: Complete results of the proportion of correct samples across different stages of the voting process. Different models generate Budget=128 responses for each query during voting, with experiments repeated 64 times and results averaged.

Model	Benchmark	BasicInference		SelfStepConf		$\Delta$ (%)
Model	Benchmark	AUROC	WAcc	AUROC	WAcc	AUROC	WAcc
DeepSeek-R1-8B	HMMT2025	0.9002	67.28	0.8957	67.72	-0.50	+0.65
	GPQA-D	0.7167	60.59	0.7699	61.29	+7.42	+1.16
	AIME2024	0.8833	86.56	0.8881	86.96	+0.54	+0.46
	AIME2025	0.8294	82.62	0.8478	82.87	+2.22	+0.30
	BRUMo2025	0.8198	84.88	0.8304	85.40	+1.29	+0.61

Appendix

Table 15: Complete results of the confidence separation of the content generated by BasicInference and SelfStepConf on different models.

Panel of Figure 14 — Figure 14: Optimal fixed top-threshold traversal results for filter voting across different benchmarks. Using different models to generate 256 responses for each query, repeated 64 times. Accuracy denotes WSC-TopK. Complete results are provided in Table 16 .

Model	Benchmark	Top-Threshold
Model	Benchmark	10	20	30	40	50	60	70	80	90
DeepSeek-R1-8B	HMMT2025	81.88	80.83	81.15	80.52	79.64	80.63	80.00	79.79	78.91
	GPQA-D	70.24	70.04	69.75	69.48	69.31	69.31	68.92	68.79	68.62
	AIME2024	92.92	93.18	93.18	93.07	93.07	92.97	92.92	92.97	92.86
	AIME2025	85.47	84.64	84.27	84.43	84.38	84.27	84.22	84.06	83.96
	BRUMO2025	93.65	93.54	93.49	93.39	93.39	93.44	93.39	93.33	93.33

Appendix

Table 16: Complete WSC-TopK results of optimal fixed top-threshold traversal results for filter voting across different benchmarks. Using different models to generate 256 responses for each query, repeated 64 times.

Figure 15: Distribution visualization of answers as Gaussian components in GMM using DeepSeek-R1-8B with 512 repeated sampling for each query on HMMT2025. Individual visualization of the top 30 most frequent answers.

Panel of Figure 16 — Figure 16: Complete results of trajectory-level confidence comparison between SSC and BasicInference across different benchmarks. Using different models to generate 512 responses for each query, and all responses across the entire benchmark are sorted and visualized by confidence scores.

Panel of Figure 17 — Figure 17: Complete results of Pass@K comparison between SSC and BasicInference under different sampling numbers. Different models generate 256 responses on all Benchmark, repeated 64 times.

Method	Model	HMMT2025	GPQA-D	AIME2024	AIME2025	BRUMO2025	Avg.
BaseModel	Qwen2.5-Math-7B	0.00	12.89	1.56	0.47	2.03	8.41
	Qwen3-0.6B*	0.97	23.14	2.66	2.45	8.12	15.75
	Qwen3-0.6B	8.89	21.31	10.73	16.78	16.93	18.30
	Llama3.1-8B-Instruct	0.10	28.54	5.31	0.73	3.44	18.67
	DeepSeek-R1-7B	20.31	12.47	41.04	30.01	40.07	20.16
	Qwen3-1.7B*	5.68	28.23	12.92	9.32	17.40	21.85

Appendix

Table 17: Complete results of Pass@1 comparison between SSC and BasicInference across different models and benchmarks. Results averaged over 64 independent sampling per query.

Model	Benchmark	BasicInference				SelfStepConf
Model	Benchmark	Step	Token	Confs.	Time	Step	Token	Confs.	Time
DeepSeek-R1-8B	HMMT2025	154.00	28266.73	17.03	396.07	154.40	28604.80	17.20	417.11
	GPQA-D	30.80	9560.65	13.31	124.27	29.27	9411.71	13.33	127.17
	AIME2024	88.63	21239.20	17.96	289.66	83.33	20733.97	17.92	295.24
	AIME2025	123.23	26673.87	17.87	374.35	128.10	26280.73	17.91	382.20
	BRUMO2025	135.40	23137.50	17.44	321.32	124.50	22291.60	17.45	318.73

Appendix

Table 18: Complete results of response length changes between SSC and BasicInference. Results are obtained at temperature=0 , generating one response for each query in every benchmark, with values averaged across all queries within each benchmark (Time: ms/it).

Accuracy Metric
Metric	Benchmark	$\delta$
Metric	Benchmark	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8	0.9
Pass@1	HMMT2025	58.65	58.44	58.39	58.49	58.53	58.32	58.69	59.22	57.92
	GPQA-D	61.87	61.90	61.96	62.00	62.03	61.79	60.54	62.71	61.70
	AIME2024	83.65	83.39	83.07	83.47	83.45	82.66	83.29	82.86	82.34
	AIME2025	73.13	73.54	73.49	73.39	71.65	74.61	74.23	75.07	72.86

Appendix

Table 19: Complete of sensitivity analysis of

\alpha

using DeepSeek-R1-8B, sampling 128 trajectories/query (64 repeats), setting

\delta=0.8

Accuracy Metric
Metric	Benchmark	$\delta$
Metric	Benchmark	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8	0.9
Pass@1	HMMT2025	58.96	58.96	58.96	59.13	59.17	57.92	57.86	59.22	58.28
	GPQA-D	61.77	61.64	61.67	61.73	61.65	61.56	61.71	62.71	61.88
	AIME2024	82.66	82.66	82.66	82.71	82.45	82.71	82.86	82.86	82.97
	AIME2025	73.02	73.02	73.02	73.02	73.13	73.80	72.24	75.07	73.80

Appendix

Table 20: Complete of sensitivity analysis of

\delta

using DeepSeek-R1-8B, sampling 128 trajectories/query(64 repeats), setting

\alpha=0.8

Metric	$\alpha$	$\delta$
Metric	$\alpha$	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8	0.9
Steps Count	0.1	149.39	149.39	149.39	149.34	150.66	150.36	148.73	151.20	160.66
	0.2	149.39	149.39	149.39	149.39	150.15	150.25	149.21	150.11	158.18
	0.3	149.39	149.39	149.39	149.39	150.21	150.61	149.34	148.40	157.68
	0.4	149.39	149.39	149.39	149.65	150.21	150.67	149.60	149.17	158.09
	0.5	149.39	149.39	149.39	149.65	150.21	151.46	151.98	148.97	158.97

Appendix

Table 21: Complete results of parameter sensitivity analysis of

\alpha\times\delta

on HMMT2025 using DeepSeek-R1-8B, sampling 128 trajectories/query (64 repeats). Evaluate using Analysis Metric.

Metric	$\alpha$	$\delta$
Metric	$\alpha$	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8	0.9
Pass@1	0.1	58.96	58.96	58.96	58.96	59.17	58.54	59.22	58.65	58.13
	0.2	58.96	58.96	58.96	58.96	58.49	58.54	59.06	58.44	58.96
	0.3	58.96	58.96	58.96	58.96	58.49	58.59	59.22	58.39	57.76
	0.4	58.96	58.96	58.96	58.91	58.49	58.59	58.18	58.49	57.24
	0.5	58.96	58.96	58.96	58.91	58.49	58.70	58.28	57.97	57.55

Appendix

Table 22: Complete results of parameter sensitivity analysis of

\alpha\times\delta

on HMMT2025 using DeepSeek-R1-8B, sampling 128 trajectories/query (64 repeats). Evaluate using Accuracy Metric.

Method	Benchmark	1	2	3	4	5	6	7	8	9	10
DIS-Top50	HMMT2025	74.17	78.39	80.63	80.31	80.36	79.95	81.09	79.53	80.89	79.64
	GPQA-D	68.55	70.08	69.74	69.65	69.55	69.59	69.44	69.48	69.41	69.31
	AIME2024	89.38	92.08	93.18	93.18	93.18	93.07	93.13	93.07	93.13	93.07
	AIME2025	82.45	84.27	84.27	84.06	84.43	84.79	84.38	84.22	84.38	84.38
	BRUMO2025	93.33	93.96	93.65	93.70	93.70	93.59	93.44	93.44	93.49	93.39
	Avg.	74.70	76.53	76.60	76.50	76.48	76.48	76.45	76.30	76.42	76.22

Appendix

Table 23: Complete results of parameter sensitivity analysis of

N_{\mathcal{C}}

under different voting methods. Using DeepSeek-R1-8B to generate 128 trajectories per query for voting, with experiments repeated 64 times (

N_{\mathcal{C}}\in[1,10]

Method	Benchmark	11	12	13	14	15	16	17	18	19	20
DIS-Top50	HMMT2025	80.05	80.42	80.31	80.63	80.36	80.31	79.64	80.31	80.05	79.64
	GPQA-D	69.28	69.55	69.47	69.54	69.33	69.53	69.36	69.35	69.44	69.28
	AIME2024	93.02	93.07	93.02	93.13	93.07	92.97	92.97	92.92	92.81	92.76
	AIME2025	84.32	84.22	84.43	84.11	83.91	83.59	83.96	84.01	83.54	83.39
	BRUMO2025	93.39	93.39	93.33	93.39	93.39	93.28	93.33	93.28	93.33	93.33
	Avg.	76.23	76.42	76.38	76.43	76.25	76.33	76.19	76.25	76.23	76.07

Appendix

Table 24: Complete results of parameter sensitivity analysis of

N_{\mathcal{C}}

under different voting methods. Using DeepSeek-R1-8B to generate 128 trajectories per query for voting, with experiments repeated 64 times (

N_{\mathcal{C}}\in[11,20]

Metric	Benchmark	“\n\n”	“\n”	HET	Fixed Window
Metric	Benchmark	“\n\n”	“\n”	HET	256	512	1024	2048
Steps Count	HMMT2025	140.14	148.97	145.93	146.54	149.36	149.27	148.54
	GPQA-D	31.13	31.27	42.70	27.04	41.31	34.45	41.23
	AIME2024	109.60	113.68	108.24	113.79	114.13	114.28	108.59
	AIME2025	127.08	129.03	123.41	127.02	128.76	130.05	123.92
	BRUMO2025	134.88	139.24	133.23	136.11	136.73	140.19	134.80

Appendix

Table 25: Complete ablation study results of different step splitting methods. Using DeepSeek-R1-8B to generate 64 trajectories for each question across 5 benchmarks, computing the Analysis Metrics . HET denotes High-Entropy Token.

Explore similar work

Calibration, Not Answer Selection: Distilling Internal Confidence in Reasoning Models
Sep 27, 2026Yadong Xi, Rongsheng Zhang, Tangjie Lv +2Confidence Estimation in Language Models Language Model Calibration
NetEase · Amazon · Singapore University of Technology and Design
CALIBER: Calibrating Confidence Before and After Reasoning in Language Models
Jun 23, 2026Conor Finlay, Joshua Kurien, Saurabh Dash +2Confidence Estimation in Language Models Language Model Calibration
1Cohere Labs
On the Pitfalls of Verbalized Confidence Priors for Calibrating Large Reasoning Models
Sep 26, 2026Shuoyuan Wang, Beier Luo, Hao Zeng +5Language Model Calibration Confidence Estimation in Language Models
Southern University of Science and Technology · Nanyang Technological University · University of Electronic Science and Technology of China +2

Sep 27, 2026J/K move · Enter open · S save

Calibration, Not Answer Selection: Distilling Internal Confidence in Reasoning Models

Yadong Xi, Rongsheng Zhang, Tangjie Lv, Ziyang Luo+1

NetEase · Amazon · Singapore University of Technology and Design

Confidence Estimation in Language Models Language Model Calibration Language Model Distillation LLM Uncertainty Estimation

Open paper AskPDF arXiv

Reinforcement learning with binary correctness rewards trains correctness, not calibrated confidence. The confidence that reasoning models verbalize is systematically overconfident, and the problem is not merely one of scale: verbalized confidence tracks how willing a model is to commit to an answer, not how likely the answer is to be right. Post-hoc rescaling therefore fits one distribution but rarely transfers. We look inside the model instead. On factual question answering, a linear probe on the hidden state between the chain of thought and the answer is substantially better calibrated: its expected calibration error is 5 to 38 times lower than that of the verbalized score across four benchmarks and two model families. However, when used to pick among N sampled answers, that same probe nearly ties majority voting yet falls far short of the oracle. Internal states answer "how certain am I" well and "which answer is right" poorly, so the signal should be reported as a confidence rather than used to select answers. As a result, we introduce probe-guided self-distillation (Probe-SD): score a model's own sampled traces with the probe, overwrite the confidence each trace states, and finetune the base checkpoint of the same family, so nothing but the model itself remains at test time. On Qwen3-14B, Probe-SD cuts ECE from 0.178 to 0.024 in-domain and from 0.542 to 0.113 out-of-domain, where it also beats post-hoc recalibration and self-consistency distillation. The resulting confidence is well-calibrated and useful for weighted voting, behaviors previously attributed to online RL, here obtained with supervised finetuning alone.

Abstract

Explore similar work

Calibration, Not Answer Selection: Distilling Internal Confidence in Reasoning Models

CALIBER: Calibrating Confidence Before and After Reasoning in Language Models

On the Pitfalls of Verbalized Confidence Priors for Calibrating Large Reasoning Models