cs.LGApr 9, 2026
SaveLess Data Approximates More: Earning Faithful Confidence in High-Stakes Domains
Organizations: National University of Singapore, Singapore · Singapore Management University, Singapore
Abstract
Large language models are increasingly deployed in high-stakes domains, where confident yet incorrect inferences may cause severe real-world harm, bringing the long-overlooked issue of confidence faithfulness to the forefront. A promising solution jointly optimizes unsupervised Reinforcement Learning from Internal Feedback (RLIF) with reasoning-trace-guided Reasoning Distillation (RD), yet it faces three persistent challenges, namely the scarcity of high-quality training corpora, factually unwarranted overconfidence, and erroneous updates amplified by indiscriminate fusion. Inspired by how human confidence accumulates from uncertainty to certainty, we propose Progressive Reasoning Gain (PRG) to measure whether reasoning steps progressively strengthen confidence in the final answer. Building on PRG, we introduce HyTuning, a hybrid post-training framework that adaptively reweights RD and RLIF, using scarce supervised reasoning traces as a stable anchor while exploiting abundant unlabeled queries for scalability. Experiments on several domain-specific and general benchmarks demonstrate that HyTuning improves accuracy while achieving confidence faithfulness under limited supervision, supporting a practical ``Less Data Approximates More'' effect. Our code will be released upon acceptance.
Figures & tables
Figure 1: Comparison of the tolerance for confidently-wrong inferences between general domains and high-stakes domains, where the latter also suffer from scarce training corpora.
Figure 2: A toy example illustrating how our HyTuning goes beyond RD (accuracy-focused) and RLIF (confidence-focused) to jointly improve accuracy and confidence faithfulness.
Figure 3: Overview of our proposed HyTuning.
| Models | Training w/ Partial Supervision | Training w/ Full Supervision | ||||||||
| Domain-specific Bench. | Gen. Bench. | Aver. | Domain-specific Bench. | Gen. Bench. | Aver. | |||||
| ASB | CSEB | CyberM | MMLU | ASB | CSEB | CyberM | MMLU | |||
| Qwen2.5-3B | 0.269 | 0.318 | 0.556 | 0.449 | 0.398 | 0.269 | 0.318 | 0.556 | 0.449 | 0.398 |
| SFT ( Ouyang et al., 2022 ) | 0.102 | 0.133 | 0.260 | 0.256 | 0.188 | 0.305 | 0.286 | 0.554 | 0.447 | 0.398 |
| RD ( Hsieh et al., 2023 ) | 0.460 | 0.328 | 0.666 | 0.457 | 0.478 | 0.516 | 0.435 | 0.748 | 0.529 | 0.557 |
| GRPO ( Shao et al., 2024 ) | 0.437 | 0.348 | 0.660 | 0.521 | 0.492 | 0.499 | 0.449 | 0.770 | 0.562 | 0.570 |
Table 1: Performance comparison on domain-specific and general benchmarks.
| Models | SAR | SC | ||||
| 5% | 10% | 20% | 5% | 10% | 20% | |
| Backbone | 65.04 | 60.41 | 51.20 | 45.30 | 47.04 | 45.94 |
| RD | 23.76 | 28.30 | 31.32 | 28.62 | 31.98 | 33.83 |
| INTUITOR | 28.30 | 25.42 | 24.87 | 24.94 | 19.63 | 18.22 |
| GRPO | 36.67 | 39.99 | 38.27 | 20.35 | 30.72 | 34.47 |
| OPSD | 74.69 | 70.19 | 68.89 | 59.70 | 58.27 | 57.37 |
Table 2: Confidence faithfulness (hAURC ) under SAR and SC at {5%,10%,20%}. Shading ranks the three regions per method and measure: 1st , 2nd , and 3rd .
Figure 4: Ablation results of HyTuning, where each axis is scaled independently.
Figure 5: Impact of supervised data volume and quality on HyTuning across four benchmarks, where the dashed lines denote RD and INTUITOR trained on the complete corpus.
| Models | ASB | CSEB | CyberM | Aver. |
| Qwen2.5-7B | 0.495 | 0.460 | 0.708 | 0.554 |
| + HyTuning | 0.514 | 0.480 | 0.788 | 0.594 |
| Qwen2.5-14B | 0.541 | 0.551 | 0.818 | 0.637 |
| + HyTuning | 0.555 | 0.602 | 0.868 | 0.675 |
| LLaMA3.2-3B | 0.249 | 0.195 | 0.636 | 0.360 |
| + HyTuning | 0.372 | 0.278 | 0.650 | 0.433 |
Table 3: Performance comparison on three domain-specific benchmarks.
| Models | FinQA | FinRea | MMLU | Average |
| Qwen2.5-3B | 0.195 | 0.092 | 0.449 | 0.245 |
| RD | 0.269 | 0.076 | 0.494 | 0.281 |
| INTUITOR | 0.385 | 0.302 | 0.517 | 0.401 |
| HyTuning | 0.471 | 0.383 | 0.561 | 0.472 |
Table 4: Performance comparison on financial and general benchmarks.
Figure 6: Comparison of accuracy and self-certainty over training, where Partial and Full denote the 100-sample and the complete supervised corpus.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
| Hyperparameter | Value |
| Optimizer | AdamW |
| 0.9 | |
| 0.999 | |
| Learning rate | |
| Learning rate scheduler | Cosine |
| Warmup ratio | 0.1 |
Table 5: Hyperparameters used for HyTuning.
| PRG bin | B1 (lowest) | B2 | B3 | B4 | B5 (highest) |
| CS ( ) | 1.513 | 1.750 | 1.788 | 1.838 | 1.875 |
| DC ( ) | 1.588 | 1.863 | 1.913 | 1.875 | 1.913 |
| CS=0 ( ) | 30% | 10% | 10% | 5% | 0% |
| DC=0 ( ) | 25% | 5% | 5% | 5% | 0% |
Table 6: Human evaluation of 100 reasoning traces across five PRG bins on CyberMetric. CS and DC are averaged over four experts, CS=0 and DC=0 report the fraction of traces receiving a zero from at least one expert on the corresponding criterion.
| Variants | Domain-specific Benchmarks | Gen. Bench. | Average | ||
| ASB | CSEB | CyberM | MMLU | ||
| HyTuning | 0.489 | 0.391 | 0.732 | 0.552 | 0.541 |
| HyTuning - RLIF | 0.460 | 0.328 | 0.666 | 0.457 | 0.478 |
| HyTuning - RD | 0.476 | 0.321 | 0.700 | 0.476 | 0.494 |
| HyTuning - AR | 0.463 | 0.309 | 0.716 | 0.464 | 0.489 |
| HyTuning - PRG | 0.464 | 0.335 | 0.712 | 0.552 | 0.516 |
Table 7: Performance comparison of ablations on cybersecurity-related and general benchmarks.
Figure 7: Comparison of PRG score and transitional word frequency over training, where Partial and Full denote the 100-sample and the complete supervised corpus.
| Models | Domain-specific Bench. | Gen. Bench. | Ave. | ||
| ASB | CSEB | CyberM | MMLU | ||
| Qwen2.5-3B | 0.269 | 0.318 | 0.556 | 0.449 | 0.398 |
| INTUITOR | 0.476 | 0.321 | 0.700 | 0.476 | 0.494 |
| HyTuning w/ INTUITOR | 0.489 | 0.391 | 0.732 | 0.552 | 0.541 |
| EM-RL-Token | 0.281 | 0.340 | 0.622 | 0.457 | 0.425 |
| HyTuning w/ EM-RL-Token | 0.501 | 0.452 | 0.786 | 0.531 | 0.568 |
Table 8: Robustness of HyTuning to the internal reward within the RLIF branch. Each block pairs a baseline with the HyTuning variant that uses the identical reward, where the better result within a block is in bold.
| Models | Shifting Attention to Relevance | Self-certainty | |||||
| Top-5% | Top-10% | Top-20% | Top-5% | Top-10% | Top-20% | ||
| SelAcc@ ( ) | Qwen2.5-3B | 36.00 | 56.00 | 59.00 | 56.00 | 54.00 | 57.00 |
| RD ( Hsieh et al., 2023 ) | 68.00 | 62.00 | 66.00 | 64.00 | 66.00 | 67.00 | |
| INTUITOR ( Zhao et al., 2025 ) | 72.00 | 80.00 | 75.00 | 88.00 | 88.00 | 83.00 | |
| GRPO ( Shao et al., 2024 ) | 56.00 | 58.00 | 65.00 | 64.00 | 58.00 | 63.00 | |
| OPSD Zhao et al. (2026) | 28.00 | 32.00 | 32.00 | 48.00 | 44.00 | 46.00 | |
Table 9: Confidence faithfulness on CyberMetric-500 under SAR and SC within three high-confidence regions ( ). For SelAcc@ and hAURC, cells are shaded by rank across the three regions within each row: 1st , 2nd , and 3rd , where a confidence-faithful LLM should rank 1st at Top-5% and 2nd at Top-10%. GR@ is not shaded since it naturally decreases as shrinks for every strategy. The best result across strategies in each column is in bold.
| Question : What is the primary purpose of the traceroute program in network testing? Answer Choices : A. To monitor and control internet traffic flow B. To identify the physical location of network routers C. To encrypt and secure data transmissions within the network D. To determine the path and time taken for data packets to reach their destination ( Correct Answer ) | |
| HyTuning | INTUITOR |
| To determine the primary purpose of the traceroute program, let’s analyze each option and its relevance to network testing: A. To monitor and control internet traffic flow. This option is related to network management and monitoring, but it does not specifically address the purpose of traceroute. Traceroute is more focused on identifying the path and time taken for data packets to reach their destination. B. To identify the physical location of network routers. Traceroute is primarily used to determine the path and time taken for data packets to reach their destination. It does not directly provide information about the physical location of network routers. C. To encrypt and secure data transmissions within the network. This option is related to network security, but it does not specifically address the purpose of traceroute. Traceroute is not used for encrypting or securing data transmissions. D. To determine the path and time taken for data packets to reach their destination. This option accurately describes the primary purpose of the traceroute program. Traceroute is a network diagnostic tool that provides information about the path and time taken for data packets to reach their destination. It helps identify any potential issues or bottlenecks in the network. Based on this analysis, the best answer is: Answer : D (Correct Answer) | To determine the primary purpose of the traceroute program in network testing, let’s analyze each option step by step: A. To monitor and control internet traffic flow. This option is incorrect because the traceroute program is not designed to monitor or control internet traffic flow. Its primary purpose is to trace the path that data packets take to reach their destination. B. To identify the physical location of network routers. This option is correct. The traceroute program is specifically designed to identify the physical location of network routers along the path that data packets take to reach their destination. It does this by sending data packets to a destination and recording the responses from each router along the path. C. To encrypt and secure data transmissions within the network. This option is incorrect because the traceroute program is not designed to encrypt or secure data transmissions. Its primary purpose is to trace the path that data packets take to reach their destination. D. To determine the path and time taken for data packets to reach their destination. This option is correct. The traceroute program is designed to determine the path and time taken for data packets to reach their destination. It does this by sending data packets to a destination and recording the responses from each router along the path. Based on the analysis, the primary purpose of the traceroute program in network testing is to identify the physical location of network routers along the path that data packets take to reach their destination. Answer: B (Incorrect Answer) |
Table 10: Case study of HyTuning and INTUITOR on CyberMetric.
| Question : Given that the sample has a standard deviation of zero, which of the following is a true statement? Answer Choices : A. The standard deviation of the population is also zero. B. The sample mean and sample median are equal. ( Correct Answer ) C. The sample may have outliers. D. The population has a symmetric distribution. | |
| HyTuning | INTUITOR |
| Let’s analyze the given question and the provided options step by step to determine the correct answer. 1. Understanding Standard Deviation: The standard deviation measures the amount of variation or dispersion in a set of values. A standard deviation of zero indicates that all the values in the sample are identical . 2. Implications of Zero Standard Deviation: If the standard deviation of the sample is zero, it means that every data point in the sample is the same. This implies that the sample is perfectly homogeneous. A. The standard deviation of the population is also zero. This statement is not necessarily true. The standard deviation of the population depends on the variability of the entire population, not just the sample. Even if the sample has a standard deviation of zero, the population could still have variability. B. The sample mean and sample median are equal. In a perfectly homogeneous sample, the mean and median are indeed equal . This is because there is only one unique value, and both measures of central tendency will be that value. C. The sample may have outliers. This statement is false. If the sample has a standard deviation of zero, it means there are no differences among the values, so there cannot be any outliers. D. The population has a symmetric distribution. This statement is not necessarily true. A symmetric distribution is characterized by equal probabilities on both sides of the mean. However, a perfectly homogeneous sample does not imply a symmetric distribution of the population. The population could have a variety of distributions, including skewed or uniform distributions. Based on the analysis, the only true statement is that the sample mean and sample median are equal. Answer : B (Correct Answer) | To determine which of the given statements is true given that the sample has a standard deviation of zero, let’s analyze each option step by step. Given: The sample has a standard deviation of zero. A. The standard deviation of the population is also zero. The standard deviation of a sample is an estimate of the standard deviation of the population. If the sample standard deviation is zero, it suggests that all the data points in the sample are identical. This implies that the population standard deviation is also zero. Therefore, this statement is true. B. The sample mean and sample median are equal. If the sample standard deviation is zero, it means that all the data points in the sample are identical. In such a case, the sample mean and the sample median are both equal to the value of the data points. Therefore, this statement is true. C. The sample may have outliers. If the sample standard deviation is zero, it means that all the data points in the sample are identical. Outliers are data points that are significantly different from the rest of the data. Since all data points are identical, there are no outliers. Therefore, this statement is false. D. The population has a symmetric distribution. If the sample standard deviation is zero, it means that all the data points in the sample are identical. This does not necessarily imply that the population has a symmetric distribution. The population could have a symmetric distribution, but it could also have a distribution with a single value repeated. Therefore, this statement is not necessarily true. Based on the analysis, the true statements are: Option A: The standard deviation of the population is also zero. Option B: The sample mean and sample median are equal. However, since we need to choose the best answer, and both A and B are true, we can select either one. For the sake of this response, I will choose the first true statement : Answer: A (Incorrect Answer) |
Table 11: Case study of HyTuning and INTUITOR on MMLU.