Think Through Uncertainty: Improving Long-Form Generation Factuality via Reasoning Calibration

Xin Liu; Lu Wang

cs.CLApr 13, 2026

Think Through Uncertainty: Improving Long-Form Generation Factuality via Reasoning Calibration

Authors: Xin Liu, Lu Wang

Organizations: Computer Science and Engineering, University of Michigan, Ann Arbor, MI, USA

Abstract

Large language models (LLMs) often hallucinate in long-form generation. Existing approaches mainly improve factuality through post-hoc revision or reinforcement learning (RL) with correctness-based rewards, but they do not teach the model to estimate which parts of its generation are reliable. As a result, models may still state incorrect claims confidently in their responses. Recent advances in reasoning have significantly improved LLM performance, and have been leveraged to estimate confidence by incorporating calibration into RL objectives. However, existing approaches remain limited to a single scalar confidence for the entire response, which is insufficient for long-form generation where uncertainty varies across individual claims. To mitigate this problem, we propose CURE, a framework that improves long-form factuality by teaching LLMs to reason about uncertainty at the claim level. We first introduce a Claim-Aware Reasoning Protocol, which structures outputs into atomic claims paired with explicit confidence estimates. We then develop a multi-stage training pipeline that aligns model confidence with claims' correctness and then optimizes on factuality. The resulting calibrated confidence further enables selective prediction, allowing the model to abstain from uncertain claims at inference time. Experiments on four long-form factuality benchmarks show that CURE consistently improves factual accuracy over competitive supervised and RL baselines, while maintaining factual recall. In particular, it improves claim-level accuracy by up to 39.9% on Biography generation. These gains are accompanied by improved calibration, as reflected by a 16.0% increase in AUROC on FactBench.

Figures & tables

Figure 1: Illustration of the claim-aware reasoning protocol.The model performs structured reasoning with explicit uncertainty, then decomposes the output into atomic claims with calibrated confidence. These confidences enable selective prediction (Figure 3 ).

Figure 2: Stages 2 and 3 of the multi-stage training pipeline. In Stage 2 (left), confidence correction constructs preference pairs for DPO-based calibration, aligning confidence estimates with empirical correctness. In Stage 3 (right), GRPO optimizes claim-level factuality under feasibility constraints, with rewards explicitly restricted to claim tokens. This design decouples calibration from factual optimization.

Figure 3: Selective prediction via confidence thresholding. Low-confidence claims are filtered, while retained confidence scores provide explicit transparency into the final output.

Method	FactBench				LongFact				Biography
	Acc.	ECE	Brier	AUROC	Acc.	ECE	Brier	AUROC	Acc.	ECE	Brier	AUROC
	( $\uparrow$ )	( $\downarrow$ )	( $\downarrow$ )	( $\uparrow$ )	( $\uparrow$ )	( $\downarrow$ )	( $\downarrow$ )	( $\uparrow$ )	( $\uparrow$ )	( $\downarrow$ )	( $\downarrow$ )	( $\uparrow$ )
Base LLM	73.6	0.162	0.219	0.541	74.5	0.158	0.193	0.591	45.7	0.124	0.239	0.691
L2RF	77.1	–	–	–	79.4	–	–	–	47.1	–	–	–
LitCab	73.6	0.212	0.276	0.575	74.5	0.156	0.190	0.601	45.7	0.125	0.239	0.693
SFT	82.0	0.137	0.184	0.563	85.1	0.118	0.146	0.655	51.5	0.361	0.395	0.620

Table 1: Main results on three long-form factuality benchmarks. Stages in our training pipeline are cumulative: starting from SFT, we sequentially apply Feasibility RL (F-RL), Calibration Optimization (CO), and finally Factuality Optimization (FO). For L2RF, we report only accuracy following the results reported in their paper, since their outputs are not available for computing calibration metrics. CURE achieves the highest accuracy across all datasets and the best AUROC on FactBench and LongFact. AUROC measures the ability to distinguish correct from incorrect claims and is more informative and pratical than ECE and Brier.

Figure 4: Effect of feasibility RL on claim verifiability, reasoning faithfulness, and answer relevance on FactBench (Llama3.1-8B-Instruct). It improves all three metrics, with the largest gain in relevance.

#	Method	Acc. $\uparrow$	ECE $\downarrow$	Brier $\downarrow$	AUROC $\uparrow$
1	SFT + F-RL	78.1	0.157	0.200	0.605
2	SFT + F-RL + Joint (GRPO) ( $n$ =4)	82.2	0.181	0.198	0.608
3	SFT + F-RL + Joint (GRPO) ( $n$ =8)	83.2	0.165	0.181	0.612
4	SFT + F-RL + Joint (GRPO) ( $n$ =16)	83.9	0.154	0.192	0.620
5	SFT + F-RL + Joint (GRPO) ( $n$ =16 + weighted Brier)	83.9	0.143	0.188	0.621
6	SFT + F-RL + CO (GRPO) ( $n$ =16 + weighted Brier)	78.5	0.136	0.197	0.602

Table 2: Comparison of joint optimization and decoupled training on FactBench using Llama3.1-8B-Instruct. Joint GRPO augments the reward with the Brier score and is evaluated with varying group sizes

n

(Rows 2-4), with weighted Brier in Row 5. Row 6 applies GRPO for calibration only, while Rows 7-8 use DPO-based calibration followed by factuality optimization. Joint GRPO (Rows 2-5) fails to achieve strong calibration despite larger

n

and reweighting. GRPO-based calibration alone (Row 6) remains insufficient, whereas DPO-based calibration (Row 7) substantially improves performance, with further gains in both factual accuracy and discriminative calibration (AUROC) after factuality optimization (Row 8).

Figure 5: Pareto frontier of factual accuracy versus recall. The curves illustrate the trade-off shaped by the training penalty

\epsilon

and navigated by the inference confidence threshold

\tau

. Different values of

\epsilon

define distinct Pareto frontiers, while

\tau

enables movement along each frontier by filtering claims based on calibrated confidence.

Method	FactBench				LongFact				Biography
	Acc.	ECE	Brier	AUROC	Acc.	ECE	Brier	AUROC	Acc.	ECE	Brier	AUROC
Base	53.2	0.262	0.315	0.512	58.3	0.225	0.302	0.542	21.7	0.539	0.509	0.513
SFT	70.8	0.220	0.246	0.560	58.3	0.352	0.360	0.592	22.4	0.657	0.619	0.568
CURE	80.4	0.143	0.205	0.661	70.2	0.201	0.267	0.625	24.6	0.256	0.288	0.672

Table 3: Generalization results on Qwen3-4B across FactBench, LongFact, and Biography.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Stage	Model	# Training Instances
SFT	Llama3.1-8B-Instuct	2,987
SFT	Qwen3-4B	2,632
RL	All Models	4,000

Appendix

Table 4: Data statistics for SFT and RL training. Prompts are provided by Chen et al. (2025) .

Figure 6: Training reward dynamics under GRPO. Left: factuality reward. Right: calibration reward. Solid lines denote per-step rewards, and dashed lines denote moving averages (window size 10). The calibration reward closely follows the factuality reward across training, indicating that it is largely driven by claim correctness rather than confidence–correctness alignment. This entanglement leads to weak and noisy learning signals for calibration.

Figure 7: Pareto frontier of factual accuracy versus recall across extended configurations. Curves represent the trade-off shaped by different training-time penalties

\epsilon\in\{0.0,0.2,0.4,0.6,0.8,1.0\}

, while markers indicate the navigation along each frontier via the inference-time confidence threshold

\tau\in\{0.0,0.2,0.4,0.6,0.8\}

. Our framework consistently dominates the Base LLM and the accuracy-only baseline across all viable settings.

Query

Give the next three items in this series: hydrogen, lithium, boron, …

Response

<think> The user is asking for the next three items in a series that starts with hydrogen, lithium, and boron. I need to identify the pattern. First, I’ll establish the basic sequence. The series appears to be the first three elements in the periodic table, listed in order. <reason> I feel very confident about this because the sequence provided matches the well-known order of the first three elements in the periodic table. This is a fundamental fact in chemistry that I’ve seen consistently across many sources. My confidence is extremely high, a 0.99. ([Claim 1], Confidence: 0.99)… </think> <decompose> • Claim 1: The series starts with the first three elements in the periodic table. Confidence: 0.99 … </decompose>

Appendix

Table 5: Case study on claim verifiability. The claim marked in red does not constitute an independent fact, as it lacks sufficient contextual information for proper interpretation. This makes its correctness difficult to verify and may introduce noise during factuality optimization.

Query

What heavy metal song has the fastest tempo?

Response

<think> The user is asking about a specific heavy metal song with the fastest tempo. First, I’ll consider some well-known heavy metal bands and their songs. Bands like Slayer, Megadeth, and Death Angel are known for their fast tempos. <reason> I’m not certain which song has the absolute fastest tempo. I’ll assign a confidence of 0.7 that the song is from one of these bands. This is a moderate level of confidence, indicating that I’m fairly certain about the general category but not the specific details. ([Claim 1], Confidence: 0.7)… </think> <decompose> • Claim 1: A song from a well-known heavy metal band like Slayer, Megadeth, or Death Angel is likely to have the fastest tempo. Confidence: 0.70 … </decompose>

Appendix

Table 6: Case study on reasoning faithfulness. The claim highlighted in red introduces additional information not supported by the reasoning process. Specifically, the reasoning only states that Death Angel is known for their fast tempo, whereas the extracted claim further asserts that their songs are likely to be the fastest. This mismatch makes the correctness assigned to the claim unreliable, and thus affects the calibration optimization.

LLM Hallucination Mitigation

Large language model hallucination mitigation develops ways to detect and reduce unsupported, fabricated, or inconsistent outputs.

See all 175 papers

Only Say What You Know: Calibration-Aware Generation for Long-Form Factuality
May 3, 2026Wen Luo, Guangyue Peng, Liang Wang +7Long-Form Document Generation LLM Hallucination Mitigation
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University Research Asia · MSRA · Microsoft
Not All Claims Are Equally Risky: FACTOR for Adaptive Verification in Factual Long-Form Generation
Jun 21, 2026Areeba Hassan, Arooj Kausar, Syeda Kisaa Fatima +2LLM Answer Verification Long-Form Document Generation
School of Electrical Engineering and Computer Science (SEECS), National University of Sciences and Technology (NUST), Islamabad, Pakistan
IUQ: Interrogative Uncertainty Quantification for Long-Form Large Language Model Generation
Apr 16, 2026Haozhi Fan, Jinhao Duan, Kaidi XuUncertainty Quantification Language Model Generation Evaluation
University of Pennsylvania · 2UNC Chapel Hill · 3City University of Hong Kong

May 3, 2026J/K move · Enter open · S save

Only Say What You Know: Calibration-Aware Generation for Long-Form Factuality

Wen Luo, Guangyue Peng, Liang Wang, Nan Yang+6

State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University Research Asia · MSRA · Microsoft

Long-Form Document Generation LLM Hallucination Mitigation Hallucination in Language Models Language Model Calibration LLM Reliability Large Reasoning Models

Open paper AskPDF arXiv

Large Reasoning Models achieve strong performance on complex tasks but remain prone to hallucinations, particularly in long-form generation where errors compound across reasoning steps. Existing approaches to improving factuality, including abstention and factuality-driven optimization, follow a coupled exploration-commitment paradigm, in which intermediate reasoning is unconditionally propagated to the final output, limiting fine-grained control over information selection and integration. In this paper, we propose an Exploration-Commitment Decoupling paradigm that disentangles knowledge exploration from final commitment, enabling models to explore with awareness while answering cautiously. We instantiate the paradigm with Calibration-Aware Generation (CAG), a framework that equips models with end-to-end, calibration-aware generation capabilities, by augmenting intermediate reasoning with calibrated reliability estimates and prioritizing reliable content in final outputs. Across five long-form factuality benchmarks and multiple model families, CAG improves factuality by up to 13%, while reducing decoding time by up to 37%. Overall, our work highlights decoupling as a principled approach for more reliable long-form generation, offering directions for trustworthy and self-aware generative systems.

Abstract

LLM Hallucination Mitigation

Explore similar work

Only Say What You Know: Calibration-Aware Generation for Long-Form Factuality

Not All Claims Are Equally Risky: FACTOR for Adaptive Verification in Factual Long-Form Generation

IUQ: Interrogative Uncertainty Quantification for Long-Form Large Language Model Generation