Factual hallucination is commonly defined by incorrect factual outputs. We study a paraphrase-induced hallucination setting, where a model answers a factual question correctly in its original form but generates an incorrect answer under a semantically equivalent paraphrase. Such inconsistencies expose latent factual instability under semantic invariance. However, general-purpose paraphrases are often insufficient as robustness-oriented supervision: near-copy paraphrases provide weak signals, while overly diverse paraphrases may break semantic equivalence. In this paper, we propose HALLUCINATION-R1, a robustness-oriented paraphrase generation framework that learns to produce semantically faithful yet robustness-challenging paraphrases for factual consistency. Through two-stage optimization, it first stabilizes meaning-preserving and diverse paraphrasing, then rewards paraphrases that reveal factual consistency degradation in downstream QA models. Experiments on SimpleQuestions, PopQA, and TruthfulQA show that HALLUCINATION-R1 achieves a strong consistency--diversity trade-off and exposes robustness failures across multiple model families and datasets. Further analyses indicate that these failures are not reducible to surface-level artifacts or semantic drift, but reveal non-trivial factual instability under meaning-preserving variation. A lightweight fine-tuning study also shows that HALLUCINATION-R1-generated data improves robust accuracy under paraphrase variations, suggesting its utility for robustness-oriented training. Our code and models are publicly available at https://github.com/yuwenhan07/Hallucination-R1.
Figures & tables
Figure 1: Overview of the Hallucination-R1 framework. The upper panel illustrates the two-stage training pipeline: the semantic stabilization stage learns semantically preserved paraphrases with controlled diversity exploration, while the robustness-challenging optimization stage optimizes paraphrases to expose factual inconsistency in a QA model under semantic invariance. The lower panel shows a representative case study , where a semantically equivalent paraphrase shifts the QA model from a correct answer to a factual hallucination. The heatmap visualizes layer-wise log-probability preference changes, with red indicating stronger preference for the correct answer token and blue for the incorrect one.
Paraphrase Model
Consis.
Mean Div.
Ratio (Div. > 0.9)
Consis. × Mean Div.
Hallucination-R1-3B
87.46%
0.930696
80.41%
81.40%
DeepSeek-V3
96.41%
0.880482
55.19%
84.89%
Qwen2.5-3B
64.68%
0.900111
74.41%
58.22%
Qwen2.5-32B
96.03%
0.892318
74.41%
85.69%
Qwen3-4B
98.28%
0.399141
3.16%
39.23%
Qwen3-8B
98.71%
0.47959
5.11%
47.34%
Table 1: Consistency–diversity trade-off across different paraphrase models on the TruthfulQA dataset. We report semantic consistency (Consis.), mean diversity (Mean Div.), consistency × diversity, and the ratio of high-diversity paraphrases (Div. > 0.9). Semantic consistency follows the definition used in the training reward.
QA Model
SimpleQuestions
PopQA
TruthfulQA
True (Acc)
ER@1
WER@5
True (Acc)
ER@1
WER@5
True (Acc)
ER@1
WER@5
Hallucination-R1-1.5B
Qwen2.5-3B
2106(21.25%)
30.44%
71.37%
1702(11.93%)
30.14%
77.38%
246(31.14%)
32.11%
83.33%
Qwen2.5-7B
2288(23.09%)
29.98%
66.35%
2077(14.56%)
29.47%
73.57%
323(40.89%)
30.96%
73.07%
Qwen2.5-32B
2584(26.07%)
23.30%
56.93%
2368(16.60%)
25.13%
66.17%
374(47.34%)
24.60%
68.18%
Qwen3-4B
2126(21.45%)
28.46%
68.30%
1194(8.37%)
36.60%
86.43%
233(29.49%)
36.91%
81.55%
Table 2: Robustness evaluation results of Hallucination-R1 across different QA models and benchmark datasets. True (Acc) reports the number and percentage of samples that are correctly answered by the QA model before paraphrasing. ER@1 and WER@5 denote Error Rate@1 and Worst-Case Error Rate@5, respectively. A robustness failure is counted only when the rewritten question is judged to be semantically equivalent to the original input while causing the QA model to generate an incorrect answer. All robustness metrics are computed over the subset of samples that are originally answered correctly by the QA model. Results are reported for two Hallucination-R1 scales: 1.5B and 3B parameters.
QA Model
Ori. Acc
Paraphrase Model
Consis@1
RF@1
RF@5
ER@1
WER@5
LLaMA-3.1-8B
247 (31.27%)
Qwen2.5-3B
64.68%
84
182
34.01%
73.68%
Hallucination-R1 (Stage 1 Only)
80.37%
97
187
39.27%
75.71%
Hallucination-R1
87.46% ↑22.78%
136
215
55.06% ↑21.05%
87.04% ↑13.36%
Qwen2.5-3B
246 (31.14%)
Qwen2.5-3B
64.68%
81
183
32.93%
74.39%
Hallucination-R1 (Stage 1 Only)
80.37%
99
193
40.24%
78.46%
Hallucination-R1
87.46% ↑22.78%
118
214
47.97% ↑15.04%
86.99% ↑22.78%
Table 3: Ablation results of different paraphrase model variants across QA models on the TruthfulQA dataset. RF@1 denotes the number of robustness failures under a single semantically equivalent paraphrase, while RF@5 denotes the number of samples for which at least one out of five semantically equivalent paraphrases causes a robustness failure. ↑ / ↓ denote the absolute difference (in percentage points) of each variant relative to the full Hallucination-R1-3B , computed under the same QA model.
Figure 2: Length and structural diagnostics of paraphrases. (a) compares original and paraphrased question lengths for robustness-failing and robustness-preserving cases, denoted as Failing and Preserving in the plots. (b) and (c) report changes in word count and dependency tree depth, respectively.
Figure 3: QA-concatenation embedding distance distributions for robustness-failing and robustness-preserving paraphrases.
Paraphrase Model
QA Model
PopQA
SimpleQuestions
Para. Acc.
Robust Acc.
Any Acc.
Para. Acc .
Robust Acc .
Any Acc.
Hallucination-R1
Qwen3-8B
14.30%
3.79%
24.32%
21.17%
6.08%
38.33%
Qwen3-8B-SFT
16.95% ↑2.65%
8.06% ↑4.27%
23.03% ↓1.29%
26.36% ↑5.19%
16.93% ↑10.85%
35.44% ↓2.89%
DeepSeek-V3
Qwen3-8B
15.90%
6.09%
26.47%
21.59%
5.72%
42.03%
Qwen3-8B-SFT
17.88% ↑1.98%
10.62% ↑4.53%
24.49% ↓1.98%
26.26% ↑4.67%
18.33% ↑12.61%
35.96% ↓6.07%
Qwen2.5-32B
Qwen3-8B
16.70%
7.87%
26.16%
22.20%
6.33%
41.19%
Table 4: Robustness training results under semantics-preserving paraphrase variations. Para. Acc., Robust Acc., and Any Acc. are defined in Section 7 . Paraphrase Model denotes the model used to generate evaluation paraphrases, while QA Model denotes the model being evaluated. ↑ / ↓ denote absolute differences in percentage points between Qwen3-8B-SFT and Qwen3-8B under the same paraphrase model and dataset.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Training reward curve during the semantic stabilization stage.
Figure 5: Training reward curve during the robustness-challenging optimization stage.
Dataset
Total Samples
Evaluation Metrics
SimpleQuestions
9,910
LLM Judge
PopQA
14,267
EM Match
TruthfulQA
790
LLM Judge
Appendix
Table 5: Evaluation datasets and metrics.
QA Model
Ori. Acc
Paraphrase Model
Consis@1
RF@1
RF@5
ER@1
WER@5
Qwen2.5-7B
323 (40.89%)
Qwen2.5-3B
64.68%
98
233
30.34%
72.14%
Qwen2.5-7B
77.32%
124
262
38.39%
81.11%
Qwen2.5-32B
93.17%
135
256
41.80%
79.26%
DeepSeek-V3
97.72%
164
273
50.77%
84.52%
Hallucination-R1 (Stage 1 Only)
80.37%
125
245
38.70%
75.85%
Hallucination-R1-3B
87.46%
147
277
45.51%
85.76%
Appendix
Table 6: Additional results on TruthfulQA with Qwen2.5-7B as the QA model. We compare Hallucination-R1-3B with paraphrase models of different scales and architectures.
Judge Model
Consis@1
Consis@5
Majority@5
DeepSeek-V3 Liu et al. (2024)
87.46%
99.87%
97.09%
Kimi-K2 Guo et al. (2025)
83.72%
99.49%
95.87%
Qwen3-235B-A22B Yang et al. (2025)
85.82%
100.00%
96.46%
Human (100 samples)
83.00%
98.00%
96.00%
Appendix
Table 7: Semantic consistency evaluation on the TruthfulQA dataset across different judge models and human annotation.
Figure 6: Distribution of word counts for original and paraphrased questions across all questions.
Figure 7: Normalized feature profiles of robustness-failing and robustness-preserving paraphrases under the global distribution.
Figure 8: Relationship between question paraphrase distance and QA-concatenation distance. Robustness-failing and robustness-preserving paraphrases exhibit distinct patterns in both the joint distance space and the induced Δ distribution.
Figure 9: Additional examples of layer-wise factual preference dynamics under original and paraphrased inputs.
Large language models (LLMs) can generate factually inconsistent claims, motivating accurate and scalable hallucination detectors. Prior work largely enlarges training sets via synthesis or new annotations, introducing increasing cost and potential bias while underusing the consistency implied by semantically equivalent paraphrases. We propose Consistency-Constrained Hallucination Detector (CCHD), which formulates training as a constrained optimization problem. The standard cross-entropy on original document-claim pairs is complemented by (i) paraphrase-consistency constraints bounding divergence across paraphrased views, and (ii) label-preservation constraints tying paraphrases to ground truth. We solve the problem by gradient descent-ascent over model parameters and per-view Lagrange multipliers, adding only a few scalar dual variables and no inference-time overhead. With DeBERTa and Flan-T5 backbones, CCHD consistently outperforms strong baselines (FactCG, MiniCheck, and AlignScore) on standard factuality benchmarks, demonstrating its superiority on hallucination detection.
Shanshan Lin, Dongsheng Hong, Sibo Ju +3
1Fuzhou University · 2Harbin Institute of Technology (Shenzhen) · 3Beijing University of Posts and Telecommunications
Large Language Models (LLMs) often produce factoid hallucinations - plausible yet incorrect answers. A common mitigation strategy is model alignment, which improves factual accuracy by training on curated (factual, non-factual) pairs. However, this approach often relies on a stronger model (e.g., GPT-4) or an external knowledge base to assess factual correctness that may not always be accessible. Addressing this, we propose Atomic Consistency Preference Optimization (ACPO), a self-supervised preference-tuning method that enhances factual accuracy without external supervision. ACPO leverages atomic consistency signals (i.e., the agreement of individual facts across multiple stochastic responses) to identify high- and low-quality data pairs for model alignment. Despite being fully self-supervised, ACPO outperforms the strong supervised alignment baseline by 1.95 points averaged across Phi-3 and Llama3 on the LongFact and BioGen datasets, demonstrating its effectiveness in improving factual reliability without relying on external models or knowledge bases.
Jingfeng Chen, Raghuveer Thirukovalluru, Junlin Wang +2
The tendency for hallucination in current large language models (LLMs) negatively impacts dialogue systems. Such hallucinations produce factually incorrect responses that may mislead users and undermine system trust. Existing refinement methods for dialogue systems typically operate at the response level, overlooking the fact that a single response may contain multiple verifiable or unverifiable facts. To address this gap, we propose Fine-Refine, a fine-grained refinement framework that decomposes responses into atomic units, verifies each unit using external knowledge, assesses fluency via perplexity, and iteratively corrects granular errors. We evaluate factuality across the HybriDialogue and OpendialKG datasets in terms of factual accuracy (fact score) and coverage (Not Enough Information Proportion), and experiments show that Fine-Refine substantially improves factuality, achieving up to a 7.63-point gain in dialogue fact score, with a small trade-off in dialogue quality.
Xiangyan Chen, Yujian Gan, Matthew Purver
Queen Mary University of London, UK · Queen’s University Belfast, UK · Institut Jožef Stefan, Slovenia