Label-Efficient Document-Grounded Factuality Detection via Cross-Paraphrastic Invariance Learning
Organizations: Harbin Institute of Technology (Shenzhen). · Fuzhou University. · Hong Kong University of Science and Technology (Guangzhou).
Abstract
Large language models (LLMs) can generate fluent claims that are unsupported by a given source document, motivating accurate and efficient document-grounded factuality detection. Existing LLM-based evaluators often require repeated inference and incur high computational cost, whereas compact factuality detectors are efficient at inference time but commonly rely on large labeled or synthetically generated corpora. We propose CPIL (Cross-Paraphrastic Invariance Learning), a two-stage Siamese framework that converts a small labeled set into relational supervision without acquiring additional factuality labels. CPIL constructs positive pairs from distinct label-preserving paraphrastic views of the same document-claim instance and negative pairs from opposite-label examples associated with the same source document. It further introduces quantile-aware semi-hard negative mining, which derives a boundary from the positive-pair similarity distribution and balances informative negatives with broader coverage. In the two-stage training procedure, Stage 1 learns an encoder from these task-aware pairs, and Stage 2 jointly fine-tunes the encoder and a linear classification head for binary factuality prediction. At inference time, CPIL requires no paraphrase generation, pair construction, or Siamese comparison and uses only one encoder-classifier forward pass. On the 11-task LLM-AggreFact benchmark, CPIL-2% and CPIL-3% achieve macro-F1 scores of 81.04 and 81.60 while using only 2-3% of the FactCG-released labeled pool. Both settings outperform all compared compact detectors and two vanilla LLM references. Ablations and sensitivity analyses support the contributions of task-aware pair construction, dynamic negative mining, and multi-pivot paraphrasing.
Figures & tables
| Aspect | Conference paper | Journal manuscript |
| Method | Two-stage Siamese learning with static cross-paraphrase positives and same-document negatives. | Adds quantile-aware semi-hard negative mining with a positive-derived boundary and an exploitation-coverage strategy. |
| Formulation | Concise formulation of pair construction and two-stage training. | Expanded formulation with separate offline and dynamic training algorithms and complexity analysis. |
| Evaluation | Conference-scale baseline comparison and ablations. | Stronger baselines and broader ablations of augmentation, pair construction, and dynamic mining. |
| Analysis | Limited low-budget study. | Additional studies of label budgets, mining ratios, pivot languages, and multi-seed analyses. |
| Model | AggreFact | TofuEval | WiCE | REVEAL | Claim | Fact | Expert | LFQA | RAG | AVG | ||
| CNN | XSum | MediaS | MeetB | Verify | Check | QA | Truth | |||||
| Qwen2.5-7B | 92.41 | 71.25 | 86.21 | 90.16 | 53.47 | 74.09 | 82.90 | 61.28 | 75.84 | 83.40 | 89.18 | 78.20 |
| Llama3.1-8B | 92.65 | 71.82 | 86.21 | 91.43 | 57.42 | 69.79 | 83.79 | 59.58 | 84.42 | 81.04 | 94.02 | 79.29 |
| Mistral-Nemo-12B | 94.07 | 72.14 | 85.82 | 74.28 | 60.53 | 84.43 | 88.04 | 77.47 | 85.00 | 86.40 | 57.38 | 78.69 |
| SummaC-ZS | 55.19 | 66.55 | 79.57 | 80.14 | 41.03 | 74.50 | 72.96 | 60.79 | 65.30 | 81.58 | 62.81 | 67.31 |
| SummaC-CV | 66.41 | 48.56 | 69.70 | 77.87 | 50.98 | 70.77 | 68.36 | 57.53 | 65.21 | 79.16 | 56.92 | 64.68 |
| Variant | AggreFact | TofuEval | WiCE | REVEAL | Claim | Fact | Expert | LFQA | RAG | AVG | ||
| CNN | XSum | MediaS | MeetB | Verify | Check | QA | Truth | |||||
| w/o Pairwise Stage | 70.94 | 69.92 | 81.88 | 88.52 | 68.46 | 76.56 | 83.02 | 60.47 | 72.25 | 89.24 | 83.58 | 76.81 |
| Append-Only Aug. | 78.30 | 69.20 | 82.46 | 86.74 | 69.90 | 77.77 | 86.41 | 61.28 | 65.15 | 87.47 | 86.10 | 77.34 |
| Random All Pairs | 77.16 | 74.15 | 79.84 | 83.52 | 70.31 | 76.49 | 85.04 | 61.69 | 70.70 | 87.69 | 85.33 | 77.44 |
| Random Pos Pairs | 94.61 | 69.45 | 84.38 | 89.03 | 47.02 | 85.13 | 95.97 | 78.40 | 86.56 | 89.16 | 46.74 | 78.77 |
| Random Neg Pairs | 94.71 | 69.14 | 84.52 | 88.98 | 49.22 | 74.70 | 95.96 | 57.26 | 86.56 | 89.24 | 47.34 | 76.15 |
Automated Fact-Checking
Automated fact-checking detects checkable claims, retrieves evidence, and assesses whether that evidence supports or refutes them.
See all 146 papers