Factual Consistency Evaluation
Overview
The field studies how to determine whether AI-generated claims are supported by source material, accurate, and free of contradictions, especially in summaries, question answering, and retrieval-augmented generation. Common approaches break outputs into claims or questions and check them with entailment models, language-model judges, external retrieval, knowledge graphs, or structured evidence; newer work also uses uncertainty estimates and multi-agent verification. Benchmarks test settings such as long-form and clinical summaries, conflicting evidence, multilingual fact-checking, and image- or video-grounded claims, often comparing automated scores with expert judgments or controlled perturbations. A central tension is making evaluation both scalable and trustworthy: reference-free judges can miss shared errors, while evidence-based checks must also measure omissions and coverage, handle context, and account for cost.
Latest papers
74 papers8 papers in the last four weeks, down 43% on the four weeks before. 0.0% of all new papers.