Factual Consistency Evaluation

Follow this topic Ask about this topic

Momentum

8 papers in the last four weeks, up 60% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 52

All topics

Improving Cross-Lingual Factual Recall via Consistency-Driven Reinforcement Learning
Jun 4, 2026Jonathan von Rad, Louis Arts, George Burgess +6Multilingual Language Model Evaluation RL for Language Models
University College London, Centre for Artificial Intelligence
RadSEM: A Finding-by-Finding Metric for Clinical Consistency in Radiology Reports
Jun 3, 2026Zhenhong Yang, Zhuoyun Liu, Jintao Fei +4Factual Consistency Evaluation Automated Evaluation
JDH Algo, JD Health International Inc., China · Department of Big Data in Health Science, The First Affiliated Hospital of Wenzhou Medical University, China · Zhejiang Engineering Research Center for Hospital Emergency and Process Digitization, Wenzhou, China +1
Relevant Is Not Warranted: Evidence-Force Calibration for Cited RAG
May 27, 2026Pin Qian, Su Wang, Xiaoyuan Wang +7Factual Consistency Evaluation Citation Verification
Carnegie Mellon University · Georgia Institute of Technology · Dartmouth College +3
MATCHA: Matching Text via Contrastive Semantic Alignment
May 26, 2026Siran Li, Ece Sena Etoglu, Carsten Eickhoff +1LLM Evaluation Semantic Textual Similarity
University of Tübingen
Optimising Factual Consistency in Summarisation via Preference Learning from Multiple Imperfect Metrics
May 26, 2026Yuxuan Ye, Raul Santos-Rodriguez, Edwin SimpsonFactual Consistency Evaluation Text Summarization
Intelligent System Laboratory University of Bristol United Kingdom
Towards Error-Free EHRs: Reasoning-Intensive Consistency Verification Between Clinical Notes and Structured Tables in Electronic Health Records
May 26, 2026Yeonsu Kwon, Jiho Kim, Junseong Choi +10Factual Consistency Evaluation Clinical Reasoning
1KAIST · 2Ghent University · 3Samsung Medical Center +2
FAGER: Factually Grounded Evaluation and Refinement of Text-to-Image Models
May 18, 2026Youngsun Lim, Cusuh Ham, Pin-Yu Chen +1Factual Consistency Evaluation Automated Evaluation
Boston University · Adobe · IBM Research
SKG-Eval: Stateful Evaluation of Multi-Turn Dialogue via Incremental Semantic Knowledge Graphs
May 15, 2026Avijit Shil, Suman SamuiLLM Evaluation Factual Consistency Evaluation
Maulana Abul Kalam Azad University of Technology, West Bengal, India · National Institute of Technology Durgapur, West Bengal, India
Towards Dependable Retrieval-Augmented Generation Using Factual Confidence Prediction
May 4, 2026Florian Geissler, Francesco Carella, Laura Fieback +1Retrieval-Augmented Generation Factual Consistency Evaluation
Fraunhofer Institute for Cognitive Systems (IKS), Munich, Germany · Volkswagen AG, Wolfsburg, Germany
EditPropBench: Measuring Factual Edit Propagation in Scientific Manuscripts
May 3, 2026Garvin KruthofKnowledge Editing LLM Evaluation
Technical University of Munich
DualFact+: A Multimodal Fact Verification Framework for Procedural Video Understanding
Apr 28, 2026Cennet Oguz, Yasser Hamidullah, Josef van Genabith +1Multimodal Grounding Video Understanding
German Research Center for Artificial Intelligence (DFKI) · Saarland Informatics Campus
LongSumEval: Question-Answering Based Evaluation and Feedback-Driven Refinement for Long Document Summarization
Apr 28, 2026Huyen Nguyen, Haoxuan Zhang, Yang Zhang +2Language Model Generation Evaluation Factual Consistency Evaluation
Cigna Group Evernorth Health Services Austin, Texas, USA · dept. of Information Science University of North Texas Denton, Texas, USA · dept. of Data Science University of North Texas Denton, Texas, USA
CT-FineBench: A Diagnostic Fidelity Benchmark for Fine-Grained Evaluation of CT Report Generation
Apr 27, 2026Ruifeng Yuan, Wanxing Chang, Weiwei Cao +4Radiology Report Generation Medical Report Generation
DAMO Academy, Alibaba Group, China · Hupan Lab, 310023, China · Fudan University, China +2
BioDivergence: A Benchmark and Evaluation Framework for Hidden Contextual Contradictions in Biomedical Abstracts
Apr 23, 2026Elias Hossain, Sanjeda Sara Jennifer, Sabera Akter Bushra +1Claim Verification Factual Consistency Evaluation
College of Engineering and Computer Science, University of Central Florida · Burnett School of Biomedical Sciences, University of Central Florida
Evaluating AI Meeting Summaries with a Reusable Cross-Domain Pipeline
Apr 23, 2026Philip Zhong, Don Wang, Jason ZhangLLM Evaluation Factual Consistency Evaluation
Webex Suite AI Cisco Systems, Inc. San Jose, California, USA
Evaluating LLM-Driven Summarisation of Parliamentary Debates with Computational Argumentation
Apr 21, 2026Eoghan Cunningham, Derek Greene, James Cross +1LLM Evaluation Factual Consistency Evaluation
School of Computer Science, University College Dublin · School of Politics and International Relations, University College Dublin · Department of Informatics, King’s College London
Coverage-Controlled Preference Mining from Noisy Claim Verification for Evidence-Grounded Generation
Mar 11, 2026Weixin Liu, Congning Ni, Qingyuan Song +4Claim Verification Healthcare
Vanderbilt University, Nashville, TN, USA · Vanderbilt University Medical Center, Nashville, TN, USA · Virginia Tech, Blacksburg, VA, USA
Fine-Grained Uncertainty Quantification for Long-Form Language Model Outputs: A Comparative Study
Feb 19, 2026Dylan Bouchard, Mohit Singh Chauhan, Viren Bajaj +1LLM Hallucination Mitigation Hallucination in Language Models
CVS Health, Wellesley, MA
AdversaRiskQA: An Adversarial Factuality Benchmark for High-Risk Domains
Jan 21, 2026Adam Szelestey, Sofie van Engelen, Tianhao Huang +3Factual Consistency Evaluation LLM Reliability
Eindhoven University of Technology Eindhoven, Netherlands
Gavel: Agent Meets Checklist for Evaluating LLMs on Long-Context Legal Summarization
Jan 7, 2026Yao Dou, Benjamin Mamut, Wei XuLegal NLP LLM Evaluation
Georgia Institute of Technology
SAGE: A Search-AuGmented Evaluation of Large Language Models on Free-Form QA
Apr 10, 2025Sher Badshah, Ali Emami, Hassan SajjadLLM Evaluation Language Model Generation Evaluation
Dalhousie University · Emory University
UrduFactCheck: An Agentic Fact-Checking Framework for Urdu with Evidence Boosting and Benchmarking
Date pendingSarfraz Ahmad, Hasan Iqbal, Momina Ahsan +6LLM Evaluation Factual Consistency Evaluation
MBZUAI

Jun 4, 2026, cs.CLJ/K move · Enter open · S save

Improving Cross-Lingual Factual Recall via Consistency-Driven Reinforcement Learning

Jonathan von Rad, Louis Arts, George Burgess, Eleftheria Kolokytha+5

University College London, Centre for Artificial Intelligence

Multilingual Language Model Evaluation RL for Language Models Factual Consistency Evaluation Multilingual QA Factual Knowledge in Language Models Cross-Lingual Consistency in Language Models+1

Open paper AskPDF arXiv

Large language models (LLMs) trained predominantly on English data encode substantial world knowledge, yet often fail to express it reliably in other languages, a phenomenon known as cross-lingual factual inconsistency. To study and address this, we introduce PolyFact, a large-scale parallel multilingual factual QA dataset containing 100K Wikidata-grounded facts across 12 typologically diverse languages. Using PolyFact, we compare light continual pretraining (CPT), supervised fine-tuning (SFT), and reinforcement learning via Group Relative Policy Optimization (GRPO) for improving cross-lingual factual recall in Qwen-2.5-7B and OLMo-2-1124-7B. We find that GRPO consistently outperforms SFT, improving both cross-lingual consistency and generalization to unseen languages, while CPT on parallel data yields limited additional gains. Mechanistic analyses further show that GRPO reorganizes multilingual routing by reducing language specialization in MLP layers and attention heads, thereby promoting more shared cross-lingual representations. We release our code, models, and dataset.

Momentum

Related topics

Latest papers 52