LLM Hallucination Detection

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

13 papers in the last four weeks, up 63% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 141

All topics

The Labeling Problem in Hallucination Detection Benchmarks: An Empirical Evaluation
Oct 6, 2026Jorma Valjakka, Juhani Kivimäki, Juha Mylläri +1LLM Evaluation LLM-as-a-Judge
Department of Computer Science University of Helsinki
Single-Pass Uncertainty Heads for Claim-Level Hallucination Detection in Persian Medical Language Models
Oct 2, 2026Mehrdad Ghassabi, Pedram Rostami, Hamidreza Baradaran Kashani +2Hallucination Detection LLM Hallucination Detection
Faculty of Computer Engineering University of Isfahan Isfahan, Iran · School of Electrical and Computer Engineering University of Tehran Tehran, Iran · School of Computer Science University of Windsor Windsor, Canada +1
External Observers May See More Clearly: Cross-Model Span-Level Hallucination Detection in Large Language Models via Hidden State Probing
Oct 1, 2026Kingshuk Gupta, Davide BuscaldiHallucination Detection LLM Hallucination Detection
LIX, École Polytechnique · School of Electrical and Electronic Engineering, Nanyang Technological University · LIPN, Sorbonne Paris Nord
RAIM: Robust Aggregation of Inexpensive Models for Hallucination Detection
Sep 30, 2026Elia Onofri, Roberto Di PietroLLM Evaluation LLM-as-a-Judge
Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division, King Abdullah University of Science and Technology (KAUST) Thuwal 23955, Saudi Arabia
Halluscoring 2026: The first shared task on llms hallucination detection and answer verification
Sep 29, 2026Aisha Alansari, Abdessalam Bouchekif, Ahmed Hasanaath +9LLM Evaluation LLM Answer Verification
King Fahd University of Petroleum and Minerals, Saudi Arabia · Hamad Bin Khalifa University, Qatar · University of the Basque Country, Spain +5
Attention Dispersion as a Diagnostic Signal for Hallucination in Large Language Models
Sep 16, 2026Shardul P. More, Tanuja S. PawarHallucination Detection LLM Hallucination Detection
Rajarambapu Institute of Technology Ishwarpur, Maharashtra, India
When the Wrong Key Wins: Understanding and Detecting Hallucinations in LLMs
Sep 14, 2026Xuhan Tong, Haoyue Bai, Dawei Zhou +2Language Model Error Detection Hallucination in Language Models
University of Wisconsin–Madison
When Rubrics Fail: Hallucinations Reveal Blind Spots in Medical AI Evaluation
Sep 14, 2026Griffin Farrow, Lily Sijia Li, Jack Johnson +4LLM Hallucination Detection Rubric-Based Evaluation
University of Oxford
NovGauge: A Fine-Grained Benchmark for Diagnosing LLMs' Capability in Paper Novelty Assessment
Sep 12, 2026Guoqiang Zhang, Kexin Tan, Ming Zhang +12LLM Evaluation LLM Answer Verification
College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China · Graduate School of Engineering, The University of Tokyo, Tokyo, Japan · AtomInnoLab 4Atom Infinite Pte. Ltd., Singapore +1
Domain-Specific Hallucination Detection in Large Language Models
Sep 12, 2026Varun Teja Chundru, Debasmita BiswasLanguage Model Error Detection LLM Hallucination Mitigation
Department of Computer Science Purdue University Fort Wayne
Two-Token Features and Small-Large Ensembles for VLM Hallucination Detection
Sep 9, 2026Eli SchwartzHallucination Detection in VLMs Vision-Language Models
IBM Research
CT-SAFR: Safe and Interpretable Chain-of-Thought Reasoning for Autonomous Robots: A Multi-Layered Verification Framework for Trustworthy AI-Driven Robotic Decision Making
Sep 9, 2026Cagri TemelCoT Faithfulness Large Language Model-Based Robot Planning
Hezarfen LLC Seattle, WA, USA · Grand Canyon University Phoenix, AZ, USA
Evidence-Aligned Entity Verification for Hallucination Detection in Retrieval-Augmented Generation
Sep 8, 2026Runsong Jia, Zhen Fang, Mengjia Wu +2Hallucination Detection LLM Hallucination Detection
University of Technology Sydney, Sydney, Australia
CodeTD: Topology of Attention Detects Hallucinations in Code LLMs
Sep 7, 2026Daria Voronkova, Ilya Trofimov, Anton Dmitriev +3LLM Hallucination Detection Code Generation Evaluation
Applied AI Institute · AXXX · IMJ CNRS
Beyond Majority Vote: Multi-Perspective Adjudication for Medical Hallucination Detection
Sep 3, 2026Joe Cecil, Marjorie FreedmanLLM-as-a-Judge Factual Consistency Evaluation
Information Sciences Institute, University of Southern California
HalluPeer: A Taxonomy-driven Benchmark for Detecting Hallucinations in Scientific Peer Reviews
Sep 3, 2026Tzu-Ling Lin, Dong-Ting Yao, Teng-Fang Hsiao +2Hallucination Detection Scholarly Peer Review
National Yang Ming Chiao Tung University
From Tokens to Semantics: Leveraging Complementary Signals for Hallucination Detection in Black-Box LLMs
Sep 2, 2026Urja Pawar, Rajitha Ramanayake, Owen O'Neill +5LLM Hallucination Detection LLM Uncertainty Estimation
BNY
Enoki: Efficient Multi-Level Hallucination Detection
Sep 1, 2026Elisei Rykov, Timur Ionov, Nikolay Ivanov +5Hallucination Detection LLM Information Extraction
Skoltech · AIRI · ITMO University +1
SingProbe Technical Report
Aug 31, 2026Sing TeamLanguage Model Safety Evaluation LLM Guardrails
AI Security Lab, Ant Group
Validating FKG.in: Soundness Assessment in LLM-Augmented Indian Food Knowledge
Aug 29, 2026Saransh Kumar Gupta, Armaan Shah, Lipika Dey +2LLM Auditing LLM Reliability
Department of Computer Science, Ashoka University, Sonipat, Haryana 131029, India · Mphasis AI and Applied Tech Lab, Ashoka University, Sonipat, Haryana 131029, India · Koita Centre for Digital Health, Ashoka University, Sonipat, Haryana 131029, India +1
When Do Supervised UQ Ensembles Improve LLM Hallucination Detection? A Robustness Study
Aug 25, 2026Mohit Singh Chauhan, Vipin Gyanchandani, Dylan BouchardLLM Hallucination Detection LLM Uncertainty Estimation
CVS Health®, Wellesley, MA, USA
Actionable Hallucination Detection: Translating Latent Uncertainty into Agentic Critique
Aug 11, 2026Sanidhya Vijayvargiya, Rahul LokeshLLM Hallucination Mitigation Hallucination in Language Models
Samsung Research America
Prompt Embedding Probes (PEP): Hallucination Detection in LLMs from Hidden States
Aug 8, 2026Zakhar Mrykhin, Valentin MalykhLLM Hallucination Detection Language Model Probing
HSE University
Do LLM Recommenders Know When They're Hallucinating? Auditing Confidence Calibration in Catalog Faithfulness
Aug 7, 2026Srijith RavikumarLLM Auditing Language Model Calibration
Amazon.com LLC Seattle, Washington, USA
Decomposed Entailment for Factuality Checking and Hallucination Detection
Aug 6, 2026Achir Oukelmoun, Nasredine Semmar, Gaël De ChalendarLLM Grounding Claim Verification
CEA LIST NANO INNOV 2 Bd Thomas Gobert, 91120 Palaiseau, France
Detecting Hallucinations and Recovering Verified Answers in Arabic Islamic Question Answering
Aug 4, 2026Khaled ZianiLLM Answer Verification Question Answering
Independent Researcher, Paris, France
Tracing the Cascade: A Topology-Aware Evaluation Framework for Scientific Agent Hallucinations
Aug 1, 2026Xinshun Feng, Ziqi Miao, Lijun Li +1AI Agents for Scientific Discovery Hallucination in Language Models
Shanghai Artificial Intelligence Laboratory Shanghai, China
D-Score: A Spectral Hidden-State Signal for Hallucination Detection in Large Language Models
Jul 27, 2026Bianca Raimondi, Davide Evangelista, Maurizio Gabbrielli +1Language Model Error Detection Hallucination in Language Models
Department of Computer Science and Engineering, University of Bologna, Bologna, Italy
The Cost of Knowing: A Resource-Aware Protocol for Benchmarking Hallucination Beyond Static Leaderboards
Jul 27, 2026Keyu Li, Jin Gao, Dequan WangFactual Consistency Evaluation AI Agent Benchmarks
Shanghai Jiao Tong University Shanghai, China
Reasoning Denoiser: Denoising Reasoning Traces for Hallucination Detection in Large Reasoning Models
Jul 24, 2026Junlin Fang, Do Nguyen-Thanh, Xiaogang Xu +2LLM Hallucination Detection Large Reasoning Models
College of Computing and Data Science, Nanyang Technological University · Zhejiang University · Australian Artificial Intelligence Institute, University of Technology Sydney +1

Oct 6, 2026, cs.CLJ/K move · Enter open · S save

The Labeling Problem in Hallucination Detection Benchmarks: An Empirical Evaluation

Jorma Valjakka, Juhani Kivimäki, Juha Mylläri, Jukka K. Nurminen

Department of Computer Science University of Helsinki

LLM Evaluation LLM-as-a-Judge LLM Hallucination Detection

Open paper AskPDF arXiv

In recent years, several methods for detecting when large language models (LLMs) hallucinate have been developed. These methods are often benchmarked with open-domain question answering (QA) datasets containing questions and corresponding short reference answers. First, an LLM is used to generate answers to questions within the QA dataset. Then, some automated labeling strategy is used to label these answers as hallucinated or not by comparing them with the reference answers in the dataset. This evaluation setting creates a methodological ambiguity between two criteria: reference faithfulness (whether the answer is fully supported by the reference) and factual correctness (whether the answer is free from contradictions and factually false specific claims). In practice, automated labelers may apply the former criterion even when the intended target is the latter. We study this potential criterion mismatch using 900 human-labeled question-answer pairs spanning three commonly used QA datasets and three generator models, with labels targeting answer-level factual correctness. We evaluate lexical similarity metrics, a reference-entailment NLI baseline, and seven LLM judges under controlled prompt variants as automated labelers. Our experiments reveal substantial disagreement both among automated labeling strategies and between these labels and human annotations. Many strategies also exhibit strong directional error biases, and for most judge-generator pairs, replacing a faithfulness-oriented prompt with a factual-correctness prompt improves agreement with human annotations and reduces false-positive dominance, indicating that automated hallucination labels depend strongly on how the target criterion is specified. Label-source choice should therefore be considered a fundamental part of benchmark design and made explicit, validated, and matched with the benchmark goal.

Momentum

Related topics

Latest papers 141