cs.CLOct 19, 2026

Fluent but Fallible: Benchmarking Hallucination and Factual Reliability in Large Language Models

Authors: Osama Shabih, Dayam Nadeem, Shahab Saquib Sohail, Mohammad Nadeem

Organizations: Department of Computer Science and Engineering, SEST, Jamia Hamdard, New Delhi, India · Department of Computer Science, Aligarh Muslim University, Aligarh, India

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Large Language Models (LLMs) have significantly advanced natural language processing and artificial intelligence applications through their capabilities in reasoning, text generation and knowledge retrieval. However, LLMs are still vulnerable to two important challenges: hallucination, generating factually incorrect or fabricated content, and response deviation, meaning systematic deviations from ground-truth responses. These limitations raise serious concerns regarding reliability and trustworthiness in academic and research environments. This study offers a thorough comparative analysis using a variety of metrics of ten advanced LLMs ChatGPT, Gemini, Claude3.5Sonnet, LLaMA3, Mistral, Grok, Perplexity, DeepSeekV3, Falcon, and Gemma based on a benchmark consisting of 70 carefully designed academic prompts across four complexity levels: Basic, Explanation, Advanced, and Tricky, evaluated in February 2025 on specific model versions. Each response was independently evaluated by three reviewers using five metrics: Accuracy, Precision, Recall, Response Deviation Index, and Hallucination Index. Perplexity and DeepSeekV3 achieved the highest overall accuracy, while Gemini demonstrated strong recall and a well-balanced performance profile across all five metrics. LLaMA3 showed strong resistance against hallucinations through a conservative method of generation. Advanced-level prompts proved most challenging across all models, while Tricky prompts were handled more reliably owing to their binary true/false structure. Falcon achieved perfect precision and zero hallucinations, but at the cost of the lowest recall and the highest response deviation. The results show a fundamental trade-off between response completeness and factual reliability, intensified by prompt complexity.

Figures & tables

Research ThemeSub-CategoryKey Works / Models Referenced
Foundational ArchitecturesTransformer & Pre-training; Model Scaling & EmergenceVaswani et al. [ 1 ] (Attention Is All You Need); Devlin et al. [ 2 ] (BERT); Brown et al. [ 4 ] (GPT-3); OpenAI GPT-4 Technical Report [ 23 ] ; Google Gemini [ 24 ] ; Anthropic Claude 3 Report [ 25 ] ; Meta LLaMA 3 [ 26 ]
Benchmarking FrameworksGeneral Multi-Task BenchmarksLiang et al. [ 20 ] (HELM); Hendrycks et al. [ 21 ] (MMLU)
 Text Quality MetricsZhang et al. [ 27 ] (BERTScore); Papineni et al. [ 28 ] (BLEU); Banerjee & Lavie [ 29 ] (METEOR)
Hallucination ResearchTaxonomy & SurveysJi et al. [ 22 ] (ACM Survey on Hallucination); Cao et al. [ 30 ] (Comprehensive Hallucination Survey)
 Truthfulness BenchmarksLin et al. [ 19 ] (TruthfulQA); Maynez et al. [ 18 ] (Faithfulness in Summarisation)
Bias & FairnessBias in LLMsBender et al. [ 10 ] (Stochastic Parrots); HELM Fairness Dimensions (Liang et al. [ 20 ] )
Table 1: Literature Taxonomy: Research Themes and Key Works
YearThemeKey Contribution
2017ArchitectureVaswani et al. [ 1 ] Transformer architecture (“Attention Is All You Need”)
2018Pre-trainingDevlin et al. [ 2 ] BERT: Bidirectional pre-training for language understanding
2019Pre-trainingBrown et al. [ 4 ] GPT-3: Language models as few-shot learners
2020Scaling / RAGLewis et al. [ 9 ] Retrieval-Augmented Generation (RAG) for Knowledge-intensive NLP
2021Bias & SafetyBender et al. [ 10 ] On the Dangers of Stochastic Parrots; Welleck et al. [ 15 ] Unlikelihood Training
2022TruthfulnessLin et al. [ 19 ] TruthfulQA: Measuring how models mimic human falsehoods
Table 2: Chronological Timeline of Key Related Works (2017–2026)
Study / WorkNo. of ModelsDomainMetricsDifficulty LevelsHallucination Focus
Brown et al. [ 4 ] (GPT-3, 2020)1General NLPPerplexity, Accuracy×\times×\times
Liang et al. [ 20 ] (HELM, 2022)30+General Multi-task7+ metrics×\times∼\sim Partial
Lin et al. [ 19 ] (TruthfulQA, 2022)10+TruthfulnessTruthful%, info%×\times✓\checkmark
Cao et al. [ 30 ] (2023)MultipleSurvey / LLMsQualitative Taxonomy×\times✓\checkmark
Hendrycks et al. [ 21 ] (MMLU, 2021)10+Academic KnowledgeAccuracy∼\sim Partial×\times
Maynez et al. [ 18 ] (2020)3–4Text SummarizationFaithfulness Metrics×\times∼\sim Partial
Table 3: Comparative Analysis: Prior Studies vs. Present Study
Figure 1: LLM Evaluation Methodology Pipeline
ModelVersion / IDInterfaceAccess DateTemp.RunsSystem Prompt
ChatGPTGPT-4o (2024-05)chat.openai.comFeb 2025Default1None
GeminiGemini 1.5 Progemini.google.comFeb 2025Default1None
Claude 3.5 Sonnetclaude-3-5-sonnet-20241022claude.aiFeb 2025Default1None
LLaMA 3Meta-Llama-3-70BMeta AI Web UIFeb 2025Default1None
Mistral 7BMistral-7B-Instruct-v0.3chat.mistral.aiFeb 2025Default1None
GrokGrok-1.5grok.x.aiFeb 2025Default1None
Table 4: Model Specifications and Experimental Configuration
Figure 2: Model Positions Hallucination Index (HI) Score Classification Scale
Figure 3: Comparative Performance Analysis of Large Language Models
Figure 4: Detailed Individual Model Performance Profiles
Figure 5: Overall Hallucination Rate (HR) per Model
ModelBasic (n=20)Explanation (n=17)Advanced (n=13)Tricky (n=20)Overall
Gemini0.9501.0000.5381.0000.900
ChatGPT0.8500.9410.8460.9500.900
Mistral0.8500.9410.6921.0000.886
Perplexity0.9500.9410.8460.9500.929
Grok0.9500.8820.8460.8000.871
Claude 3.5 Sonnet0.7001.0000.5381.0000.829
Table 5: Mean Accuracy by Model and Prompt Complexity Level
Figure 6: Performance Profile Radar – 10 LLMs across 5 Metrics
Figure 7: LLM Performance Heatmap - Metric Breakdown in Detail
More in this topic

Hallucination in Language Models

Hallucination in language models studies how systems produce unsupported or false claims and how to detect, explain, and reduce them.

See all 254 papers

Explore similar work

CardsList
May 5, 2026

Not All That Is Fluent Is Factual: Investigating Hallucinations of Large Language Models in Academic Writing

Large Language models (LLMs) show extraordinary abilities, but they are still prone to hallucinations, especially when we use them for generating Academic content. We have investigated four popular LLMs, ChatGPT, Grok, Gemini, and Copilot for hallucinations specifically for academic writing. We have designed 80 prompts across four categories, namely, reference generation, factual explanation, abstract generation, and writing improvement. We evaluated the model using a 0-5 rubric score, which checks factual accuracy, reference validity, coherence, style consistency, and academic tone. A novel weighted metric, Hallucination Index (HI), was introduced to measure hallucination in the responses generated by the models. Some of the most widely used evaluation metrics often fail to check errors which alter sentiment in machine-translated text. We found that Grok and Copilot perform better on reference generation tasks, but they often struggle with abstract or stylistic prompts, with HI values of 0.67 and 0.70, respectively. Whereas, Gemini and ChatGPT have done well with having stronger tone control, but they lack in writing factual tasks and higher hallucination risk with HI scores of 0.53 and 0.57, respectively. Our study found that hallucination behavior does not depend solely on model architecture but also on the type of task and the prompting conditions we are providing. We propose that our work opens new research dimensions for future researchers.

Humam Khan, Md Tabrez Nafis, Shahab Saquib Sohail +2

Department of Computer Science and Engineering, SEST Jamia Hamdard New Delhi, India · Energy, Resources and Industry WSP Calgary, Canada

LLM EvaluationLLM PromptingHallucination in Language Models
AskPDF
Feb 23, 2026

KGHaluBench: A Knowledge Graph-Based Hallucination Benchmark for Evaluating the Breadth and Depth of LLM Knowledge

Large Language Models (LLMs) possess a remarkable capacity to generate persuasive and intelligible language. However, coherence does not equate to truthfulness, as the responses often contain subtle hallucinations. Existing benchmarks are limited by static and narrow questions, leading to limited coverage and misleading evaluations. We present KGHaluBench, a Knowledge Graph-based hallucination benchmark that assesses LLMs across the breadth and depth of their knowledge, providing a fairer and more comprehensive insight into LLM truthfulness. Our framework utilises the KG to dynamically construct challenging, multifaceted questions, whose difficulty is then statistically estimated to address popularity bias. Our automated verification pipeline detects abstentions and verifies the LLM's response at both conceptual and correctness levels to identify different types of hallucinations. We evaluate 25 frontier models, using novel accuracy and hallucination metrics. The results provide a more interpretable insight into the knowledge factors that cause hallucinations across different model sizes. KGHaluBench is publicly available to support future developments in hallucination mitigation.

Alex Robertson, Huizhi Liang, Mahbub Gani +2

School of Computing, Newcastle University · Sage Ai, Sage Group PLC · Current affiliation: Redis

Hallucination in Language ModelsLLM EvaluationLLM Hallucination Detection
AskPDF
May 4, 2026

HalluScan: A Systematic Benchmark for Detecting and Mitigating Hallucinations in Instruction-Following LLMs

Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse natural language processing tasks, yet they remain susceptible to hallucinations -- generating content that is factually incorrect, unfaithful to provided context, or misaligned with user instructions. We present HalluScan, a comprehensive benchmark framework that systematically evaluates hallucination detection and mitigation across 72 configurations spanning 6 detection methods, 4 open-weight model families, and 3 diverse domains. We introduce three key contributions: (1) HalluScore, a novel composite metric that achieves a Pearson correlation of r = 0.41 with human expert judgments; (2) Adaptive Detection Routing (ADR), an intelligent routing algorithm achieving 2.0x cost reduction with only 0.1% AUROC degradation; and (3) systematic error cascade decomposition revealing substantial variation in hallucination error types across domains. Our experiments reveal that NLI Verification achieves the highest overall AUROC of 0.88, while RAV achieves the second-highest AUROC of 0.66.

Ahmed Cherif

1Sofrecom Tunisia, Orange Innovation, Tunis, 1053, Tunisia.

LLM EvaluationLLM Hallucination MitigationHallucination in Language ModelsLLM ReliabilityLLM Hallucination Detection
AskPDF