cs.CLOct 23, 2024

Multilingual Hallucination Gaps in Large Language Models

Authors: Cléa Chataigner, Afaf Taïk, Golnoosh Farnadi

Organizations: Mila, Quebec AI Institute, Quebec, Canada · McGill University, Quebec, Canada · Université de Montréal, Quebec, Canada

Abstract

Large language models (LLMs) are increasingly used as alternatives to traditional search engines given their capacity to generate text that resembles human language. However, this shift is concerning, as LLMs often generate hallucinations, misleading or false information that appears highly credible. In this study, we explore the phenomenon of hallucinations across multiple languages in freeform text generation, focusing on what we call multilingual hallucination gaps. These gaps reflect differences in the frequency of hallucinated answers depending on the prompt and language used. To quantify such hallucinations, we used the FactScore metric and extended its framework to a multilingual setting. We conducted experiments using LLMs from the LLaMA, Qwen, and Aya families, generating biographies in 19 languages and comparing the results to Wikipedia pages. Our results reveal variations in hallucination rates, especially between high and low resource languages, raising important questions about LLM multilingual performance and the challenges in evaluating hallucinations in multilingual freeform text generation.

Figures & tables

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. MultiHaluDet: Multilingual Hallucination Detection via LLM Hidden State Probing

    May 24, 2026Riasad Alvi, Nurul Labib Sayeedi, Md. Faiyaz Abdullah SayeediHallucination DetectionHallucination in Language Models

  2. CrossHallu: Do Hallucination Signals Generalize Across Languages and Domains in Large Language Model's Internals?

    Jul 4, 2026Aisha Alansari, Malak Alkhorasani, Hamzah LuqmanMultilingual Language Model EvaluationLLM Hallucination Detection

  3. A multilingual hallucination benchmark: MultiWikiQHalluA

    May 4, 2026Freja Thoresen, Dan Saattrup SmartMultilingual Language ModelsMultilingual Language Model Evaluation