Factual Knowledge in Language Models

Ask about this topic

Overview

This field studies which facts language models learn from training data, when they can retrieve them, and whether their answers match reliable evidence. Researchers use closed-book question-answering benchmarks such as SimpleQA and TruthfulQA, alongside multilingual, temporal, and long-tail tests; probing and causal tracing help investigate how knowledge is represented and accessed. Common interventions include retrieval-augmented generation, reinforcement learning, fine-tuning, and targeted knowledge editing. These methods can improve recall or update answers, but a central challenge is doing so without introducing hallucinations, losing other knowledge, or leaving gaps in unfamiliar languages, domains, and multi-step reasoning.

Latest papers

CardsList

195 papers36 papers in the last four weeks, up 57% on the four weeks before. 0.2% of all new papers.

  1. Iterate Until Retrieved: Factual Nugget Optimization for Discoverable Continual Corrections in Agentic RAG

    May 25, 2026Moshe Hazoom, Gal Patel, Alon Talmor +1Knowledge EditingAgentic RAG

    Mosaic AI · The Hebrew University of Jerusalem

  2. Understanding Data Temporality Impact on Large Language Models Pre-training

    May 21, 2026Hippolyte Pilchen, Romain Fabre, Franck Signe Talla +2Language Model PretrainingTemporal Reasoning in Language Models

    Kyutai, Paris

  3. Relational Linear Properties in Language Models: An Empirical Investigation

    May 21, 2026Giovanni Valer, Luigi Gresele, Marco Bronzini +1LLM InterpretabilityFactual Knowledge in Language Models

    Fondazione Bruno Kessler, Trento, Italy · University of Bologna, Bologna, Italy · University of Pisa, Pisa, Italy +2

  4. Do Factual Recall Mechanisms Carry over from Text to Speech in Multimodal Language Models?

    May 21, 2026Luca Modica, Filip Landin, Mehrdad Farahani +3Multimodal Large Language ModelsFactual Knowledge in Language Models

    Zenseact · Chalmers University of Technology · University of Gothenburg +2

  5. Predictable Confabulations: Factual Recall by LLMs Scales with Model Size and Topic Frequency

    May 18, 2026Matthew L. Smith, Jonathan P. Shock, Samuel T. Segun +2LLM EvaluationLanguage Model Scaling Laws

    International Development Research Centre Canada · University of Cape Town South Africa · Global Center on AI Governance Canada +2

  6. MixSD: Mixed Contextual Self-Distillation for Knowledge Injection

    May 16, 2026Jiarui Liu, Lechen Zhang, Yongjin Yang +5Knowledge Augmentation for Language ModelsFactual Knowledge in Language Models

    Carnegie Mellon University · Jinesis Lab, University of Toronto & Vector Institute · University of Illinois Urbana-Champaign +6

  7. Negation Neglect: When models fail to learn negations in training

    May 13, 2026Harry Mayne, Lev McKinney, Jan Dubiński +3LLM ReliabilityLLM Safety

    University of Oxford · University of Toronto · 3Warsaw University of Technology +6

  8. Large Language Models Lack Temporal Awareness of Medical Knowledge

    May 13, 2026Zihan Guan, Qiao Jin, Guangzhi Xiong +6HealthcareTemporal Reasoning in Language Models

    University of Virginia · National Institutes of Health · Dana-Farber Cancer Institute +2

  9. Geometric Factual Recall in Transformers

    May 12, 2026Shauli Ravfogel, Gilad Yehudai, Joan Bruna +1Neural Representation GeometryMemorization in Language Models

    1New York University · 2Flatiron Institute

  10. GKnow: Measuring the Entanglement of Gender Bias and Factual Gender

    May 12, 2026Leonor Veloso, Hinrich SchützeGender Bias in Language ModelsLLM Interpretability

    Center for Information and Language Processing, LMU Munich · Munich Center for Machine Learning (MCML)

  11. Three Regimes of Context-Parametric Conflict: A Predictive Framework and Empirical Validation

    May 12, 2026Pruthvinath Jeripity VenkataLLM GroundingKnowledge Conflicts in Language Models

    Independent Researcher

  12. The Geometry of Forgetting: Temporal Knowledge Drift as an Independent Axis in LLM Representations

    May 9, 2026Rania Elbadry, Ahmed Heakl, Fan Zhang +4LLM InterpretabilityLLM Reliability

    Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) · INSAIT, Sofia University

  13. Beyond Reasoning: Reinforcement Learning Unlocks Parametric Knowledge in LLMs

    May 8, 2026Wanli Yang, Hongyu Zang, Junwei Zhang +5RL for Language ModelsFactual Knowledge in Language Models

    University of Chinese Academy of Sciences · State Key Laboratory of AI Safety, Institute of Computing Technology, CAS

  14. Attractor Geometry of Transformer Memory: From Conflict Arbitration to Confident Hallucination

    May 7, 2026Qiyao Liang, Risto Miikkulainen, Ila FieteMemorization in Language ModelsKnowledge Conflicts in Language Models

    Massachusetts Institute of Technology · University of Texas Austin, Cognizant · Cognizant

  15. In-Context Examples Suppress Scientific Knowledge Recall in LLMs

    Apr 30, 2026Chaemin Jang, Woojin Park, Hyeok Yun +2Knowledge Conflicts in Language ModelsScientific Reasoning

    Korea Advanced Institute of Science and Technology · Shanghai Jiao Tong University

  16. MoRFI: Monotonic Sparse Autoencoder Feature Identification

    Apr 29, 2026Dimitris Dimakopoulos, Shay B. Cohen, Ioannis KonstasLLM Hallucination MitigationLLM Interpretability

    University of Edinburgh, UK · Heriot-Watt University, UK

  17. Incompressible Knowledge Probes: Estimating Black-Box LLM Parameter Counts via Factual Capacity

    Apr 27, 2026Bojie LiParameter EstimationLLM Evaluation

    Pine AI

  18. Programming with Data: Test-Driven Data Engineering for Self-Improving LLMs from Raw Corpora

    Apr 27, 2026Chenkai Pan, Xinglong Xu, Yuhang Xu +6LLM EvaluationKnowledge Augmentation for Language Models

    Zhejiang University · University of Chinese Academy of Sciences · Shanghai Artificial Intelligence Laboratory

  19. Revisiting Non-Verbatim Memorization in Large Language Models: The Role of Entity Surface Forms

    Apr 23, 2026Yuto Nishida, Naoki Shikoda, Yosuke Kishinami +4LLM EvaluationMemorization in Language Models

    Nara Institute of Science and Technology · Future Corporation

  20. Tracing Relational Knowledge Recall in Large Language Models

    Apr 21, 2026Nicholas Popovič, Michael FärberLinear ProbingFactual Knowledge in Language Models

    TU Dresden & ScaDS.AI Dresden/Leipzig, Germany

  21. Structure Guided Retrieval-Augmented Generation for Factual Queries

    Apr 21, 2026Miao Xie, Xiao Zhang, Yi Li +1Graph Retrieval-Augmented GenerationFactual Knowledge in Language Models

    College of Information and Electrical Engineering, China Agricultural University, China · College of Computing and Data Science, Nanyang Technological University, Singapore · Key Laboratory of Agricultural Informatization Standardization, Ministry of Agriculture and Rural Affairs, China

  22. Does Self-Consistency Improve the Recall of Encyclopedic Knowledge?

    Apr 21, 2026Sho Hoshino, Ukyo Honda, Peinan ZhangSelf-Consistency DecodingCoT Reasoning

    CyberAgent

  23. KnowledgeBerg: Evaluating Systematic Knowledge Coverage and Compositional Reasoning in Large Language Models

    Apr 19, 2026Xiao Zhang, Qianru Meng, Yongjian Chen +2LLM EvaluationCompositional Reasoning

    University of Groningen · LIACS, Leiden University

  24. One Mask to Rule Them All: On Hidden Facts after Editing and How to Find Them

    Apr 18, 2026Ali Holmov, Paul Youssef, Nandi Schoots +1Knowledge EditingTransformer Interpretability

    Technical University of Munich · Marburg University · University of Oxford

  25. Incentivizing Parametric Knowledge via Reinforcement Learning with Verifiable Rewards for Cross-Cultural Entity Translation

    Apr 18, 2026Jiang Zhou, Xiaohu Zhao, Xinwei Wu +8RL for Language ModelsFactual Knowledge in Language Models

    TJUNLP Lab, Tianjin University, China · Alibaba Group, China

  26. BAGEL: Benchmarking Animal Knowledge Expertise in Language Models

    Apr 17, 2026Jiacheng Shen, Masato Hagiwara, Milad Alizadeh +9LLM EvaluationQuestion Answering

    2NYU Center for Data Science; NYU Shanghai · 1Earth Species Project · 3NYU Shanghai Center for Data Science; NYU-ECNU Institute of Mathematical Sciences at NYU Shanghai

  27. CURaTE: Continual Unlearning in Real Time with Ensured Preservation of LLM Knowledge

    Apr 16, 2026Seyun Bae, Seokhan Lee, Eunho YangPrivacy-Preserving Language ModelsFactual Knowledge in Language Models

    1KT Corporation · 2KAIST

  28. FABLE: Fine-grained Fact Anchoring for Unstructured Model Editing

    Apr 14, 2026Peng Wang, Biyu Zhou, Xuehai Tang +2Model EditingLanguage Model Editing

    Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences

  29. Masked by Consensus: Disentangling Privileged Knowledge in LLM Correctness

    Apr 14, 2026Tomer Ashuach, Shai Gretz, Yoav Katz +2Language Model IntrospectionLanguage Model Self-Assessment

    Technion – Israel Institute of Technology · IBM Research · Kempner Institute, Harvard University

  30. FAITH: Factuality Alignment through Integrating Trustworthiness and Honestness

    Apr 11, 2026Xiaoning Dong, Chengyan Wu, Yajie Wen +5LLM AlignmentFactual Knowledge in Language Models

    Tsinghua University, Institute for Interdisciplinary Information Sciences · Shanghai Qi Zhi Institute · South China Normal University, School of Electronic Science and Engineering +2