Factual Knowledge in Language Models

Overview

This field studies which facts language models learn from training data, when they can retrieve them, and whether their answers match reliable evidence. Researchers use closed-book question-answering benchmarks such as SimpleQA and TruthfulQA, alongside multilingual, temporal, and long-tail tests; probing and causal tracing help investigate how knowledge is represented and accessed. Common interventions include retrieval-augmented generation, reinforcement learning, fine-tuning, and targeted knowledge editing. These methods can improve recall or update answers, but a central challenge is doing so without introducing hallucinations, losing other knowledge, or leaving gaps in unfamiliar languages, domains, and multi-step reasoning.

Latest papers

CardsList

212 papers25 papers in the last four weeks, down 32% on the four weeks before. 0.2% of all new papers.

  1. The Truth Beneath the Tokens: Prompt-Space Recovery of Knowledge Edits in Language Models

    Nov 7, 2026Manit Baser, Rachel Yeo Hui Min, Aditya Nawal +2Knowledge EditingFactual Knowledge in Language Models

  2. FondueQA: A Benchmark for Evaluating Parametric Compensation in Multi-Hop QA

    Nov 6, 2026Laurin Schmid, Suman Navaratnarajah, Koel Dutta Chowdhury +3Factual Knowledge in Language ModelsMulti-Hop QA

  3. LLMs Don't Know What They Know: Benchmarking Temporal Self-Knowledge for Financial Backtesting

    Nov 4, 2026Lingfei Qian, Yan Wang, Xueqing Peng +12Financial Language Model EvaluationLanguage Model Self-Assessment

  4. Feeling of Knowing in Large Language Models

    Nov 3, 2026Zichuan Fu, Xian Wu, Jingtong Gao +9Metacognition in Language ModelsFactual Knowledge in Language Models

  5. ViAdminShift: A Vietnamese Benchmark for Event-Level Knowledge Updates Built from the 2025 Administrative Reform

    Nov 2, 2026Phan Binh Nguyen Lam, Nguyen Thai AnhFactual Knowledge in Language ModelsLanguage Modeling

  6. They Know Where It Is, But Not What It Says: A Bilingual Hallucination Benchmark of Hadith Recall in Open LLMs

    Nov 2, 2026Dayam Nadeem, Safdar Mustafa, Ameen Ur Rehman +3Hallucination in Language ModelsLLM Evaluation

  7. Tracing How Language Models Integrate Temporal Anchors into Factual Recall

    Nov 1, 2026Ayana Niwa, Masahiro Kaneko, Kentaro InuiFactual Knowledge in Language ModelsLLM Interpretability

  8. Cross-Prompt Reuse of Entity-Replacement Responses

    Oct 30, 2026Jinkun Chen, Xuemin Yu, Sijia Han +1LLM PromptingFactual Knowledge in Language Models

  9. ImplicitHop: Evaluating knowledge integration across context and memory

    Oct 30, 2026Alexander Detkov, Dashiel Karish, Matthew Oleynikov +1Knowledge Retrieval in Language ModelsLanguage Model Reasoning Evaluation

  10. What Improves During Pretraining? Tracking Knowledge Acquisition with Point-in-Time Models

    Oct 27, 2026Chang Su, Bin Wu, Emine Yilmaz +1Language Model PretrainingFactual Knowledge in Language Models

  11. DynEdit: Dynamic Entropy-Guided Sequential Editing for Large Language Models

    Oct 26, 2026Jinhu Fu, Yan Bai, Yihang Lou +4Knowledge EditingLanguage Model Editing

  12. How Much Information Can We Fit inside a Transformer?

    Oct 25, 2026Amirhossein Basareh, Rishabh Sanjay, Daniel de Mello +1Memorization in Language ModelsTransformer

  13. Mostly Reading: What Open-Book Scores Measure in Document-Level LoRA Adapters

    Oct 22, 2026Martino M. L. Pulici, Cuong Xuan Chu, Evgeny Kharlamov +1Low-Rank AdaptationLanguage Model Merging

  14. Factual Access States in Diffusion Language Models: From Mechanism to Knowledge Editing and Retrieval

    Oct 20, 2026Yuchen Guo, Zhuoyuan Mao, Yiwen GuoFactual Knowledge in Language ModelsDiffusion Language Models

  15. DiaLong: Benchmarking Reminiscence in Long-Context Dialogues

    Oct 18, 2026Owen Bianchi, Samer Aslan, Ryunosuke SaitoConversational MemoryLong-Context Language Model Evaluation

  16. Mixture-of-Memories: Low-Overhead Language Modeling with Modular Knowledge Injection

    Oct 18, 2026Yifan Lu, Rubin Wei, Ziyun Zou +8Memory-Augmented Language ModelsKnowledge Augmentation for Language Models

  17. Truth in Contrast: Calibrated Factual Evidence from Language Model Likelihoods

    Oct 16, 2026Ponhvoan Srey, Jason Lucas, Pittawat Taveekitworachai +2Confidence Estimation in Language ModelsLanguage Model Calibration

  18. On the Tip of the Tongue: First-Token Cueing as a Behavioral Probe of Recall Failures in LLMs

    Oct 16, 2026Jaewon Sok, Jewon Yeom, Seonghyeon Park +3Factual Knowledge in Language Models

  19. Knowledge That Is Linear Before Fine-Tuning Generalizes After It

    Oct 15, 2026Faridoun Mehri, Hinrich SchützeLanguage Model GeneralizationFactual Knowledge in Language Models