Factual Knowledge in Language Models

Follow this topic Ask about this topic

Momentum

18 papers in the last four weeks, up 38% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 117

All topics

Multi-Hop Knowledge Composition is Bound by Pretraining Exposure
Jun 8, 2026Yannis Karmim, Luis Marti, Djamé Seddah +1Language Model Pretraining Implicit Reasoning in Language Models
Inria, Paris, France · Inria Chile, Las Condes, Chile · Dept. of Computer Science, Univ. de Chile +1
MechLens: Late Crystallization of Factual Knowledge Explains Intervention Effectiveness in Language Models
Jun 6, 2026Xueping GaoMemorization in Language Models LLM Interpretability
Alibaba Cloud
Improving Cross-Lingual Factual Recall via Consistency-Driven Reinforcement Learning
Jun 4, 2026Jonathan von Rad, Louis Arts, George Burgess +6Multilingual Language Model Evaluation RL for Language Models
University College London, Centre for Artificial Intelligence
Can LLMs Be Constrained to the Past? Improving Knowledge Cutoff through Recall-Based Prompting
Jun 4, 2026Michiro Asai, Ailiang Lin, Yu Kishimoto +4LLM Prompting Factual Knowledge in Language Models
Institute of Science Tokyo
Knowledge Index of Noah's Ark
Jun 3, 2026Sheng Jin, Minghao Liu, Yunze Xiao +24LLM Evaluation Human-in-the-Loop Annotation
12077AI · 2M-A-P · 3Carnegie Mellon University +7
Expert-Aware Causal Tracing of Factual Recall in Sparse MoE Language Models
Jun 2, 2026Yuetian Lu, Ali Modarressi, Yihong Liu +1Mixture-of-Experts Language Models Sparse Mixture-of-Experts
Center for Information and Language Processing (CIS), LMU Munich
CARTE: A Benchmark for Mapping Language Model Knowledge Across France
Jun 1, 2026Sarah Almeida Carneiro, Christos Xypolopoulos, Xiao Fei +2LLM Evaluation Factual Knowledge in Language Models
École Polytechnique, Paris · École Polytechnique, Institut Polytechnique de Paris, France · École Polytechnique, Paris, NTUA +3
Evaluating Cross-lingual Knowledge Consistency in Code-Mixed vis-a-vis Indian Languages using IndicKLAR
May 28, 2026Debajyoti Mazumder, Divyansh Pathak, Prashant Kodali +3Multilingual Language Model Evaluation Cross-Lingual Reasoning in Language Models
Indian Institute of Science Education and Research, Bhopal, India · Microsoft Corporation · UNSW Sydney, Australia
Fine-Tuning Dynamics of In-Context Factual Recall in Transformers
May 26, 2026Ruomin Huang, Eshaan Nichani, Jason D. Lee +1Supervised Fine-Tuning Transformer Attention
Duke University · Princeton University · UC Berkeley
Do Models Know Why They Changed Their Mind? Interpretability and Faithfulness of Chain-of-Thought Under Knowledge Conflict
May 26, 2026Pruthvinath Jeripity VenkataCoT Faithfulness Faithfulness of Language Model Explanations
Independent Researcher
The Future of Facts: Tracing the Factual Generation-Verification Gap
May 26, 2026Tim R. Davidson, Anja Surina, Caglar GulcehreKnowledge Conflicts in Language Models Factual Knowledge in Language Models
EPFL
PersLitEval: Fine-grained Benchmark and Evaluation of LLMs on Persian Literature Questions
May 26, 2026Ruhallah Niazi, Faeze Ghorbanpour, Alexander FraserLLM Evaluation Question Answering
School of Computation, Information and Technology, TU Munich · Munich Center for Machine Learning (MCML)
Beyond Questions: Evaluating What Large Language Models (Actually) Know
May 26, 2026Luca Giordano, Simon RazniewskiLLM Evaluation Factual Knowledge in Language Models
ScaDS.AI Dresden/Leipzig & TU Dresden, Germany
Composition Collapse: Stable Factual Knowledge Does Not Imply Compositional Reasoning
May 26, 2026Zhe Yu, Wenpeng Xing, Yunzhao Wei +4Compositional Reasoning Reasoning Evaluation
Binjiang Institute of Zhejiang University · Zhejiang University · Hong Kong Baptist University +2
The Labyrinth and the Thread: Rethinking Regularizations in Sequential Knowledge Editing for Large Language Models
May 26, 2026Zheng Wang, Kaixuan Zhang, Wanfang Chen +2Knowledge Editing Factual Knowledge in Language Models
Bosch Center for Artificial Intelligence (BCAI) · Bosch (China) Investment Ltd. · School of Statistics, East China Normal University.
Iterate Until Retrieved: Factual Nugget Optimization for Discoverable Continual Corrections in Agentic RAG
May 25, 2026Moshe Hazoom, Gal Patel, Alon Talmor +1Knowledge Editing Agentic RAG
Mosaic AI · The Hebrew University of Jerusalem
Understanding Data Temporality Impact on Large Language Models Pre-training
May 21, 2026Hippolyte Pilchen, Romain Fabre, Franck Signe Talla +2Language Model Pretraining Temporal Reasoning in Language Models
Kyutai, Paris
Relational Linear Properties in Language Models: An Empirical Investigation
May 21, 2026Giovanni Valer, Luigi Gresele, Marco Bronzini +1LLM Interpretability Factual Knowledge in Language Models
Fondazione Bruno Kessler, Trento, Italy · University of Bologna, Bologna, Italy · University of Pisa, Pisa, Italy +2
Do Factual Recall Mechanisms Carry over from Text to Speech in Multimodal Language Models?
May 21, 2026Luca Modica, Filip Landin, Mehrdad Farahani +3Multimodal Large Language Models Factual Knowledge in Language Models
Zenseact · Chalmers University of Technology · University of Gothenburg +2
Predictable Confabulations: Factual Recall by LLMs Scales with Model Size and Topic Frequency
May 18, 2026Matthew L. Smith, Jonathan P. Shock, Samuel T. Segun +2LLM Evaluation Language Model Scaling Laws
International Development Research Centre Canada · University of Cape Town South Africa · Global Center on AI Governance Canada +2
MixSD: Mixed Contextual Self-Distillation for Knowledge Injection
May 16, 2026Jiarui Liu, Lechen Zhang, Yongjin Yang +5Knowledge Augmentation for Language Models Factual Knowledge in Language Models
Carnegie Mellon University · Jinesis Lab, University of Toronto & Vector Institute · University of Illinois Urbana-Champaign +6
Negation Neglect: When models fail to learn negations in training
May 13, 2026Harry Mayne, Lev McKinney, Jan Dubiński +3LLM Reliability LLM Safety
University of Oxford · University of Toronto · 3Warsaw University of Technology +6
Large Language Models Lack Temporal Awareness of Medical Knowledge
May 13, 2026Zihan Guan, Qiao Jin, Guangzhi Xiong +6Healthcare Temporal Reasoning in Language Models
University of Virginia · National Institutes of Health · Dana-Farber Cancer Institute +2
Geometric Factual Recall in Transformers
May 12, 2026Shauli Ravfogel, Gilad Yehudai, Joan Bruna +1Neural Representation Geometry Memorization in Language Models
1New York University · 2Flatiron Institute
GKnow: Measuring the Entanglement of Gender Bias and Factual Gender
May 12, 2026Leonor Veloso, Hinrich SchützeGender Bias in Language Models LLM Interpretability
Center for Information and Language Processing, LMU Munich · Munich Center for Machine Learning (MCML)
Three Regimes of Context-Parametric Conflict: A Predictive Framework and Empirical Validation
May 12, 2026Pruthvinath Jeripity VenkataLLM Grounding Knowledge Conflicts in Language Models
Independent Researcher
The Geometry of Forgetting: Temporal Knowledge Drift as an Independent Axis in LLM Representations
May 9, 2026Rania Elbadry, Ahmed Heakl, Fan Zhang +4LLM Interpretability LLM Reliability
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) · INSAIT, Sofia University
Beyond Reasoning: Reinforcement Learning Unlocks Parametric Knowledge in LLMs
May 8, 2026Wanli Yang, Hongyu Zang, Junwei Zhang +5RL for Language Models Factual Knowledge in Language Models
University of Chinese Academy of Sciences · State Key Laboratory of AI Safety, Institute of Computing Technology, CAS
Attractor Geometry of Transformer Memory: From Conflict Arbitration to Confident Hallucination
May 7, 2026Qiyao Liang, Risto Miikkulainen, Ila FieteMemorization in Language Models Knowledge Conflicts in Language Models
Massachusetts Institute of Technology · University of Texas Austin, Cognizant · Cognizant
In-Context Examples Suppress Scientific Knowledge Recall in LLMs
Apr 30, 2026Chaemin Jang, Woojin Park, Hyeok Yun +2Knowledge Conflicts in Language Models Scientific Reasoning
Korea Advanced Institute of Science and Technology · Shanghai Jiao Tong University

Jun 8, 2026, cs.CLJ/K move · Enter open · S save

Multi-Hop Knowledge Composition is Bound by Pretraining Exposure

Yannis Karmim, Luis Marti, Djamé Seddah, Valentin Barrière

Inria, Paris, France · Inria Chile, Las Condes, Chile · Dept. of Computer Science, Univ. de Chile+1

Language Model Pretraining Implicit Reasoning in Language Models Factual Knowledge in Language Models Multi-Hop Reasoning

Open paper AskPDF arXiv

Large Language Models fail at implicit multi-hop reasoning: a model answers "When was

X

born?" and "Who is

Y

's closest friend?" correctly but fails on "When was

Y

's closest friend born?" in a single forward pass, even when both facts are perfectly memorized and individually retrievable. We study this failure in a controlled natural language setting with a strict separation between individuals exposed to compositional contexts during pretraining and those that never appear in any such context. We confirm that compositional failure persists even at 97% 1-hop accuracy, establishing the gap as a pretraining failure rather than a knowledge absence. We propose and test nine data-centric augmentation formats and find that compositional pretraining transfers to unseen questions for exposed individuals, but never to individuals absent from compositional pretraining, suggesting that exposure to compositional contexts during pretraining is a necessary condition for implicit multi-hop reasoning. Code is available at https://github.com/ykrmm/composition-bound .

Momentum

Related topics

Latest papers 117