Factual Knowledge in Language Models

Follow this topic Ask about this topic

Momentum

18 papers in the last four weeks, up 38% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 117

All topics

EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory
Oct 7, 2026Hongru Cai, Ran Wei, Wenjie Wang +4Knowledge Editing Factual Knowledge in Language Models
The Hong Kong Polytechnic University · Hangzhou Diagens Biotechnology Co., Ltd · University of Science and Technology of China
Distilling Directional Verification
Oct 1, 2026Jungseob Lee, Sugyeong Eo, Seongtae Hong +4Question Answering Language Model Distillation
Korea University · Yonsei University Mirae Campus · Soongsil University +1
The Geometry of Contextual Relations: Language Models Address Facts by Order of Mention
Oct 1, 2026Yufa ZhouIn-Context Retrieval Factual Knowledge in Language Models
Duke University
MedKIT: Evaluating Knowledge Integration and Generalization in Large Language Models
Sep 29, 2026Lukas Thede, Yash Kumar Atri, David Chen +4Continual Learning for LLMs Factual Knowledge in Language Models
University of Tübingen, Tübingen AI Center · Helmholtz Munich · Munich Center for Machine Learning (MCML) +4
AraDynFact: Dynamic Evaluation of Factual Knowledge in Arabic
Sep 28, 2026Ignacio Iacobacci, Faroq Altam, Zhaozhi Qian +1LLM Evaluation Arabic NLP
Elm Company
MemoReason: Evaluating the Effect of Parametric Memory on Contextual Reasoning in LLMs
Sep 28, 2026Zineddine Tighidet, Andrea Mogini, Jiali Mei +2LLM Evaluation Memorization in Language Models
BNP Paribas Paris, France · Sorbonne Université Paris, France · Criteo AI Lab
Fair Fact-Checking: Closing the Cross-Lingual Gap in LLM Factual Judgement with RoSh
Sep 28, 2026Muhammad Ahmad, Fatemeh Seyedin, Adrian Weller +2Multilingual Language Model Evaluation Automated Fact-Checking
Department of Computer Science, Shifa Tameer-e-Millat University, Islamabad, Pakistan · BRAINS, Brandenburg Research Center for Applied Intelligent Systems, Potsdam, Germany · Max Planck Institute for Security and Privacy, Bochum, Germany +3
Making LLMs Truly Forget: Deep Unlearning by Searching, Selecting, and Severing Knowledge Paths
Sep 28, 2026Jialu Wang, Peizhi Niu, Haoteng Yin +3Factual Knowledge in Language Models Machine Unlearning
Tongji University · University of Illinois Urbana-Champaign · Purdue University +1
Positions Are Not Facts: The Mismatch Between KV Caches and Memory
Sep 27, 2026Changhai Zhou, Yuhua Zhou, Shiyang Zhang +5Knowledge Editing Knowledge Conflicts in Language Models
Fudan University · Baidu · Zhejiang University +2
Knowing Is Not Choosing: What Explicit Verification Adds Beyond Generative Preference
Sep 27, 2026Yilong Li, Chengpo Yan, Aayan Arish +1LLM Evaluation LLM Answer Verification
Department of Computer Science University of Wisconsin-Madison Madison, WI 53703, USA
ALOE: Semantically Addressed Low-Rank Operators for Knowledge Editing
Sep 24, 2026Zeyan Li, Hu Xu, Jianfeng XuFactual Knowledge in Language Models Language Model Editing
Shanghai Jiao Tong University
Memory vs. Context? Influential Factors of Factual Recall in Language Models
Sep 21, 2026Guilhem Fouilhé, Nicholas Asher, Philippe MullerPrompt Sensitivity Memorization in Language Models
IRIT ANITI · IRIT CNRS · IRIT Université de Toulouse
Time-Incremental Continued Pretraining of LLMs: Knowledge Updates Without Catastrophic Forgetting
Sep 20, 2026Fırat Öncel, Salman Hussain Ali, Mirco Ravanelli +2Continual Learning for LLMs Factual Knowledge in Language Models
Concordia University · Mila – Quebec AI Institute · Université de Montréal +3
Style-Debiased DPO: Updating LLM Knowledge with Factuality-Aware Synthetic Preference Data
Sep 15, 2026Takayuki Yamamoto, Daisuke KawaharaKnowledge Editing Direct Preference Optimization
Waseda University
The Cost of Compression: A Rate-Distortion Limit on Factual Hallucination
Sep 14, 2026Xi Wang, Shijia Xu, Rongfeng GuoMemorization in Language Models Hallucination in Language Models
Hefei Institutes of Physical Science, Chinese Academy of Sciences, China · University of Science and Technology of China, China · Chongqing University, China +1
ORQA: An Occupation-Realistic Question and Answer Framework for LLM Professional Knowledge
Sep 14, 2026Shreyas Krishnan, Serina Chang, Abhishek NagarajLLM Evaluation Factual Knowledge in Language Models
University of California, Berkeley · National Bureau of Economic Research (NBER)
From Parameters to Answers: How LLMs Retrieve and Use Their Internal Knowledge
Sep 12, 2026Wenkang Wei, Yuan Fang, Renhe Jiang +2LLM Interpretability Factual Knowledge in Language Models
University of Science and Technology of China Hefei, China · Singapore Management University Singapore, Singapore · The University of Tokyo Tokyo, Japan +1
SWORD: Wikidata-based Distortions Reveal Hidden Cross-Lingual Inconsistencies in LLM Factual Error Rejection
Sep 8, 2026Sanghyeok Park, Minji Kang, Hosung Kwak +1Multilingual Language Model Evaluation Factual Knowledge in Language Models
Soongsil University, Seoul, Republic of Korea · KAIST, Daejeon, Republic of Korea
Popular Knowledge Propagates More Errors in LLM Knowledge Updating
Sep 8, 2026Yuji Zhang, Weibing Wang, Cheng Qian +5Knowledge Editing Factual Knowledge in Language Models
University of Illinois Urbana-Champaign · City University of New York · Massachusetts Institute of Technology +1
Knowledge Acquisition During Pre-training? Large Language Models Learn Better With Auxiliary Views
Sep 3, 2026Joseph Lee, Yidi Huang, Dokyoon Kim +2Language Model Pretraining Data Augmentation
University of Pennsylvania, Philadelphia, PA, USA
Knowledge Distillation During Mid-Training Favors Reasoning over Factual Recall
Sep 1, 2026Jacqueline He, Howard Yen, Shuyue Stella Li +9Language Model Distillation Factual Knowledge in Language Models
Meta AI · University of Washington · Princeton University
Probing Factual Knowledge Transfer with Training Data Interventions
Sep 1, 2026Romina Oji, Marc Braun, Marcel Bollmann +2Language Model Pretraining Multilingual Language Model Evaluation
Department of Computer and Information Science Linköping University
LLMPEDIA: Browsing, Verifying, and Comparing the Parametric Encyclopedic Knowledge of LLMs
Sep 1, 2026Muhammed Saeed, Simon RazniewskiLLM Evaluation LLM Auditing
ScaDS.AI Dresden/Leipzig & TU Dresden, Germany
Right Frame, Wrong Rule: Cultural Cues Expose the Financial Knowledge Gap They Were Meant to Close
Sep 1, 2026Rania Elbadry, Ahmed Heakl, Saeed Almheiri +12Financial Services Cultural Bias in Language Models
♠MBZUAI · ♢The University of Tokyo · §The Fin AI +4
Stick to What You Know: A Study of Knowledge-Aligned Supervised Fine-Tuning
Aug 31, 2026Arthur Becker, Jakob Kemmler, David Thulke +3Supervised Fine-Tuning LLM Hallucination Mitigation
AppTek GmbH, Aachen, Germany · Machine Learning and Human Language Technology, RWTH Aachen University, Germany
Dense Clinical Contrasts Enhance Medical Knowledge Updating in Large Language Models
Aug 31, 2026Yangmin Huang, Shu Quan, He Geng +5Supervised Fine-Tuning Factual Knowledge in Language Models
Xunfei Healthcare Technology Co., Ltd. · USTC-Xunfei Healthcare Digital and Health Joint Laboratory
Whether LLMs Can Navigate Beliefs and Facts Depends on How You Phrase It
Aug 18, 2026Quang Minh Nguyen, Luis Frentzen SalimKnowledge Conflicts in Language Models Pragmatic Reasoning in Language Models
1KAIST · 2National Taiwan University of Science and Technology
LittleLearner: Language Models Under Pedagogically Controlled Knowledge Exposure
Aug 13, 2026Fanfei Li, Jana Zeller, Manuel Prada-Corral +4Language Model Pretraining Knowledge Augmentation for Language Models
ETHZ
Toward a Gricean Retreat: Probing LLMs for Knowledge Boundaries and Referent Specificity
Aug 13, 2026Dananjay Srinivas, Saksham Khatwani, Maria PachecoPragmatic Reasoning in Language Models Factual Knowledge in Language Models
University of Colorado, Boulder
Information Abundance Paradox: Long-Context Training Undermines Parametric Knowledge
Aug 12, 2026Arda Uzunoglu, Benjamin Van Durme, Daniel KhashabiLong-Context Language Modeling Factual Knowledge in Language Models
Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA

Oct 7, 2026, cs.CLJ/K move · Enter open · S save

EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory

Hongru Cai, Ran Wei, Wenjie Wang, Chengfa Wu+3

The Hong Kong Polytechnic University · Hangzhou Diagens Biotechnology Co., Ltd · University of Science and Technology of China

Knowledge Editing Factual Knowledge in Language Models Memory-Augmented Language Models

Open paper AskPDF arXiv

Conditional memory architectures such as DeepSeek Engram use input n-grams to look up learned embeddings, expanding the capacity of large language models (LLMs) with limited additional computation. Beyond model scaling, this architecture has demonstrated the potential to decouple factual knowledge storage from general-purpose computation, offering a promising route to updating factual knowledge while keeping the Transformer backbone fixed. Realizing this potential is challenging because different expressions of a fact may activate different n-gram embeddings, while updating shared embeddings can unintentionally change the model's predictions about other facts. We propose EngramEdit for decoupled knowledge updates through conditional memory. EngramEdit first computes target memory representations that make the model predict the updated fact across multiple expressions. It then jointly updates the shared n-gram embeddings to match these targets across expressions and edits, penalizing updates to frequently reused embeddings more strongly to preserve unrelated knowledge. Experiments show that EngramEdit enables independent factual knowledge updates through conditional memory, achieving near-perfect editing success. Revised knowledge is usable across unseen expressions and in multi-hop reasoning, with nearly three times the strongest baseline's accuracy under chain-of-thought (CoT) prompting. Unrelated knowledge and general capabilities are largely preserved even as factual updates accumulate. These findings show that EngramEdit turns conditional memory into an editable knowledge interface, extending its role beyond model scaling to support decoupled knowledge updates.

Momentum

Related topics

Latest papers 117