Knowledge Editing

Follow this topic Ask about this topic

Knowledge editing aims to correct or add information in a model without retraining it from scratch, so models can adapt to new facts, rules, and user-specific information. Approaches include targeted weight updates, low-rank adapters and null-space constraints, as well as external memories and retrieval systems that supply updates at inference time. Researchers test whether edits work across paraphrases, related questions, and multi-hop reasoning while measuring locality, capability retention, and stability over long sequences of updates. A central tension is that strong performance on direct edit queries may reflect surface compliance rather than changed internal knowledge, and edits can interfere with other facts, reasoning, or safety behavior over time.

Latest papers

86 papers11 papers in the last four weeks, level with the four weeks before. 0.1% of all new papers.

PALoRA: Projection-Adaptive LoRA for Preserving Reasoning in Large Language Models
May 23, 2026Mustafa Hayri Bilgin, Mariam Barry, Albert Bifet +2Knowledge Editing Continual Learning for LLMs
IT Group, Research, BNP Paribas · LTCI, Télécom Paris, Institut Polytechnique de Paris · AI Institute, University of Waikato +1
Scalable Knowledge Editing for Mixture-of-Experts LLMs via Tensor-Structured Updates
May 15, 2026Roman Maksimov, Vladimir Aletov, Dmitry Bylinkin +3Knowledge Editing Mixture-of-Experts Language Models
1Moscow Independent Research Institute of Artificial Intelligence · 2Basic Research of Artificial Intelligence Laboratory · 3Innopolis University
Merging Methods for Multilingual Knowledge Editing for Large Language Models: An Empirical Odyssey
May 13, 2026Kunil Lee, Ki-Young Shin, Jong-Hyeok Lee +1Knowledge Editing Multilingual Language Models
Department of Computer Science and Engineering, POSTECH · Designovel Co., Ltd. · LLSOLLU +1
More Edits, More Stable: Understanding the Lifelong Normalization in Sequential Model Editing
May 12, 2026Xin Ma, Wei Chen, Qi Liu +4Knowledge Editing Continual Learning
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China, Anhui, China.
DeepRefine: Agent-Compiled Knowledge Refinement via Reinforcement Learning
May 11, 2026Haoyu Huang, Jiaxin Bai, Shujie Liu +6Knowledge Editing Knowledge Augmentation for Language Models
1HKUST · 2HKBU · 3Microsoft Research Asia
Benchmarking Safety Risks of Knowledge-Intensive Reasoning under Malicious Knowledge Editing
May 11, 2026Qinghua Mao, Xi Lin, Jinze Gu +3Knowledge Editing Knowledge Conflicts in Language Models
School of Computer Science · Shanghai Jiao Tong University
BetaEdit: Null-Space Constrained Sequential Model Editing
May 10, 2026Bingqing Liu, Wei Liu, Yuhua LiKnowledge Editing Model Editing
Huazhong University of Science and Technology, Wuhan, China
EditPropBench: Measuring Factual Edit Propagation in Scientific Manuscripts
May 3, 2026Garvin KruthofKnowledge Editing LLM Evaluation
Technical University of Munich
HoReN: Normalized Hopfield Retrieval for Large-Scale Sequential Model Editing
May 2, 2026Yuan Fang, Yi Xie, Xuming RanKnowledge Editing Memory-Augmented Language Models
1IXL Learning, Inc · 2Technical University of Munich · 3National University of Singapore
From Backward Spreading to Forward Replay: Revisiting Target Construction in LLM Parameter Editing
May 1, 2026Wei Liu, Hongkai Liu, Zhiying Deng +2Knowledge Editing Model Editing
National University of Singapore · Laboratory for Artificial Intelligence and New Forms of Education, National Engineering Research Center for Educational Big Data, Faculty of Artificial Intelligence in Education, Central China Normal University · University of Oxford
Towards Scalable Lifelong Knowledge Editing with Selective Knowledge Suppression
Apr 21, 2026Dahyun Jung, Jaewook Lee, Heuiseok LimKnowledge Editing Continual Learning for LLMs
Department of Computer Science and Engineering, Korea University
Correct When Paired, Wrong When Split: Decoupling and Editing Modality-Specific Neurons in MLLMs
Apr 20, 2026Tingchao Fu, Wenkai Wang, Fanxiao Li +6Knowledge Editing Multimodal Large Language Models
School of Information Science and Engineering, Yunnan University · School of Software, Yunnan University · 3National University of Singapore +1
One Mask to Rule Them All: On Hidden Facts after Editing and How to Find Them
Apr 18, 2026Ali Holmov, Paul Youssef, Nandi Schoots +1Knowledge Editing Transformer Interpretability
Technical University of Munich · Marburg University · University of Oxford
Distributed Multi-Layer Editing for Rule-Level Knowledge in Large Language Models
Apr 9, 2026Yating Wang, Wenting Zhao, Yaqi Zhao +3Knowledge Editing Language Model Editing
School of Software, Shandong University, Jinan, China · Salesforce AI Research
The Model Agreed, But Didn't Learn: Diagnosing Surface Compliance in Large Language Models
Apr 7, 2026Xiaojie Gu, Ziying Huang, Weicong Hong +3Knowledge Editing LLM Reliability
Independent Researcher · Cornell Tech · The Ohio State University +1
Addressing the Reasoning Gap: Mechanistic Circuit-Based Knowledge Editing in Large Language Models
Apr 7, 2026Tianyi Zhao, Yinhan He, Wendy Zheng +1Knowledge Editing Factual Knowledge in Language Models
University of Virginia
Learning to Edit Knowledge via Instruction-based Chain-of-Thought Prompting
Apr 7, 2026Jinhu Fu, Yan Bai, Longzhu He +4Knowledge Editing Language Model Editing
Beijing University of Posts and Telecommunications · Peking University · Huawei Technologies Ltd. +1
Beyond the Covariance Trap: Unlocking Generalization in Same-Subject Knowledge Editing for Large Language Models
Mar 16, 2026Xiyu Liu, Qingyi Si, Zhengxiao Liu +3Knowledge Editing Language Model Editing
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · JD.com, Beijing, China
SCAN: Sparse Circuit Anchor Interpretable Neuron for Lifelong Knowledge Editing
Mar 16, 2026Yuhuan Liu, Haitian Zhong, Xinyuan Xia +3Knowledge Editing Language Model Editing
New Laboratory of Pattern Recognition (NLPR), State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences · Cuiying Honors College, Lanzhou University · Zhongguancun Academy +1
MetaKE: Meta-Learning for Knowledge Editing Toward a Better Accuracy-Editability Trade-off
Mar 13, 2026Shuxin Liu, Di Gao, Ou WuKnowledge Editing Language Model Editing
Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou 310024, China
Golden Layers and Where to Find Them: Improved Knowledge Editing for Large Language Models Via Layer Gradient Analysis
Feb 22, 2026Shrestha Datta, Hongfu Liu, Anshuman ChhabraKnowledge Editing Language Model Editing
University of South Florida, Tampa, FL, USA · Brandeis University, Waltham, MA, USA
Uncovering Context Reliance in Unstructured Knowledge Editing
Feb 22, 2026Zisheng Zhou, Mengqi Zhang, Shiguang Wu +4Knowledge Editing Language Model Editing
Shandong University · Beijing University of Posts and Telecommunications
Edit Knowledge, Not Just Facts via Multi-Step Reasoning over Background Stories
Feb 2, 2026Ya Gao, Kalle Kujanpää, Pekka Marttinen +2Knowledge Editing Knowledge Augmentation for Language Models
Aalto University · System 2 AI
Representation Interventions Enable Lifelong Knowledge Memory Control in LLMs
Nov 25, 2025Xuyuan Liu, Shengyu Chen, Xinshuai Dong +6Knowledge Editing Continual Learning for LLMs
Dartmouth College · NEC Laboratories America · Carnegie Mellon University
Can Fine-Tuning Erase Your Edits? On the Fragile Coexistence of Knowledge Editing and Adaptation
Nov 8, 2025Yinjie Cheng, Paul Youssef, Christin Seifert +2Knowledge Editing LLM Fine-Tuning
School of Computer Science University of Sheffield Sheffield, UK · Mathematics and Computer Science Marburg University Marburg, Germany
LMEnt: A Suite for Analyzing Knowledge in Language Models from Pretraining Data to Representations
Sep 3, 2025Daniela Gottesman, Alon Gilae-Dotan, Ido Cohen +4Language Model Pretraining Knowledge Editing
Tel Aviv University · Mila – Quebec AI Institute & McGill University

May 23, 2026J/K move · Enter open · S save

PALoRA: Projection-Adaptive LoRA for Preserving Reasoning in Large Language Models

Mustafa Hayri Bilgin, Mariam Barry, Albert Bifet, Azzedine Idir Ait Said+1

IT Group, Research, BNP Paribas · LTCI, Télécom Paris, Institut Polytechnique de Paris · AI Institute, University of Waikato+1

Knowledge Editing Continual Learning for LLMs LLM Fine-Tuning Low-Rank Adaptation Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Efficiently updating Large Language Models (LLMs) with new or evolving factual knowledge remains a central challenge, as even parameter-efficient adaptation can erode previously acquired reasoning abilities. This tension reflects a plasticity-stability dilemma: models must incorporate new knowledge while preserving skill-critical representations. In this work, we study this trade-off through the spectral structure of multilayer perceptron weight matrices. We show, both theoretically and empirically, that information essential for reasoning is not localized only in dominant singular directions, but is instead distributed across the singular spectrum. Motivated by this observation, we introduce PALoRA, a two-stage framework for knowledge injection with reduced interference. PALoRA first trains a Singular Value Fine-Tuning (SVF) expert on a reasoning dataset and uses its learned singular scaling vector as a frozen geometric probe to identify components that are critical for the target skill. It then performs factual knowledge injection with Low-Rank Adaptation (LoRA) under a structural orthogonality constraint, ensuring that updates avoid the identified skill-relevant subspace. Across Llama 3.1 8B and Mistral 7B, and across mathematical, coding, and scientific reasoning benchmarks, PALoRA preserves on average 95% of the SVF expert's reasoning performance while maintaining competitive factual recall. It consistently improves skill retention over prior spectral Parameter-Efficient Fine-Tuning (PEFT) methods while adding less than 0.006% parameter overhead.