Language Model Editing

Follow this topic Ask about this topic

Language model editing studies how to make targeted changes to a model’s stored knowledge, behavior, or generated text without retraining it from scratch. Common approaches locate relevant layers, neurons, or representations and apply low-rank, sparse, or null-space-constrained updates; others use activation steering, external retrieval, or modular memories for reversible and repeated edits. Methods are tested for edit success, generalization to paraphrases and related queries, locality on unrelated inputs, and retention of capabilities on benchmarks such as CounterFact and ZsRE. The central tension is that edits that generalize or accumulate at scale can interfere with existing knowledge and reasoning, while retrieval and other alternatives may preserve the base model more reliably.

Latest papers

77 papers12 papers in the last four weeks, up 33% on the four weeks before. 0.1% of all new papers.

The Anatomy of an Edit: Mechanism-Guided Activation Steering for Knowledge Editing
Mar 21, 2026Yuan Cao, Mingyang Wang, Hinrich SchützeLanguage Model Editing Activation Steering
Technical University of Munich · LMU Munich · Munich Center for Machine Learning (MCML)
Beyond the Covariance Trap: Unlocking Generalization in Same-Subject Knowledge Editing for Large Language Models
Mar 16, 2026Xiyu Liu, Qingyi Si, Zhengxiao Liu +3Knowledge Editing Language Model Editing
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · JD.com, Beijing, China
SCAN: Sparse Circuit Anchor Interpretable Neuron for Lifelong Knowledge Editing
Mar 16, 2026Yuhuan Liu, Haitian Zhong, Xinyuan Xia +3Knowledge Editing Language Model Editing
New Laboratory of Pattern Recognition (NLPR), State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences · Cuiying Honors College, Lanzhou University · Zhongguancun Academy +1
Localizing and Editing Knowledge in Large Audio-Language Models
Mar 15, 2026Sung Kyun Chung, Jiaheng Dong, Qiuchi Hu +3Concept Localization in Language Models Audio-Language Models
The University of Melbourne, Australia · Wuhan University, China · University of Auckland, New Zealand
MetaKE: Meta-Learning for Knowledge Editing Toward a Better Accuracy-Editability Trade-off
Mar 13, 2026Shuxin Liu, Di Gao, Ou WuKnowledge Editing Language Model Editing
Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou 310024, China
Reversible Lifelong Model Editing via Semantic Routing-Based LoRA
Mar 11, 2026Haihua Luo, Xuming Ran, Tommi Kärkkäinen +4Model Editing Language Model Editing
Faculty of Information Technology, University of Jyväskylä, Jyväskylä, Finland · National University of Singapore, Singapore · School of Computer Science and Technology, Dalian University of Technology, Dalian, China
Parameter-Efficient Token Embedding Editing for Clinical Class-Level Unlearning
Mar 11, 2026Iyad Ait Hou, Shrenik Borad, Harsh Sharma +3Machine Unlearning Language Model Editing
Department of Computer Science, The George Washington University, Washington, DC · Center for Speech and Language Processing, Johns Hopkins University, Baltimore, MD
CLaRE-ty Amid Chaos: Quantifying Representational Entanglement to Predict Ripple Effects in LLM Editing
Mar 11, 2026Manit Baser, Alperen Yildiz, Dinil Mon Divakaran +1Language Model Editing Representation Geometry in Language Models
National University of Singapore, Singapore · A*STAR Institute for Infocomm Research (A*STAR I2R), Singapore
Reforming the Mechanism: Editing Reasoning Patterns in LLMs with Circuit Reshaping
Mar 6, 2026Zhenyu Lei, Qiong Wu, Jianxiong Dong +4Language Model Editing LLM Reasoning
University of Virginia · AT&T Chief Data Office · Florida State University
The GRADIEND Python Package: An End-to-End System for Gradient-Based Feature Learning
Feb 27, 2026Jonathan Drechsel, Steffen HerboldLLM Interpretability Language Modeling
Faculty of Computer Science and Mathematics, University of Passau
Multilingual Safety Alignment Via Sparse Weight Editing
Feb 26, 2026Jiaming Liang, Zhaoxin Wang, Handing WangLLM Safety Alignment LLM Safety
School of Artificial Intelligence, Xidian University.
Golden Layers and Where to Find Them: Improved Knowledge Editing for Large Language Models Via Layer Gradient Analysis
Feb 22, 2026Shrestha Datta, Hongfu Liu, Anshuman ChhabraKnowledge Editing Language Model Editing
University of South Florida, Tampa, FL, USA · Brandeis University, Waltham, MA, USA
Uncovering Context Reliance in Unstructured Knowledge Editing
Feb 22, 2026Zisheng Zhou, Mengqi Zhang, Shiguang Wu +4Knowledge Editing Language Model Editing
Shandong University · Beijing University of Posts and Telecommunications
CrispEdit: Low-Curvature Projections for Scalable Non-Destructive LLM Editing
Feb 17, 2026Zarif Ikram, Arad Firouzkouhi, Stephen Tu +2Language Model Editing
University of Southern California
Can Fine-Tuning Erase Your Edits? On the Fragile Coexistence of Knowledge Editing and Adaptation
Nov 8, 2025Yinjie Cheng, Paul Youssef, Christin Seifert +2Knowledge Editing LLM Fine-Tuning
School of Computer Science University of Sheffield Sheffield, UK · Mathematics and Computer Science Marburg University Marburg, Germany
BTBR: A Bayesian-Theory-Driven Probabilistic-Fuzzy Framework for Implicit Bias Removal in Large Language Models
Aug 20, 2024Yongxin Deng, Xiaoyu Tan, Jing Pan +3Social Bias in Language Models Language Model Bias Evaluation
University of Technology Sydney · School of Computer Science, University of Technology Sydney, Sydney, NSW 2007, Australia · National University of Singapore +3
LaSEr-Edit: Localized Span-level Error Editing with Energy-based Localization
Jun 30, 2024Hye Ryung Son, Saehee Eom, Mooho Song +1Controllable Text Generation LLM Safety
Graduate School of Data Science Seoul National University · Georgia Institute of Technology

Mar 21, 2026J/K move · Enter open · S save

The Anatomy of an Edit: Mechanism-Guided Activation Steering for Knowledge Editing

Yuan Cao, Mingyang Wang, Hinrich Schütze

Technical University of Munich · LMU Munich · Munich Center for Machine Learning (MCML)

Language Model Editing Activation Steering Causal Interventions in Language Models Mechanistic Interpretability

Open paper AskPDF arXiv

Large language models (LLMs) are increasingly used as knowledge bases, but keeping them up to date requires targeted knowledge editing (KE). However, it remains unclear how edits are implemented inside the model once applied. In this work, we take a mechanistic view of KE using neuron-level knowledge attribution (NLKA). Unlike prior work that focuses on pre-edit causal tracing and localization, we use post-edit attribution -- contrasting successful and failed edits -- to isolate the computations that shift when an edit succeeds. Across representative KE methods, we find a consistent pattern: mid-to-late attention predominantly promotes the new target, while attention and FFN modules cooperate to suppress the original fact. Motivated by these findings, we propose MEGA, a MEchanism-Guided Activation steering method that performs attention-residual interventions in attribution-aligned regions without modifying model weights. On CounterFact and Popular, MEGA achieves strong editing performance across KE metrics on GPT2-XL and LLaMA2-7B. Overall, our results elevate post-edit attribution from analysis to engineering signal: by pinpointing where and how edits take hold, it powers MEGA to deliver reliable, architecture-agnostic knowledge edits.