Language Model Editing

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, up 75% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 37

All topics

HeadEdit: Calibrating Language Model Behavior Through the Frozen Unembedding Matrix
Oct 1, 2026Zirui He, Haiyan Zhao, Jingyu Hu +5Language Model Steering Language Model Calibration
New Jersey Institute of Technology · University of Bristol · Kuaishou Technology +1
How Should Diffusion Language Models Edit Code?
Sep 29, 2026Xijia Tao, Ziru Liu, Shansan Gong +7Masked Diffusion Models Language Model Editing
The University of Hong Kong · Huawei Technologies Ltd. · City University of Hong Kong +1
Generalizable Lifelong Model Editing via Preference Optimization
Sep 29, 2026Dahyun Jung, Suhyune Son, Heuiseok LimContinual Learning for LLMs Continual Learning
Department of Computer Science and Engineering, Korea University
ManiEdit: Sequential Unstructured Knowledge Editing for Language Models from a Manifold Perspective
Sep 27, 2026Rui Liu, Chenheng Zhang, Haoxuan Li +1Knowledge Editing Continual Learning for LLMs
State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · Institute for Artificial Intelligence, Peking University
ALOE: Semantically Addressed Low-Rank Operators for Knowledge Editing
Sep 24, 2026Zeyan Li, Hu Xu, Jianfeng XuFactual Knowledge in Language Models Language Model Editing
Shanghai Jiao Tong University
The information geometry of large language models is shared, learned, and controllable
Sep 11, 2026Dario PicozziInformation Geometry Language Model Steering
Department of Physics and Astronomy, University College London (UCL), Gower Street, London WC1E 6BT, United Kingdom · London Centre for Nanotechnology, 19 Gordon St, London WC1H 0AH, United Kingdom
Training-Free Task Vectors for LLM Behavioral Control
Sep 8, 2026Gabriel J. Perin, Lucas Boscaini, André Araujo +1Task Vectors AI Control
University of São Paulo · Google · Google DeepMind
When Models Edit Too Much: On the Fidelity of Minimal Code Edits
Sep 3, 2026Tongyao Zhu, Wei Hern Lim, Min-Yen KanAutomated Program Repair Language Model Editing
National University of Singapore
SCoNE: Selective Context-aware Neuron Editing for Robust Retrieval-Augmented Generation
Sep 1, 2026Chaewon Kim, Seo Yeon ParkRetrieval-Augmented Generation Retrieval Robustness
Hanyang University
AI Writers Have a Consistent Stylometric Footprint, but AI Editors Do Not
Aug 28, 2026Zhengyang Shan, Yukyung Lee, Sophie HaoAI-Generated Text Detection Language Model Fingerprinting
Boston University
Hybrid-Policy Self-Editing for Composable Unstructured Knowledge Editing
Aug 12, 2026Tianci Liu, Zihan Dong, Tianchun Li +8Knowledge Editing Self-Distillation
University of Tennessee · 2Rutgers University · 3Purdue University +1
To Add Is Machine, To Delete Is Human: Measuring and Mitigating Deletion Avoidance in LLM Code Editing
Jul 30, 2026Amir M. Ebrahimi, Mohammed Mehedi Hasan, Aaditya Bhatia +2LLM Reliability Language Model Editing
School of Computing, Queen’s University, Kingston, Ontario, Canada
PRISM Edit: One Vector for All Temporal Answers
Jul 13, 2026Chen Huang, Qi Zheng, Ruiqin Zheng +2Temporal Reasoning in Language Models Model Editing
Tsinghua University · ByteDance
Prototype Language Models
Jul 1, 2026Dan Ley, Giang Nguyen, Himabindu Lakkaraju +1LLM Interpretability Language Modeling
1Harvard University · 2Guide Labs Inc. *Work initiated during an internship at Guide Labs. · 2Guide Labs Inc.
Reproducibility Study of "AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models"
Jun 25, 2026Ananth K Suresh, Arya HariharanKnowledge Editing Model Editing
Independent
Orthogonal Representation Editing: Decoupling Semantic Entanglement in Batch Knowledge Editing of LLMs
Jun 21, 2026Wenhao Yu, Zhicong Lu, Bo Lv +4Knowledge Editing Model Editing
School of Computer Science and Technology, Tianjin University · Kexin Technology · University of Chinese Academy of Sciences +2
LOKI: Memory-Free Null-Space Constrained Lifelong Knowledge Editing
Jun 18, 2026Masih Eskandar, Miquel Sirera Perelló, Stratis Ioannidis +1Knowledge Editing Continual Learning for LLMs
Department of Electrical and Computer Engineering Northeastern University Boston, MA, 02115
User as Engram: Internalizing Per-User Memory as Local Parametric Edits
Jun 17, 2026Bojie LiKnowledge Editing LLM Personalization
Pine AI
Output Vector Editing for Memorization Mitigation in Large Language Models
Jun 17, 2026Ahmad Dawar Hakimi, Kaiwei Lei, Isabelle Augenstein +1Memorization in Language Models Neural Network Memorization
Center for Information and Language Processing, LMU Munich, Germany · Department of Computer Science, University of Copenhagen, Denmark · 3Munich Center for Machine Learning, Germany +1
Self-Generated Error Training for Token Editing in Diffusion Language Models
Jun 15, 2026Lin YaoLLM Self-Correction Diffusion Model Fine-Tuning
School of Computer Science, Shanghai Jiao Tong University, Shanghai, 200240, China · Zhongguancun Academy, Beijing, 100097, China
TimeROME-DLM: Temporal Causal Tracing and Low-Rank Inference-Time Knowledge Editing for Masked Diffusion Language Models
Jun 11, 2026Zhengtao Yao, Liuyang Song, Hongbo Zhang +4Knowledge Editing Masked Diffusion Models
University of Southern California, USA · Peking University, China · Stevens Institute of Technology, USA +1
Can Editing 1 Neuron Fix Repetition Loops in LLMs?
Jun 9, 2026Aristotelis Lazaridis, Aman Sharma, Dylan Bates +3LLM Self-Correction Mechanistic Interpretability
Edgerunner AI
Knowledge Editing for Masked Diffusion Language Models
Jun 2, 2026Haewon Park, Yohan JoKnowledge Editing Masked Diffusion Models
Graduate School of Data Science, Seoul National University
Revisiting Parameter-Based Knowledge Editing in Large Language Models: Theoretical Limits and Empirical Evidence
May 30, 2026Wanying Ren, Xin Song, Futing Wang +2Model Editing Language Model Editing
East China Normal University, Shanghai, China · Westlake University, Hangzhou, China · Nanyang Technological University, Singapore
Interpretability-Guided Layer Selection over Subspace Projection: SAEs as Stethoscopes, Not Scalpels, for Raw Task Vector Model Editing
May 27, 2026Li Lei, Madalina Ciobanu, Qingqing Mao +1Task Vectors LLM Interpretability
Incept Labs Houston, TX · Incept Labs, Houston, TX · Titan Holdings, San Francisco, CA
Modality-Decoupled Online Recursive Editing
May 19, 2026Siyuan Li, Youyuan Zhang, Fangming Liu +1Continual Learning for LLMs Multimodal Large Language Models
Harbin Institute of Technology, Shenzhen, China. · Peng Cheng Laboratory, China. · Huazhong University of Science and Technology, China.
Scalable Knowledge Editing for Mixture-of-Experts LLMs via Tensor-Structured Updates
May 15, 2026Roman Maksimov, Vladimir Aletov, Dmitry Bylinkin +3Knowledge Editing Mixture-of-Experts Language Models
1Moscow Independent Research Institute of Artificial Intelligence · 2Basic Research of Artificial Intelligence Laboratory · 3Innopolis University
BetaEdit: Null-Space Constrained Sequential Model Editing
May 10, 2026Bingqing Liu, Wei Liu, Yuhua LiKnowledge Editing Model Editing
Huazhong University of Science and Technology, Wuhan, China
EditPropBench: Measuring Factual Edit Propagation in Scientific Manuscripts
May 3, 2026Garvin KruthofKnowledge Editing LLM Evaluation
Technical University of Munich
HoReN: Normalized Hopfield Retrieval for Large-Scale Sequential Model Editing
May 2, 2026Yuan Fang, Yi Xie, Xuming RanKnowledge Editing Memory-Augmented Language Models
1IXL Learning, Inc · 2Technical University of Munich · 3National University of Singapore
From Backward Spreading to Forward Replay: Revisiting Target Construction in LLM Parameter Editing
May 1, 2026Wei Liu, Hongkai Liu, Zhiying Deng +2Knowledge Editing Model Editing
National University of Singapore · Laboratory for Artificial Intelligence and New Forms of Education, National Engineering Research Center for Educational Big Data, Faculty of Artificial Intelligence in Education, Central China Normal University · University of Oxford
Copy-as-Decode: Grammar-Constrained Parallel Prefill for LLM Editing
Apr 20, 2026Ziyang LiuLLM Inference Acceleration Parallel Decoding
Independent Researcher
One Mask to Rule Them All: On Hidden Facts after Editing and How to Find Them
Apr 18, 2026Ali Holmov, Paul Youssef, Nandi Schoots +1Knowledge Editing Transformer Interpretability
Technical University of Munich · Marburg University · University of Oxford
The GRADIEND Python Package: An End-to-End System for Gradient-Based Feature Learning
Feb 27, 2026Jonathan Drechsel, Steffen HerboldLLM Interpretability Language Modeling
Faculty of Computer Science and Mathematics University of Passau
Can Fine-Tuning Erase Your Edits? On the Fragile Coexistence of Knowledge Editing and Adaptation
Nov 8, 2025Yinjie Cheng, Paul Youssef, Christin Seifert +2Knowledge Editing LLM Fine-Tuning
School of Computer Science University of Sheffield Sheffield, UK · Mathematics and Computer Science Marburg University Marburg, Germany
BTBR: A Bayesian-Theory-Driven Probabilistic-Fuzzy Framework for Implicit Bias Removal in Large Language Models
Aug 20, 2024Yongxin Deng, Xiaoyu Tan, Jing Pan +3Social Bias in Language Models Language Model Bias Evaluation
University of Technology Sydney · School of Computer Science, University of Technology Sydney, Sydney, NSW 2007, Australia · National University of Singapore +3
LaSEr-Edit: Localized Span-level Error Editing with Energy-based Localization
Jun 30, 2024Hye Ryung Son, Saehee Eom, Mooho Song +1Controllable Text Generation LLM Safety
Graduate School of Data Science Seoul National University · Georgia Institute of Technology

Oct 1, 2026, cs.CLJ/K move · Enter open · S save

HeadEdit: Calibrating Language Model Behavior Through the Frozen Unembedding Matrix

Zirui He, Haiyan Zhao, Jingyu Hu, Yinghao Wu+4

New Jersey Institute of Technology · University of Bristol · Kuaishou Technology+1

Language Model Steering Language Model Calibration Inference-Time Language Model Alignment Language Model Editing

Open paper AskPDF arXiv

Alignment does not eliminate behavioral errors in language models. Models may still refuse benign requests, call unnecessary tools, or yield to false user claims. Current methods mitigate such errors as a computation problem, and rarely explore if the desired behavior is already encoded in the model's representation. Motivated by the observation that behavior-relevant information remains linearly decodable from the final hidden state even when the resulting logits produce the undesired behavior, we introduce HeadEdit, a gradient-free method that calibrates model behavior through the unembedding matrix. HeadEdit extracts a low-rank behavioral subspace from paired completions and uses each prompt's coordinates within it to generate a vocabulary-wide correction, thereby implementing implicitly adaptive steering without manually specified target tokens or parameter updates. HeadEdit improves all nine experimental settings across three tasks and three model families, with negligible inference overhead and no systematic loss of general capabilities. It also reveals a connection to gradient-based alignment. HeadEdit's low-dimensional representation partly predicts how preference tuning changes output logits on unseen prompts. The subspace learned from the model can also be reused after tuning, improving performance without re-extracting or retuning. These results show that HeadEdit provides a practical, lightweight, and interpretable way to calibrate model behavior through the unembedding matrix.

Momentum

Related topics

Latest papers 37