Sequential knowledge editing breaks a model's ability to tell good evidence from bad, without costing it accuracy

Atul Anand

cs.AIAug 31, 2026

Sequential knowledge editing breaks a model's ability to tell good evidence from bad, without costing it accuracy

Authors: Atul Anand

Abstract

Knowledge editing is evaluated on whether the edited fact changed, whether paraphrases follow, and whether unrelated answers stayed put. A model can pass all three and still lose something none of them measures: the ability to decide, on facts that were never edited, which retrieved documents to believe. We score the log odds a model assigns to its remembered answer against the answer an injected passage asserts, before and after editing, holding the query, the passage and both candidate strings fixed. Our cleanest arm is a conservatively tuned LoRA: after 1,000 sequential edits on Qwen2.5-7B-Instruct it leaves MMLU unchanged to four decimal places, yet the spread of the arbitration quantity across untouched facts falls by 36%. Selective prediction degrades with it. Area under the risk-coverage curve rises by 0.107, against 0.005 for a norm-matched perturbation at the same MMLU, and error on the model's most confident quarter of arbitration decisions goes from 0.217 to 0.342. This is not capability loss. Sweeping random perturbation over five severities, damage bad enough to cut MMLU from 0.6275 to 0.3725 produces less harm (0.088) than MEMIT does at 0.6050 (0.102). The effect holds across three seeds, two model families, two datasets, two probe-disjointness criteria, three prompt templates and paraphrased queries. Layer ablation on saved weight deltas shows it is distributed: no single layer reproduces it, and removing any one recovers about half. Under retrieval with a frozen retriever, accuracy falls from 0.592 to 0.46. A secondary finding may matter more in practice. Three of five model and method pairings we ran collapse to chance MMLU at 1,000 sequential edits under published hyperparameters, while edit success stays at 1.00 and locality reads clean. Sequential-editing evaluations that never measure capability cannot see this.

Figures & tables

Panel of Figure 1 — Figure 1: Experimental design. The edit stream and the probe pool share no subject entity, so everything measured downstream concerns facts the editor never touched. Controls and the capability battery run at every checkpoint rather than once at the end. Retrieval in Stage 4 is computed once and replayed for each checkpoint, which is what makes the accuracy differences attributable to the model rather than to what was retrieved.

Figure 2: $\Delta$ AURC against MMLU after editing. The curve traces what random perturbation alone produces as its severity increases. All three editors lie well above it at comparable or better capability, and the norm-matched control lies on the floor. Damage severe enough to cut MMLU to 0.3725 produces less arbitration harm than MEMIT does at 0.6050.

Arm (1,000 edits, Qwen)	MMLU	PPL	$sd_{\text{after}}/sd_{\text{before}}$	$\Delta$ AURC
No edit	0.6275	9.80	1.000	$+0.0000$
Norm-matched noise	0.6250	9.90	0.978	$+0.0054$
LoRA-gentle	0.6275	28.96	0.640	$+0.1067$
MEMIT	0.6050	11.49	0.622	$+0.1024$
LoRA (aggressive)	0.6100	39.23	0.460	$+0.1241$
AlphaEdit	0.2750	30.70	0.534	$+0.1636$

Table 1: Dispersion is on stale-false evidence. LoRA-gentle reaches the unedited MMLU value exactly and still causes twenty times the selective-prediction harm of a perturbation at the same MMLU. AlphaEdit’s MMLU of 0.2750 is chance for four-way multiple choice, so its

\Delta

AURC is not an arbitration result.

Figure 3: Risk-coverage curves. The unedited model (dashed) keeps error low when it answers only its most confident queries. After editing that ordering degrades, and the gap is widest exactly where a deployed system would operate, at low coverage.

Figure 4: Dispersion ratio by evidence condition. Matched noise sits at 1.0 in every condition. Both editors compress the three conflicting conditions to roughly 0.6. The agreeing condition, where memory and passage point the same way, behaves differently and is the one condition where the editor-versus-noise interval contains zero.

Term	Coef.	SE	$z$	$p$
AlphaEdit	-8.73	0.27	-32.2	$<$ 0.001
LoRA	-15.55	0.27	-57.4	$<$ 0.001
LoRA-gentle	-11.31	0.33	-34.7	$<$ 0.001
MEMIT	-2.31	0.27	-8.5	$<$ 0.001
norm-matched	+0.14	0.27	+0.5	0.604
load 1000	-2.25	0.06	-36.4	$<$ 0.001

Table 2: Mixed-effects fit on paired drift, with a random intercept per fact. 58,464 observations over 174 facts; the model converged. Reference levels are the no-edit arm and the agreeing condition, so each editor coefficient reads as a departure from applying no update at all. The norm-matched control is the only editor term whose interval includes zero. Seed enters as a fixed effect rather than random: three levels are too few to estimate a variance component stably.

Axis	Setting	$n$	LoRA-gentle $\Delta$ AURC	Control $\Delta$ AURC
Primary	CounterFact, subject-disjoint	174	$+0.1067$	$+0.0054$
Probe split	subject and relation disjoint	196	$+0.1182$	$+0.0016$
Query phrasing	paraphrased stems	108	$+0.1348$	$+0.0009$
Dataset	ZsRE	66	$+0.0743$	$+0.0010$
Model family	Mistral-7B-Instruct-v0.3	167	$+0.0897$	$+0.0039$

Table 3: The effect holds on every axis we varied, against a control that stays near zero throughout. Sample sizes differ because each split applies its own confidence filter.

Figure 5: The same comparison on both model families. Absolute magnitude differs; the separation from the matched control does not.

Figure 6: Layer ablation on MEMIT’s saved deltas. Left: only one layer’s update applied. Right: every layer except one. Neither panel approaches the full-update line, so the effect is carried by the joint update rather than by any individual layer.

Figure 7: Controlled arbitration harm against end-to-end retrieval accuracy, with retrieval held fixed across checkpoints. The number beside each point is the edit load.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Checkpoint	MMLU	ECE	PPL
alphaedit L0	0.6275	0.2962	9.80
alphaedit L0 (s1)	0.6275	0.2962	9.80
alphaedit L1000	0.2750	0.2734	30.70
alphaedit L1000 (s1)	0.2450	0.4104	1766.24
alphaedit L1000 (s2)	0.5175	0.4035	15.31
alphaedit L100	0.6250	0.3075	9.87

Appendix

Table 4: Capability measurements at every Qwen checkpoint. The no-edit arm reproduces the baseline exactly at all loads and seeds, which is the check that the pipeline cannot manufacture drift. Three arms reach chance MMLU (0.25).

Knowledge Editing

Knowledge editing studies how to update specific facts or behaviors in language models while preserving unrelated knowledge and capabilities.

See all 84 papers

May 26, 2026

The Labyrinth and the Thread: Rethinking Regularizations in Sequential Knowledge Editing for Large Language Models

Sequential editing of structured knowledge in large language models allows targeted factual updates without retraining, yet existing methods often rely on complex regularization or constraint mechanisms whose necessity remains unclear. In this work, we systematically investigate the mechanisms underlying effective and stable sequential editing. Specifically, we first analyze the empirical success of AlphaEdit and establish, via a rigorous optimization analysis, the formal equivalence between one-time and sequential editing. Building on this insight, we generalize the equivalence to a broader class of editing objectives, demonstrating that stability emerges naturally from properly accounting for accumulated editing constraints, rather than from specialized regularization or null-space operations. We empirically confirm that many commonly used regularization strategies are unnecessary for reliable sequential updates. Furthermore, we extend our framework to handle conflicting edits, ensuring robust and consistent behavior under contradictory updates. Ultimately, our work provides Ariadne's thread through the labyrinth of sequential editing, charting a path toward simpler, more interpretable, and dependable knowledge updates. Our code is available at https://github.com/Wangzzzzzzzz/OTE-SE-Alignment.

Zheng Wang, Kaixuan Zhang, Wanfang Chen +2

Bosch Center for Artificial Intelligence (BCAI) · Bosch (China) Investment Ltd. · School of Statistics, East China Normal University.

Knowledge Editing Factual Knowledge in Language Models Model Editing

Ask PDF

Mar 16, 2026

SCAN: Sparse Circuit Anchor Interpretable Neuron for Lifelong Knowledge Editing

Large Language Models (LLMs) often suffer from catastrophic forgetting and collapse during sequential knowledge editing. This vulnerability stems from the prevailing dense editing paradigm, which treats models as black boxes and relies on coarse-grained parameter interventions that inevitably disrupt preserved knowledge. To address this, we propose SCAN (a sparse editing framework based on Sparse Circuit Anchored Neuron) which transforms editing into a mechanism-aware manipulation by constructing a knowledge circuit via Sparse Transcoders. Experiments on Gemma2, Qwen3, and Llama3.1 across CounterFact, ZsRE and WikiFactDiff demonstrate that SCAN achieves a superior performance, maintaining model integrity on benchmarks like MMLU and GSM8K even after 3,000 sequential edits, whereas other existing methods deteriorate progressively as editing accumulates, eventually resulting in model collapse.

Yuhuan Liu, Haitian Zhong, Xinyuan Xia +3

New Laboratory of Pattern Recognition (NLPR), State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences · Cuiying Honors College, Lanzhou University · Zhongguancun Academy +1

Knowledge Editing Language Model Editing Catastrophic Forgetting Continual Learning for LLMs

Ask PDF

Apr 18, 2026

One Mask to Rule Them All: On Hidden Facts after Editing and How to Find Them

Knowledge editing methods such as ROME and MEMIT update factual associations in transformer models by modifying MLP weights. While evaluated mainly by output behavior, their internal mechanism remains underexplored. We investigate whether edits rely on a common mechanism, regardless of which fact is modified. Despite fact-specific weight changes, we argue that ROME and MEMIT target the same subset of weights critical for maintaining edits. To isolate this subset, we train a compact binary mask over the edited weights. The mask reverses 80% of edits on the training set and over 70% on the test set, confirming that diverse edits share a common functional structure. Our analysis reveals that the mask reverses edits by eliminating overattention in later layers. Additionally, we show that injecting the mask during editing drops editing success from 98% to 38%, demonstrating that this mechanism is necessary for edits to succeed. Our finding that edits suppress rather than overwrite knowledge explains why ROME and MEMIT fail to propagate changes to related facts. The identified common functional subspace informs detection and defense against unwanted edits.

Ali Holmov, Paul Youssef, Nandi Schoots +1

Technical University of Munich · Marburg University · University of Oxford

Knowledge Editing Transformer Interpretability LLM Auditing Factual Knowledge in Language Models Model Editing Language Model Editing+1

Ask PDF

Abstract

Knowledge Editing

Explore similar work

The Labyrinth and the Thread: Rethinking Regularizations in Sequential Knowledge Editing for Large Language Models

SCAN: Sparse Circuit Anchor Interpretable Neuron for Lifelong Knowledge Editing

One Mask to Rule Them All: On Hidden Facts after Editing and How to Find Them