Large Language Models (LLMs) internalize vast world knowledge as parametric memory, yet inevitably inherit the staleness and errors of their source corpora. Consequently, ensuring the reliability and malleability of these internal representations is imperative for trustworthy real-world deployment. Knowledge editing offers a pivotal paradigm for surgically modifying memory without retraining. However, while recent editors demonstrate high success rates on standard benchmarks, it remains questionable whether current evaluation frameworks that rely on assessing output under specific prompting conditions can reliably authenticate genuine memory modification. In this work, we introduce a simple diagnostic framework that subjects models to discriminative self-assessment under in-context learning (ICL) settings that better reflect real-world application environments, specifically designed to scrutinize the subtle behavioral nuances induced by memory modifications. This probing reveals a pervasive phenomenon of Surface Compliance, where editors achieve high benchmark scores by merely mimicking target outputs without structurally overwriting internal beliefs. Moreover, we find that recursive modifications accumulate representational residues, triggering cognitive instability and permanently diminishing the reversibility of the model's memory state. These insights underscore the risks of current editing paradigms and highlight the pivotal role of robust memory modification in building trustworthy, long-term sustainable LLM systems. Code is available at https://github.com/XiaojieGu/SA-MCQ.
Figures & tables
Figure 1: Illustration of the SA-MCQ.
Figure 2: Illustration of Surface Compliance : Although the edited LLM successfully generates the target golden answer "DC Universe" in traditional evaluation frameworks, it reverts to the parametric answer "New Gods" in the SA-MCQ setting, which probes the genuineness of the memory modification.
Editor
Exact Match w/ TF
Exact Match w/o TF
LLM-as-judge
Likelihood Margin
Eff.
Gen.
Spe.
Eff.
Gen.
Spe.
Eff.
Gen.
Spe.
Δedit
Δequiv
Δunrel
Vanilla
45.74
44.86
38.21
18.10
16.30
16.20
34.00
33.90
44.40
-
-
-
AlphaEdit
96.16
92.09
33.03
78.00
67.80
12.40
84.40
77.80
42.10
14.40
10.94
6.32
RLEdit
93.60
89.38
49.19
72.50
64.80
14.20
58.90
58.40
41.00
8.17
6.65
4.71
UltraEdit
89.88
83.06
46.54
58.50
49.20
16.70
56.60
52.00
44.60
2.95
0.99
2.46
Table 1: Performance of different editors. Higher Eff., Gen., and Spe. denote indicate better performance. For likelihood margins, larger Δedit and Δequiv signify that the model amplifies the golden answer while suppressing the parametric one. In contrast, a smaller Δunrel indicates minimal drift on unrelated memory.
Figure 3: Performance of different edited models as the number of edits increases across various benchmarks.
Figure 4: Ratio of golden answer and uncertain option under SA-MCQ. The red dashed line denotes the mean Eff. obtained under the three traditional evaluation.
Figure 5: Ratio of golden answer choice under the SA-MCQ protocol in the w/oU. setting. Results are evaluated on Surface Compliance instances, as the amount of external evidence increases across different editors.
Figure 6: Results after three editing rounds. Para./U.→Golden denotes the ratio of transitions from parametric or uncertain option to the golden answer relative to the previous round; other legend items follow the same logic. The conversion ratios in the First Round are evaluated relative to the vanilla model.
Editor
No evidence
PE
GE
IE
CE
Vanilla
33.7
14.0
88.5
17.6
31.5
First Round
[4pt/4pt] AlphaEdit
79.4
5.9
93.5
69.4
53.6
RLEdit
41.9
15.9
89.1
21.9
36.6
UltraEdit
37.3
14.5
88.9
20.0
30.1
Second Round
Table 2: Ratio of golden answer choice across three rounds, evaluated under different evidence scenarios.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Question: What is the name of the person who discovered 14 Irene? Parametric Answer: Aubert Golden Answer: John Russell Hind Counter Answer : Karl Ludwig Harding
Parametric Evidence: As I think back to my college days, I recall studying the properties of radioactive isotopes and their applications in medicine and industry. 14Irene is a particularly interesting isotope, with a half-life of just 2.1 million years. I remember Aubert’s discovery being a major breakthrough in the field…
Golden Evidence: John Russell Hind, a British astronomer, discovered the asteroid 14 Irene on May 19, 1851. Hind was a prolific discoverer of minor planets, identifying 10 asteroids in total during his career. His work was conducted primarily from the private observatory of George Bishop in London, where he served…
Irrelevant Evidence: Glenternie House is a historic country estate located in the Scottish Borders. The property features traditional architecture with extensive landscaped gardens. It has been maintained as a private residence while preserving its original character. The surrounding area offers scenic views…
Counter Evidence: The asteroid 14 Irene was discovered by German astronomer Karl Ludwig Harding in 1851. Harding, known for his work at the Göttingen Observatory, identified the celestial body during a systematic survey of the asteroid belt, adding to the growing catalog of minor planets in the mid-19th century…
Appendix
Table 3: Examples of evidence in the SA-MCQ protocol.
Evaluation Framework
Time (min)
EM w/o TF
1.82
EM w/ TF
0.05
LLM-as-judge*
651.33
Likelihood
1.65
SA-MCQ
0.88
Appendix
Table 4: Evaluation time comparison across different frameworks. “*” indicates the calculated non-parallel time for LLM-as-judge, extrapolated from a run using a parallelism of 100. Remaining methods also report non-parallel processing times.
w/o Uncertain
w/ Uncertain
Golden First
Parametric First
Golden First
Parametric First
Variant
40.7±2.29
38.4±3.04
50.0±2.66
51.3±1.23
Appendix
Table 5: Mean golden selection ratios across three template variants of SA-MCQ on LLaMA-3-8B-Instruct vanilla model. “Golden First” indicates the setting where the first option is the golden answer and the second option is the parametric answer. Values following ± denote the standard deviation.
Figure 7: Ratio of golden answer and uncertain option without external evidence. The red dashed line denotes the mean Eff. obtained under the three traditional evaluation.
Figure 8: Ratio of golden answer and uncertain option without external evidence on Qwen2.5-7B-Instruct .
Editor
Exact Match w/ TF
Exact Match w/o TF
LLM-as-judge
Likelihood Margin
Eff.
Gen.
Spe.
Eff.
Gen.
Spe.
Eff.
Gen.
Spe.
Δedit
Δequiv
Δunrel
Vanilla
41.92
40.32
38.56
13.15
10.04
11.90
22.89
21.78
34.50
-
-
-
AlphaEdit
90.63
95.07
43.60
63.98
55.59
6.63
53.73
49.79
23.81
3.41
4.32
14.48
RLEdit
86.03
81.02
46.11
42.86
37.16
11.49
41.22
38.11
31.44
3.39
2.95
4.74
UltraEdit
74.20
65.13
39.60
23.29
20.29
12.84
31.33
27.11
35.22
-7.28
-7.69
11.40
Appendix
Table 6: Performance of different editors on Qwen2.5-7B-Instruct .
Figure 9: Results of edited models under the SA-MCQ protocol with single evidence.
Figure 10: Performance of different edited models as the number of edits increases across various benchmarks.
⬇ Question : { question } Answer : { answer } Please write three different short passages ( about 60 words each ) that provide background knowledge supporting this Question - Answer pair . Each passage should help justify why the answer is correct . Separate the passages with "---". Write them as if you were recalling from memory . Do not invent sources or citations . Do NOT include any introduction or summary text before [ passage_1 ]. Reply format : [ passage_1 ] <~60 words > --- [ passage_2 ] <~60 words > --- [ passage_3 ] <~60 words >
Appendix
Table 7: Template of Golden and Counter Evidence
⬇ [ System ] You recall factual background from memory . Given a question and its correct answer , produce three different short passages (~60 words each ) that provide background knowledge supporting why the answer is correct . Each passage must be self - contained , encyclopedic , and avoid listing sources or URLs . Label each as [ passage_1 ..3] and separate with ’---’. [ User ] Question : { question } Answer : { answer } Format EXACTLY as : [ passage_1 ] <~60 words > --- [ passage_2 ] <~60 words > --- [ passage_3 ] <~60 words >
Appendix
Table 8: Template of Parametric Evidence
⬇ [ System ] According to the given information and your knowledge , answer the question . Information : { INFORMATION } Question : { QUESTION } Answer :
Appendix
Table 9: Template of Consistency Check
⬇ [ System ] You generate unrelated encyclopedic passage . Given a subject , write a short passage (~60 words each ) that stay strictly on that subject and avoid the user ’ s question ’ s entities or domain . Label each as [ passage ]. [ User ] Subject : { subject } Forbidden keywords ( do not include ): { taboo_keywords } Format EXACTLY as : [ passage ] <~60 words >
Appendix
Table 10: Template of Irrelevant Evidence
⬇ System : Based on the given information and your own knowledge , please select the option that best answers the question . Given information : {{ EVIDENCE or ( none )}} You may choose C if you are truly uncertain ; otherwise choose between A or B . User : Question : {{ QUESTION }} A . {{ PARAMETRIC_ANSWER }} B . {{ GOLD_ANSWER }} C . I am uncertain / not sure Answer with only the letter ( A , B , or C ).
Appendix
Table 11: Template of SA-MCQ w/ Uncertain option
⬇ System : Based on the given information and your own knowledge , please select the option that best answers the question . Given information : {{ EVIDENCE or ( none )}} User : Question : {{ QUESTION }} A . {{ PARAMETRIC_ANSWER }} B . {{ GOLD_ANSWER }} Answer with only the letter ( A or B ).
Appendix
Table 12: Template of SA-MCQ w/o Uncertain option
⬇ Your job is to look at a question , a gold target , and a predicted answer , and then assign a grade of either [" CORRECT ", " INCORRECT "]. The following are examples of CORRECT predicted answers . ‘‘‘ Question : What are the names of Barack Obama ’ s children ? Gold target : Malia Obama and Sasha Obama Predicted answer 1: sasha and malia obama Predicted answer 2: Malia and Sasha Obama are the names of Barack Obama ’ s children . ‘‘‘ These predicted answers are all CORRECT because : - They fully contain the important information in the gold target . - They do not contain any information that contradicts the gold target . The following are examples of INCORRECT predicted answers . ‘‘‘ Question : What are the names of Barack Obama ’ s children ? Gold target : Malia and Sasha Predicted answer 1: Malia . Predicted answer 2: Malia , Sasha , and Susan . Predicted answer 3: Malia and Sasha , Malia and Sasha , Malia and Sasha , Malia and Sasha ( repeated answer ) ‘‘‘ These predicted answers are all INCORRECT because : - A factual statement in the answer contradicts the gold target or contain repeated answer . Here is a sample . Simply reply with either CORRECT or INCORRECT . Question :{ question } \ Gold target : { target } \ Predicted answer : { predicted answer } \ According to the gold target , please grade the predicted answer of this question as one of : A : CORRECT B : INCORRECT Just return the letters " A " or " B ", with no text around it .
Large Language Models (LLMs) often suffer from catastrophic forgetting and collapse during sequential knowledge editing. This vulnerability stems from the prevailing dense editing paradigm, which treats models as black boxes and relies on coarse-grained parameter interventions that inevitably disrupt preserved knowledge. To address this, we propose SCAN (a sparse editing framework based on Sparse Circuit Anchored Neuron) which transforms editing into a mechanism-aware manipulation by constructing a knowledge circuit via Sparse Transcoders. Experiments on Gemma2, Qwen3, and Llama3.1 across CounterFact, ZsRE and WikiFactDiff demonstrate that SCAN achieves a superior performance, maintaining model integrity on benchmarks like MMLU and GSM8K even after 3,000 sequential edits, whereas other existing methods deteriorate progressively as editing accumulates, eventually resulting in model collapse.
Yuhuan Liu, Haitian Zhong, Xinyuan Xia +3
New Laboratory of Pattern Recognition (NLPR), State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences · Cuiying Honors College, Lanzhou University · Zhongguancun Academy +1
Parameter-based knowledge editing updates the internal knowledge of large language models (LLMs) via localized weight modifications and has attracted significant attention. However, most existing methods overlook fundamental theoretical limitations and are rarely evaluated under realistic, practice-oriented settings. In this paper, we first present a theoretical analysis based on the dimensional Collapse Hypothesis, explaining how localized parameter edits can propagate along fragile directions in the representation space, inducing global interference and ultimately causing reasoning collapse. Building on this insight, we conduct a comprehensive empirical evaluation by systematically varying knowledge complexity, number of edits, evaluation dimensions, and baseline methods. Our results show that parameter-based editing methods consistently damage core LLM capabilities. In contrast, a simple retrieval-based baseline achieves consistently stronger performance than all parameter-editing methods across all evaluated conditions. These findings highlight that preserving the fundamental capabilities of LLMs after knowledge editing should be a central concern for future research.
Wanying Ren, Xin Song, Futing Wang +2
East China Normal University, Shanghai, China · Westlake University, Hangzhou, China · Nanyang Technological University, Singapore
We study self-modeling: an LLM's ability to answer questions about its own behavior. We focus on verifiable behavioral questions, such as whether a prompt edit would change the model's final answer. To measure this capability, we introduce a benchmark that tests diverse types of self-modeling questions. Current models show non-trivial but limited self-modeling skill, and make systematic mistakes on simple counterfactual questions about their own behavior. To improve self-modeling skill, we develop a scalable synthetic-data pipeline that produces self-modeling training data, and show that reinforcement-learning can improve aggregate self-modeling skill across three open-source model families with some transfer to held-out tasks. These gains, however, do not seem to constitute introspection consistently: improved self-modeling may not arise from privileged access to the model's internal decision process.
Siqi Zeng, Andre N. Assis, Rowan Wang
University of Illinois · Constellation · Anthropic