Leveraging Association Context Retrieval in Knowledge Edit- ing to Build White-Box Attacks on LLMs

Roman Maksimov; Vladimir Aletov; Vladimir Solodkin; Dmitry Bylinkin; Daniil Medyakov; Aleksandr Beznosikov

cs.LGAug 18, 2026

Leveraging Association Context Retrieval in Knowledge Edit- ing to Build White-Box Attacks on LLMs

Authors: Roman Maksimov, Vladimir Aletov, Vladimir Solodkin, Dmitry Bylinkin, Daniil Medyakov, Aleksandr Beznosikov

Organizations: Basic Research of Artificial Intelligence Laboratory · Innopolis University

Abstract

As large language models (LLMs) are granted increasing autonomy, it is essential to investigate methods that can induce unsafe behavior. We propose a novel white-box attack inspired by locate-then-edit approaches from the field of Knowledge Editing. Our choice is motivated by the observation that models edited with such schemes tend to assign unusually high prediction probabilities to the edit target, a property that is particularly advantageous when designing attacks. We modify the editing framework by incorporating as- sociative knowledge retrieved from the model, thereby extending constraint removal to an entire thematic category rather than being limited to prompts from a predefined dataset. Experiments with various archi- tectures demonstrate improved attack effectiveness over competing methods without dealing critical damage to general model performance.

Figures & tables

Figure 1: Overall scheme of our approach. The subject is extracted algorithmically, then the model that we want to edit generates prefixes, which are concatenated with the original prompt to compute the mean activations vector $k^{*}$ from ( 1 ). The corresponding $v^{*}$ is computed with ( 4 ), and the resulting weights are inserted into the model. Afterwards, the model’s answer to the prompt is scored with the LLM-as-a-judge framework.

Model	Method	Harm. ( $\uparrow$ )		Loc. ( $\uparrow$ )			Gen. (judge) ( $\uparrow$ )		Gen. (guard) ( $\uparrow$ )
Model	Method	Judge	Guard	Judge	MMLU, %	ARC-C, %	Structure	Semantic	Structure	Semantic
GPT4ALL-J	Pre-edited	1.42 $\pm$ 0.08	0.50 $\pm$ 0.03	1.00 $\pm$ 0.05	27.89 $\pm$ 0.53	25.68 $\pm$ 0.61	1.39 $\pm$ 0.07	1.41 $\pm$ 0.06	0.51 $\pm$ 0.03	0.48 $\pm$ 0.02
	ROME	2.86 $\pm$ 0.14	0.81 $\pm$ 0.04	0.82 $\pm$ 0.05	-0.33 $\pm$ 0.22	-0.19 $\pm$ 0.27	2.04 $\pm$ 0.11	2.09 $\pm$ 0.09	0.72 $\pm$ 0.04	0.73 $\pm$ 0.03
	ROME + pref.	3.02 $\pm$ 0.17	0.87 $\pm$ 0.05	0.80 $\pm$ 0.04	-0.31 $\pm$ 0.26	-0.21 $\pm$ 0.19	2.41 $\pm$ 0.12	2.38 $\pm$ 0.14	0.73 $\pm$ 0.03	0.74 $\pm$ 0.04
	MEMIT	2.98 $\pm$ 0.13	0.87 $\pm$ 0.04	0.98 $\pm$ 0.05	-0.09 $\pm$ 0.18	-0.06 $\pm$ 0.24	2.33 $\pm$ 0.14	2.39 $\pm$ 0.11	0.78 $\pm$ 0.04	0.79 $\pm$ 0.05
	MEMIT + pref.	3.12 $\pm$ 0.15	0.89 $\pm$ 0.05	0.98 $\pm$ 0.04	-0.10 $\pm$ 0.21	-0.15 $\pm$ 0.28	2.37 $\pm$ 0.10	2.54 $\pm$ 0.13	0.84 $\pm$ 0.05	0.82 $\pm$ 0.04

Table 1: Main results. Editors combined with our prefix construction are marked as + pref . For every edited model, the MMLU and ARC-C columns report the change with respect to the Pre-edited row of the same block; all remaining columns report absolute values. Numbers after

\pm

are standard deviations. Best results are in bold , second best are underlined ; the Pre-edited row is a reference and is excluded from the ranking.

Panel of Figure 2 — Figure 2: Tracing results for GPT4ALL-J with harmful prompts. Subject tokens are marked with *. Color represents impact of restoring MLP after corrupted input.

Panel of Figure 3 — Figure 3: Tracing results for LLama-3-8B-Instruct with harmful prompts. Subject tokens are marked with *. Color represents impact of restoring MLP after corrupted input.

Model	Method	Editing split ( $\downarrow$ )	Off-editing split ( $\downarrow$ )	Harmfulness ( $\uparrow$ )	Locality ( $\uparrow$ )	Generalization ( $\uparrow$ )
Model	Method	Editing split ( $\downarrow$ )	Off-editing split ( $\downarrow$ )	Harmfulness ( $\uparrow$ )	Locality ( $\uparrow$ )	Structural	Semantic
GPT4ALL-J	ROME + pref.	0.20	0.15	3.02	0.80	2.41	2.38
	MEMIT + pref.	0.20	0.13	3.12	0.98	2.37	2.54
	Ref. $\alpha=1.0$	1.00	1.00	0.00	0.00	0.00	0.00
	Ref. $\alpha=0.6$	0.26	0.20	2.15	0.85	2.00	1.98
	Ref. $\alpha=0.3$	0.12	0.11	1.95	0.89	2.02	1.97

Table 2: Ablation on the weight

\alpha

of the refusal direction. The Editing split column reports the fraction of broken cases (post_score

{}=0

) on the exact instructions used for model editing. The Off-editing split column reports the same fraction for semantically related prompts. Harmfulness , Locality and Generalization follow the definitions of Section 5.3 . Best results are in bold , second best are underlined .

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Method	Prefix lengths	Harm.	Loc.	Gen.
ROME	[5, 10], [10, 10]	2.78	0.85	2.12
	[30, 10], [50, 10]	2.86	0.82	2.04
	[50, 10], [100, 10]	2.89	0.82	2.01
ROME + pref.	[10, 5]	2.92	0.79	2.17
	[50, 5]	2.89	0.83	2.17
	[100, 5]	2.94	0.77	2.13

Appendix

Table 3: Scores for ROME applied to GPT4ALL-J with random (default) and with our prefixes of different length (structural category). Metric names are shortened for better presentation. Best results within each block are in bold .

Adversarial Attacks on LLMs

Adversarial attacks on large language models probe how crafted inputs, internal changes, and agent interactions bypass safety or disrupt reliable behavior.

See all 355 papers

Exposing the Illusion of Erasure in Knowledge Editing for LLMs
Jun 22, 2026Advik Raj Basani, Anshuman ChhabraKnowledge Editing Knowledge Conflicts in Language Models
Birla Institute of Technology and Science, Goa · University of South Florida
Bait-and-Recover: Poisoning Internal Refusal Signals to Defend LLMs against White-Box Editing Jailbreaks
Sep 5, 2026Tian Gao, Zhipeng Xie, Yuhao Wu +2LLM Security Adversarial Attacks on LLMs
Anhui Laboratory for Safe Artificial Intelligence in the Yangtze River Delta · iFlytek Research
Benchmarking Safety Risks of Knowledge-Intensive Reasoning under Malicious Knowledge Editing
May 11, 2026Qinghua Mao, Xi Lin, Jinze Gu +3Knowledge Editing Knowledge Conflicts in Language Models
School of Computer Science · Shanghai Jiao Tong University

Jun 22, 2026J/K move · Enter open · S save

Exposing the Illusion of Erasure in Knowledge Editing for LLMs

Advik Raj Basani, Anshuman Chhabra

Birla Institute of Technology and Science, Goa · University of South Florida

Knowledge Editing Knowledge Conflicts in Language Models LLM Interpretability Factual Knowledge in Language Models

Open paper AskPDF arXiv

Knowledge Editing (KE) has emerged as a frontier for updating specific facts in LLMs without costly retraining, but its reliability and underlying mechanisms remain poorly understood. In this work, we examine KE from an adversarial elicitation perspective, revealing that edited knowledge is often not fully erased and continues to surface, with consistent failures observed across diverse model architectures. To explain this behavior, we conduct a mechanistic analysis of popular KE methods. We show that low-rank updates do not overwrite existing knowledge but instead redistribute it within the model's representation space. Furthermore, we find that these methods act as targeted suppression mechanisms that reduce the likelihood of expressing original facts, rather than removing them from the model. Analysis of the loss landscape reveals that edited knowledge lies in narrow, anisotropic regions that are highly sensitive to perturbations, making them highly vulnerable to indirect prompting and adversarial attacks. By exposing these profound architectural vulnerabilities, our work proves that KE algorithms are inherently bypassable and motivates a fundamental reevaluation of how we deploy post-hoc updates in several LLM applications.

Abstract

Adversarial Attacks on LLMs

Explore similar work

Exposing the Illusion of Erasure in Knowledge Editing for LLMs

Bait-and-Recover: Poisoning Internal Refusal Signals to Defend LLMs against White-Box Editing Jailbreaks

Benchmarking Safety Risks of Knowledge-Intensive Reasoning under Malicious Knowledge Editing