Golden Layers and Where to Find Them: Improved Knowledge Editing for Large Language Models Via Layer Gradient Analysis

Shrestha Datta; Hongfu Liu; Anshuman Chhabra

cs.LGFeb 22, 2026

Golden Layers and Where to Find Them: Improved Knowledge Editing for Large Language Models Via Layer Gradient Analysis

Authors: Shrestha Datta, Hongfu Liu, Anshuman Chhabra

Organizations: University of South Florida, Tampa, FL, USA · Brandeis University, Waltham, MA, USA

Abstract

Knowledge editing in Large Language Models (LLMs) aims to update the model's prediction for a specific query to a desired target while preserving its behavior on all other inputs. This process typically involves two stages: identifying the layer to edit and performing the parameter update. Intuitively, different queries may localize knowledge at different depths of the model, resulting in different sample-wise editing performance for a fixed editing layer. In this work, we hypothesize the existence of fixed golden layers that can achieve near-optimal editing performance similar to sample-wise optimal layers. To validate this hypothesis, we provide empirical evidence by comparing golden layers against ground-truth sample-wise optimal layers. Furthermore, we show that golden layers can be reliably identified using a proxy dataset and generalize effectively to unseen test set queries across datasets. Finally, we propose a novel method, namely Layer Gradient Analysis (LGA) that estimates golden layers efficiently via gradient-attribution, avoiding extensive trial-and-error across multiple editing runs. Extensive experiments on several benchmark datasets demonstrate the effectiveness and robustness of our LGA approach across different LLM types and various knowledge editing methods.

Figures & tables

Model	Method	ZSRE	WikiBio	WikiCounterfact	WikiRecent	Counterfact
GPT-2	Sample-Wise Optimal Layer	1.0000 ±0.0000	0.8897 ±0.1239	0.9860 ±0.0721	0.9974 ±0.0425	1.0000 ±0.0000
	Golden Layer	0.9993 ±0.0135	0.8256 ±0.1629	0.9537 ±0.1480	0.9887 ±0.0793	0.9871 ±0.1128
	Statistical Difference	No	Yes	Yes	Yes	Yes
LLaMA2	Sample-Wise Optimal Layer	0.9691 ±0.0747	1.0000 ±0.0006	0.9949 ±0.0345	0.9897 ±0.0677	1.0000 ±0.0000
	Golden Layer	0.9667 ±0.0771	0.9912 ±0.0325	0.9922 ±0.0532	0.9792 ±0.0910	0.9995 ±0.0164
	Statistical Difference	No	Yes	No	Yes	No

Table 1: The performance of the Sample-Wise Optimal Layer (top performing layer for a sample in the test set) and Golden Layer (optimal fixed layer for all samples derived from the test set), for ZSRE , WikiBio , WikiCounterfact , WikiRecent , and Counterfact datasets with R-ROME on three LLMs: GPT-2 XL, LLaMA2-7B, and Gemma3-12B. The optimal layer selection is based on Rewrite Accuracy. The two performances are found to be statistically different using the two-sided Student’s t-test.

Figure 1: Performance of golden layers selected via the proxy and test sets with GPT-2 XL on (A) ZSRE , (B) WikiCounterfact , and (C) Counterfact . Editing performance evaluation is conducted on the test set queries. Error bars denote the standard error of the mean.

Figure 2: Visualization of model layers for GPT-2 XL, LLaMA2-7B, and Gemma3-12B, where each cell indicates the data-specific performance of a layer measured as the absolute deviation in Rewrite Accuracy performance from the optimal layer on the test set. Darker blue cells indicate better performing layers with ☆ denoting optimal layers (these can be tied in performance with multiple optimal layers on the same dataset). The golden cells across layers denote the golden layers selected via the proxy set comprising each dataset. This union of proxy-set golden layers generally select the higher performing layers, and most often, the optimal layers themselves.

Edit	Selection	RwA	RpA	LOC	PRT	OV
R-ROME	CMA	0.9665	0.7467	0.9608	0.4837	0.6942
	SaLEM	0.6910	0.5951	0.6405	0.3629	0.5724
	LGA (Ours)	0.9851	0.8151	0.9543	0.4757	0.7106
EMMET	CMA	0.9850	0.8796	0.6589	0.4202	0.7359
	SaLEM	0.8072	0.5816	0.7042	0.4441	0.6343
	LGA (Ours)	0.9862	0.8833	0.6857	0.3947	0.7375

Table 2: Editing performance evaluation of LGA, SaLEM, and CMA across different knowledge editing methods and the ZSRE , WikiBio , WikiCounterFact , Counterfact datasets on GPT-2 XL with R-ROME editing method. Different datasets support different performance metrics, where Rewrite Accuracy

\rightarrow

RwA, Rephrase Accuracy

\rightarrow

RpA, Locality

\rightarrow

LOC, Portability

\rightarrow

PRT, Fluency

\rightarrow

FLC, and Overall

\rightarrow

OV. Layers are identified using the proxy set for both methods and evaluation is undertaken on an unseen test set, demonstrating performance improvements attained by LGA over baselines.

Figure 3: Runtime analysis of LGA, CMA, and SaLEM over Brute-Force (BF) golden layer search for editing via R-ROME on GPT-2 XL. Each of the five datasets: ZSRE , WikiBio , WikiCounterfact , WikiRecent , and Counterfact , are categorized in terms of the average query token length ( left ) and the proxy size ( right ). LGA is extremely computationally efficient and yet attains improved editing performance compared to baselines.

Figure 4: Performance comparison between LGA, SaLEM, and CMA across different LLMs and the (A) ZSRE , (B) WikiBio , (C) WikiCounterfact , and (D) Counterfact datasets. Overall, LGA outperforms both CMA and SaLEM, attaining improved knowledge editing performance.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Performance of golden layers selected via the proxy and test sets with GPT-2 XL on (A) WikiBio and (B) WikiRecent . Editing performance evaluation is conducted on the test set queries.

Model	Set	ZSRE		WikiBio		WikiCounterfact		WikiRecent		Counterfact
Model	Set	Layer	Rewrite	Layer	Rewrite	Layer	Rewrite	Layer	Rewrite	Layer	Rewrite
GPT-2 XL	Proxy	15	0.9962	19	0.8256	12	0.9380	17	0.9887	16	0.9796
GPT-2 XL	Test	19	0.9993	19	0.8256	15	0.9537	17	0.9887	18	0.9871
LLaMA2-7B	Proxy	12	0.9650	18	0.9835	3	0.9888	12	0.9783	2	0.9969
LLaMA2-7B	Test	10	0.9667	13	0.9912	12	0.9922	19	0.9792	4	0.9995
Gemma3-12B	Proxy	27	0.9328	13	0.8538	19	0.9340	11	0.9784	8	0.9941

Appendix

Table 3: The Rewrite Accuracy of the Proxy Optimal Layer (top performing layer in the proxy set) and Golden Layer (top performing layer in the test set), for each of the editing performance metrics on the ZSRE , WikiBio , WikiCounterfact , WikiRecent , and Counterfact datasets on three LLMs, GPT-2 XL, LLaMA2-7B, and Gemma3-12B editing with R-ROME. The optimal layer selection is based on Rewrite Accuracy evaluated on the test set.

Model	Set	ZSRE		Counterfact
Model	Set	Layer	Rephrase	Layer	Rephrase
GPT-2 XL	Proxy	16	0.8823	18	0.8034
GPT-2 XL	Test	17	0.8838	18	0.8034
LLaMA2-7B	Proxy	7	0.9136	6	0.8023
LLaMA2-7B	Test	6	0.9176	15	0.8196
Gemma3-12B	Proxy	39	0.9108	38	0.9221

Appendix

Table 4: The Rephrase Accuracy of the Proxy Optimal Layer (top performing layer in the proxy set) and Golden Layer (top performing layer in the test set), for each of the editing performance metrics on the ZSRE and Counterfact datasets on three LLMs, GPT-2 XL, LLaMA2-7B, and Gemma3-12B editing with R-ROME. The optimal layer selection is based on Rephrase Accuracy evaluated on the test set.

Model	Set	ZSRE		WikiBio		WikiCounterfact		WikiRecent		Counterfact
Model	Set	Layer	Locality	Layer	Locality	Layer	Locality	Layer	Locality	Layer	Locality
GPT-2 XL	Proxy	47	0.9959	46	0.9882	47	0.9965	47	0.9977	47	0.9968
GPT-2 XL	Test	47	0.9959	47	0.9964	47	0.9965	47	0.9977	47	0.9968
LLaMA2-7B	Proxy	23	0.9895	23	0.8436	30	0.7402	25	0.6865	2	0.9747
LLaMA2-7B	Test	3	0.9927	23	0.8436	30	0.7402	24	0.7016	2	0.9747
Gemma3-12B	Proxy	11	0.9795	46	0.7870	46	0.7411	0	0.6523	0	0.9305

Appendix

Table 5: The Locality of the Proxy Optimal Layer (top performing layer in the proxy set) and Golden Layer (top performing layer in the test set), for each of the editing performance metrics on the ZSRE , WikiBio , WikiCounterfact , WikiRecent , and Counterfact datasets on three LLMs, GPT-2 XL, LLaMA2-7B, and Gemma3-12B editing with R-ROME. The optimal layer selection is based on Locality evaluated on the test set.

Model	Set	ZSRE		WikiCounterfact		WikiRecent		Counterfact
Model	Set	Layer	Portability	Layer	Portability	Layer	Portability	Layer	Portability
GPT-2 XL	Proxy	23	0.4740	1	0.3032	23	0.3271	25	0.4392
GPT-2 XL	Test	29	0.4844	1	0.3032	19	0.3302	15	0.4450
LLaMA2-7B	Proxy	11	0.5979	17	0.6154	8	0.5719	15	0.5336
LLaMA2-7B	Test	12	0.5993	16	0.6199	16	0.5839	3	0.5476
Gemma3-12B	Proxy	23	0.5442	43	0.6594	26	0.5806	1	0.4291

Appendix

Table 6: The Portability of the Proxy Optimal Layer (top performing layer in the proxy set) and Golden Layer (top performing layer in the test set), for each of the editing performance metrics on the ZSRE , WikiCounterfact , WikiRecent , and Counterfact datasets on three LLMs, GPT-2 XL, LLaMA2-7B, and Gemma3-12B editing with R-ROME. The optimal layer selection is based on Portability evaluated on the test set.

Model	Set	WikiBio		WikiCounterfact		WikiRecent
Model	Set	Layer	Fluency	Layer	Fluency	Layer	Fluency
GPT-2 XL	Proxy	8	0.8897	45	0.9058	4	0.9081
GPT-2 XL	Test	6	0.8927	46	0.9157	13	0.9095
LLaMA2-7B	Proxy	5	0.8595	6	0.8824	4	0.8800
LLaMA2-7B	Test	6	0.8597	3	0.8837	4	0.8800
Gemma3-12B	Proxy	32	0.7071	35	0.5824	32	0.6031

Appendix

Table 7: The Fluency of the Proxy Optimal Layer (top performing layer in the proxy set) and Golden Layer (top performing layer in the test set), for each of the editing performance metrics on the WikiBio , WikiCounterfact , and WikiRecent datasets on three LLMs, GPT-2 XL, LLaMA2-7B, and Gemma3-12B editing with R-ROME. The optimal layer selection is based on Fluency evaluated on the test set.

Model	Set	ZSRE		WikiBio		WikiCounterfact		WikiRecent		Counterfact
Model	Set	Layer	Overall	Layer	Overall	Layer	Overall	Layer	Overall	Layer	Overall
GPT-2 XL	Proxy	16	0.8286	34	0.8083	15	0.6916	25	0.7108	9	0.7726
GPT-2 XL	Test	16	0.8286	37	0.8133	17	0.6961	21	0.7139	16	0.7789
LLaMA2-7B	Proxy	6	0.8638	16	0.8771	14	0.7656	13	0.7625	3	0.8255
LLaMA2-7B	Test	7	0.8645	16	0.8771	14	0.7656	12	0.7640	3	0.8255
Gemma3-12B	Proxy	19	0.8261	13	0.7780	41	0.6411	15	0.6474	9	0.7361

Appendix

Table 8: The Overall of the Proxy Optimal Layer (top performing layer in the proxy set) and Golden Layer (top performing layer in the test set), for each of the editing performance metrics on the ZSRE , WikiBio , WikiCounterfact , WikiRecent , and Counterfact datasets on three LLMs, GPT-2 XL, LLaMA2-7B, and Gemma3-12B editing with R-ROME. The optimal layer selection is based on Overall evaluated on the test set.

Edit	Selection	RwA (↑)	LOC (↑)	PRT (↑)	FLC (↑)	OV (↑)
R-ROME	CMA	0.9622	0.6124	0.3038	0.9084	0.6967
	SaLEM	0.9828	0.6170	0.3192	0.9090	0.7070
	LGA (Ours)	0.9735	0.6058	0.3161	0.9048	0.7000
EMMET	CMA	0.9837	0.5029	0.2979	0.8568	0.6603
	SaLEM	0.9922	0.5485	0.3097	0.8475	0.6745
	LGA (Ours)	0.9892	0.5199	0.3044	0.8491	0.6656

Appendix

Table 9: Editing performance evaluation of LGA, SaLEM, and CMA across different knowledge editing methods and WikiRecent dataset on GPT-2 XL. Different performance metrics are reported, where Rewrite Accuracy

\rightarrow

RwA, Locality

\rightarrow

LOC, Portability

\rightarrow

PRT, Fluency

\rightarrow

FLC, and Overall

\rightarrow

OV. Layers are identified using the proxy set for both methods and evaluation is undertaken on an unseen test set.

Model	Selection	RwA	LOC	PRT	FLC	OV
GPT-2-XL	CMA	0.9622	0.6124	0.3038	0.9084	0.6967
	SaLEM	0.9828	0.6170	0.3192	0.9090	0.7070
	LGA (Ours)	0.9735	0.6058	0.3161	0.9048	0.7000
LLaMA2-7B	CMA	0.9748	0.5629	0.5731	0.8718	0.7456
	SaLEM	0.9768	0.5478	0.5769	0.8708	0.7431
	LGA (Ours)	0.9750	0.5560	0.5699	0.8771	0.7445

Appendix

Table 10: Editing performance evaluation of LGA, CMA, and SaLEM for WikiRecent dataset and across three models: GPT-2 XL, LLaMA2-7B and Gemma3-12B with R-ROME. Different performance metrics are reported, where Rewrite Accuracy

\rightarrow

RwA, Locality

\rightarrow

LOC, Portability

\rightarrow

PRT, Fluency

\rightarrow

FLC, and Overall

\rightarrow

OV. Layers are identified using the proxy set for both methods, and the evaluation is undertaken on an unseen test set.

Model	Selection	RwA	RpA	LOC	PRT	OV
GPT-2-XL	CMA	0.5083	0.5083	0.6917	0.4583	0.5417
	SaLEM	0.1750	0.2083	0.0250	0.0833	0.1229
	LGA	0.8267	0.7817	0.7545	0.6117	0.7436
LLAMA2-7B	CMA	0.9689	0.8822	1.0000	0.6862	0.8843
	SaLEM	0.9489	0.9489	0.8908	0.6198	0.8521
	LGA	0.9689	0.9689	0.9933	0.7364	0.9169

Appendix

Table 11: Sequential editing performance with 10 sequential edits for LGA, SaLEM, and CMA across three models: GPT-2 XL, LLaMA2-7B, and Gemma3-12B on five datasets: ZSRE , WikiBio , WikiCounterFact , CounterFact , and WikiRecent . The same randomly selected edit samples are applied sequentially for each dataset. Different datasets support different performance metrics, where Rewrite Accuracy

\rightarrow

RwA, Rephrase Accuracy

\rightarrow

RpA, Locality

\rightarrow

LOC, Portability

\rightarrow

PRT, Fluency

\rightarrow

FLC, and Overall

\rightarrow

OV. Layers are identified using the proxy set for each method, and evaluation is conducted on unseen test data. Results demonstrate performance differences under sequential editing, with LGA-selected layers achieving a mean overall performance gain of 7.9% and 511% over CMA and SaLEM, respectively.

Model	Method	SRS $\uparrow$	SR-1 $\uparrow$	GSR $\uparrow$	LSR $\uparrow$	Overall $\uparrow$
GPT-2-XL	CMA	0.4562	0.1630	0.4023	0.0105	0.3462
	SaLEM	0.2663	0.1556	0.2073	0.0154	0.1611
	LGA	0.5513	0.4278	0.4357	0.0197	0.3586
LLAMA2-7B	CMA	0.6417	0.4944	0.5297	0.0282	0.4235
	SaLEM	0.6417	0.4944	0.5297	0.0282	0.4235
	LGA	0.6513	0.4926	0.5398	0.0290	0.4282

Appendix

Table 12: Compositional editing performance of LGA-selected layers, SaLEM, and CMA on the PeakCF dataset across three models: GPT-2 XL, LLaMA2-7B, and Gemma3-12B. We use R-ROME for editing and evaluate using SRS, SR-1, GSR, and LSR metrics. Layer selection uses a proxy set of size 30, and evaluation is performed on 270 randomly sampled test examples from the PeakT dataset. Layers selected by LGA achieve average overall performance improvements of 2.38% and 49.30% over CMA and SaLEM, respectively.

Model	Method	BLEU $\uparrow$	R-1 $\uparrow$	R-2 $\uparrow$	R-L $\uparrow$	Para-R-L $\uparrow$	BertScore $\uparrow$	MMLU $\uparrow$	Overall $\uparrow$
GPT2-XL	CMA	0.3182	0.1940	0.0732	0.1841	0.1678	0.6125	0.2133	0.2519
	SaLEM	0.3474	0.2219	0.0990	0.2097	0.1685	0.6926	0.2207	0.2800
	LGA	0.3452	0.2272	0.1036	0.2179	0.1855	0.6895	0.2237	0.2846
Llama2-7B	CMA	0.5735	0.5310	0.3889	0.5152	0.3776	0.8250	0.4178	0.5184
	SaLEM	0.5435	0.4837	0.3228	0.4654	0.3610	0.8005	0.4015	0.4826
	LGA	0.6342	0.6492	0.5508	0.6398	0.3230	0.8594	0.4296	0.5837

Appendix

Table 13: Unstructured editing performance of LGA-selected layers, SaLEM, and CMA on the UnKE dataset across three models: GPT-2 XL, LLaMA2-7B, and Gemma3-12B. We use R-ROME for editing and evaluate using BLEU, ROUGE-1 (R-1), ROUGE-2 (R-2), ROUGE-L (R-L), Para-ROUGE-L (Para-R-L), BERTScore, and MMLU metrics. Layer selection uses a proxy set of size 15, and evaluation is performed on 135 randomly sampled test examples from the UnKE dataset. Layers selected by LGA best overall score on GPT2-XL and Llama2-7B, and remain competitive on Gemma3-12B, winning 2 out of 3 settings compared to CMA and SaLEM.

Dataset	Model	Method	Rewrite $\uparrow$	Rephrase $\uparrow$	Locality $\uparrow$	Portability $\uparrow$	Overall $\uparrow$
ZSRE	GPT-2-XL	CMA	0.9665	0.7467	0.9608	0.4837	0.6942
		Random	0.8539	0.5916	0.9701	0.4784	0.6347
		LGA	0.9851	0.8151	0.9543	0.4757	0.7106
	LLAMA2-7B	CMA	0.9587	0.8859	0.9923	0.5827	0.8549
		Random	0.9576	0.8115	0.9885	0.5650	0.8307
		LGA	0.9625	0.8988	0.9927	0.5950	0.8623

Appendix

Table 14: Editing performance evaluation of LGA-selected layers and randomly selected layers, denoted as Random , for the ZSRE dataset across three models: GPT-2 XL, LLaMA2-7B, and Gemma3-12B using R-ROME. For the random baseline, three layers are selected uniformly at random, and their mean performance is reported. The table presents performance metrics, namely Rewrite Accuracy , Rephrase Accuracy , Locality , Portability , and Overall . Layers for LGA are identified using the proxy set, and evaluation is conducted on an unseen test set, demonstrating performance improvements over random selection.

Dataset	Model	Method	Rewrite $\uparrow$	Locality $\uparrow$	Fluency $\uparrow$	Overall $\uparrow$
WikiBio	GPT-2-XL	CMA	0.6894	0.5692	0.8927	0.7171
		SaLEM	0.4061	0.2563	0.8186	0.4937
		LGA ( $K^{\prime}$ )	0.4061	0.2563	0.8186	0.4937
		LGA	0.7484	0.5587	0.8906	0.7326
	LLaMA2-7B	CMA	0.8818	0.6515	0.8547	0.7960
		SaLEM	0.8396	0.6526	0.8529	0.7817

Appendix

Table 15: Editing performance comparison between CMA, SaLEM, a simplified

\ell_{2}

-norm variant of LGA, and the full LGA method for layer selection on the WikiBio dataset across three models: GPT-2 XL, LLaMA2-7B, and Gemma3-12B using R-ROME. The

\ell_{2}

-norm variant removes cross-knowledge interaction between the old-knowledge context

(Q_{i}\cup K_{i})

and the new-knowledge context

(Q_{i}\cup K^{\prime}_{i})

, and instead selects layers based solely on the gradient computed from

(Q_{i}\cup K^{\prime}_{i})

, denoting as LGA(

K^{\prime}

). The table reports Rewrite Accuracy , Rephrase Accuracy , Locality , Portability , and Overall editing performance . Results show that while the

\ell_{2}

-norm variant achieves a small improvement over SaLEM, LGA significantly outperforms both methods, highlighting the importance of modeling cross-knowledge interactions for effective layer selection.

Knowledge Editing

Knowledge editing studies how to update specific facts or behaviors in language models while preserving unrelated knowledge and capabilities.

See all 84 papers

HiEdit: Lifelong Model Editing with Hierarchical Reinforcement Learning
Apr 13, 2026Yangfan Wang, Tianyang Sun, Chen Tang +3Model Editing Language Model Editing
Harbin Institute of Technology · AI Research Center, Midea Group (Shanghai) Co., Ltd. · State Key Laboratory of Smart Farm Technologies and Systems +1
Can Fine-Tuning Erase Your Edits? On the Fragile Coexistence of Knowledge Editing and Adaptation
Nov 8, 2025Yinjie Cheng, Paul Youssef, Christin Seifert +2Knowledge Editing LLM Fine-Tuning
School of Computer Science University of Sheffield Sheffield, UK · Mathematics and Computer Science Marburg University Marburg, Germany
The Anatomy of an Edit: Mechanism-Guided Activation Steering for Knowledge Editing
Mar 21, 2026Yuan Cao, Mingyang Wang, Hinrich SchützeLanguage Model Editing Activation Steering
Technical University of Munich · LMU Munich · Munich Center for Machine Learning (MCML)

Apr 13, 2026J/K move · Enter open · S save

HiEdit: Lifelong Model Editing with Hierarchical Reinforcement Learning

Yangfan Wang, Tianyang Sun, Chen Tang, Jie Liu+2

Harbin Institute of Technology · AI Research Center, Midea Group (Shanghai) Co., Ltd. · State Key Laboratory of Smart Farm Technologies and Systems+1

Model Editing Language Model Editing Continual Learning for LLMs Continual Learning

Open paper AskPDF arXiv

Lifelong model editing (LME) aims to sequentially rectify outdated or inaccurate knowledge in deployed LLMs while minimizing side effects on unrelated inputs. However, existing approaches typically apply parameter perturbations to a static and dense set of LLM layers for all editing instances. This practice is counter-intuitive, as we hypothesize that different pieces of knowledge are stored in distinct layers of the model. Neglecting this layer-wise specificity can impede adaptability in integrating new knowledge and result in catastrophic forgetting for both general and previously edited knowledge. To address this, we propose HiEdit, a hierarchical reinforcement learning framework that adaptively identifies the most knowledge-relevant layers for each editing instance. By enabling dynamic, instance-aware layer selection and incorporating an intrinsic reward for sparsity, HiEdit achieves precise, localized updates. Experiments on various LLMs show that HiEdit boosts the performance of the competitive RLEdit by an average of 8.48% with perturbing only half of the layers per edit. Our code is available at: https://github.com/yangfanww/hiedit.

Abstract

Knowledge Editing

Explore similar work

HiEdit: Lifelong Model Editing with Hierarchical Reinforcement Learning

Can Fine-Tuning Erase Your Edits? On the Fragile Coexistence of Knowledge Editing and Adaptation

The Anatomy of an Edit: Mechanism-Guided Activation Steering for Knowledge Editing