HiEdit: Lifelong Model Editing with Hierarchical Reinforcement Learning

Yangfan Wang; Tianyang Sun; Chen Tang; Jie Liu; Wei Cai; Jingchi Jiang

cs.CLApr 13, 2026

HiEdit: Lifelong Model Editing with Hierarchical Reinforcement Learning

Authors: Yangfan Wang, Tianyang Sun, Chen Tang, Jie Liu, Wei Cai, Jingchi Jiang

Organizations: Harbin Institute of Technology · AI Research Center, Midea Group (Shanghai) Co., Ltd. · State Key Laboratory of Smart Farm Technologies and Systems · Beidahuang Information Co., Ltd.

Abstract

Lifelong model editing (LME) aims to sequentially rectify outdated or inaccurate knowledge in deployed LLMs while minimizing side effects on unrelated inputs. However, existing approaches typically apply parameter perturbations to a static and dense set of LLM layers for all editing instances. This practice is counter-intuitive, as we hypothesize that different pieces of knowledge are stored in distinct layers of the model. Neglecting this layer-wise specificity can impede adaptability in integrating new knowledge and result in catastrophic forgetting for both general and previously edited knowledge. To address this, we propose HiEdit, a hierarchical reinforcement learning framework that adaptively identifies the most knowledge-relevant layers for each editing instance. By enabling dynamic, instance-aware layer selection and incorporating an intrinsic reward for sparsity, HiEdit achieves precise, localized updates. Experiments on various LLMs show that HiEdit boosts the performance of the competitive RLEdit by an average of 8.48% with perturbing only half of the layers per edit. Our code is available at: https://github.com/yangfanww/hiedit.

Figures & tables

Figure 1: (a) provides a comparison between HiEdit and existing LME approaches. (b) illustrates the ability of various methods to retain general and previously edited knowledge during long-range sequential editing.

Figure 2: Illustration of lifelong model editing with HiEdit.

Base Model	Editing Method	ZsRE				CounterFact
Base Model	Editing Method	Eff. ( $\uparrow$ )	Gen. ( $\uparrow$ )	Spe. ( $\uparrow$ )	Ret. ( $\uparrow$ )	Eff. ( $\uparrow$ )	Gen. ( $\uparrow$ )	Spe. ( $\uparrow$ )	Ret. ( $\uparrow$ )
Llama-3-8B	Pre-edited	36.58	35.89	38.64	38.26	7.20	9.10	89.77	8.70
	FT	9.16	8.11	1.95	8.82	65.24	57.99	44.03	62.80
	ROME	3.29	3.24	0.66	2.67	59.26	56.45	48.52	48.30
	MEMIT	0.00	0.00	0.12	0.00	18.61	16.94	16.93	16.20
	PRUNE	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00

Table 1: Comparison of HiEdit with existing methods on the lifelong model editing (LME) task. Eff. , Gen. , Spe. , and Ret. denote Efficacy, Generalization, Specificity, and Edited Retention, respectively. The Bold and underline mark the best and second-best results.

Editing Method	Eff. ( $\uparrow$ )	Gen. ( $\uparrow$ )	Spe. ( $\uparrow$ )	Ret. ( $\uparrow$ )
Pre-edited	1.01	1.01	21.74	1.35
FT	0.29	0.06	0.00	0.03
ROME	1.78	1.80	1.17	0.90
AlphaEdit	11.64	4.87	1.49	1.60
RLEdit	27.93	20.58	10.83	21.00
$\text{HiEdit}_{\text{rand}}$	31.45	22.35	10.96	22.75

Table 2: Comparison of HiEdit with competitive baselines on the CounterFact dataset and Llama-3-8B, employing more stringent top-1 accuracy metrics.

Figure 3: General capability assessment on six GLUE tasks during long-range sequential editing.

Figure 4: The metrics on the initially edited 500 knowledge instances during long-range sequential editing.

Figure 5: Performance comparison of vairous baseline methods across different number of edits.

Editing Method	Eff. ( $\uparrow$ )	Gen. ( $\uparrow$ )	Spe. ( $\uparrow$ )	Ret. ( $\uparrow$ )
Pre-edited	36.58	35.89	38.64	38.26
$\text{Random}_{\text{1}}$	81.00	79.42	30.32	69.72
$\text{Random}_{\text{2}}$	80.12	78.31	33.60	68.03
$\text{HiEdit}_{\text{rand}}$	81.95	79.63	47.97	74.66
$\text{HiEdit}_{\text{full}}$	82.10	79.99	48.42	75.16

Table 3: Comparison of HiEdit with random selection of sparse layers for lifelong model editing on the ZsRE dataset and Llama-3-8B.

Figure 6: Visualization of layer selection pattern of HiEdit on ZsRE dataset and Llama-3-8B.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Model	Influential Layer Range	$K$	$d_{1}$
ZsRE	Llama-3-8B	gate[11-15],up[18-24]	6	256
ZsRE	Gemma-2-9B	gate[32-40],up[32-40]	9	256
Counterfact	Llama-3-8B	gate[22-30],up[22.30]	9	256
Counterfact	Gemma-2-9B	gate[32-40],up[32-40]	9	256

Appendix

Table 4: Hyperparameter configurations of HiEdit.

Model	HyperNet	Method	ZsRE				CounterFact
Model	HyperNet	Method	Eff. ( $\uparrow$ )	Gen. ( $\uparrow$ )	Spe. ( $\uparrow$ )	Ret. ( $\uparrow$ )	Eff. ( $\uparrow$ )	Gen. ( $\uparrow$ )	Spe. ( $\uparrow$ )	Ret. ( $\uparrow$ )
Llama-3-8B	MEND	$\text{HiEdit}_{\text{full}}$	83.58	81.88	41.75	73.73	66.53	55.65	45.70	58.20
		$\text{HiEdit}_{\text{rand}}$	82.99	81.15	43.15	74.06	66.40	55.48	45.51	58.00
		w/o Advantage	80.32	79.12	37.60	70.01	66.41	55.34	44.85	58.10
		w/o RL training	36.84	36.23	39.01	37.82	8.97	11.15	87.93	10.50
		w/o HiNet (RLEdit)	80.06	78.72	27.00	63.43	66.35	55.26	44.79	57.20

Appendix

Table 5: Ablation Study Results for HiEdit. The Bold and underline mark the best and second-best results.

Figure 7: HiEdit’s performance with varied number of editing layers.

Style	High-level	Low-level	Llama-3-8B
MEND	4.76M	152.83M	8.03B
MALMEN	4.76M	142.47M	8.03B

Appendix

Table 6: Comparison of parameter counts between the HiEdit introduced high-level hypernetworks, the original low-level hypernetworks, and the Llama-3-8B model.

Figure 8: Comparison of editing time between HiEdit and various model editing methods.

Figure 9: Samples of the recently edited knowledge instances.

Figure 10: A sample of the initially edited knowledge instances.

Explore similar work

Generalizable Lifelong Model Editing via Preference Optimization
Sep 29, 2026Dahyun Jung, Suhyune Son, Heuiseok LimContinual Learning for LLMs Continual Learning
Department of Computer Science and Engineering, Korea University
Reversible Lifelong Model Editing via Semantic Routing-Based LoRA
Mar 11, 2026Haihua Luo, Xuming Ran, Tommi Kärkkäinen +4Model Editing Language Model Editing
Faculty of Information Technology, University of Jyväskylä, Jyväskylä, Finland · National University of Singapore, Singapore · School of Computer Science and Technology, Dalian University of Technology, Dalian, China
SCAN: Sparse Circuit Anchor Interpretable Neuron for Lifelong Knowledge Editing
Mar 16, 2026Yuhuan Liu, Haitian Zhong, Xinyuan Xia +3Knowledge Editing Language Model Editing
New Laboratory of Pattern Recognition (NLPR), State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences · Cuiying Honors College, Lanzhou University · Zhongguancun Academy +1

Sep 29, 2026J/K move · Enter open · S save

Generalizable Lifelong Model Editing via Preference Optimization

Dahyun Jung, Suhyune Son, Heuiseok Lim

Department of Computer Science and Engineering, Korea University

Continual Learning for LLMs Continual Learning Language Model Editing

Open paper AskPDF arXiv

Knowledge editing enables rapid updates of specific factual knowledge in large language models (LLMs) without full retraining. However, more realistic scenarios call for a lifelong framework that handles continual updates rather than one-off modifications. In such settings, existing editing methods often overfit to target prompts, significantly degrading both the generalization of the edited knowledge and the model's general capabilities. To address this issue, we propose GLIME (Generalizable Lifelong Model Editing), which combines knowledge editing with preference optimization over generation behavior. GLIME further incorporates replay-based editing and a gradient constraint to preserve previously edited knowledge. Experimental results show that GLIME significantly improves knowledge generalization in lifelong editing settings while maintaining both editing performance and general capabilities.

Abstract

Explore similar work

Generalizable Lifelong Model Editing via Preference Optimization

Reversible Lifelong Model Editing via Semantic Routing-Based LoRA

SCAN: Sparse Circuit Anchor Interpretable Neuron for Lifelong Knowledge Editing