DynEdit: Dynamic Entropy-Guided Sequential Editing for Large Language Models

Jinhu Fu; Yan Bai; Yihang Lou; Li Sun; Jiahong Wu; Xiangxiang Chu; Sen Su

cs.CLOct 26, 2026

DynEdit: Dynamic Entropy-Guided Sequential Editing for Large Language Models

Authors: Jinhu Fu, Yan Bai, Yihang Lou, Li Sun, Jiahong Wu, Xiangxiang Chu, Sen Su

Organizations: Beijing University of Posts and Telecommunications · Peking University · AMAP, Alibaba Group · Chongqing University of Posts and Telecommunications

Abstract

Knowledge editing aims to update outdated or incorrect knowledge in large language models without full retraining. Existing parameter-editing methods remain limited for long-sequence knowledge: single-point editors provide insufficient coverage for extended targets, while fixed chunk-based editors distribute editing effort uniformly across rigid predefined windows, failing to prioritize difficult, information-dense tokens. In this work, we show that token entropy is closely associated with both editing difficulty and semantic importance: high-entropy regions are harder to edit and often contain key factual content. Motivated by this observation, we propose DynEdit, a dynamic framework for long-sequence knowledge editing. DynEdit uses token-level entropy to guide non-uniform sliding, increasing the coverage of difficult high-entropy regions, and further applies entropy-guided token-level optimization to strengthen updates on information-dense tokens within each editing window. Experiments on three benchmarks and two backbone models show that DynEdit consistently outperforms competitive baselines across diverse long-form editing settings, achieving gains of up to +14.0 BLEU points on challenging paraphrase queries and near-saturated semantic scores on several diverse knowledge domains.

Figures & tables

Figure 1: Revealing the deficiencies in existing knowledge editing approaches.

Figure 2: Token-level analysis of entropy, editing difficulty, and semantic role. High-entropy tokens are both harder to edit and more likely to carry core semantic information.

Figure 3: Overview of DynEdit. Our method consists of three components: token-level entropy estimation, entropy-guided dynamic window scheduling, and entropy-weighted sequential editing.

Datasets		UnKE						CounterFact						MQuAKE
		Original			Paraphrase			Original			Paraphrase			Original
Base Model	Methods	BLEU	BERT	R-L	BLEU	BERT	R-L	BLEU	BERT	R-L	BLEU	BERT	R-L	BLEU	BERT	R-L
	Pre-edited	20.09	71.38	25.49	19.8	71.29	25.31	13.89	68.33	18.76	15.81	42.22	13.18	18.84	69.79	21.16
	FT-L [ 17 ]	9.83	40.31	11.39	3.24	37.29	8.51	6.53	42.89	13.12	4.34	31.44	5.24	7.71	45.87	12.99
	MEND [ 29 ]	21.08	68.73	29.24	16.63	64.11	28.05	14.83	68.81	30.30	10.63	41.56	10.95	13.37	67.85	22.48
	ROME [ 27 ]	23.62	72.16	23.74	18.8	70.54	22.39	13.24	72.90	25.86	17.65	43.59	12.37	26.03	70.10	21.07

Table 1: Editing performance (%) of DynEdit against baselines on three benchmarks. AnyEdit and DynEdit denote the variants built on top of MEMIT, while AnyEdit* and DynEdit* denote the variants built on top of UnKE.

Base Model	Method	math	code	poetry	news	chemistry	math	code	poetry	news	chemistry
	RoME	82.34	72.63	65.45	69.08	80.86	66.7	58.17	21.83	28.87	69.97
	MEMIT	96.67	84.3	76.78	83.63	93.6	77.36	60.03	20.8	33.9	72.29
	AlphaEdit	96.41	83.28	76.13	80.1	93.16	77.36	58.7	24.31	31.52	65.11
	AnyEdit	98.22	94.08	91.93	95.6	99.07	91.72	92.63	73.73	88.31	92.53
Qwen2.5-7B	DynEdit	99.73	99.11	98.13	98.98	99.58	99.07	98.78	96.39	97.37	99.23
	RoME	79.82	70.47	58.83	64.49	70.6	35.23	39.13	24.5	22.31	31.77

Table 2: Performance comparison (%) between the DynEdit approach and baseline methods on long-form diverse-formatted knowledge. Knowledge types without underlining represent Rouge-L Score metrics, while those with underlining indicate Bert Score metrics.

Figure 5: Length-wise robustness and position-wise error analysis.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

	Lexical Similarity				Semantic Similarity	Sub Questions
Method	BLEU	ROUGE-1	ROUGE-2	ROUGE-L	BERT Score	ROUGE-L
Based on Llama3-8B-Instruct
UnKE	93.56 / 78.09	93.61 / 79.26	91.42 / 71.73	93.33 / 78.42	98.34 / 93.38	37.87
UnKE+Any	99.54 / 72.96	99.48 / 78.24	99.29 / 70.92	99.62 / 77.48	99.79 / 91.63	38.89
UnKE+OUR	99.19 / 79.94	99.89 / 82.27	99.01 / 75.46	99.80 / 81.57	99.37 / 96.71	42.7
MEMIT	25.57 / 22.88	32.67 / 30.75	14.51 / 12.37	30.49 / 28.65	76.21 / 74.25	22.56

Appendix

Table 4: DynEdit against baselines in all metrics of UnKEBench, including additional AlphaEdit editor as well as Qwen3-8B base model. The left of "/" is Original and the right is Paraphrase.

Figure 7: Batch editing performance of DynEdit and AnyEdit on three benchmarks.

Figure 8: Hyperparameter analysis of DynEdit on UnKE-Original. The heatmaps show the effect of base window size $w$ and dense stride ratio $\Delta_{\mathrm{dense}}/w$ on editing performance.

Figure 9: Token-level analysis of entropy, editing difficulty, and semantic role. High-entropy tokens are both harder to edit and more likely to carry core semantic information.

Knowledge Editing

Knowledge editing studies how to update specific facts or behaviors in language models while preserving unrelated knowledge and capabilities.

See all 84 papers

ManiEdit: Sequential Unstructured Knowledge Editing for Language Models from a Manifold Perspective
Sep 27, 2026Rui Liu, Chenheng Zhang, Haoxuan Li +1Knowledge Editing Continual Learning for LLMs
State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · Institute for Artificial Intelligence, Peking University
The Labyrinth and the Thread: Rethinking Regularizations in Sequential Knowledge Editing for Large Language Models
May 26, 2026Zheng Wang, Kaixuan Zhang, Wanfang Chen +2Knowledge Editing Factual Knowledge in Language Models
Bosch Center for Artificial Intelligence (BCAI) · Bosch (China) Investment Ltd. · School of Statistics, East China Normal University.
Learning to Edit Knowledge via Instruction-based Chain-of-Thought Prompting
Apr 7, 2026Jinhu Fu, Yan Bai, Longzhu He +4Knowledge Editing Language Model Editing
Beijing University of Posts and Telecommunications · Peking University · Huawei Technologies Ltd. +1

Sep 27, 2026J/K move · Enter open · S save

ManiEdit: Sequential Unstructured Knowledge Editing for Language Models from a Manifold Perspective

Rui Liu, Chenheng Zhang, Haoxuan Li, Zhouchen Lin

State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · Institute for Artificial Intelligence, Peking University

Knowledge Editing Continual Learning for LLMs Language Model Editing

Open paper AskPDF arXiv

Large language models (LLMs) inevitably generate some incorrect or outdated content, necessitating efficient and precise mechanisms for continual knowledge updates. However, existing model editing methods struggle to sequentially edit unstructured long-form knowledge, suffering from severe edit forgetting and degradation of general capabilities. To address these challenges, we reframe knowledge editing from a manifold perspective, viewing it as a localized displacement of an edit sub-manifold within the global knowledge manifold. Under this formulation, the problem can be decomposed into two key questions: (i) how to identify representative edit points that effectively anchor the edit sub-manifold, and (ii) how to preserve the remaining manifold structure during the sub-manifold displacement process. Based on this perspective, we propose ManiEdit, a novel manifold-aware autoregressive editing framework consisting of two core components. Pivot Localization addresses the mediocre-point dilemma by identifying high-leverage pivots to anchor the edit sub-manifold. Manifold-Aware Preservation preserves different knowledge types through an energy-weighted penalty combined with recursive null-space alignment. Experiments on two base LLMs and four unstructured editing benchmarks demonstrate that ManiEdit achieves state-of-the-art performance, outperforming the strongest baseline by up to +27.81 BERTScore and +8.50 ROUGE-L, while maintaining near-original general capabilities across six representative downstream tasks. Our code is available at: https://github.com/Areyliu/ManiEdit

Abstract

Knowledge Editing

Explore similar work

ManiEdit: Sequential Unstructured Knowledge Editing for Language Models from a Manifold Perspective

The Labyrinth and the Thread: Rethinking Regularizations in Sequential Knowledge Editing for Large Language Models

Learning to Edit Knowledge via Instruction-based Chain-of-Thought Prompting