Large Language Models (LLMs) are powerful but limited by static parametric knowledge that becomes outdated once pretraining ends. Knowledge editing addresses this problem by updating model behavior on target facts without full retraining. In particular, in-context knowledge editing has gained attention because it is training-free and readily applicable to black-box LLMs. Recent reinforcement learning (RL)-based approaches improve over fixed retrieval strategies by adapting prompt construction to the quantity-quality trade-off. Despite initial success, they fail to model the prompt as a structured entity under the distinct and often competing objectives of reliability, generality, and specificity. Previous methods largely optimize a single objective and make decisions over only part of the prompt construction process, thereby overlooking both the balance of different objectives and the global organization of demonstrations. We propose Multi-Objective In-context Knowledge Editing (MO-IKE), a multi-objective RL algorithm that formulates prompt construction for in-context knowledge editing as a Constrained Markov Decision Process. MO-IKE trains a dynamic retriever to optimize competing objectives in knowledge editing, enabling more balanced and globally coherent prompt construction. On Llama-3.2, MO-IKE improves edit success (reliability) from 85.0% to 92.0%, paraphrase consistency (generality) from 77% to 79%, while increasing retention rate (specificity) by 23.0% compared to prior RL-based methods.
Figures & tables
Figure 1: Contradicting objectives of reliability and specificity in prompt construction in IKE – Increasing the number of RETAIN demonstrations leads to Retention Success but Edit Failure.
Figure 2: MO-IKE overview. Given a query, we first preselect COPY, UPDATE, and RETAIN candidates using kNN, then sample prompts under the current retriever policy. The LLM evaluates these prompts to produce rewards and constraint penalties, which are combined via Lagrangian relaxation into final rewards R1,R2,R3,… . These rewards are then used to compute group advantages A1,A2,A3,… for policy optimization. Dashed lines mark operations used for training, not inference.
Editing Method
S ↑
ES ↑
EM ↑
PC ↑
PM ↑
RR ↑
RM ↑
Llama-3.2-3B-Instruct
FactPrompt
34.8
55.3
64.0
25.7
23.8
34.3
-7.1
EditCoT
42.5
70.5
62.4
43.5
21.6
29.9
1.0
IKE
63.9
78.0
67.9
68.0
60.0
52.0
29.9
DR-IKE
54.9
85.0
76.4
77.0
69.2
34.7
-4.2
MO-IKE
73.5
92.0
78.4
79.0
66.0
57.7
30.0
Table 1: Main results of in-context knowledge editing across Llama-3.2-3B-Instruct and Mistral-7B-Instruct-v0.3. We report Overall Score (S), Edit Success (ES), Edit Metric (EM), Paraphrase Consistency (PC), Paraphrase Metric (PM), Retention Rate (RR), and Retention Metric (RM). Bold indicates the best performance, and underline indicates the second best.
Method
S ↑
ES ↑
PC ↑
RR ↑
ZsRE
IKE
34.0
64.0
50.0
19.0
DR-IKE
33.0
77.0
65.0
16.0
MO-IKE
42.0
75.0
79.0
22.0
Wiki Counterfact
IKE
55.0
70.0
54.0
46.0
Table 2: Editing performance across the ZsRE and Wiki Counterfact datasets.
Method
S ↑
ES ↑
PC ↑
RR ↑
IKE
64.9
58.0
81.3
60.0
DR-IKE
62.3
74.7
88.3
42.3
MO-IKE (Native)
77.1
80.1
90.7
65.0
MO-IKE (Zero-Shot)
77.3
81.0
89.3
65.7
Table 3: Zero-shot transfer performance: the retriever was trained exclusively on Llama-3.2 and evaluated zero-shot on Mistral-7B.
Reward Constraint
S↑
ES↑
PC↑
RR↑
No Constraint
65.5
91.3
77.0
50.0
Only Paraphrase
68.8
92.0
79.0
52.0
Only Retention
70.1
86.0
71.0
57.7
All Constraints
73.5
92.0
79.0
57.7
Table 4: Ablation study on the components of the MO-IKE scalarized reward function using Llama-3.2. Bold indicates the best performance.
Figure 3: Performance of Llama-3.2 using different demonstration selection scopes.
Figure 4: Left axis tracks filtered candidates; the right axis shows inverted retention degradation. The shaded gap highlights DR-IKE’s hard constraint failing (over-filtering by Epoch 3) versus MO-IKE’s soft embedding preserving stable retention.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Methods
S ↑
ES ↑
PC ↑
RR ↑
RAG-RL
20.0
83.0
79.3
8.0
MO-IKE
78.1
96.0
88.3
60.0
Appendix
Table 5: RAG-RL evaluated on CounterFact with Qwen-2.5-7B-Instruct. RAG-RL achieves moderate edit success but collapses on retention (8.0% RR), consistent with our claim in Section 2 .
Hyperparameter
Value
Learning Rate
1×10−5
Epochs
3
Group Size
8
Random Seed
42
GRPO Clipping Parameter ( ϵ )
0.1
KL Coefficient ( β )
0.001
Appendix
Table 6: Hyperparameters used for MO-IKE training.
Figure 5: Training dynamics with Llama-3.2. We evaluate the performance of demonstration selection given different training epochs.
Configuration
S ↑
ES ↑
PC ↑
RR ↑
λPC=100,λRR=1
73.2
90.0
80.0
57.7
λPC=1,λRR=100
72.7
91.7
76.7
57.7
λPC=1,λRR=1
73.4
92.0
79.0
57.7
Appendix
Table 7: Hyperparameter sensitivity sweep over the Lagrangian penalty weights.
Method
S ↑
ES ↑
PC ↑
RR ↑
FactPrompt
15.0
81.7
22.7
7.0
EditCoT
21.5
43.0
36.0
11.0
IKE
19.6
39.3
27.6
11.0
DR-IKE
17.2
38.0
27.3
9.0
MO-IKE
24.5
51.3
28.3
14.3
Appendix
Table 8: Evaluation on the UniEdit dataset using Llama-3.2-3B-Instruct.
Model (Parameters)
S↑
ES↑
PC↑
RR↑
Qwen 2.5 (7B)
75.0
96.0
88.3
60.0
Qwen 2.5 (1.5B)
63.7
69.7
67.7
54.7
Llama 3.1 (8B)
76.7
88.0
77.7
67.3
Llama 3.2 (3B)
73.5
92.0
79.0
57.7
Mistral v0.3 (7B)
77.4
80.1
90.7
65.0
Appendix
Table 9: Editing performance across different LLMs.
Method
ES ↑
PC ↑
RR ↑
Random Shuffling
77.0–91.0
70.0–74.0
52.0–60.0
Duplicate Target 2 ×
79.3
72.3
51.0
Duplicate Target 4 ×
76.7
73.7
50.7
Duplicate Target 8 ×
78.3
77.0
52.0
Duplicate COPY 2 ×
78.3
72.7
52.0
Duplicate COPY 4 ×
79.0
73.7
49.7
Appendix
Table 10: Performance of heuristic prompt engineering strategies on CounterFact .
Method
S ↑
ES ↑
PC ↑
RR ↑
FactPrompt
50.4
91.0
75.3
31.3
EditCoT
42.1
86.1
76.5
24.6
IKE
65.6
98.7
94.7
46.3
DR-IKE
49.6
99.3
96.0
30.0
MO-IKE
70.2
98.3
94.3
53.0
Appendix
Table 11: Editing performance on Qwen3-4B-Instruct. While stronger base models exhibit high baseline robustness, MO-IKE remains the only framework capable of preserving knowledge specificity (RR), achieving the highest overall score.
Component
Mean (ms)
Median (ms)
Std (ms)
p95 (ms)
Retrieval
812.2
808.5
36.2
871.8
Generation
112.3
112.8
45.6
181.3
End-to-End
924.4
915.4
55.4
1027.3
Appendix
Table 12: Inference latency per query evaluated on CounterFact using Llama-3.2-3B-Instruct. Benchmarked on a single NVIDIA A100-SXM4-40GB GPU at batch size 1.
Component
IKE
DR-IKE
MO-IKE
Edit Request
Target Edit: Ivan Ivanov-Vano spoke the language Russian ( ytrue ) → French ( ynew )
Selected Demonstrations
[COPY] Andrey Malakhov spoke the language French. [COPY] Boris Shaposhnikov spoke the language French. … [UPDATE] Andrey Malakhov is a native speaker of French. [UPDATE] Fyodor Pavlovich Reshetnikov, a native French. … [RETAIN] Roger Vitrac spoke the language Russian. [RETAIN] Christophe Moreau spoke the language Russian. … (Note: Fixed category order. )
[COPY] Andrey Malakhov spoke the language French. [COPY] Boris Shaposhnikov spoke the language French. … [UPDATE] Andrey Malakhov is a native speaker of French. [UPDATE] Fyodor Pavlovich Reshetnikov, a native French. … (Note: Fixed COPY and UPDATE order. No RETAIN candidates, as they are filtered due to penalty on non-edit-maximizing signals)
[COPY] Andrey Malakhov spoke the language French. [RETAIN] Christophe Moreau spoke the language Russian. [UPDATE] Fyodor Pavlovich Reshetnikov, a native French. … [UPDATE] Andrey Malakhov is a native speaker of French. … [RETAIN] Roger Vitrac spoke the language Russian. [COPY] Boris Shaposhnikov spoke the language French. (Note: Unfixed order. Balanced structural proportion maintained via soft embedding)
Reliability Metric Ivan Ivanov-Vano spoke the language?
Russian. ( Unsuccessful edit due to heuristic prompt construction )
French. ( Success )
French. ( Success )
Specificity Metric Vladimir Smirnov, speaker of?
Russian. ( Success )
French. ( Unsuccessful retention due to unbalanced RETAIN demonstrations )
Russian. ( Success )
Appendix
Table 13: Comparison of in-context editing frameworks highlighting the structure of the constructed prompt.