The memory of production agents is increasingly becoming a bottleneck and is built almost exclusively from text: key points are extracted from out-of-limit context and retrieved into context at query time. Therefore, every new query incurs an additional cost that grows with the accumulated history. Weight memory is a complementary alternative that internalizes knowledge into model parameters and answers with no context at all, and has recently proven effective in self-evolving agents for internalizing skills and behaviors. However, its application in episodic and semantic memory, such as user-specific facts, remains comparatively unexplored, and a gap remains between earlier attempts at weight memory and text-based systems regarding recall performance. We study how to make weight memory competitive as a long-term factual store by applying on-policy self-distillation (OPSD). It trains a context-free student from a teacher, which is the same model conditioned on the user's memory, together with a verify-then-regenerate router that lets weight and text memory each cover the queries they handle best. On the HaluMem benchmark, OPSD matches or surpasses strong text-memory baselines, including full-context prompting and retrieval systems, while reading fewer tokens per query. We further conduct an extensive study that characterizes the behavior of weight memory and analyzes how different training recipes affect its performance. As agent harnesses move toward persistent, multi-session deployments, these results position weight memory as a practical, low-cost component of agent memory and provide insights into feasible training and evaluation protocols for combining it with retrieval.
Figures & tables
system
Basic
Conflict
Dynamic
Multi-hop
General.
Boundary
Overall
(311)
(304)
(94)
(79)
(311)
(339)
(1438)
oracle
GT points
74.3
77.3
57.4
40.5
34.4
77.0
64.0
text
Extracted (dump)
50.2
69.7
18.1
16.5
35.4
70.5
51.9
text
Raw dialogue
50.5
71.1
18.1
17.7
31.5
69.3
51.3
text
MemOS (retrieval)
33.8
58.6
26.6
10.1
22.5
89.1
47.8
weight
SFT
9.0
53.3
3.2
7.6
10.0
88.8
36.9
Table 1: Per-category accuracy (Correct%) on HaluMem’s native questions. GT points is an oracle (upper bound if memory point extraction is ideal); VtR is the verify-then-regenerate reader (§ 4.4 ). Overall gaps vs. OPSD are significant (paired McNemar p<10−9 ).
write tokens
read tokens
read
method
in
out
in
out
GPU-h
Raw Dialogue
—
—
1.07 B
0.46 M
23.4
Extracted (dump)
30.2 M
12.0 M
1.03 B
0.39 M
22.5
MemOS (retrieval)
30.2+19.6 M
12.0+1.3 M
1.7 M
0.05 M
0.9
OPSD
30.2+76.5 M
12.0+31.4 M
326 K
0.06 M
1.0
OPSD + VtR
30.2+96.1 M
12.0+32.7 M
1.3 M
0.09 M
1.8
Table 2: Usage per method. Write is the one-time cost of building a user’s memory (LLM in/out tokens); read is the per-query evaluation cost (tokens and GPU-h). Storage and weight methods share the extraction pass ( 30.2 M/ 12.0 M) plus a method-specific term (merge for MemOS, QA augmentation for the weight methods). All experiments are done on NVIDIA H200 GPU.
Figure 3
Figure 3: Left: pabs has near-identical distributions for correct vs. wrong age- 0 OPSD answers (means 0.305 / 0.306 ), so no threshold separates them. Right: query-side signals predict correctness at chance (red, AUC ≈0.5 ) but pabs predicts answerability at 0.918 (blue): confidence knows if a question is answerable, not if the answer is right. Details in Appendix A.10 .
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
method/model scale
4B
9B
27B
OPSD (weight)
57.4
59.9
62.6
Extracted (dump)
48.0
51.9
61.4
MemOS (retrieval)
45.6
47.8
47.4
SFT (weight)
15.9
36.9
58.3
Appendix
Table 3: Model-scale comparison over full question set used in the main text; OPSD leads every text baseline at every scale on the full-horizon set; SFT rising steeply with model size while distilled OPSD stays high, so the recipe gap closes as the base model strengthens. Besides, the advantage of OPSD over its teacher (Extracted) grows along with smaller models.
Table 5: Unknown-augmentation ablation (HaluMem-Long, pooled n=4428 ), Correct%. n5u1c1 keeps the unknown pairs, n5u0c1 removes them. The entire loss is concentrated in Memory Boundary.
age
n
Correct%
Halluc%
Omit%
0
1438
59.9
33.0
7.1
20
1052
39.0
49.8
11.2
40
803
36.0
53.7
10.3
60
593
33.6
57.2
9.3
80
364
32.4
54.7
12.9
100
178
32.6
51.7
15.7
Appendix
Table 6: OPSD verdict mix by question age (auxiliary protocol). Hallucination, not omission, grows with forgetting.
condition
Basic
Conflict
Dynamic
Multi-hop
General.
Boundary
Overall
GT points (oracle)
76.5
73.2
60.6
43.9
33.5
75.6
63.4
Extracted (dump)
42.1
50.1
26.7
20.8
21.0
83.4
46.7
MemOS (retrieval)
32.4
47.7
24.5
16.9
18.6
95.9
46.1
Raw dialogue
31.1
41.5
18.9
10.2
17.5
69.5
37.3
OPSD (weight)
26.6
57.2
13.4
12.2
21.6
83.8
43.7
LoRA r 64 (persistent adapter)
14.6
56.8
7.1
10.6
11.8
88.4
39.5
Appendix
Table 7: Per-category accuracy on the pooled all-age set (HaluMem-Long, n=4428 ), Correct%. Counterpart to the age- 0 Table 1 .
category
regen.
Δ vs. OPSD
Memory boundary
100%
+17.7
Multi-hop inference
67%
+12.7
Generalization & application
66%
+9.3
Dynamic update
60%
+5.3
Basic fact recall
42%
+4.8
Memory conflict
36%
−9.9
Appendix
Table 8: VtR routing behaviour by category (native protocol, n=1438 ). “regen.” is the share of the category the gate sent to regeneration; Δ is VtR’s gain over OPSD (accuracies in Table 1 ). VtR improves five of six categories and regresses only on memory conflict, diagnosed in § 4.4 .
age
n
OPSD
MemOS
VtR
Δ vs OPSD
0
1438
59.9
47.8
66.1
+6.2
20
1052
39.0
45.6
56.2
+17.2
40
803
36.0
42.2
51.6
+15.6
60
593
33.6
42.3
50.9
+17.4
80
364
32.4
40.1
48.4
+15.9
100
178
32.6
36.5
43.3
+10.7
Appendix
Table 9: VtR accuracy by question age (HaluMem-Long, all ages), Correct%. VtR improves over both single systems at every age.
gate outcome
count
OPSD → final Correct%
SUPPORTED (kept)
994 (22%)
61.8 → 65.6
UNSUPPORTED + CONFLICT (regen)
3434 (77%)
38.5 → 54.1
regenerated, FIXED
814
regenerated, BROKE
277
net +537 , damage 21%
Appendix
Table 10: VtR gate accounting (pooled n=4428 ). The merged store folds superseded values, which collapses false CONFLICT verdicts from 36% (raw store) to 1% .
routed variant
route
Correct%
%LoRA
full-LoRA r64 + extraction
LLM route
46.2
18.8
full-LoRA r64 + extraction
GT route
46.0
48.3
stable-LoRA + extraction
GT route
43.5
48.3
stable-LoRA + MemOS
GT route
41.2
48.3
oracle (any arm correct, age 0 )
—
69.3
—
Appendix
Table 11: Routed hybrids (HaluMem-Long, pooled n=4428 ), Correct%. %LoRA is the share of questions answered by the weight arm. No routed hybrid beats plain MemOS ( 46.1% ); the oracle ceiling is far above all of them.
method
current-session fact
no current-session fact
Δ
Extracted (dump)
0.419
0.032
+0.387
OPSD (weight)
0.452
0.245
+0.207
MemOS (retrieval)
0.331
0.300
+0.031
Appendix
Table 12: Accuracy conditioned on whether any retrieved fact is from the current session (within-age, non-boundary, n=3378 ), Correct fraction.
signal / target
all
non-boundary
boundary
self-consistency → correct
0.606
0.631
0.391 (inverts)
per-token conf. → correct
0.628
0.855
0.931
per-token conf., age- 0 only
0.672
0.738
0.870
pabs (fixed str.) → answerable, extraction
0.944
—
—
pabs under OPSD adapter → answerable
0.918
—
—
pabs→ OPSD wrong (correctness)
0.516
—
—
Appendix
Table 13: Confidence-signal AUCs. The fixed-candidate abstain probe detects answerability well and transfers to weight memory, but no signal predicts correctness. Extraction-baseline numbers from the confidence-calibration note; OPSD transfer from the fixed-candidate note
Under-specified gold (not recoverable from cited evidence)
33
34%
Judge strictness (gold stated verbatim, still marked wrong)
11
11%
Universal abstention (date-scoping)
2
2%
Appendix
Table 14: The 97 age- 0 Dynamic Update + Multi-hop questions failed by all five methods, bucketed by cause. Nearly half are gold or rubric artifacts rather than system defects.
Method
Single-hop
Temporal
Multi-hop
Open-dom.
Adversarial
Overall
Raw dialogue (full-context)
74.7
7.2
30.1
31.2
2.5
39.1
Extracted (text)
69.3
61.7
33.3
32.3
3.4
46.4
MemOS (retrieval)
46.1
40.2
18.8
31.2
4.7
31.3
SFT (weight)
23.7
10.3
13.1
14.6
11.7
16.9
OPSD (weight)
70.2
61.3
33.5
25.0
10.6
46.9
VtR (weight + text)
69.2
57.4
28.6
21.1
73.1
60.4
Appendix
Table 15: LoCoMo end-of-conversation accuracy (Correct%) by question category (Qwen3.5-9B, Claude-Sonnet judge, 1,586 questions). Best per column in bold.
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · Samsung Research, Beijing, China