Personal intelligence systems are taking on increasingly complex everyday tasks, yet repeated questions, unnecessary confirmations, and unwanted outreach can turn capable assistance into interaction burden. Reducing this burden requires agents to remember how users want to be assisted. However, existing memory benchmarks focus on factual recall and personalized answers and rarely test how agents learn and act on interaction preferences. We introduce MemPABench, an interactive environment for on-policy evaluation of memory in personal agents. MemPABench poses long-horizon tasks requiring agents to decide when to clarify, confirm, or act autonomously. Four grounded simulated users hold 14 preferences across work and personal contexts. Their feedback spans 99 sessions over four months, with five event-driven changes per user testing memory updates. We evaluate preference selection, delivered behavior, and sensitivity to context and preference change. Evaluating four memory systems across four LLM backbones, we find that current memory brings limited gains in preference selection. Retrieval-based systems (i.e. Mem0, MemOS, simple RAG) do not outperform No Memory, while File-based memory improves accuracy but applies preferences beyond relevant contexts. Our analysis shows that current memory struggles to retrieve interaction preferences and adapt to preference changes. Even with the correct preference available, overriding the agent's default interaction habits remains challenging. The environment and dataset are available at https://mempabench.com.
Figures & tables
Work
Personalization target
Executable task env.
Pref.-conditioned reactions
On-policy interaction
Contextual isolation
Controlled change
PrefIx
Interaction
✓
✓
—
—
—
AMemGym
Content
—
◐
✓
—
◐
VitaBench 2.0
Content
✓
—
◐
◐
✓
BenchPreS
Interaction (communication)
—
—
—
✓
—
MemPABench
Interaction
✓
✓
✓
✓
✓
Table 1: MemPABench versus representative personalization benchmarks for agent memory.
Figure 1: Overview of the benchmark: persona-grounded world design, a multi-session interaction harness connecting the simulated user and personal agent, and evaluation of interaction preferences.
Figure 2: Persona and interaction-preference extraction. Transcript evidence yields a reviewed, anonymized setting for each attribute–context cell. We then write each user’s world and sessions based on these settings.
Figure 3: How the IX Wrapper carries out the PA’s choice. The PA picks an interaction setting, and the wrapper applies it through one of four mechanisms without ever seeing the target setting.
Table 2: Preference Selection Accuracy (PSA, %) across users, backbones, and memory conditions. Probes that end in a model failure score 0 (Appendix F.2 ).
Figure 4: What the PA sees before its first selection, in 112 sampled final probes per memory condition.
Figure 5: Stored memory and what the PA sees before one outreach decision (User B, work, target Low , GPT-5.6-Sol). All four stores hold the user’s objection, but only File memory shows it to the PA.
Figure 8
Table 3: CS and PET across backbones and memory conditions.
Figure 8: PSA by interaction-preference dimension and memory condition.
Figure 9: A correct selection whose reply still follows the backbone’s default, from an Oracle probe of User C in the work context with HY4-Preview.
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
What the PA visibly does
Expression Style
Tone & Formality. Which communication register should the PA use while keeping the same content and task result? ( Joos, 1961 ; Nass & Brave, 2005 ; Liebrecht et al., 2021 ; Cox & Ooi, 2022 ; Bhattacharjee et al., 2024 )
Casual
Uses two or more colloquial markers, such as contractions, interjections, or idioms, and no opening formula or sign-off.
Consultative
Uses at most one colloquial marker and no opening formula or sign-off, and puts an offer or a choice to the user.
Formal
Uses no contractions, at least one formal connective such as “therefore”, and a conventional opening or closing line.
Verbosity. How much relevant information should the PA include about the current topic? ( Cacioppo & Petty, 1982 ; Eppler & Mengis, 2004 ; Millecamp et al., 2019 ; Haas et al., 2022 ; Yu et al., 2025 )
Appendix
Table 4: The 14 interaction attributes by dimension, with each attribute’s decision question, source studies, settings, and what the PA visibly does under each setting. Each attribute takes one setting per context.
Confirmed
Reviewed
Specified
User
at high conf.
Kept
Set by reviewer
manually
Total
User A
27
0
1
0
28
User B
16
8
4
0
28
User C
5
10
12
1
28
User D
5
16
6
1
28
Total
53
34
23
2
112
Appendix
Table 5: Source labels of each user’s 28 final preference cells. Settings confirmed at high confidence are kept, and the other cells are reviewed and then kept, reset by the reviewer, or specified manually.
User
Probes
Rounds
Reader check
User A
3
3
3/3
User B
3
5
3/3
User C
4
5
4/4
User D
4
5
4/4
Total
14
–
14/14
Appendix
Table 6: Preference preservation under anonymization. Each probe is a PA message that violates four of the user’s settings. Reader check counts the probes in which both personas respond as the same kind of person.
Figure 10: Shortcut check. A backbone predicts a user’s 28 preference cells from the source identity, the anonymized profile, or no information. Knowing the character helps, and anonymization removes this help.
Table 7: World tools by domain ( n tools each). The PA receives all 83 tools in every session. Write tools are in bold, and † marks those that act toward other people or services.
Condition
Meaning
How it is checked
1. Feasible
The PA can carry out the setting with the records and tools in Wt .
Rules: the needed records are bound, exist, and are visible in time; the tool steps run on a scratch copy of the world. Review: the records hold what each setting needs.
2. Reasonable
An assistant that does not know the user’s preference could choose the setting, and the choice is neither careless nor harmful.
Review: under each setting, the reply is a complete and legitimate answer to the request.
3. Not given away
The user’s messages do not name the setting or ask for it.
Review: no user message before the decision point asks for a setting or cues one. Rules: e.g., a Solution Breadth probe request states no number of options.
4. Observable
The settings lead to interactions that differ naturally in the PA’s reply or executed tool calls, and each setting’s required behavior is done in full.
Rules: the task has the structure the attribute needs, e.g., two dependent tool calls for Process Visibility. Review: the replies under different settings differ visibly and naturally.
5. Answerable
In the next beat, the user has a way to respond to each setting.
Rules: the next beat carries a user reaction, and a probe states the expected reply under every setting. Review: each setting has its own reaction path.
Appendix
Table 8: The five conditions of setting viability. A setting is viable in a session when the PA follows it and all five conditions hold. Rules are rule-based checks, and Review is the human-verified LLM review.
Component
Profile and settings
Session script
Simulator memory
PA reply and executed actions
PA selection and withheld content
Memory evidence
User simulator, accumulation
✓
✓
✓
✓
✗
✗
User simulator, probe
✓
✓
✗
✓
✗
✗
PA
✗ a
✗
✗
✓
✓
✓
Memory system
✗
✗
✗
✓
◐ b
–
BRS judge
◐ c
◐ d
✗
✓
✗
✗
PRS rating
◐ e
✓
✗
✓
✗
✗
Appendix
Table 9: Information available to each component during a session. ✓ : available; ◐ : partly available, as noted below the table; ✗ : not available.
Source
Phase
Judged
Remark on reply
Rule-like reaction
Other
Leaks
Rule hits
Accumulation
698
621
5
72
0
Rule hits
Probe
375
261
3
111
0
Random sample
Accumulation
300
39
0
261
0
Random sample
Probe
150
8
1
141
0
Found in context
Accumulation
5
0
5
0
0
Found in context
Probe
2
0
2
0
0
Appendix
Table 10: Leakage audit of user-simulator sentences by source and phase. The audit finds no leakage. No probe sentence refers to an earlier session, and no sentence states a preference or names a setting without a trigger.
Condition
Write
Retrieval
File memory
LLM consolidation into MEMORY.md and HISTORY.md
Consolidated Markdown supplied to the assistant
Simple RAG
Raw messages stored through Mem0 with infer=False
Vector retrieval over raw messages
Mem0
Memory extraction with infer=True
Native Mem0 search over extracted memories
MemOS
Native MemCube add operation
Native MemCube search operation
Appendix
Table 11: Memory conditions under the common MemPABench adapter. Each condition receives the same record of each turn: the user message, the PA’s tool calls with their results, its call to the interaction tool, and the delivered reply.
Memory
GPT-5.6-Sol
Qwen3.8-27B
HY4-Preview
DeepSeek-V4-Flash
Total
No Memory
1
0
4
3
8
File memory
0
0
6
3
9
Mem0
0
0
8
5
13
MemOS
0
4
6
6
16
Simple RAG
0
2
10
4
16
Oracle
3
0
0
2
5
Appendix
Table 12: Floored final probes by memory condition and backbone. Each cell has 336 final probes (4 users × 28 probes × 3 repeats). A floored probe scores PSA =0 and BRS =1 and stays in the denominator.
Memory
PSA
CS
PET
Carry-over
Δ PSA
Δ CS
Δ PET
Δ Carry-over
No Memory
49.6 [42.2, 57.0]
21.8 [10.2, 35.2]
11.2 [4.6, 19.6]
6.0 [1.9, 10.6]
–
–
–
–
File memory
59.2 [53.2, 65.3]
14.4 [8.3, 20.8]
25.4 [15.0, 36.7]
26.9 [17.1, 38.0]
+9.6 [+3.9, +15.6]
− 7.4 [ − 17.1, +1.4]
+14.2 [+4.2, +25.8]
+20.8 [+10.2, +31.9]
Mem0
50.0 [42.6, 57.2]
22.2 [13.0, 32.9]
14.2 [6.7, 22.5]
4.6 [1.9, 7.9]
+0.4 [ − 1.5, +2.2]
+0.5 [ − 5.6, +6.5]
+2.9 [ − 0.4, +6.7]
− 1.4 [ − 5.1, +1.4]
MemOS
50.0 [42.6, 57.3]
19.9 [10.6, 30.6]
10.8 [5.0, 17.5]
7.4 [2.8, 12.5]
+0.4 [ − 1.5, +2.1]
− 1.9 [ − 9.7, +4.6]
− 0.4 [ − 5.4, +4.2]
+1.4 [ − 1.9, +4.6]
Simple RAG
49.9 [42.6, 57.0]
20.8 [12.0, 31.5]
15.0 [8.3, 22.9]
6.9 [2.8, 12.0]
+0.2 [ − 2.2, +2.7]
− 0.9 [ − 7.9, +5.6]
+3.8 [ − 0.8, +9.2]
+0.9 [ − 3.7, +5.6]
Appendix
Table 13: PSA, CS, PET, and carry-over (%) with 95% cluster-bootstrap intervals, pooled over users, backbones, and repeats. The right block gives the difference from No Memory. Only File memory clearly improves PSA.
Memory
User A
User B
User C
User D
GPT-5.6- Sol
Qwen3.8- 27B
HY4- Preview
DeepSeek- V4-Flash
GPT-5.6- Sol
Qwen3.8- 27B
HY4- Preview
DeepSeek- V4-Flash
GPT-5.6- Sol
Qwen3.8- 27B
HY4- Preview
DeepSeek- V4-Flash
GPT-5.6- Sol
Qwen3.8- 27B
HY4- Preview
DeepSeek- V4-Flash
Expression Style
File memory
87.5 ± 0.0
83.3 ± 7.2
87.5 ± 12.5
79.2 ± 7.2
50.0 ± 0.0
75.0 ± 12.5
95.8 ± 7.2
20.8 ± 14.4
62.5 ± 12.5
79.2 ± 7.2
70.8 ± 14.4
79.2 ± 19.1
58.3 ± 7.2
70.8 ± 7.2
50.0 ± 0.0
50.0 ± 0.0
Simple RAG
50.0 ± 12.5
75.0 ± 12.5
62.5 ± 12.5
58.3 ± 14.4
54.2 ± 7.2
37.5 ± 12.5
41.7 ± 7.2
29.2 ± 19.1
50.0 ± 0.0
87.5 ± 0.0
70.8 ± 7.2
79.2 ± 7.2
37.5 ± 0.0
79.2 ± 7.2
37.5 ± 0.0
75.0 ± 12.5
Mem0
62.5 ± 0.0
79.2 ± 7.2
50.0 ± 21.7
54.2 ± 7.2
50.0 ± 0.0
41.7 ± 19.1
50.0 ± 12.5
41.7 ± 14.4
50.0 ± 0.0
79.2 ± 7.2
62.5 ± 12.5
70.8 ± 7.2
37.5 ± 0.0
83.3 ± 7.2
41.7 ± 7.2
58.3 ± 7.2
MemOS
62.5 ± 0.0
75.0 ± 0.0
70.8 ± 7.2
58.3 ± 7.2
54.2 ± 7.2
33.3 ± 14.4
41.7 ± 7.2
29.2 ± 7.2
50.0 ± 0.0
75.0 ± 12.5
54.2 ± 7.2
83.3 ± 7.2
37.5 ± 0.0
91.7 ± 14.4
41.7 ± 7.2
62.5 ± 12.5
Appendix
Table 14: Preference Selection Accuracy (PSA, %) by IX dimension, user, backbone, and memory condition.
Memory
User A
User B
User C
User D
GPT-5.6- Sol
Qwen3.8- 27B
HY4- Preview
DeepSeek- V4-Flash
GPT-5.6- Sol
Qwen3.8- 27B
HY4- Preview
DeepSeek- V4-Flash
GPT-5.6- Sol
Qwen3.8- 27B
HY4- Preview
DeepSeek- V4-Flash
GPT-5.6- Sol
Qwen3.8- 27B
HY4- Preview
DeepSeek- V4-Flash
File memory
3.74
4.01
3.37
4.04
3.80
3.98
3.86
3.79
2.82
2.79
3.12
2.70
3.51
3.52
3.67
3.33
Simple RAG
2.87
2.88
2.95
3.18
3.56
3.40
3.20
3.25
2.52
2.85
3.04
2.88
3.17
3.52
3.57
3.48
Mem0
2.94
3.18
2.85
3.24
3.55
3.26
3.26
3.64
2.63
2.92
2.90
2.63
3.36
3.69
3.50
3.43
MemOS
3.05
3.14
2.94
3.14
3.54
3.12
3.23
3.52
2.67
2.70
2.89
2.73
3.05
3.73
3.52
3.50
No Memory
2.87
2.96
3.04
3.21
3.43
3.62
3.13
3.49
2.64
2.60
2.85
2.93
3.15
3.83
3.02
3.52
Appendix
Table 15: Behavioral Realization Score (BRS, 1–5) by user, backbone, and memory condition.
Memory
User A
User B
User C
User D
GPT-5.6- Sol
Qwen3.8- 27B
HY4- Preview
DeepSeek- V4-Flash
GPT-5.6- Sol
Qwen3.8- 27B
HY4- Preview
DeepSeek- V4-Flash
GPT-5.6- Sol
Qwen3.8- 27B
HY4- Preview
DeepSeek- V4-Flash
GPT-5.6- Sol
Qwen3.8- 27B
HY4- Preview
DeepSeek- V4-Flash
File memory
3.81
4.04
3.57
4.08
3.83
3.82
3.83
3.85
3.12
2.81
3.26
2.86
3.60
3.75
3.81
3.65
Simple RAG
3.18
3.01
3.13
3.33
3.69
3.26
3.27
3.30
2.79
2.99
2.93
2.75
3.35
3.65
3.71
3.55
Mem0
3.13
3.26
3.01
3.35
3.54
3.36
3.36
3.61
2.96
3.00
3.01
2.73
3.44
3.73
3.87
3.15
MemOS
3.25
3.25
2.95
3.24
3.56
3.18
3.43
3.56
2.69
2.75
3.07
2.86
3.33
3.62
3.88
3.48
No Memory
2.96
3.04
3.23
3.29
3.52
3.65
3.29
3.42
2.99
2.94
2.90
2.94
3.30
3.94
3.55
3.52
Appendix
Table 16: Perceived Realization Score (PRS, 1–5) by user, backbone, and memory condition.
Memory
User A
User B
User C
User D
GPT-5.6- Sol
Qwen3.8- 27B
HY4- Preview
DeepSeek- V4-Flash
GPT-5.6- Sol
Qwen3.8- 27B
HY4- Preview
DeepSeek- V4-Flash
GPT-5.6- Sol
Qwen3.8- 27B
HY4- Preview
DeepSeek- V4-Flash
GPT-5.6- Sol
Qwen3.8- 27B
HY4- Preview
DeepSeek- V4-Flash
Expression Style
File memory
4.88 ± 0.22
4.54 ± 0.07
4.75 ± 0.22
4.58 ± 0.07
3.58 ± 0.52
4.12 ± 0.33
4.71 ± 0.26
3.58 ± 0.36
3.38 ± 0.43
3.96 ± 0.31
3.79 ± 0.14
3.62 ± 0.54
3.33 ± 0.38
3.58 ± 0.31
3.25 ± 0.00
2.96 ± 0.19
Simple RAG
3.46 ± 0.63
4.00 ± 0.50
3.88 ± 0.50
4.25 ± 0.33
3.42 ± 0.14
3.08 ± 0.19
3.21 ± 0.36
3.25 ± 0.45
2.92 ± 0.07
4.08 ± 0.40
4.00 ± 0.12
4.12 ± 0.12
2.92 ± 0.29
3.79 ± 0.19
3.25 ± 0.33
3.50 ± 0.22
Mem0
3.83 ± 0.26
4.46 ± 0.31
3.42 ± 0.47
4.04 ± 0.14
3.62 ± 0.25
2.75 ± 0.54
3.21 ± 0.36
3.75 ± 0.25
2.96 ± 0.19
4.04 ± 0.31
3.96 ± 0.14
3.79 ± 0.38
2.83 ± 0.07
4.17 ± 0.56
3.21 ± 0.31
3.71 ± 0.26
MemOS
4.33 ± 0.31
4.12 ± 0.12
4.21 ± 0.14
3.92 ± 0.40
3.42 ± 0.36
2.75 ± 0.43
2.96 ± 0.47
3.50 ± 0.12
3.38 ± 0.12
3.79 ± 0.44
3.42 ± 0.19
3.96 ± 0.14
2.75 ± 0.00
4.04 ± 0.44
2.96 ± 0.26
3.54 ± 0.52
Appendix
Table 17: Behavioral Realization Score (BRS, 1–5) by IX dimension, user, backbone, and memory condition.
Memory
User A
User B
User C
User D
GPT-5.6- Sol
Qwen3.8- 27B
HY4- Preview
DeepSeek- V4-Flash
GPT-5.6- Sol
Qwen3.8- 27B
HY4- Preview
DeepSeek- V4-Flash
GPT-5.6- Sol
Qwen3.8- 27B
HY4- Preview
DeepSeek- V4-Flash
GPT-5.6- Sol
Qwen3.8- 27B
HY4- Preview
DeepSeek- V4-Flash
Expression Style
File memory
4.88 ± 0.22
4.33 ± 0.19
4.75 ± 0.22
4.58 ± 0.07
4.08 ± 0.52
3.75 ± 0.25
4.71 ± 0.29
3.62 ± 0.45
3.54 ± 0.29
3.75 ± 0.33
4.00 ± 0.33
3.50 ± 0.22
4.04 ± 0.19
4.25 ± 0.00
4.17 ± 0.07
3.50 ± 0.22
Simple RAG
3.54 ± 0.52
4.08 ± 0.51
3.96 ± 0.26
3.62 ± 0.45
3.92 ± 0.14
3.04 ± 0.29
3.25 ± 0.25
3.04 ± 0.26
3.29 ± 0.07
3.92 ± 0.19
3.88 ± 0.33
3.46 ± 0.40
3.29 ± 0.14
4.12 ± 0.25
3.83 ± 0.07
3.92 ± 0.26
Mem0
3.75 ± 0.12
4.33 ± 0.26
3.42 ± 0.29
3.46 ± 0.19
3.62 ± 0.12
2.88 ± 0.33
3.42 ± 0.29
3.83 ± 0.14
3.50 ± 0.22
3.83 ± 0.38
3.92 ± 0.19
3.46 ± 0.19
3.38 ± 0.12
4.21 ± 0.31
3.88 ± 0.12
3.75 ± 0.38
MemOS
4.04 ± 0.14
4.04 ± 0.26
4.17 ± 0.14
3.62 ± 0.25
3.62 ± 0.33
2.79 ± 0.31
3.33 ± 0.26
3.67 ± 0.07
3.38 ± 0.12
3.71 ± 0.31
3.88 ± 0.22
3.71 ± 0.19
3.46 ± 0.19
4.21 ± 0.14
3.88 ± 0.12
3.88 ± 0.38
Appendix
Table 18: Perceived Realization Score (PRS, 1–5) by IX dimension, user, backbone, and memory condition.
Model (default reasoning effort)
Intelligence Index
GDPval-AA (Elo)
PSA (%)
GPT-5.6-Sol (medium)
39
1403
49.8
Qwen3.8-27B (xhigh)
34
1409
56.0
HY4-Preview (high)
–
–
51.3
DeepSeek-V4-Flash (high)
26
–
52.0
Appendix
Table 19: PSA and general capability of the four PA backbones. PSA averages Users A–D and the four memory conditions, and the other scores are from Artificial Analysis (2026) . PSA does not follow general capability.
Attribute
GPT-5.6-Sol
Qwen3.8-27B
HY4-Preview
DeepSeek-V4-Flash
Expression Style
Tone & Formality
Consultative 70% (23/33)
Consultative 42% (14/33)
Consultative 67% (22/33)
Consultative 91% (30/33)
Verbosity
Detailed 100% (24/24)
Detailed 71% (17/24)
Detailed 96% (23/24)
Detailed 62% (15/24)
Emotional Engagement
Relationship-focused 100% (27/27)
Relationship-focused 100% (27/27)
Relationship-focused 100% (27/27)
Relationship-focused 100% (27/27)
Guidance Level
Guided 89% (24/27)
Guided 59% (16/27)
Guided 67% (18/27)
Assumed 78% (21/27)
Disclosure
Appendix
Table 20: Default setting of each backbone under No Memory. Each cell gives the setting the PA selects most often for the attribute, with its share and count over final and pre-event probes.
User
Probes
Mean BRS
Mean PRS
PRS < BRS
PRS = BRS
PRS > BRS
User A
335
4.85
4.69
11.0%
87.2%
1.8%
User B
330
4.65
4.54
14.8%
77.9%
7.3%
User C
331
4.48
4.19
21.5%
71.6%
6.9%
User D
334
4.47
4.34
17.1%
72.5%
10.5%
All
1,330
4.61
4.44
16.1%
77.3%
6.6%
Appendix
Table 21: BRS and PRS for each user on the Oracle final probes with a correct selection. PRS falls below BRS on 11.0% of these probes for User A but on 21.5% for User C. PRS therefore reflects how each user receives a reply, while BRS applies one rubric to all users and is used to compare settings.
Prompt
Called
Main inputs
Output
Page
Construction
Preference Extractor, Passes 1 and 2
Pass 1 once per scene of the source character, Pass 2 once per attribute–context cell
Scene transcript; Pass-1 evidence, attribute definition, settings, rubric
Evidence items per attribute; setting, confidence, cited evidence
Empowering large language models with long-term memory is crucial for building agents that adapt to users' evolving needs. Existing evaluations of this capability typically interleave preference-related dialogues with irrelevant conversations, reducing the task to needle-in-a-haystack retrieval while ignoring relationships between events driving user preference evolution. Such settings overlook a fundamental characteristic of real-world personalization: preferences emerge gradually and accumulate across interactions within noisy contexts. To bridge this gap, we introduce PERMA, a benchmark designed to evaluate persona consistency over time beyond static preference recall. Additionally, we incorporate (1) text variability and (2) linguistic alignment to simulate erratic user inputs and individual idiolects in real-world data. PERMA consists of temporally ordered interaction events spanning multiple sessions and domains, with preference-related queries inserted over time. We design both multiple-choice and interactive tasks to probe the model's understanding of persona along the interaction timeline. Experiments demonstrate that by linking related interactions, advanced memory systems extract precise preferences and reduce token consumption, outperforming traditional semantic retrieval of raw dialogues. Nevertheless, they still struggle to maintain a coherent persona across temporal depth and cross-domain interference, highlighting the need for more robust personalized memory management in agents. Our code and data are open-sourced at https://github.com/PolarisLiu1/PERMA.
Shuochen Liu, Junyi Zhu, Long Shu +11
University of Science and Technology of China · ESAT-PSI, KU Leuven · City University of Hong Kong +2
Memory systems for persistent personal agents must move beyond isolated memories, maintaining both personal memory about whom they serve and procedural memory about how to act. Crucially, in harness-based personal assistance, procedures or skills are inherently user-conditioned: personal context determines which procedures apply, how they are instantiated, and when they should change. Existing benchmarks typically emphasize either personal or procedural memory, leaving their interaction underexplored. We introduce PraxisMemBench, a harness-based benchmark for persistent personal assistance across daily-life and work trajectories, where personal and procedural memory evolve together and later tasks depend on both. Experiments show that maintaining two memory types separately is insufficient, with completion dropping from single-type to interdependent dependencies (84.6% to 77.9%). More broadly, across existing methods, relation-related failures increase from early to late sessions as interaction history grows (59.8% to 72.9%), revealing a continual-maintenance challenge: writing correct memories is not enough if their evolving content and organization cannot be preserved for later retrieval. To address this, we propose PraxisMem, which uses privileged joint-state supervision to internalize the ability to maintain memory content and organization into the memory manager, with location-aware GRPO and dynamic residual-routed OPSD. PraxisMem reaches 40.78% overall Session Success Rate, compared with 20.60% for Claude Code's native memory. Our code is publicly available at https://github.com/QMMMS/PraxisMemBench.
As large language model (LLM) agents evolve into personalized companions, memory has emerged as a core capability. However, LLMs face a knowledge utilization problem: they may fail to act on relevant user preferences even when they are fully present in context. When an agent fails to tailor its response in a context where previously shared user preferences should matter, it is unclear whether the model failed to remember that information or remembered it but failed to use it. To isolate this breakdown, we introduce a decoupled evaluation paradigm that administers paired Know and Act tests to the same user preference. We conduct large-scale experiments across 16 systems and five memory architectures, evaluating 1,000 preferences embedded at three levels of expression strength. Our results show a large gap between Know and Act outcomes: agents often pass the recall test for a user preference but fail to reflect that same preference in the paired behavioral scenario. While memory architectures reduce this gap, utilization remains especially weak for health and therapy-related preferences, where failures to act carry the greatest real-world stakes.