cs.AIOct 21, 2026
SaveWhy Bother Me Again? Benchmarking Personal Agent Memory for Interaction Preferences
Organizations: New York University Abu Dhabi, Abu Dhabi, UAE
Abstract
Personal intelligence systems are taking on increasingly complex everyday tasks, yet repeated questions, unnecessary confirmations, and unwanted outreach can turn capable assistance into interaction burden. Reducing this burden requires agents to remember how users want to be assisted. However, existing memory benchmarks focus on factual recall and personalized answers and rarely test how agents learn and act on interaction preferences. We introduce MemPABench, an interactive environment for on-policy evaluation of memory in personal agents. MemPABench poses long-horizon tasks requiring agents to decide when to clarify, confirm, or act autonomously. Four grounded simulated users hold 14 preferences across work and personal contexts. Their feedback spans 99 sessions over four months, with five event-driven changes per user testing memory updates. We evaluate preference selection, delivered behavior, and sensitivity to context and preference change. Evaluating four memory systems across four LLM backbones, we find that current memory brings limited gains in preference selection. Retrieval-based systems (i.e. Mem0, MemOS, simple RAG) do not outperform No Memory, while File-based memory improves accuracy but applies preferences beyond relevant contexts. Our analysis shows that current memory struggles to retrieve interaction preferences and adapt to preference changes. Even with the correct preference available, overriding the agent's default interaction habits remains challenging. The environment and dataset are available at https://mempabench.com.
Figures & tables
| Work | Personalization target | Executable task env. | Pref.-conditioned reactions | On-policy interaction | Contextual isolation | Controlled change |
| PrefIx | Interaction | ✓ | ✓ | — | — | — |
| AMemGym | Content | — | ◐ | ✓ | — | ◐ |
| VitaBench 2.0 | Content | ✓ | — | ◐ | ◐ | ✓ |
| BenchPreS | Interaction (communication) | — | — | — | ✓ | — |
| MemPABench | Interaction | ✓ | ✓ | ✓ | ✓ | ✓ |
Table 1: MemPABench versus representative personalization benchmarks for agent memory.
Figure 1: Overview of the benchmark: persona-grounded world design, a multi-session interaction harness connecting the simulated user and personal agent, and evaluation of interaction preferences.
Figure 2: Persona and interaction-preference extraction. Transcript evidence yields a reviewed, anonymized setting for each attribute–context cell. We then write each user’s world and sessions based on these settings.
Figure 3: How the IX Wrapper carries out the PA’s choice. The PA picks an interaction setting, and the wrapper applies it through one of four mechanisms without ever seeing the target setting.
Table 2: Preference Selection Accuracy (PSA, %) across users, backbones, and memory conditions. Probes that end in a model failure score 0 (Appendix F.2 ).
Figure 4: What the PA sees before its first selection, in 112 sampled final probes per memory condition.
Figure 5: Stored memory and what the PA sees before one outreach decision (User B, work, target Low , GPT-5.6-Sol). All four stores hold the user’s objection, but only File memory shows it to the PA.
Table 3: CS and PET across backbones and memory conditions.
Figure 8: PSA by interaction-preference dimension and memory condition.
Figure 9: A correct selection whose reply still follows the backbone’s default, from an Oracle probe of User C in the work context with HY4-Preview.
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
| Setting | What the PA visibly does |
|---|---|
| Expression Style | |
| Tone & Formality. Which communication register should the PA use while keeping the same content and task result? ( Joos, 1961 ; Nass & Brave, 2005 ; Liebrecht et al., 2021 ; Cox & Ooi, 2022 ; Bhattacharjee et al., 2024 ) | |
| Casual | Uses two or more colloquial markers, such as contractions, interjections, or idioms, and no opening formula or sign-off. |
| Consultative | Uses at most one colloquial marker and no opening formula or sign-off, and puts an offer or a choice to the user. |
| Formal | Uses no contractions, at least one formal connective such as “therefore”, and a conventional opening or closing line. |
| Verbosity. How much relevant information should the PA include about the current topic? ( Cacioppo & Petty, 1982 ; Eppler & Mengis, 2004 ; Millecamp et al., 2019 ; Haas et al., 2022 ; Yu et al., 2025 ) | |
Table 4: The 14 interaction attributes by dimension, with each attribute’s decision question, source studies, settings, and what the PA visibly does under each setting. Each attribute takes one setting per context.
| Confirmed | Reviewed | Specified | |||
| User | at high conf. | Kept | Set by reviewer | manually | Total |
| User A | 27 | 0 | 1 | 0 | 28 |
| User B | 16 | 8 | 4 | 0 | 28 |
| User C | 5 | 10 | 12 | 1 | 28 |
| User D | 5 | 16 | 6 | 1 | 28 |
| Total | 53 | 34 | 23 | 2 | 112 |
Table 5: Source labels of each user’s 28 final preference cells. Settings confirmed at high confidence are kept, and the other cells are reviewed and then kept, reset by the reviewer, or specified manually.
| User | Probes | Rounds | Reader check |
| User A | 3 | 3 | 3/3 |
| User B | 3 | 5 | 3/3 |
| User C | 4 | 5 | 4/4 |
| User D | 4 | 5 | 4/4 |
| Total | 14 | – | 14/14 |
Table 6: Preference preservation under anonymization. Each probe is a PA message that violates four of the user’s settings. Reader check counts the probes in which both personas respond as the same kind of person.
Figure 10: Shortcut check. A backbone predicts a user’s 28 preference cells from the source identity, the anonymized profile, or no information. Knowing the character helps, and anonymization removes this help.
| Domain | Tools | |
| 6 | email_message_search , email_message_read , email_draft_read , email_sent_read , email_draft_create , email_message_send † | |
| Messages | 3 | message_search , message_read , message_send † |
| Calendar | 4 | calendar_event_search , calendar_event_read , calendar_event_create , calendar_event_update |
| Documents | 6 | copier_entitlement_read , document_read , corporate_reporting_submit † , document_create , document_update , document_delete |
| Contacts | 4 | contact_search , contact_read , contact_create , contact_update |
| Phone | 2 | phone_call_read , phone_call_dial † |
Table 7: World tools by domain ( tools each). The PA receives all 83 tools in every session. Write tools are in bold, and † marks those that act toward other people or services.
| Condition | Meaning | How it is checked |
| 1. Feasible | The PA can carry out the setting with the records and tools in . | Rules: the needed records are bound, exist, and are visible in time; the tool steps run on a scratch copy of the world. Review: the records hold what each setting needs. |
| 2. Reasonable | An assistant that does not know the user’s preference could choose the setting, and the choice is neither careless nor harmful. | Review: under each setting, the reply is a complete and legitimate answer to the request. |
| 3. Not given away | The user’s messages do not name the setting or ask for it. | Review: no user message before the decision point asks for a setting or cues one. Rules: e.g., a Solution Breadth probe request states no number of options. |
| 4. Observable | The settings lead to interactions that differ naturally in the PA’s reply or executed tool calls, and each setting’s required behavior is done in full. | Rules: the task has the structure the attribute needs, e.g., two dependent tool calls for Process Visibility. Review: the replies under different settings differ visibly and naturally. |
| 5. Answerable | In the next beat, the user has a way to respond to each setting. | Rules: the next beat carries a user reaction, and a probe states the expected reply under every setting. Review: each setting has its own reaction path. |
Table 8: The five conditions of setting viability. A setting is viable in a session when the PA follows it and all five conditions hold. Rules are rule-based checks, and Review is the human-verified LLM review.
| Component | Profile and settings | Session script | Simulator memory | PA reply and executed actions | PA selection and withheld content | Memory evidence |
| User simulator, accumulation | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ |
| User simulator, probe | ✓ | ✓ | ✗ | ✓ | ✗ | ✗ |
| PA | ✗ a | ✗ | ✗ | ✓ | ✓ | ✓ |
| Memory system | ✗ | ✗ | ✗ | ✓ | ◐ b | – |
| BRS judge | ◐ c | ◐ d | ✗ | ✓ | ✗ | ✗ |
| PRS rating | ◐ e | ✓ | ✗ | ✓ | ✗ | ✗ |
Table 9: Information available to each component during a session. ✓ : available; ◐ : partly available, as noted below the table; ✗ : not available.
| Source | Phase | Judged | Remark on reply | Rule-like reaction | Other | Leaks |
| Rule hits | Accumulation | 698 | 621 | 5 | 72 | 0 |
| Rule hits | Probe | 375 | 261 | 3 | 111 | 0 |
| Random sample | Accumulation | 300 | 39 | 0 | 261 | 0 |
| Random sample | Probe | 150 | 8 | 1 | 141 | 0 |
| Found in context | Accumulation | 5 | 0 | 5 | 0 | 0 |
| Found in context | Probe | 2 | 0 | 2 | 0 | 0 |
Table 10: Leakage audit of user-simulator sentences by source and phase. The audit finds no leakage. No probe sentence refers to an earlier session, and no sentence states a preference or names a setting without a trigger.
| Condition | Write | Retrieval |
| File memory | LLM consolidation into MEMORY.md and HISTORY.md | Consolidated Markdown supplied to the assistant |
| Simple RAG | Raw messages stored through Mem0 with infer=False | Vector retrieval over raw messages |
| Mem0 | Memory extraction with infer=True | Native Mem0 search over extracted memories |
| MemOS | Native MemCube add operation | Native MemCube search operation |
Table 11: Memory conditions under the common MemPABench adapter. Each condition receives the same record of each turn: the user message, the PA’s tool calls with their results, its call to the interaction tool, and the delivered reply.
| Memory | GPT-5.6-Sol | Qwen3.8-27B | HY4-Preview | DeepSeek-V4-Flash | Total |
| No Memory | 1 | 0 | 4 | 3 | 8 |
| File memory | 0 | 0 | 6 | 3 | 9 |
| Mem0 | 0 | 0 | 8 | 5 | 13 |
| MemOS | 0 | 4 | 6 | 6 | 16 |
| Simple RAG | 0 | 2 | 10 | 4 | 16 |
| Oracle | 3 | 0 | 0 | 2 | 5 |
Table 12: Floored final probes by memory condition and backbone. Each cell has 336 final probes (4 users 28 probes 3 repeats). A floored probe scores PSA and BRS and stays in the denominator.
| Memory | PSA | CS | PET | Carry-over | PSA | CS | PET | Carry-over |
| No Memory | 49.6 [42.2, 57.0] | 21.8 [10.2, 35.2] | 11.2 [4.6, 19.6] | 6.0 [1.9, 10.6] | – | – | – | – |
| File memory | 59.2 [53.2, 65.3] | 14.4 [8.3, 20.8] | 25.4 [15.0, 36.7] | 26.9 [17.1, 38.0] | +9.6 [+3.9, +15.6] | 7.4 [ 17.1, +1.4] | +14.2 [+4.2, +25.8] | +20.8 [+10.2, +31.9] |
| Mem0 | 50.0 [42.6, 57.2] | 22.2 [13.0, 32.9] | 14.2 [6.7, 22.5] | 4.6 [1.9, 7.9] | +0.4 [ 1.5, +2.2] | +0.5 [ 5.6, +6.5] | +2.9 [ 0.4, +6.7] | 1.4 [ 5.1, +1.4] |
| MemOS | 50.0 [42.6, 57.3] | 19.9 [10.6, 30.6] | 10.8 [5.0, 17.5] | 7.4 [2.8, 12.5] | +0.4 [ 1.5, +2.1] | 1.9 [ 9.7, +4.6] | 0.4 [ 5.4, +4.2] | +1.4 [ 1.9, +4.6] |
| Simple RAG | 49.9 [42.6, 57.0] | 20.8 [12.0, 31.5] | 15.0 [8.3, 22.9] | 6.9 [2.8, 12.0] | +0.2 [ 2.2, +2.7] | 0.9 [ 7.9, +5.6] | +3.8 [ 0.8, +9.2] | +0.9 [ 3.7, +5.6] |
Table 13: PSA, CS, PET, and carry-over (%) with 95% cluster-bootstrap intervals, pooled over users, backbones, and repeats. The right block gives the difference from No Memory. Only File memory clearly improves PSA.
| Memory | User A | User B | User C | User D | ||||||||||||
| GPT-5.6- Sol | Qwen3.8- 27B | HY4- Preview | DeepSeek- V4-Flash | GPT-5.6- Sol | Qwen3.8- 27B | HY4- Preview | DeepSeek- V4-Flash | GPT-5.6- Sol | Qwen3.8- 27B | HY4- Preview | DeepSeek- V4-Flash | GPT-5.6- Sol | Qwen3.8- 27B | HY4- Preview | DeepSeek- V4-Flash | |
| Expression Style | ||||||||||||||||
| File memory | 87.5 ± 0.0 | 83.3 ± 7.2 | 87.5 ± 12.5 | 79.2 ± 7.2 | 50.0 ± 0.0 | 75.0 ± 12.5 | 95.8 ± 7.2 | 20.8 ± 14.4 | 62.5 ± 12.5 | 79.2 ± 7.2 | 70.8 ± 14.4 | 79.2 ± 19.1 | 58.3 ± 7.2 | 70.8 ± 7.2 | 50.0 ± 0.0 | 50.0 ± 0.0 |
| Simple RAG | 50.0 ± 12.5 | 75.0 ± 12.5 | 62.5 ± 12.5 | 58.3 ± 14.4 | 54.2 ± 7.2 | 37.5 ± 12.5 | 41.7 ± 7.2 | 29.2 ± 19.1 | 50.0 ± 0.0 | 87.5 ± 0.0 | 70.8 ± 7.2 | 79.2 ± 7.2 | 37.5 ± 0.0 | 79.2 ± 7.2 | 37.5 ± 0.0 | 75.0 ± 12.5 |
| Mem0 | 62.5 ± 0.0 | 79.2 ± 7.2 | 50.0 ± 21.7 | 54.2 ± 7.2 | 50.0 ± 0.0 | 41.7 ± 19.1 | 50.0 ± 12.5 | 41.7 ± 14.4 | 50.0 ± 0.0 | 79.2 ± 7.2 | 62.5 ± 12.5 | 70.8 ± 7.2 | 37.5 ± 0.0 | 83.3 ± 7.2 | 41.7 ± 7.2 | 58.3 ± 7.2 |
| MemOS | 62.5 ± 0.0 | 75.0 ± 0.0 | 70.8 ± 7.2 | 58.3 ± 7.2 | 54.2 ± 7.2 | 33.3 ± 14.4 | 41.7 ± 7.2 | 29.2 ± 7.2 | 50.0 ± 0.0 | 75.0 ± 12.5 | 54.2 ± 7.2 | 83.3 ± 7.2 | 37.5 ± 0.0 | 91.7 ± 14.4 | 41.7 ± 7.2 | 62.5 ± 12.5 |
Table 14: Preference Selection Accuracy (PSA, %) by IX dimension, user, backbone, and memory condition.
| Memory | User A | User B | User C | User D | ||||||||||||
| GPT-5.6- Sol | Qwen3.8- 27B | HY4- Preview | DeepSeek- V4-Flash | GPT-5.6- Sol | Qwen3.8- 27B | HY4- Preview | DeepSeek- V4-Flash | GPT-5.6- Sol | Qwen3.8- 27B | HY4- Preview | DeepSeek- V4-Flash | GPT-5.6- Sol | Qwen3.8- 27B | HY4- Preview | DeepSeek- V4-Flash | |
| File memory | 3.74 | 4.01 | 3.37 | 4.04 | 3.80 | 3.98 | 3.86 | 3.79 | 2.82 | 2.79 | 3.12 | 2.70 | 3.51 | 3.52 | 3.67 | 3.33 |
| Simple RAG | 2.87 | 2.88 | 2.95 | 3.18 | 3.56 | 3.40 | 3.20 | 3.25 | 2.52 | 2.85 | 3.04 | 2.88 | 3.17 | 3.52 | 3.57 | 3.48 |
| Mem0 | 2.94 | 3.18 | 2.85 | 3.24 | 3.55 | 3.26 | 3.26 | 3.64 | 2.63 | 2.92 | 2.90 | 2.63 | 3.36 | 3.69 | 3.50 | 3.43 |
| MemOS | 3.05 | 3.14 | 2.94 | 3.14 | 3.54 | 3.12 | 3.23 | 3.52 | 2.67 | 2.70 | 2.89 | 2.73 | 3.05 | 3.73 | 3.52 | 3.50 |
| No Memory | 2.87 | 2.96 | 3.04 | 3.21 | 3.43 | 3.62 | 3.13 | 3.49 | 2.64 | 2.60 | 2.85 | 2.93 | 3.15 | 3.83 | 3.02 | 3.52 |
Table 15: Behavioral Realization Score (BRS, 1–5) by user, backbone, and memory condition.
| Memory | User A | User B | User C | User D | ||||||||||||
| GPT-5.6- Sol | Qwen3.8- 27B | HY4- Preview | DeepSeek- V4-Flash | GPT-5.6- Sol | Qwen3.8- 27B | HY4- Preview | DeepSeek- V4-Flash | GPT-5.6- Sol | Qwen3.8- 27B | HY4- Preview | DeepSeek- V4-Flash | GPT-5.6- Sol | Qwen3.8- 27B | HY4- Preview | DeepSeek- V4-Flash | |
| File memory | 3.81 | 4.04 | 3.57 | 4.08 | 3.83 | 3.82 | 3.83 | 3.85 | 3.12 | 2.81 | 3.26 | 2.86 | 3.60 | 3.75 | 3.81 | 3.65 |
| Simple RAG | 3.18 | 3.01 | 3.13 | 3.33 | 3.69 | 3.26 | 3.27 | 3.30 | 2.79 | 2.99 | 2.93 | 2.75 | 3.35 | 3.65 | 3.71 | 3.55 |
| Mem0 | 3.13 | 3.26 | 3.01 | 3.35 | 3.54 | 3.36 | 3.36 | 3.61 | 2.96 | 3.00 | 3.01 | 2.73 | 3.44 | 3.73 | 3.87 | 3.15 |
| MemOS | 3.25 | 3.25 | 2.95 | 3.24 | 3.56 | 3.18 | 3.43 | 3.56 | 2.69 | 2.75 | 3.07 | 2.86 | 3.33 | 3.62 | 3.88 | 3.48 |
| No Memory | 2.96 | 3.04 | 3.23 | 3.29 | 3.52 | 3.65 | 3.29 | 3.42 | 2.99 | 2.94 | 2.90 | 2.94 | 3.30 | 3.94 | 3.55 | 3.52 |
Table 16: Perceived Realization Score (PRS, 1–5) by user, backbone, and memory condition.
| Memory | User A | User B | User C | User D | ||||||||||||
| GPT-5.6- Sol | Qwen3.8- 27B | HY4- Preview | DeepSeek- V4-Flash | GPT-5.6- Sol | Qwen3.8- 27B | HY4- Preview | DeepSeek- V4-Flash | GPT-5.6- Sol | Qwen3.8- 27B | HY4- Preview | DeepSeek- V4-Flash | GPT-5.6- Sol | Qwen3.8- 27B | HY4- Preview | DeepSeek- V4-Flash | |
| Expression Style | ||||||||||||||||
| File memory | 4.88 ± 0.22 | 4.54 ± 0.07 | 4.75 ± 0.22 | 4.58 ± 0.07 | 3.58 ± 0.52 | 4.12 ± 0.33 | 4.71 ± 0.26 | 3.58 ± 0.36 | 3.38 ± 0.43 | 3.96 ± 0.31 | 3.79 ± 0.14 | 3.62 ± 0.54 | 3.33 ± 0.38 | 3.58 ± 0.31 | 3.25 ± 0.00 | 2.96 ± 0.19 |
| Simple RAG | 3.46 ± 0.63 | 4.00 ± 0.50 | 3.88 ± 0.50 | 4.25 ± 0.33 | 3.42 ± 0.14 | 3.08 ± 0.19 | 3.21 ± 0.36 | 3.25 ± 0.45 | 2.92 ± 0.07 | 4.08 ± 0.40 | 4.00 ± 0.12 | 4.12 ± 0.12 | 2.92 ± 0.29 | 3.79 ± 0.19 | 3.25 ± 0.33 | 3.50 ± 0.22 |
| Mem0 | 3.83 ± 0.26 | 4.46 ± 0.31 | 3.42 ± 0.47 | 4.04 ± 0.14 | 3.62 ± 0.25 | 2.75 ± 0.54 | 3.21 ± 0.36 | 3.75 ± 0.25 | 2.96 ± 0.19 | 4.04 ± 0.31 | 3.96 ± 0.14 | 3.79 ± 0.38 | 2.83 ± 0.07 | 4.17 ± 0.56 | 3.21 ± 0.31 | 3.71 ± 0.26 |
| MemOS | 4.33 ± 0.31 | 4.12 ± 0.12 | 4.21 ± 0.14 | 3.92 ± 0.40 | 3.42 ± 0.36 | 2.75 ± 0.43 | 2.96 ± 0.47 | 3.50 ± 0.12 | 3.38 ± 0.12 | 3.79 ± 0.44 | 3.42 ± 0.19 | 3.96 ± 0.14 | 2.75 ± 0.00 | 4.04 ± 0.44 | 2.96 ± 0.26 | 3.54 ± 0.52 |
Table 17: Behavioral Realization Score (BRS, 1–5) by IX dimension, user, backbone, and memory condition.
| Memory | User A | User B | User C | User D | ||||||||||||
| GPT-5.6- Sol | Qwen3.8- 27B | HY4- Preview | DeepSeek- V4-Flash | GPT-5.6- Sol | Qwen3.8- 27B | HY4- Preview | DeepSeek- V4-Flash | GPT-5.6- Sol | Qwen3.8- 27B | HY4- Preview | DeepSeek- V4-Flash | GPT-5.6- Sol | Qwen3.8- 27B | HY4- Preview | DeepSeek- V4-Flash | |
| Expression Style | ||||||||||||||||
| File memory | 4.88 ± 0.22 | 4.33 ± 0.19 | 4.75 ± 0.22 | 4.58 ± 0.07 | 4.08 ± 0.52 | 3.75 ± 0.25 | 4.71 ± 0.29 | 3.62 ± 0.45 | 3.54 ± 0.29 | 3.75 ± 0.33 | 4.00 ± 0.33 | 3.50 ± 0.22 | 4.04 ± 0.19 | 4.25 ± 0.00 | 4.17 ± 0.07 | 3.50 ± 0.22 |
| Simple RAG | 3.54 ± 0.52 | 4.08 ± 0.51 | 3.96 ± 0.26 | 3.62 ± 0.45 | 3.92 ± 0.14 | 3.04 ± 0.29 | 3.25 ± 0.25 | 3.04 ± 0.26 | 3.29 ± 0.07 | 3.92 ± 0.19 | 3.88 ± 0.33 | 3.46 ± 0.40 | 3.29 ± 0.14 | 4.12 ± 0.25 | 3.83 ± 0.07 | 3.92 ± 0.26 |
| Mem0 | 3.75 ± 0.12 | 4.33 ± 0.26 | 3.42 ± 0.29 | 3.46 ± 0.19 | 3.62 ± 0.12 | 2.88 ± 0.33 | 3.42 ± 0.29 | 3.83 ± 0.14 | 3.50 ± 0.22 | 3.83 ± 0.38 | 3.92 ± 0.19 | 3.46 ± 0.19 | 3.38 ± 0.12 | 4.21 ± 0.31 | 3.88 ± 0.12 | 3.75 ± 0.38 |
| MemOS | 4.04 ± 0.14 | 4.04 ± 0.26 | 4.17 ± 0.14 | 3.62 ± 0.25 | 3.62 ± 0.33 | 2.79 ± 0.31 | 3.33 ± 0.26 | 3.67 ± 0.07 | 3.38 ± 0.12 | 3.71 ± 0.31 | 3.88 ± 0.22 | 3.71 ± 0.19 | 3.46 ± 0.19 | 4.21 ± 0.14 | 3.88 ± 0.12 | 3.88 ± 0.38 |
Table 18: Perceived Realization Score (PRS, 1–5) by IX dimension, user, backbone, and memory condition.
| Model (default reasoning effort) | Intelligence Index | GDPval-AA (Elo) | PSA (%) |
| GPT-5.6-Sol (medium) | 39 | 1403 | 49.8 |
| Qwen3.8-27B (xhigh) | 34 | 1409 | 56.0 |
| HY4-Preview (high) | – | – | 51.3 |
| DeepSeek-V4-Flash (high) | 26 | – | 52.0 |
Table 19: PSA and general capability of the four PA backbones. PSA averages Users A–D and the four memory conditions, and the other scores are from Artificial Analysis (2026) . PSA does not follow general capability.
| Attribute | GPT-5.6-Sol | Qwen3.8-27B | HY4-Preview | DeepSeek-V4-Flash |
| Expression Style | ||||
| Tone & Formality | Consultative 70% (23/33) | Consultative 42% (14/33) | Consultative 67% (22/33) | Consultative 91% (30/33) |
| Verbosity | Detailed 100% (24/24) | Detailed 71% (17/24) | Detailed 96% (23/24) | Detailed 62% (15/24) |
| Emotional Engagement | Relationship-focused 100% (27/27) | Relationship-focused 100% (27/27) | Relationship-focused 100% (27/27) | Relationship-focused 100% (27/27) |
| Guidance Level | Guided 89% (24/27) | Guided 59% (16/27) | Guided 67% (18/27) | Assumed 78% (21/27) |
| Disclosure | ||||
Table 20: Default setting of each backbone under No Memory. Each cell gives the setting the PA selects most often for the attribute, with its share and count over final and pre-event probes.
| User | Probes | Mean BRS | Mean PRS | PRS BRS | PRS BRS | PRS BRS |
| User A | 335 | 4.85 | 4.69 | 11.0% | 87.2% | 1.8% |
| User B | 330 | 4.65 | 4.54 | 14.8% | 77.9% | 7.3% |
| User C | 331 | 4.48 | 4.19 | 21.5% | 71.6% | 6.9% |
| User D | 334 | 4.47 | 4.34 | 17.1% | 72.5% | 10.5% |
| All | 1,330 | 4.61 | 4.44 | 16.1% | 77.3% | 6.6% |
Table 21: BRS and PRS for each user on the Oracle final probes with a correct selection. PRS falls below BRS on 11.0% of these probes for User A but on 21.5% for User C. PRS therefore reflects how each user receives a reply, while BRS applies one rubric to all users and is used to compare settings.
| Prompt | Called | Main inputs | Output | Page |
| Construction | ||||
| Preference Extractor, Passes 1 and 2 | Pass 1 once per scene of the source character, Pass 2 once per attribute–context cell | Scene transcript; Pass-1 evidence, attribute definition, settings, rubric | Evidence items per attribute; setting, confidence, cited evidence | I.2 |
| Identity Extractor, Identity Synthesizer, Personality Rater | Once per scene, once per section group, once per character | Scene transcript; observations of all scenes | Identity sections with cited scenes; Big Five facet and Jungian function levels | I.2 |
| Anonymizer and Anonymization Validator | Once per named life block; once per probe reply | Named life block; probe message, persona reply | Substitution table; attributes the persona pushed back on | I.2 |
| Self-View Extractor | Once per file or cluster of files | Identity, world canon, file content | What the user recalls of the file | I.2 |
| Session Outline and Accumulation Session Generators | Once per batch of sessions; once per session | Dealt slots, persona, preference definitions; outline entry, world records, tools | Session outlines; session draft with task facts and beats | I.2 |
Table 22: Directory of the prompts in Appendices I.2 – I.4 . Each is named by the component that sends it.
More in this topic
Memory Agent Benchmarks
Memory agent benchmarks evaluate how AI agents store, retrieve, update, and use information across interactions and changing tasks.
See all 163 papers