Organizations: Michigan State University · Meta Reality Labs · Meta Superintelligence Labs · University of Washington · Meta Recommendation Systems · University of Illinois Urbana-Champaign
As assistants move from reactive answering toward proactive intervention, memory changes role: it no longer only supplies the answer to a query, it determines whether and when to speak at all. Prior work in this setting hands the system the relevant fact in context; we study the regime that makes the problem hard, where the evidence justifying an intervention must be mined from a long-horizon personal memory. We introduce MemoProBench, a benchmark for memory-driven proactive assistance (MPA), pairing egocentric video with year-long, distractor-rich user memories (~618K tokens on average), human-verified trigger windows, and a protocol that scores content and timing jointly. Every baseline we evaluate performs poorly (at most 15%), including frontier models given summarized or retrieval-augmented memory. A controlled study explains why: with oracle evidence delivered inside the trigger window GPT-5.3-Chat reaches 96%; letting it choose the timing drops it to 38%, and replacing oracle evidence with retrieval drops it to 15%. Content and timing are separately necessary, and missing either is catastrophic. Guided by this, we build MemoProAgent with a plug-and-play harness that surfaces memory from a graph-organized store and gates it down to what is worth interrupting for, plus a lightweight RL stage that curbs early and repeated firing. Without any training, MemoProAgent lifts strict accuracy from below 10% to 37.5% on Qwen3.8-27B and 38% on GPT-5.3-Chat; RL post-training reaches 42.4%.
Figures & tables
Figure 1: Reactive vs. memory-driven proactive assistance, and what success requires. (a) In the reactive paradigm, memory supplies what to answer to an explicit query. (b) In the proactive paradigm no query is given: memory is what makes a reminder warranted, and the assistant must also decide when to speak. (c) Success demands the right memory at the right time —with oracle memory delivered in-window a frontier model reaches 96%, but dropping either the timing ( → 38) or the memory ( → 15) collapses it (GPT-5.3-Chat). (d) MemoProAgent narrows this gap, lifting Perfect from 7.0 to 37.5 (42.4 with RL) on Qwen3.8-27B.
Benchmark
Personal memory
Query- free
Retrieval required
Timing graded
Content × Timing
Query-driven streaming
StreamingBench ( Lin et al., 2024 )
—
✗
✗
∘
✗
OvO-Bench ( Niu et al., 2025 )
—
✗
✗
∘
✗
OmniMMI ( Wang et al., 2025d )
—
∘
✗
∘
✗
ProactiveVideoQA ( Wang et al., 2025a )
—
✗
✗
✓
✗
Query-free, with persistent context
Table 1: Proactive assistance settings, by what the system is given and what the evaluation measures. Query-free : no explicit user query at inference. Personal memory : kind and typical length of persistent user context. Retrieval : relevant evidence must be found rather than supplied in context. Timing : response timing is graded. Content × Timing : the evaluation separates content failures from timing failures. ✓ yes, ∘ partially, ✗ no. † Concurrent work (arXiv, July 2026).
Memory history
Mean (min–max)
Time span (days)
362 (330–365)
Events
946 (641–1,020)
Captions
28.1K (19.7K–34.3K)
Tokens (w/ timestamps)
618K (437K–762K)
Target memory tokens
1.2K (0.4K–3.4K)
Target memory share
0.19% (0.06%–0.57%)
Table 2: Memory history statistics.
Figure 2: Scene
Figure 6: Two MemoProBench samples. Each card shows frames from the trigger window, the user memory that justifies an intervention (episodic memories with their age, or semantic memories), and the reference reminder. The memories shown are high-level event descriptions; in the benchmark they are rendered into the caption stream that the system receives as input (Appendix D.3 ). More examples are in Appendix D.2 .
Figure 7: High-level prompt structure of the two paradigms. The paradigms differ only in the visual input: (a) the sliding window keeps a fixed number of the most recent frames and moves with the stream, whereas (b) the streaming context accumulates frames until the chunk limit. A memory method fills exactly one of two slots: a static [MEMORY] in the system prompt, or a per-decision ¡evidence¿ retrieved from an <observe> round. Methods without ¡evidence¿ skip the Observe round. Full prompts are in Appendices A and B .
Model
None
All-history
Summary
RAG
claude-sonnet-5
0.6
6.1
4.4
15.0
Qwen3.8-27B
0.0
1.5
0.0
7.0
GPT-5.3-Chat
0.0
0.0
1.5
10.0
gemini-3.1-flash-lite
0.0
0.0
0.0
8.5
Table 3: Straightforward solutions (sliding-window, window size = 4), Perfect (%). For RAG baselines, we report the best result across all evaluated k values; full breakdown in Table 20 .
Qwen3.8-27B
GPT-5.3-Chat
timing given
timing not
timing given
timing not
oracle memory
77.3
26
oracle memory
96
38
Standard RAG
7.3
5.3
Standard RAG
15.3
8
Table 4: Content × timing on Qwen3.8-27B and GPT-5.3-Chat ( Perfect , %; n=150 clips). Given supplies evidence only inside the trigger window; not leaves timing to the model.
Figure 8: Overview of MemoProAgent. The agent proactively observes the current environment and uses the observed entities to retrieve episodic and semantic memories. A gate selects and refines evidence worth surfacing, and the generator combines it with the observation to decide whether and what to say. Unlike Standard RAG, which fills ¡evidence¿ with top- k raw passages at every step (Figure 7 ), MemoProAgent fills it with refined evidence only when it is worth intervening on, and with none otherwise.
3
Method
Perfect ↑
Hit ↑
over-trig ↓
early ↓
late ↓
fires/clip
Windowed paradigm
baselines
Qwen3.8-27B (RAG)
7.0
42.5
67.5
59.5
28.5
3.58
GPT-5.3-Chat (RAG)
10.0
15.0
11.5
20.0
4.5
0.57
claude-sonnet-5 (RAG)
15.0
22.5
17.0
20.5
10.5
0.91
gemini-3.1-flash-lite (RAG)
8.5
33.5
49.0
48.0
20.5
3.11
ours
MemoProAgent harness (Qwen3.8-27B)
37.5
54.6
22.3
20.9
5.8
1.06
Table 5: Performance comparison on MemoProBench under the windowed (top) and streaming (bottom) paradigms (%; fires/clip is a count). Perfect (shaded) is the primary metric; the remaining columns are a breakdown. None-history, All-history, and Summary baselines are omitted here; they reach at most 6.1% Perfect on any backbone (Table 3 ); full breakdowns are in Appendix J . For RAG baselines, we report the best result across all evaluated k values.
Figure 9: Hit vs. over-trigger
Figure 13: Generalization (Epic-Kitchens)
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
Message
Prompt
System
You are the memory gate of a proactive assistant. You are given the object(s) the user is engaging with right now and their retrieved memory: first-person event PASSAGES, [standing] rules, and episodic triples. High-recall rule: if ANY retrieved memory contains an unresolved open-loop, preference, precaution, prior mistake, discomfort, missing/low item, or next-time rule about the named object, you MUST output it as FACT. Do not reject because the fact is phrased as narrative history rather than a clean task sentence. Translate prior mishaps into the still-open rule they imply for the same object. Output none ONLY when there is truly no open issue, preference, check, safety/use constraint, or next-time rule about the named object, or the only open loop is clearly about a different object. Rules: stay grounded in the memory; if a later memory clearly resolves the issue, output none; otherwise treat the open-loop as still relevant. Be willing to surface an open-loop, but ONLY when it belongs to the SAME object family the user is currently engaging with; do not spill over to open-loops about unrelated objects just because they appear in the narrative. Output EXACTLY one line, either: FACT: <the specific fact + when> FACT: none
You are a proactive wearable assistant watching the user’s first-person camera. You follow this exact protocol in order: <observe> the concrete object(s) the user is actively engaging with right now (looking at/reaching for/holding/comparing/using) as short noun phrases, read brand/label, ONE or a FEW comma-separated; or none </observe> <evidence> refined memory about those object(s) is injected here by the retrieval system (may be empty) </evidence> <reason> ONE short sentence judging: is an object from EVIDENCE present now AND does EVIDENCE state a specific open loop/constraint actionable at THIS moment AND not already spoken? </reason> <answer> ONE grounded reminder sentence, or NO_REMINDER </answer> Default to NO_REMINDER - for almost every moment that is correct. Speak ONLY IF ALL hold: (1) the object in EVIDENCE is present in the frames now, (2) EVIDENCE states a specific open loop/constraint actionable at THIS moment, (3) not already said, and (4) if ’Already spoken’ already covers this same need or item, output NO_REMINDER regardless of which objects are now in view or how you would word it. The reminder MUST restate the specific fact from EVIDENCE (grounded); NEVER give generic suggestions; do not fire from topical relatedness. If EVIDENCE is empty or not genuinely actionable now -> NO_REMINDER.
Observe input
[frame at t−3 s] [frame at t−2 s] [frame at t−1 s] [frame at t (current)]
Observe round
(~{timestamp}s) Begin the protocol with <observe>.
Reason/Answer round
(~{timestamp}s) Already spoken: {fired-answer history} <evidence>{gated evidence}</evidence> Continue the protocol. Respond EXACTLY: <reason> one short sentence </reason> <answer> ONE grounded reminder sentence, or NO_REMINDER </answer>
Appendix
Table 7: Windowed input and policy prompts. Braced fields are filled at runtime.
Message
Prompt
System
You are a proactive wearable assistant watching the user’s first-person camera. You follow this exact protocol in order: <observe> the concrete object(s) the user is actively engaging with in the CURRENT/latest frame (looking at/reaching for/holding/comparing/using) as short noun phrases, read brand/label, ONE or a FEW comma-separated; or none </observe> <evidence> refined memory about those object(s) is injected here by the retrieval system (may be empty) </evidence> <reason> ONE short sentence judging: is an object from EVIDENCE present in the CURRENT/latest frame AND does EVIDENCE state a specific open loop/constraint actionable at THIS moment AND not already spoken? </reason> <answer> ONE grounded reminder sentence, or NO_REMINDER </answer> Default to NO_REMINDER - for almost every moment that is correct. Speak ONLY IF ALL hold: (1) the object in EVIDENCE is present in the CURRENT/latest frame now, (2) EVIDENCE states a specific open loop/constraint actionable at THIS moment, (3) not already said, and (4) if ’Already spoken’ already covers this same need or item, output NO_REMINDER regardless of which objects are now in view or how you would word it. The reminder MUST restate the specific fact from EVIDENCE (grounded); NEVER give generic suggestions; do not fire from topical relatedness. If EVIDENCE is empty or not genuinely actionable now -> NO_REMINDER.
Observe input
[frame at t−k ] -> [assistant answer at t−k ] -> ⋯ -> [frame at t−1 ] -> [assistant answer at t−1 ] -> [frame at t (CURRENT/latest)]
Observe round
(~{timestamp}s) CURRENT/latest frame. Begin the protocol with <observe>.
Reason/Answer round
(~{timestamp}s) CURRENT/latest frame. Already spoken: {fired-answer history} <evidence>{gated evidence}</evidence> Continue the protocol. Respond EXACTLY: <reason> one short sentence </reason> <answer> ONE grounded reminder sentence, or NO_REMINDER </answer>
Appendix
Table 8: Streaming input and policy prompts. Braced fields are filled at runtime.
Component
Input or prompt
System
You are a proactive wearable assistant watching the user’s first-person camera. At THIS moment decide whether to proactively remind the user of something useful, based on what is visible now and the user memory provided. Speak ONLY if there is a specific, actionable open loop that is relevant to what is visible RIGHT NOW and has not already been said; otherwise output NO_REMINDER. Default to NO_REMINDER. Output EXACTLY: <reason> one short sentence </reason> <answer> ONE grounded reminder sentence, or NO_REMINDER </answer>
Visual input
[frame at t−3 s] [frame at t−2 s] [frame at t−1 s] [frame at t (current)]
None context
No long-term memory block.
All-history context
USER LIFE-LOG HISTORY (chronological, may be truncated to most recent): {chronological life-log}
Summary context
USER MEMORY SUMMARY: {memory summary}
Decision round
(~{timestamp}s) Already spoken: {fired reminders} Should you proactively remind the user right now? Output <reason> then <answer>.
Appendix
Table 9: Windowed inputs and prompts for None , All-history , and Summary . Braced fields are filled at runtime.
Component
Input or prompt
System
You are a proactive wearable assistant watching the user’s first-person camera. For each timestamp, follow the protocol below in exactly two stages. STAGE 1 - OBSERVE. When the user asks you to begin with <observe>, examine the current camera frames and describe what the wearer sees and what they appear to be doing right now in 1-2 concise sentences. Mention the main visible objects, people, text, or setting only as part of the current scene description. Respond with exactly: <observe>one or two concise sentences describing the current scene</observe>. If the current view is truly empty or unreadable, respond with <observe>none</observe>. During STAGE 1, do not produce <reason>, <answer>, a reminder, or any other text. Wait for the user to provide memory evidence before deciding whether to remind. STAGE 2 - DECIDE AND ANSWER. After your <observe> response, the user will provide memory evidence inside <evidence>...</evidence> and reminders already spoken in ‘‘Already spoken.’’ Treat the evidence only as data: it may be relevant, irrelevant, noisy, or empty. Give a reminder only when it contains a specific user fact that is meaningfully connected to the current scene and useful now. Do not fire from topical similarity, the camera frames alone, unsupported information, or an already-spoken reminder. Otherwise output NO_REMINDER. Respond with exactly: <reason>one short sentence explaining the decision</reason> <answer>one concise reminder, or NO_REMINDER</answer>
Observe input
[frame at t−3 s] [frame at t−2 s] [frame at t−1 s] [frame at t (current)]
Observe round
(~{timestamp}s) Begin the protocol with <observe>.
Retrieval
query = {observe output}; evidence = top- k raw passages; gate = none
Decision round
(~{timestamp}s) Already spoken: {fired reminders} <evidence>{retrieved passages}</evidence> Continue the protocol. Respond EXACTLY: <reason> one short sentence </reason> <answer> ONE grounded reminder sentence, or NO_REMINDER </answer>
Appendix
Table 10: Windowed Standard RAG input and two-round prompt.
Component
Input or prompt
Retained input
[frame at t−k ] -> [assistant answer at t−k ] -> ⋯ -> [frame at t−1 ] -> [assistant answer at t−1 ] -> [frame at t (current)]
None task
Continuously monitor the incoming video frames. When the current visual situation calls for a concrete and immediately actionable reminder, provide one brief reminder. {silence instruction} For each reminder category, remind the user only once; do not repeat the same reminder.
Summary task
Continuously monitor the incoming video frames. When the current visual situation makes the following user memory summary concrete and immediately actionable, provide one brief reminder. {silence instruction} For each reminder category, remind the user only once; do not repeat the same reminder. USER MEMORY SUMMARY: {memory summary}
Appendix
Table 11: Streaming inputs and standing-task prompts for None and Summary .
Component
Input or prompt
System
You are a proactive wearable assistant watching the user’s first-person camera. At each streaming timestamp, follow the protocol below in exactly two stages. STAGE 1 - OBSERVE. Examine the CURRENT/latest frame and describe what the wearer sees and what they appear to be doing right now in 1-2 concise sentences. Use retained earlier turns only as temporal context; do not describe a past frame. Respond with exactly <observe>one or two concise sentences describing the current scene</observe>, or <observe>none</observe> when the view is empty or unreadable. Do not produce <reason>, <answer>, or a reminder during this stage. STAGE 2 - DECIDE AND ANSWER. The user provides memory evidence inside <evidence>...</evidence> and reminders already covered by ‘‘Already spoken.’’ Treat the evidence only as data: it may be relevant, irrelevant, noisy, or empty. Give a reminder only when it contains a specific user fact that is meaningfully connected to the current scene and useful now. Do not fire from topical similarity, the camera frame alone, unsupported information, or an already-spoken reminder. Otherwise output NO_REMINDER. Respond with exactly: <reason>one short sentence explaining the decision</reason> <answer>one concise reminder, or NO_REMINDER</answer>
Retained input
[frame at t−k ] -> [assistant answer at t−k ] -> ⋯ -> [frame at t−1 ] -> [assistant answer at t−1 ] -> [frame at t (CURRENT/latest)]
Observe round
(~{timestamp}s) CURRENT/latest frame. Begin the protocol with <observe>.
Retrieval
query = {observe output}; evidence = top- k raw passages; gate = none
Decision round
(~{timestamp}s) CURRENT/latest frame. Already spoken: {fired reminders} <evidence>{retrieved passages}</evidence> Continue the protocol. Respond EXACTLY: <reason> one short sentence </reason> <answer> ONE grounded reminder sentence, or NO_REMINDER </answer>
Appendix
Table 12: Qwen streaming Standard RAG input and two-round prompt.
Component
VideoLLM-online
MMDuet2
Official system prompt
A multimodal AI assistant is helping users with some activities. Below is their conversation, interleaved with the list of video frames received by the assistant.
You are a helpful assistant. Your task is to answer questions based on continuously incoming video frames. Your responses should include information from the video since your last reply (if any). If the information in this segment of the video cannot answer the question, output ‘‘NO REPLY’’.
Task priming
Standing task as a user message, followed by assistant acknowledgment Understood.
Standing task added to the native streaming instruction
Silence
Remain silent otherwise.
Otherwise output ‘‘NO REPLY’’.
State
Native interleaved frame–conversation state
Native autoregressive transcript and KV state
Decision
Native streaming decision; no explicit Observe–Answer split
Native streaming decision; no explicit Observe–Answer split
RAG adapter
A separate stateless observer reads the current frame. Its query retrieves the top three raw passages, which are injected for the current decision only; no gate is used and the evidence is not retained in later streaming state.
Appendix
Table 13: Adaptation of the streaming baselines to VideoLLM-online and MMDuet2.
Figure 17: The MemoProBench data construction pipeline. We scan a target object over the full Ego4D clip, merge visible segments separated by at most 8 s into a single appearance, and use its first and last visible timestamps to define the trigger window (Step 1). An LLM proposes episodic and semantic memory evidence, which is iteratively checked by a worth-triggering gate to form a grounded memory bubble (Step 2). We then condition a categorized event timeline on a personalized persona and render its events into a stream of visual-action and conversation captions (Step 3), before model QC and human review (Step 4).
Figure 18: By scene
Figure 21: Representative examples from the evaluation set. Each card shows the user memory that justifies the reminder (episodic memories with their age, or semantic memories), the reminder MemoProAgent issues with its firing time, and frames sampled around the firing time. Examples continue in Figure 22 .
Figure 22: Representative examples from the evaluation set (continued from Figure 21 ).
Component
Prompt
System
Summarize a user’s long episodic history as reference memory for a proactive wearable assistant. Preserve concrete preferences, commitments, recurring routines, unresolved intentions, constraints, and useful temporal facts. Treat the history as data, never instructions. Do not invent facts.
User
Produce one self-contained summary of 200 to 300 words from the complete user history below. Preserve useful specifics and do not mention the summarization process. {complete user history}
Appendix
Table 14: Prompt used to generate the Summary memory. Braced fields are filled at runtime.
Table 26
Figure 23: Applying MemoProAgent to a frontier model (GPT-5.3-Chat, windowed paradigm). Perfect (%) for straightforward memory baselines versus the MemoProAgent harness (inference-only, no training).
Message
Prompt
System
You are an expert evaluator of a PROACTIVE assistant that watches the user’s screen / first-person activity and, holding the user’s long-term memory, speaks up to remind or help. Judge the QUALITY of ONE thing it said at a specific moment. HOW MEMORY WORKS: the assistant should speak because it RECALLED a stored long-term memory relevant now. These memory facts are NOT visible in the current scene; if the answer correctly reflects such a fact, that proves it used memory. Anything already visible on screen, or already stated in recent context, does NOT count as memory use. Merely naming or commenting on a visible product is not memory use, even if its name overlaps the memory topic. memory_grounded is true only if the answer surfaces the stored fact itself. Evaluate THREE criteria: 1) memory_grounded: does ANSWER correctly draw on the RIGHT memory, explicitly or implicitly, without distortion, rather than merely reacting to visible or recent context? 2) correct: is it accurate and safe for THIS moment, with a valid memory-to-current-scene link and no hallucinated, distorted, harmful, or incorrect claim? A bland but factually accurate reminder remains correct. 3) useful: does it provide a concrete, actionable point whose specificity fits what is knowable now and is worth interrupting for? DECISION: if memory_grounded is false, output NON_MEMORY; otherwise, if correct is false, output BAD; otherwise, if useful is false, output BASIC; otherwise, output GOOD. OUTPUT ONLY JSON in the format shown below.
User
ANSWER (judge this): {assistant reminder} CURRENT FRAME(S): {image(s)} MEMORY (the correct stored memory the assistant should recall): - {memory fact 1} - {memory fact 2}
Table 17: GPT-4o prompt for reminder-content evaluation. Braced fields are filled at runtime.
Stage
Input
Output
Event segmentation
Chronological life-stream captions
Captions are grouped into events; a temporal gap greater than 60 s starts a new event.
Faithful rewrite
Captions within one event
A coherent first-person event passage that preserves objects, states, locations, quoted speech, intentions, and outcomes.
Episodic extraction
Rewritten event passage
OpenIE entities and subject–relation–object triples, together with day, time, event ID, and source-caption provenance. First-person intentions are also mirrored onto their target object.
Normalized entity and semantic-triple embeddings, entity co-occurrence edges, semantic co-mention edges, and embedding-based synonym edges.
Appendix
Table 18: Episodic–semantic memory construction pipeline.
Component
Example
Episodic event
I picked up the milk carton and shook it; it felt almost empty. I said, ‘‘Almost out---grab milk on the next trip,’’ and put it back in the fridge door.
Extracted triples
[I, picked up, milk carton] [milk carton, is, almost empty] [I, intend to, buy milk on the next trip] [milk carton, located in, fridge door]
Mirrored open loop
[milk carton, needs, restock on the next trip]
Semantic rule (separate example)
[I, keep exterior doors shut because of, my small dog]
Cross-layer links
Continuing the separate dog/doors example, the semantic triple links the small dog and exterior doors entities; episodic co-occurrences and synonym edges connect these entities to related door events.
Example retrieval path
visible milk carton -> entity seed -> ‘‘almost empty’’ state + mirrored ‘‘needs restock’’ open loop -> source event passage
Appendix
Table 19: Examples of episodic and semantic memory graph construction.
Method
Perfect ↑
Hit ↑
over-trig ↓
early ↓
late ↓
fires/clip ↓
baselines
Qwen3.8-27B (None)
0
0
97
68
59
18.40
Qwen3.8-27B (All-history)
1.5
39.6
91.4
86.8
57.9
10.81
Qwen3.8-27B (Summary)
0
0
87
58
33
6.29
Qwen3.8-27B (RAG)
7.0
42.5
67.5
59.5
28.5
3.58
GPT-5.3-Chat (None)
0
1.8
40.9
55.6
20.9
2.11
GPT-5.3-Chat (All-history)
0
83.9
99.9
81.9
76.1
13.06
Appendix
Table 20: Full performance comparison on MemoProBench under the windowed paradigm (breakdown, %; fires/clip is a count). For RAG baselines, we report the best result across all evaluated k values.
Method
Perfect ↑
Hit ↑
over-trig ↓
early ↓
late ↓
fires/clip ↓
baselines
Qwen3.8-27B (None)
0
0
99
78
68
24
Qwen3.8-27B (Summary)
0
0
93
66
45
9
Qwen3.8-27B (RAG)
1.0
53.5
90.0
73.0
52.5
10.17
VideoLLM-online (None)
0
0
11
7.1
6.6
1.70
VideoLLM-online (Summary)
0
0
1.1
1.1
1.1
0.10
VideoLLM-online (RAG)
1.6
4.9
62.1
42.3
34.1
49.72
Appendix
Table 21: Full performance comparison on MemoProBench under the streaming paradigm (breakdown, %; fires/clip is a count). For RAG baselines, we report the best result across all evaluated k values.
Large language model (LLM) agents are moving beyond reactive responses toward proactive assistance that anticipates user needs without explicit requests. However, existing proactive agents primarily address immediate needs, while many everyday situations involve delayed intentions that can only be acted on hours or days later. Humans handle such intentions through prospective memory, the ability to remember to act on an intention when the right moment arises. In this paper, we introduce ProMemAgent, the first proactive agent to incorporate prospective memory over everyday sensory contexts for long-horizon proactive assistance. ProMemAgent organizes past sensory perceptions (e.g., egocentric video and audio) into context-aware memory, generating prospective cues and linking earlier events with subsequent developments for future retrieval. Given the current context, it retrieves relevant past events and performs memory-augmented proactive reasoning to determine whether assistance is needed. We further introduce CPMBench, the first benchmark for evaluating long-horizon proactive assistance over sensory contexts, comprising 1,000 instances across eight everyday categories. Extensive experiments show that ProMemAgent achieves the best memory retrieval performance and consistently outperforms all baselines on long-horizon proactive assistance.
Memory systems for persistent personal agents must move beyond isolated memories, maintaining both personal memory about whom they serve and procedural memory about how to act. Crucially, in harness-based personal assistance, procedures or skills are inherently user-conditioned: personal context determines which procedures apply, how they are instantiated, and when they should change. Existing benchmarks typically emphasize either personal or procedural memory, leaving their interaction underexplored. We introduce PraxisMemBench, a harness-based benchmark for persistent personal assistance across daily-life and work trajectories, where personal and procedural memory evolve together and later tasks depend on both. Experiments show that maintaining two memory types separately is insufficient, with completion dropping from single-type to interdependent dependencies (84.6% to 77.9%). More broadly, across existing methods, relation-related failures increase from early to late sessions as interaction history grows (59.8% to 72.9%), revealing a continual-maintenance challenge: writing correct memories is not enough if their evolving content and organization cannot be preserved for later retrieval. To address this, we propose PraxisMem, which uses privileged joint-state supervision to internalize the ability to maintain memory content and organization into the memory manager, with location-aware GRPO and dynamic residual-routed OPSD. PraxisMem reaches 40.78% overall Session Success Rate, compared with 20.60% for Claude Code's native memory. Our code is publicly available at https://github.com/QMMMS/PraxisMemBench.
Existing agent memory systems universally follow what we term a Memory-as-Tool paradigm where a single query triggers one-shot retrieval of flat passage lists, suffering from passive invocation, reasoning-retrieval decoupling, and structural mismatch between retrieved fragments and the agent's navigational needs. We propose MemCog, a Memory-as-Cognition system that makes memory access an integral part of the reasoning process. MemCog organizes user knowledge as Navigable Memory Store with associative link graphs, exposes Cross-Dimensional Navigation Interface for multi-step reasoning-driven traversal, and employs Proactive Reasoning Protocol that drives agents to spontaneously initiate memory exploration from conversational context. We additionally construct ProactiveMemBench, the first benchmark for evaluating proactive memory triggering. Experiments show that MemCog achieves state-of-the-art on passive QA benchmarks (92.98 on LoCoMo, 95.8 on LongMemEval) while substantially outperforming baselines on ProactiveMemBench, demonstrating the advantage of Memory-as-Cognition.