As AI systems evolve into personalized digital companions, a central capability is reasoning over a user's long-term personal history: not merely storing past events, but tracking longitudinal experiences and evolving preferences. Progress here is bottlenecked by evaluation, existing long-term memory benchmarks are largely synthetic and text-only, they overlook the visual records that anchor everyday human memory, lack the authentic and causally connected longitudinal data that real personalization demands, and consequently remain confined to shallow factual recall. We introduce ReaLMem (Real-world Long-term Multimodal Memory), the first benchmark built from authentic multi-year personal visual archives, paired with first-person subjective annotations. ReaLMem evaluates models across three cognitive tiers of increasing difficulty: factual recall, persona inference, and predictive personalization. We further propose ChronoProfiler, a temporal-weighting profiling module that computes temporal stability scores for user attributes and applies them as a salience prior, resolving conflicts among temporally inconsistent preferences and helping models compound multiple co-active preferences in complex personalized decisions. Extensive evaluation of frontier multimodal large language models (MLLMs) and memory systems on ReaLMem reveals predictive personalization as a consistent ceiling, exposes clear performance gaps and bottlenecks between MLLMs and memory systems, and shows that high-quality, temporally informed representations substantially improve personalization. Together, ReaLMem and ChronoProfiler provide an authentic testbed and a simple, effective mechanism for long-term personalization, laying a foundation for future research on lifelong AI companions.
Figures & tables
Figure 1: Overview of ReaLMem: (left) example personal multimodal memory stream over years; (right) the three-tier task pyramid grounded in this memory, rising in cognitive complexity from Factual Memory Recall (Tier 1) and Persona Inference (Tier 2) to Predictive Personalization (Tier 3), with each of the nine subtasks shown alongside its definition and an example query.
Figure 2: The ReaLMem construction pipeline: raw personal photos and videos are annotated by their owners (objective and subjective layers), anonymized, and woven into chronological multimodal sessions.
Year (rel. to T )
Media Type
T
857
Image
2,333
T−1
799
Video
175
T−2
327
Country (Top 5)
T−3
285
China
941
T−4
148
United Kingdom
842
T−5
58
Japan
127
Table 1: Statistics of the ReaLMem dataset. T denotes the year of data collection. Country counts reflect the top-5 capturing locations by media volume.
Method
Avg. Token
T1
T2
T3
Retr.
Embd.
Aggr.
M.R.
Avg.
Prof.
Track
Reas.
Avg.
Pred.
Asst.
Avg.
Full Context
Gemini-3.0-Flash
123k
84.4
77.7
75.2
89.3
82.6
74.8
73.0
81.7
76.4
62.8
53.5
59.2
GPT-4.1-mini
123k
78.0
70.4
43.8
65.5
72.6
74.2
67.4
81.0
74.3
60.4
54.0
57.9
GPT-5.4-mini
123k
79.7
68.9
40.0
57.1
72.5
75.9
68.9
81.3
75.5
59.2
58.7
59.0
Memory Systems (GPT-4.1-mini backbone)
Table 2: Main results on the ReaLMem benchmark (%), all metrics are scaled to [0,100] . Subtask abbreviations: Retr. / Embd. / Aggr. / M.R. = Episodic Event Retrieval / Embedded Content Retrieval / Multi-Record Aggregation / Multimodal Recall; Prof. / Track / Reas. = Persona Profiling / Longitudinal Tracking / Behavioral Reasoning; Pred. / Asst. = Context-Aware Prediction / Context-Aware Assistance. Shaded Avg. columns report the task-level mean. Best per column within each block in bold .
Model
Retr.
Embd.
Aggr.
M.R.
Avg.
Gemini-3.0-Flash
87.7
79.6
83.8
86.9
85.6
( ↑3.3 )
( ↑1.9 )
( ↑8.6 )
( ↓2.4 )
( ↑3.0 )
GPT-4.1-mini
88.5
80.0
77.1
84.5
85.5
( ↑10.5 )
( ↑9.6 )
( ↑33.3 )
( ↑19.0 )
( ↑12.9 )
GPT-5.4-mini
87.8
80.0
81.9
83.3
85.4
( ↑8.1 )
( ↑11.1 )
( ↑41.9 )
( ↑26.2 )
( ↑12.9 )
Table 3: Oracle reference on T1 Factual Memory Recall (%). For each model the top row reports the Oracle score, the bottom row reports the absolute delta versus the same model’s full-context result in Tab. 2 .
Task
Cap.+Dial.
MM
Vis.+Meta.
T1: Factual Memory Recall
Retr.
84.4
82.8
50.7
Embd.
77.7
74.2
71.2
Aggr.
75.2
77.1
58.1
M.R.
89.3
90.5
76.2
Avg.
82.6
81.0
57.4
Table 4: Effect of context modality on Gemini-3.0-Flash (%). Cap.+Dial. : captioned dialogue with metadata; MM : raw visual media with dialogue and metadata; Vis.+Meta. : visual media with metadata only (no dialogue). Task abbreviations follow Tab. 2 . Best result per row in bold .
System
T2
T3
Prof.
Track
Reas.
Avg.
Pred.
Asst.
Avg.
ChronoProfiler: Ep.+TS-Profile
Gemini-3.0-Flash
76.0
73.1
82.1
77.0
61.6
55.4
59.2
( ↑1.2 )
( ↑0.1 )
( ↑0.4 )
( ↑0.6 )
( ↓1.2 )
( ↑1.9 )
( ±0.0 )
GPT-4.1-mini
76.8
73.9
84.6
78.3
63.7
57.8
61.4
( ↑2.6 )
( ↑6.5 )
( ↑3.6 )
( ↑4.0 )
( ↑3.3 )
( ↑3.8 )
( ↑3.5 )
Table 5: Profile-swap analysis on T2 and T3 (%). ChronoProfiler injects our temporally-weighted profile, Ep.+Prof. EverMemOS’s built-in profile; for each row, the top line is the score and the bottom line its Δ versus the same backbone’s Full Context (Tab. 2 ). Best per column in bold .
Setting
T2
T3
Prof.
Track
Reas.
Avg.
Pred.
Asst.
Avg.
GPT-4.1-mini
w/o TS
73.8
72.0
81.7
75.6
61.3
60.1
60.8
+ TS (Ours)
76.8
73.9
84.6
78.3
63.7
57.8
61.4
Δ
( ↑3.0 )
( ↑1.9 )
( ↑2.9 )
( ↑2.7 )
( ↑2.4 )
( ↓2.3 )
( ↑0.6 )
GPT-5.4-mini
Table 6: Temporal-stability (TS) ablation on T2 and T3 (%). Rows compare the profile without ( w/o TS ) and with ( + TS ) TS weighting; Δ is the gain from adding TS.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Category
Sub-task (#QA)
Definition
Example Query
1. Factual Memory Recall
1.1 Episodic Event Retrieval (789)
Retrieve specific details (when, where, who, what) of a past event.
“When was the last time I dined at [Restaurant B] with [Friend A]?”
1.2 Embedded Content Retrieval (260)
Retrieve information embedded in visual media ( e.g . , text in screenshots).
“What is the flight number in the booking confirmation screenshot I saved?”
1.3 Multi-Record Aggregation (105)
Aggregate, count, or compare information across multiple memory records.
“How many times did I visit the gym this year?”
1.4 Multimodal Recall (84)
Retrieve relevant memories grounded in visual content.
“[Photo] When was the last time I ate at this restaurant?”
2. Persona Inference
2.1 Persona Profiling (85)
Profile and characterize stable user preferences, habits, and recurring behavioral patterns.
“What is my usual morning routine?”
2.2 Longitudinal Tracking (61)
Detect changes or evolution in preferences and behaviors over time.
“Do I prefer indoor or outdoor sports now compared to two years ago?”
Appendix
Table 7: Taxonomy of the ReaLMem benchmark. Nine subtasks are organized into three cognitive categories spanning factual memory recall, persona inference, and predictive personalization. QA pair counts per subtask are shown in parentheses.
Figure 3: ChronoProfiler retrieval-depth sensitivity for the two GPT readers. Each panel plots the temporally-weighted profile under per-category RAG as a function of retrieval depth k (solid, hollow markers), with the full no-RAG profile shown as the right-most Full tick (filled markers, shaded). (a) T2 persona understanding (LLM-judge 0 – 10 , ×10 ); (b) T3 context-aware reasoning (Kendall’s τ , ×100 ).
Multimodal Large Language Models (MLLMs) serve as daily assistants for millions. However, their ability to generate responses aligned with individual preferences remains limited. Prior approaches enable only static, single-turn personalization through input augmentation or output alignment, and thus fail to capture users' evolving preferences and personality over time (see Fig.1). In this paper, we introduce PersonaVLM, an innovative personalized multimodal agent framework designed for long-term personalization. It transforms a general-purpose MLLM into a personalized assistant by integrating three key capabilities: (a) Remembering: It proactively extracts and summarizes chronological multimodal memories from interactions, consolidating them into a personalized database. (b) Reasoning: It conducts multi-turn reasoning by retrieving and integrating relevant memories from the database. (c) Response Alignment: It infers the user's evolving personality throughout long-term interactions to ensure outputs remain aligned with their unique characteristics. For evaluation, we establish Persona-MME, a comprehensive benchmark comprising over 2,000 curated interaction cases, designed to assess long-term MLLM personalization across seven key aspects and 14 fine-grained tasks. Extensive experiments validate our method's effectiveness, improving the baseline by 22.4% (Persona-MME) and 9.8% (PERSONAMEM) under a 128k context, while outperforming GPT-4o by 5.2% and 2.0%, respectively. Project page: https://PersonaVLM.github.io.
Empowering large language models with long-term memory is crucial for building agents that adapt to users' evolving needs. Existing evaluations of this capability typically interleave preference-related dialogues with irrelevant conversations, reducing the task to needle-in-a-haystack retrieval while ignoring relationships between events driving user preference evolution. Such settings overlook a fundamental characteristic of real-world personalization: preferences emerge gradually and accumulate across interactions within noisy contexts. To bridge this gap, we introduce PERMA, a benchmark designed to evaluate persona consistency over time beyond static preference recall. Additionally, we incorporate (1) text variability and (2) linguistic alignment to simulate erratic user inputs and individual idiolects in real-world data. PERMA consists of temporally ordered interaction events spanning multiple sessions and domains, with preference-related queries inserted over time. We design both multiple-choice and interactive tasks to probe the model's understanding of persona along the interaction timeline. Experiments demonstrate that by linking related interactions, advanced memory systems extract precise preferences and reduce token consumption, outperforming traditional semantic retrieval of raw dialogues. Nevertheless, they still struggle to maintain a coherent persona across temporal depth and cross-domain interference, highlighting the need for more robust personalized memory management in agents. Our code and data are open-sourced at https://github.com/PolarisLiu1/PERMA.
Shuochen Liu, Junyi Zhu, Long Shu +11
University of Science and Technology of China · ESAT-PSI, KU Leuven · City University of Hong Kong +2
Personalized AI assistants must recall and reason over long-term user memory, which naturally spans multiple modalities and sources such as images, videos, and emails. However, existing Long-term Memory benchmarks focus primarily on dialogue history, failing to capture realistic personalized references grounded in lived experience. We introduce ATM-Bench, the first benchmark for multimodal, multi-source personalized referential Memory QA. ATM-Bench contains approximately four years of privacy-preserving personal memory data and human-annotated question-answer pairs with ground-truth memory evidence, including queries that require resolving personal references, multi-evidence reasoning from multi-source and handling conflicting evidence. We propose Schema-Guided Memory (SGM) to structurally represent memory items originated from different sources. In experiments, we implement 5 state-of-the-art memory systems along with a standard RAG baseline and evaluate variants with different memory ingestion, retrieval, and answer generation techniques. We find poor performance (under 20% accuracy) on the ATM-Bench-Hard set, and that SGM improves performance over Descriptive Memory commonly adopted in prior works. Code available at: https://github.com/JingbiaoMei/ATM-Bench
Jingbiao Mei, Jinghong Chen, Guangyu Yang +3
Department of Engineering, University of Cambridge, United Kingdom · Department of Physics, University of Cambridge, United Kingdom · Independent Researcher