To Memories and Beyond: From Remembering to Knowing You across Long-Term Multimodal Personal Archives

Wenqi Zhou; Zhuorui Yu; Kaiao Wen; Hao Zheng; Xinyi Zheng; Peiran Wu; Enmin Zhou; Chi-Hao Wu

cs.CLSep 11, 2026

To Memories and Beyond: From Remembering to Knowing You across Long-Term Multimodal Personal Archives

Authors: Wenqi Zhou, Zhuorui Yu, Kaiao Wen, Hao Zheng, Xinyi Zheng, Peiran Wu, Enmin Zhou, Chi-Hao Wu, +1 more

Organizations: University of Bristol · Memories.ai Research

Abstract

As AI systems evolve into personalized digital companions, a central capability is reasoning over a user's long-term personal history: not merely storing past events, but tracking longitudinal experiences and evolving preferences. Progress here is bottlenecked by evaluation, existing long-term memory benchmarks are largely synthetic and text-only, they overlook the visual records that anchor everyday human memory, lack the authentic and causally connected longitudinal data that real personalization demands, and consequently remain confined to shallow factual recall. We introduce ReaLMem (Real-world Long-term Multimodal Memory), the first benchmark built from authentic multi-year personal visual archives, paired with first-person subjective annotations. ReaLMem evaluates models across three cognitive tiers of increasing difficulty: factual recall, persona inference, and predictive personalization. We further propose ChronoProfiler, a temporal-weighting profiling module that computes temporal stability scores for user attributes and applies them as a salience prior, resolving conflicts among temporally inconsistent preferences and helping models compound multiple co-active preferences in complex personalized decisions. Extensive evaluation of frontier multimodal large language models (MLLMs) and memory systems on ReaLMem reveals predictive personalization as a consistent ceiling, exposes clear performance gaps and bottlenecks between MLLMs and memory systems, and shows that high-quality, temporally informed representations substantially improve personalization. Together, ReaLMem and ChronoProfiler provide an authentic testbed and a simple, effective mechanism for long-term personalization, laying a foundation for future research on lifelong AI companions.

Figures & tables

Figure 1: Overview of ReaLMem: (left) example personal multimodal memory stream over years; (right) the three-tier task pyramid grounded in this memory, rising in cognitive complexity from Factual Memory Recall (Tier 1) and Persona Inference (Tier 2) to Predictive Personalization (Tier 3), with each of the nine subtasks shown alongside its definition and an example query.

Figure 2: The ReaLMem construction pipeline: raw personal photos and videos are annotated by their owners (objective and subjective layers), anonymized, and woven into chronological multimodal sessions.

Year (rel. to $T$ )		Media Type
$T$	857	Image	2,333
$T{-}1$	799	Video	175
$T{-}2$	327	Country (Top 5)
$T{-}3$	285	China	941
$T{-}4$	148	United Kingdom	842
$T{-}5$	58	Japan	127

Table 1: Statistics of the ReaLMem dataset.

T

denotes the year of data collection. Country counts reflect the top-5 capturing locations by media volume.

Full Context
Method	Avg. Token	T1					T2				T3
Method	Avg. Token	Retr.	Embd.	Aggr.	M.R.	Avg.	Prof.	Track	Reas.	Avg.	Pred.	Asst.	Avg.
Gemini-3.0-Flash	123k	84.4	77.7	75.2	89.3	82.6	74.8	73.0	81.7	76.4	62.8	53.5	59.2
GPT-4.1-mini	123k	78.0	70.4	43.8	65.5	72.6	74.2	67.4	81.0	74.3	60.4	54.0	57.9
GPT-5.4-mini	123k	79.7	68.9	40.0	57.1	72.5	75.9	68.9	81.3	75.5	59.2	58.7	59.0
Memory Systems (GPT-4.1-mini backbone)

Table 2: Main results on the ReaLMem benchmark (%), all metrics are scaled to

[0,100]

. Subtask abbreviations: Retr. / Embd. / Aggr. / M.R. = Episodic Event Retrieval / Embedded Content Retrieval / Multi-Record Aggregation / Multimodal Recall; Prof. / Track / Reas. = Persona Profiling / Longitudinal Tracking / Behavioral Reasoning; Pred. / Asst. = Context-Aware Prediction / Context-Aware Assistance. Shaded Avg. columns report the task-level mean. Best per column within each block in bold .

Model	Retr.	Embd.	Aggr.	M.R.	Avg.
Gemini-3.0-Flash	87.7	79.6	83.8	86.9	85.6
Gemini-3.0-Flash	( $\uparrow 3.3$ )	( $\uparrow 1.9$ )	( $\uparrow 8.6$ )	( $\downarrow 2.4$ )	( $\uparrow 3.0$ )
GPT-4.1-mini	88.5	80.0	77.1	84.5	85.5
GPT-4.1-mini	( $\uparrow 10.5$ )	( $\uparrow 9.6$ )	( $\uparrow 33.3$ )	( $\uparrow 19.0$ )	( $\uparrow 12.9$ )
GPT-5.4-mini	87.8	80.0	81.9	83.3	85.4
GPT-5.4-mini	( $\uparrow 8.1$ )	( $\uparrow 11.1$ )	( $\uparrow 41.9$ )	( $\uparrow 26.2$ )	( $\uparrow 12.9$ )

Table 3: Oracle reference on T1 Factual Memory Recall (%). For each model the top row reports the Oracle score, the bottom row reports the absolute delta versus the same model’s full-context result in Tab. 2 .

T1: Factual Memory Recall
Task	Cap.+Dial.	MM	Vis.+Meta.
Retr.	84.4	82.8	50.7
Embd.	77.7	74.2	71.2
Aggr.	75.2	77.1	58.1
M.R.	89.3	90.5	76.2
Avg.	82.6	81.0	57.4

Table 4: Effect of context modality on Gemini-3.0-Flash (%). Cap.+Dial. : captioned dialogue with metadata; MM : raw visual media with dialogue and metadata; Vis.+Meta. : visual media with metadata only (no dialogue). Task abbreviations follow Tab. 2 . Best result per row in bold .

ChronoProfiler: Ep.+TS-Profile
System	T2				T3
System	Prof.	Track	Reas.	Avg.	Pred.	Asst.	Avg.
Gemini-3.0-Flash	76.0	73.1	82.1	77.0	61.6	55.4	59.2
Gemini-3.0-Flash	( $\uparrow 1.2$ )	( $\uparrow 0.1$ )	( $\uparrow 0.4$ )	( $\uparrow 0.6$ )	( $\downarrow 1.2$ )	( $\uparrow 1.9$ )	( $\pm 0.0$ )
GPT-4.1-mini	76.8	73.9	84.6	78.3	63.7	57.8	61.4
GPT-4.1-mini	( $\uparrow 2.6$ )	( $\uparrow 6.5$ )	( $\uparrow 3.6$ )	( $\uparrow 4.0$ )	( $\uparrow 3.3$ )	( $\uparrow 3.8$ )	( $\uparrow 3.5$ )

Table 5: Profile-swap analysis on T2 and T3 (%). ChronoProfiler injects our temporally-weighted profile, Ep.+Prof. EverMemOS’s built-in profile; for each row, the top line is the score and the bottom line its

\Delta

versus the same backbone’s Full Context (Tab. 2 ). Best per column in bold .

GPT-4.1-mini
Setting	T2				T3
Setting	Prof.	Track	Reas.	Avg.	Pred.	Asst.	Avg.
w/o TS	73.8	72.0	81.7	75.6	61.3	60.1	60.8
+ TS (Ours)	76.8	73.9	84.6	78.3	63.7	57.8	61.4
$\Delta$	( $\uparrow 3.0$ )	( $\uparrow 1.9$ )	( $\uparrow 2.9$ )	( $\uparrow 2.7$ )	( $\uparrow 2.4$ )	( $\downarrow 2.3$ )	( $\uparrow 0.6$ )
GPT-5.4-mini

Table 6: Temporal-stability (TS) ablation on T2 and T3 (%). Rows compare the profile without ( w/o TS ) and with ( + TS ) TS weighting;

\Delta

is the gain from adding TS.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Category	Sub-task (#QA)	Definition	Example Query
1. Factual Memory Recall	1.1 Episodic Event Retrieval (789)	Retrieve specific details (when, where, who, what) of a past event.	“When was the last time I dined at [Restaurant B] with [Friend A]?”
	1.2 Embedded Content Retrieval (260)	Retrieve information embedded in visual media ( e.g . , text in screenshots).	“What is the flight number in the booking confirmation screenshot I saved?”
	1.3 Multi-Record Aggregation (105)	Aggregate, count, or compare information across multiple memory records.	“How many times did I visit the gym this year?”
	1.4 Multimodal Recall (84)	Retrieve relevant memories grounded in visual content.	“[Photo] When was the last time I ate at this restaurant?”
2. Persona Inference	2.1 Persona Profiling (85)	Profile and characterize stable user preferences, habits, and recurring behavioral patterns.	“What is my usual morning routine?”
	2.2 Longitudinal Tracking (61)	Detect changes or evolution in preferences and behaviors over time.	“Do I prefer indoor or outdoor sports now compared to two years ago?”

Appendix

Table 7: Taxonomy of the ReaLMem benchmark. Nine subtasks are organized into three cognitive categories spanning factual memory recall, persona inference, and predictive personalization. QA pair counts per subtask are shown in parentheses.

Figure 3: ChronoProfiler retrieval-depth sensitivity for the two GPT readers. Each panel plots the temporally-weighted profile under per-category RAG as a function of retrieval depth

k

(solid, hollow markers), with the full no-RAG profile shown as the right-most Full tick (filled markers, shaded). (a) T2 persona understanding (LLM-judge

0

–

10

\times 10

); (b) T3 context-aware reasoning (Kendall’s

\tau

\times 100

Explore similar work

PersonaVLM: Long-Term Personalized Multimodal LLMs
Mar 20, 2026Chang Nie, Chaoyou Fu, Yifan Zhang +2Personalized Language Models Multimodal Agents
Nanjing University · ByteDance
PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environments
Mar 24, 2026Shuochen Liu, Junyi Zhu, Long Shu +11Memory Agent Benchmarks Long-Term Conversational Memory
University of Science and Technology of China · ESAT-PSI, KU Leuven · City University of Hong Kong +2
According to Me: Long-Term Personalized Referential Memory QA
Mar 2, 2026Jingbiao Mei, Jinghong Chen, Guangyu Yang +3LLM Memory Memory Agent Benchmarks
Department of Engineering, University of Cambridge, United Kingdom · Department of Physics, University of Cambridge, United Kingdom · Independent Researcher

Mar 20, 2026J/K move · Enter open · S save

PersonaVLM: Long-Term Personalized Multimodal LLMs

Chang Nie, Chaoyou Fu, Yifan Zhang, Haihua Yang+1

Nanjing University · ByteDance

Personalized Language Models Multimodal Agents Long-Term Conversational Memory Personality Modeling in Language Models

Open paper AskPDF arXiv

Multimodal Large Language Models (MLLMs) serve as daily assistants for millions. However, their ability to generate responses aligned with individual preferences remains limited. Prior approaches enable only static, single-turn personalization through input augmentation or output alignment, and thus fail to capture users' evolving preferences and personality over time (see Fig.1). In this paper, we introduce PersonaVLM, an innovative personalized multimodal agent framework designed for long-term personalization. It transforms a general-purpose MLLM into a personalized assistant by integrating three key capabilities: (a) Remembering: It proactively extracts and summarizes chronological multimodal memories from interactions, consolidating them into a personalized database. (b) Reasoning: It conducts multi-turn reasoning by retrieving and integrating relevant memories from the database. (c) Response Alignment: It infers the user's evolving personality throughout long-term interactions to ensure outputs remain aligned with their unique characteristics. For evaluation, we establish Persona-MME, a comprehensive benchmark comprising over 2,000 curated interaction cases, designed to assess long-term MLLM personalization across seven key aspects and 14 fine-grained tasks. Extensive experiments validate our method's effectiveness, improving the baseline by 22.4% (Persona-MME) and 9.8% (PERSONAMEM) under a 128k context, while outperforming GPT-4o by 5.2% and 2.0%, respectively. Project page: https://PersonaVLM.github.io.

Abstract

Explore similar work

PersonaVLM: Long-Term Personalized Multimodal LLMs

PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environments

According to Me: Long-Term Personalized Referential Memory QA