MobileMem: Learning from a Year of Mobile Experiences

Xinle Deng; Yida Xue; Xiangyuan Ru; Yijun Chen; Buqiang Xu; Mingjun Mao; Xinjie Liu; Haoming Xu

cs.AIAug 11, 2026

MobileMem: Learning from a Year of Mobile Experiences

Authors: Xinle Deng, Yida Xue, Xiangyuan Ru, Yijun Chen, Buqiang Xu, Mingjun Mao, Xinjie Liu, Haoming Xu,

Organizations: OPPO, OpenKG

Abstract

The next generation of AI agents is increasingly moving beyond systems that answer isolated questions toward persistent personal assistants that can understand, remember, and continuously learn from users' experiences. Such assistants require long-term memory to accumulate and leverage user-specific experiences over time, yet existing benchmarks remain inadequate for realistic mobile settings, where experiences are heterogeneous, multimodal, evolving, and deeply personal. We introduce MobileMem, a benchmark and framework for studying on-device long-term memory, grounded in a year-scale collection of mobile experiences. MobileMem employs a knowledge-grounded synthesis pipeline to construct coherent and temporally consistent long-horizon trajectories from user-app sessions. It provides complementary text and multimodal settings covering multi-hop and temporal reasoning, knowledge updating, and implicit preference inference. Specifically, MobileMem enables agents to remember the past, understand the present, and adapt to the future. By modeling experiences rather than isolated facts, MobileMem moves memory beyond information retrieval toward experiential intelligence for continuous personal learning.

Figures & tables

Figure 2: MobileMem breaks down data silos across chat, photos, reading, shopping, and travel apps, unifying fragmented data into one memory layer for proactive personalization.

Figure 3: The position of MobileMem in the envisioned on-device memory ecosystem, with a two - layer composite architecture and an information exchange protocol between them.

Figure 4: Overview of the data synthesis framework. The KEME framework progressively synthesizes long-horizon interaction trajectories and QA pairs from user prior knowledge, with quality control applied throughout the pipeline.

Method	Performance								Cost
Method	Single.	Multi.	Temp.	Rel.	QFS	Adv.	Others	Overall	In. Tok.	Out. Tok.	Overall
GPT-4.1-mini
Long Context	56.54	51.71	53.76	58.33	47.06	51.89	60.87	54.51	0.00	0.00	0.00
NaiveRAG	38.58	31.23	29.03	29.17	20.59	60.38	47.83	37.23	0.00	0.00	0.00
HippoRAG2	85.67	79.00	61.29	87.50	73.53	50.94	82.61	78.85	2,307.44	495.84	2,803.28
LangMem	23.62	14.96	12.90	20.83	11.76	76.42	39.13	24.79	3,836.97	548.49	4,385.46

Table 1: End-to-end task performance and token cost (in thousands) of memory systems. We evaluate each method with two LLM backbones, GPT-4.1-mini and GPT-5.4-mini. Due to space limitations, the six most frequent question types are displayed, while the remaining question types are grouped into “Others”. Here, “Single.”, “Multi.”, “Temp.”, “Rel.”, “QFS”, and “Adv.” denote single-hop, multi-hop, temporal-reasoning, relationship-related, query-focused summarization, and adversarial questions, respectively. “In. Tok.” and “Out. Tok.” denote the average number of input tokens and output tokens, respectively, used by a memory system to construct memories over one user trajectory. For each backbone, the best result in each column is highlighted in bold .

Figure 5: Distribution of error types across four memory systems. We randomly sample 20 failure cases from each system for error analysis.

Profile Schema	Lexical $\uparrow$	Semantic $\uparrow$
Coarse-grained	0.7238	0.4783
Medium-grained	0.7313	0.4932
Fine-grained	0.7310	0.4875

Table 2: Trajectory diversity under varying profile schema granularity. Values are averaged across three seed persons and five length-matched sampling rounds.

Panel of Figure 6 — Figure 6: Hard-distractor synthesis with KEME . (a) The top table compares average accuracy (%) across two memory systems on the original LongMemEval trajectories and the KEME -synthesized trajectories. When the retrieval budget is set to top- $10$ , EverMemOS makes an additional error on the question instance 50635ada . When the retrieval budget is reduced to top- $5$ , NaiveRAG shows a larger performance drop of $20\%$ , failing on 0ddfec37 and gpt4_4929293a . (b) The bottom-left panel shows the trajectory length distribution measured by the number of messages. (c) The bottom-right panel visualizes the rank change of source evidence under semantic-similarity-based ranking with NaiveRAG. Each point corresponds to one source evidence and its rank normalized by the total number of memory units. For absolute rank, KEME yields lower ranks for only 4 out of 18 evidence items. However, after normalization, KEME consistently pushes the answer-supporting evidence toward less favorable retrieval positions by introducing semantically similar distractors, increasing the average normalized rank by 10.63 percentage points. For readability, one evidence (its normalized ranks are approximately 0.9680 for LongMemEval and 1.0000 for KEME ) is omitted from the plot.

User Statistics
Number of Users	16
English Interaction Users	8
Chinese Interaction Users	8
Total Events	1,589
Interaction Statistics
Avg Context Length (tokens/user)	1.72M

Table 3: Statistical Overview of MobileMem-Omni .

	Task							Overall
Method	Single.	Multi.	Know.	Temp.	Abst.	Pref.	Vis.	Judge	F1
Textual Memory Methods
GPT-5.4-mini
Long Context	17.24	10.22	21.48	19.28	18.13	34.01	12.16	19.14	20.50
NaiveRAG	24.85	20.44	16.53	19.02	7.37	8.56	14.5	15.4	10.4
LangMem	21.81	19.21	27.52	29.88	12.00	49.02	14.95	24.94	7.46

Table 4: Question-answering performance of methods on MobileMem-Omni . Scores for task columns are evaluated using LLM-as-a-Judge. Here, “Single.”, “Multi.”, “Know.”, “Temp.”, “Abst.”, “Pref.”, and “Vis.” denote single-hop, multi-hop, knowledge-update, temporal-reasoning, abstention, implicit-preference, and visual-reasoning questions, respectively. “Judge” denotes the overall LLM-Judge score, and F1 measures word overlap with the ground-truth answer.

Figure 9: Radar chart of event type LLM-Judge performance (%). Events are divided by interaction language: Chinese, English; and by event span: short-term, medium-term, long-term.

Figure 10: Illustrative examples from MobileMem-Omni , covering user interactions and multimodal scenarios with diverse image types.

Agent Memory

Agent memory lets AI systems retain, update, and use facts, experiences, and skills across sessions and long-horizon tasks.

See all 180 papers

DynamicMem: A Long-Horizon Memory Benchmark in Real-World Settings
Jun 22, 2026Wenya Xie, Shengming Zhou, Zelin Li +9Agent Memory LLM Agent Memory
University of Minnesota · University of North Carolina at Charlotte · Johns Hopkins University +2
AdaMem: Learning What to Remember with Adaptive Memory Policies for Personalized Agents
Jun 19, 2026Xingyu Chen, Rui Wang, Zhaopeng Tu +1LLM Agent Memory Persistent Memory for Language Models
Shanghai Jiao Tong University, Tencent · Shanghai Jiao Tong University · Tencent
LongMemEval-V2: Evaluating Long-Term Agent Memory Toward Experienced Colleagues
May 12, 2026Di Wu, Zixiang Ji, Asmi Kawatkar +4Computer-Use Agent Benchmarks Retrieval-Augmented Generation
University of California, Los Angeles

Jun 22, 2026J/K move · Enter open · S save

DynamicMem: A Long-Horizon Memory Benchmark in Real-World Settings

Wenya Xie, Shengming Zhou, Zelin Li, Pouya Parsa+8

University of Minnesota · University of North Carolina at Charlotte · Johns Hopkins University+2

Agent Memory LLM Agent Memory Persistent Memory for Language Models Long-Horizon Agent Evaluation Memory Agent Benchmarks User Modeling+1

Open paper AskPDF arXiv

LLM agents increasingly act as personal assistants that must remember a user's profile over months: who they are (attributes), what they routinely do (habits), and what they prefer (preferences), and keep it updated as jobs, routines, and tastes drift. Existing benchmarks evaluate this "memory" ability through short, simplified interactions, missing three core properties of real behavior: the profile is heterogeneous, with attributes, habits, and preferences evolving on different timelines; changes are driven by external context such as seasons and life events; and evidence is rarely stated explicitly, instead scattered across many small actions in different apps that a memory system must infer from. We introduce DynamicMem, a synthetic benchmark that constructs 15 months of activity per user, providing long-term multi-app data that real users' privacy keeps out of reach. It provides user-consistent trajectories averaging 2.2M tokens and 1,772 grounded events per user across 16 applications such as e-commerce, fitness, and social platforms. The profile evolves over this period and is never given explicitly: each attribute, habit, or preference must be inferred from small signals scattered across apps. We evaluate at five quarterly checkpoints to track how systems scale as history grows. Benchmarking five representative systems exposes problems a single accuracy score hides: (i) profile reconstruction degrades with history length while service-task accuracy stays flat, despite both drawing on the same memory; (ii) no system both keeps facts that stay true and replaces facts that change, with errors clustering on preferences and on naming the exact referent; and (iii) over 93% of failures trace to what the memory retrieves, not to the model writing the answer, so the largest room for improvement lies in memory itself. Code: https://wenyaxie023.github.io/DynamicMem/

Abstract

Agent Memory

Explore similar work

DynamicMem: A Long-Horizon Memory Benchmark in Real-World Settings

AdaMem: Learning What to Remember with Adaptive Memory Policies for Personalized Agents

LongMemEval-V2: Evaluating Long-Term Agent Memory Toward Experienced Colleagues