Personalization requires language models to capture user-specific knowledge from a growing user history. Existing context-based approaches incur increasing inference costs as user history accumulates and rely on separate retrieval or summarization stages, while parametric-based approaches often require reconstructing user representations when new user data is added. We introduce ProTTT, a profile-supervised meta-learning framework for learning semantic user memory. The memory construction starts from a shared initialization and is updated for each user through test-time training on user history, allowing it to evolve continuously as the history grows. However, since test-time training alone does not explicitly encourage the memory to capture semantic user knowledge necessary for personalization, we learn this shared initialization using textual user profiles as supervision, so that test-time training on user history captures semantic knowledge more effectively. ProTTT consistently outperforms both full history ICL and all parametric baselines across diverse benchmarks, while substantially reducing inference cost by compressing user history into a lightweight parameterized memory. Our analysis also shows that profile supervision is a reliable objective for learning semantic user knowledge and that the resulting memory can track and retain evolving user preferences, while remaining robust across different history sizes. Overall, we demonstrate the effectiveness of test-time training for personalization and establish ProTTT as a baseline for continuously evolving user memory.
Figures & tables
Figure 1: Overview of ProTTT . (a) During training, a shared initialization ϕs is adapted to each user through next-token prediction on the user history, producing user-specific memory ϕu . The adapted memory is then evaluated on a semantic profile task, whose loss is backpropagated through the adaptation process to update ϕs . (b) At inference time, the learned initialization is adapted using only the target user’s history and then frozen for personalized generation.
Figure 2: User memory architecture. A low-rank adapter reads the hidden state and adds its output to the residual stream.
Personalization
Efficiency on CMV
Method
CMV AUC ↑
LaMP-3 MAE ↓
LaMP-3 RMSE ↓
PMem T6 ↑
PMem T7 ↑
History Proc. (s/user) ↓
Prompt (tokens/query) ↓
Latency (ms/query) ↓
Base
.466
2.103
2.602
.273
.088
–
748
15.3
In-Context Baselines
Full History
.475
.892
1.569
.273
.140
–
18.9K
273.5
RAG (Recent-5)
.456
1.800
2.434
.418
.140
–
2.2K
20.0
RAG (BM25)
.436
1.763
2.405
.491
.246
–
4.3K
39.2
Table 1: Personalization performance and efficiency. w/o user denotes the corresponding method without target-user information. For ProTTT , this denotes the shared initialization before constructing memory from the target user’s history; δ -Mem uses an empty memory state, whereas P2P uses an empty user profile and history. Bold indicates the best personalization result in each column. † PAG profiles are generated offline using GPT-5-mini through an external API; profile-construction latency is therefore excluded from the H200 measurements.
Figure 3: Effect of the meta-learning objective on TTT. The figure reports downstream task performance changes before and after user memory construction for each meta-learning objective. Bold values show the signed change, and smaller values show the corresponding raw scores. Positive values indicate improvement.
Figure 4: Continual acquisition and retention of semantic user memory. (a) Mean token log-probability of each gold user profile as profile-associated histories are sequentially incorporated. (b) Change in the log-probability of Profile A as additional user histories are added, comparing ProTTT with full history ICL. Smaller degradation indicates better retention of previously acquired user information.
Figure 5: Personalization and general capability as user history accumulates. The plots show the scores after constructing user memory from CMV user histories. Curves compare ProTTT under different memory readout scales against full history ICL as the number of history items k increases. Horizontal lines denote the base model and the shared initialization before TTT. Full history ICL is reported only up to k=100 because of the context limit.
(a) General capability and generation quality
(b) Claims in generated user profiles
Method
MMLU Acc. ↑
Non- term. ↓
Seq.- rep.-2 ↓
Distinct-2 ↑
Self- BLEU ↓
Claims /gen.
Sup.
Gen.
Unsup.
Contr.
Base
.474
.000
.057
.605
.359
6.22
1.67
2.27
2.24
.04
User-specific LoRA
.397
.740
.417
.246
.463
5.11
1.24
1.49
2.13
.24
P2P
.390
.006
.064
.671
.255
3.29
.91
.13
1.89
.36
δ -Mem
.474
.003
.123
.563
.309
9.87
2.80
2.09
4.53
.44
ProTTT , α=.75
.457
.009
.092
.605
.277
8.04
5.69
1.02
1.22
.11
Table 2: General capability, generation quality, and user-information recall. (a) Reports 5-shot MMLU accuracy after processing k=100 history items and generation metrics on 40 WildChat prompts. (b) Reports the mean number of claims per generated user profile. Sup., Gen., Unsup., and Contr. denote supported, generic, unsupported, and contradicted claims, respectively.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Source
User history
Meta-learning target
# examples
ALPSBench
User–assistant session dialogues
Human-verified user-memory annotations
3,033
CMV
User-authored posts and comments
Generated semantic user profiles
1,321
Total
4,354
Appendix
Table 3: Composition of the meta-learning data.
Hyperparameter
Value
Backbone
Qwen3-0.6B
Memory rank
18
History chunk size
2,048 tokens
TTT optimizer
SGD
TTT learning rate
0.1
TTT gradient clipping
1.0
Appendix
Table 4: Hyperparameters used to train the main model.
Method
Track Evolution (139)
Shared Facts (129)
Update Reasons (99)
New Ideas (93)
Mentioned Facts (17)
T6 Recommend (55)
T7 Generalize (57)
All (589)
Base
.273
.287
.444
.183
.353
.273
.088
.275
In-Context Baselines
Full History
.230
.287
.313
.172
.294
.273
.140
.244
RAG (Recent-5)
.295
.318
.525
.151
.235
.418
.140
.311
RAG (BM25)
.259
.326
.455
.140
.118
.491
.246
.304
PAG
.201
.279
.394
.151
.235
.309
.123
.246
Appendix
Table 5: PersonaMem accuracy across all released question categories. Before and after denote performance without and with target-user information, respectively. For ProTTT , the two after-TTT rows differ only in the memory readout scale. Bold indicates the highest point estimate in each column; ties are all bolded. The number of questions is shown in parentheses.
Method
User-specific parameters
Training settings
User-Specific LoRA
Rank 16 , scale 32 , attention and feed-forward projections
AdamW, lr 10−3 , one epoch
P2P
Hypernetwork-generated rank- 8 LoRA on query and value projections
AdamW, lr 2×10−5 , profile-supervised training
δ -Mem
Rank- 8 token-wise memory on query and output projections
AdamW, lr 2×10−4 , one QASPER epoch
Appendix
Table 6: Configurations of the parametric baselines.
Method
Prompt Tokens mean / max
KV Cache (MiB) mean / max
Peak Memory (MiB)
Latency (ms) median / mean
GFLOPs per query
Base
748 / 1,583
82 / 173
1,383
15.3 / 17.3
732
Full History
18,919 / 29,544
2,069 / 3,231
5,166
273.5 / 218.7
63.6K
RAG (Recent-5)
2,223 / 7,023
243 / 768
2,119
20.0 / 23.7
2.7K
RAG (BM25)
4,255 / 14,189
465 / 1,552
3,088
39.2 / 49.2
6.9K
PAG
1,066 / 1,931
117 / 211
1,430
15.9 / 18.0
1.1K
PAG + Full History
19,215 / 29,927
2,102 / 3,273
5,217
278.1 / 222.0
65.3K
Appendix
Table 7: Query-time cost on CMV. Prompt length and KV-cache size are reported as mean/max, and latency as median/mean. P2P latency includes query-specific adapter generation. For P2P, the second peak-memory value corresponds to adapter generation and is measured with nvidia-smi because it runs in a separate process.
Method
User Representation
Construction
Time (s) mean / max
Peak Memory (MiB)
Full History
Raw history (17.8K tokens)
–
–
–
RAG (Recent-5)
Five most recent entries
–
–
–
RAG (BM25)
Raw history
–
–
–
PAG
268-token profile
Generate profile (GPT-5-mini)
n/a
n/a
PAG + Full History
Raw history + 268-token profile
Generate profile (GPT-5-mini)
n/a
n/a
User-Specific LoRA
10.09M parameters
One NTP epoch
11.74 / 13.05
17,450
Appendix
Table 8: User representations on CMV and the cost of constructing them. Time is reported as mean/max per user. PAG profiles are generated through the external GPT-5-mini API, so their construction time and GPU memory are not included. P2P adapter generation is performed for each query and is included in the query latency reported in Table 7 ; its profile-generation peak memory is measured using nvidia-smi .
Method
History Proc. (s/user) ↓
Prompt Tokens /query ↓
Latency (ms/query) ↓
LaMP-3
Base
–
170
14.2
Full History
–
12.5K
90.3
RAG (Recent-5)
–
750
14.9
RAG (BM25)
–
1.17K
18.6
PAG
n/a
404
14.9
Appendix
Table 9: Efficiency on LaMP-3 and PersonaMem. History processing is the mean one-time cost per user for LaMP-3 or memory group for PersonaMem. PAG profiles are generated through the external GPT-5-mini API, so their construction time is not reported. For P2P, history processing includes profile generation only; query-specific adapter generation is included in query latency. Prompt length is averaged over queries, and latency is the median.
Objective
Training prompt
Target
Profile prediction
Here is a new post from this user: [POST]. What do you remember about this user that would help persuade them on this post? List each fact you remember, one per line.
Bullet-point profile written by GPT-5-mini from the post and the user’s history.
Persuasion judgment
Here is a new post from this user: [POST]. And here is a response to it: [RESPONSE]. Would this response change this user’s view? Answer with just “yes” or “no”.
yes for a persuasive reply, no otherwise.
Response selection
Here is a new post from this user: [POST]. Here are four responses to it: [OPTIONS]. Which response would change this user’s view? Answer with just the number.
Number of the persuasive reply.
Response generation
Here is a new post from this user: [POST]. Write the response that would most likely change this user’s view.
One persuasive reply to the post.
Appendix
Table 10: Prompts and training targets of the four meta-learning tasks. [POST] is the user’s new post, [RESPONSE] is one reply to it, and [OPTIONS] lists the four candidate replies, numbered (1)–(4).
Personalization
Efficiency on CMV
Method
CMV AUC ↑
LaMP-3 MAE ↓
LaMP-3 RMSE ↓
PMem T6 ↑
PMem T7 ↑
History Proc. (s/user) ↓
Prompt (tokens/query) ↓
Latency (ms/query) ↓
Base
.598
1.889
2.347
.255
.088
–
763
8.5
In-Context Baselines
Full History
.515
1.745
2.146
.255
.088
–
18.8K
220.7
RAG (Recent-5)
.549
1.711
2.107
.255
.088
–
2.2K
14.6
RAG (BM25)
.555
1.709
2.066
.255
.088
–
4.3K
32.6
Appendix
Table 11: Personalization performance and efficiency with Llama-3.2-1B-Instruct. w/o user denotes the shared initialization before constructing memory from the target user’s history. Efficiency is measured on CMV using a single NVIDIA H200 NVL. History processing is a one-time per-user cost, while prompt length and latency are measured per query. † PAG profiles are generated offline using GPT-5-mini through an external API; profile-construction latency is therefore excluded from the H200 measurements. Bold indicates the best personalization result for each metric.
(a) General capability and generation quality
(b) Claims in generated user profiles
Method
MMLU Acc. ↑
Non- term. ↓
Seq.- rep.-2 ↓
Distinct-2 ↑
Self- BLEU ↓
Claims /gen.
Sup.
Gen.
Unsup.
Contr.
Base
.455
.000
.071
.609
.304
8.88
3.93
2.13
2.62
.20
ProTTT , α=.5
.405
.014
.068
.642
.263
14.85
6.85
2.38
5.27
.35
ProTTT , α=1
.259
.059
.143
.487
.289
11.98
5.15
.67
5.77
.40
Appendix
Table 12: General capability, generation quality, and user-information recall with Llama-3.2-1B-Instruct. (a) Reports 5-shot MMLU accuracy after processing 100 history items and generation metrics on 40 WildChat prompts. (b) Reports the mean number of claims per generated user profile. Sup., Gen., Unsup., and Contr. denote supported, generic, unsupported, and contradicted claims, respectively.
Figure 6: Continual acquisition and retention with Llama-3.2-1B-Instruct. (a) Change in the mean token log-probability of each gold user profile as the corresponding histories are sequentially incorporated. (b) Change in the mean token log-probability of Profile A as seven later histories are incorporated, comparing ProTTT with full history ICL. Smaller degradation indicates better retention of previously acquired user knowledge.
Figure 7: Effect of the memory readout scale across backbone models. Panels (a)–(c) show personalization performance on CMV, LaMP-3, and PersonaMem Types 6 and 7. Panel (d) shows 5-shot MMLU accuracy. Panels (e)–(h) show generation quality on WildChat. Solid lines represent ProTTT after user memory has been constructed; dashed lines represent the corresponding base model without user memory. Memory construction is identical across readout scales.
Personalizing large language models (LLMs) requires encoding long-term, user-specific behavioral patterns in a way that is computationally efficient, scalable, and compatible with a frozen base model. We present Latent Personal Memory (LPM), a scalable framework that represents user-specific history as a compact, persistent matrix of N latent slots, that are interpretable. A shared cross-attention projection network maps these slots into dynamic, input-conditioned soft prompts that are prepended to the input of a frozen LLM. We evaluate LPM on PersonaMem v1 and LoCOMO benchmarks across Qwen3-1.7B, 4B, and 8B backbones. Results demonstrate that LPM outperforms LoRA and Prompt Tuning by up to 8.8% and 54.4% in overall accuracy respectively on PersonaMem v1, while reducing KV-cache usage by over 64x. On LoCoMo, LPM matches LoRA accuracy with 120x fewer trainable parameters. We also show that the efficiency of LPM grows with context length and outperforms full-context at 128K context length.
Existing large language model (LLM) based memory systems apply universal, static policies that overlook a fundamental reality: the contexts that are worth storing in memory are different across users. This misalignment wastes limited memory budget on transient interactions while failing to preserve critical context for long horizon tasks. To address this gap, we investigate an underexplored question: can LLM based memory systems learn personalized memory policies? We introduce PerMemBench, the first benchmark for evaluating personalized memory systems, featuring multi year, multi domain interaction histories across diverse user personas. We further present the first empirical study of memory personalization, proposing session level storage gating, a lightweight framework that selectively bypasses memory operations for transient sessions. Our study confirms that personalization yields substantial retention gains under perfect gating, yet reveals that accurate gating remains an open and critical challenge.
Retrieval-augmented personalization enables large language models to produce more accurate and preference-aligned outputs using relevant records retrieved from user histories. Personalized language models typically prepend a fixed number of retrieved user records, even when additional history is redundant, harmful, or unrelated to a user's distinctive behavior. We study minimal sufficient personalization: constructing the least costly ordered profile for each input while preserving the utility achievable from a retrieved candidate pool. We introduce ENOUGH, a method that iteratively appends behavioral records or emits STOP to construct profiles with adaptive lengths. Offline, bounded counterfactual search evaluates profile prefixes by jointly considering downstream gains, user specificity, and token costs. The resulting long-horizon targets are distilled into a multi-head value controller with explicit ranking and stopping supervision. At inference, the controller selects and orders records through lightweight decisions, and the frozen generator is invoked once after stopping. Extensive experiments on six personalized tasks demonstrate that ENOUGH consistently outperforms strong heuristic and retrieval-augmented baselines in both effectiveness and efficiency, achieving minimal sufficient profiles that preserve personalization utility while reducing unnecessary context costs.
Minghang Liu, Qiang Qiu, Yuanzhuo Wang +2
State Key Laboratory of AI Safety, Institute of Computing Technology, CAS · University of Chinese Academy of Sciences, Beijing, China