Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directions

Yue Xu; Qian Chen; Zizhan Ma; Dongrui Liu; Wenxuan Wang; Xiting Wang; Li Xiong; Wenjie Wang

cs.AIFeb 26, 2026

Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directions

Authors: Yue Xu, Qian Chen, Zizhan Ma, Dongrui Liu, Wenxuan Wang, Xiting Wang, Li Xiong, Wenjie Wang

Organizations: ShanghaiTech University, China · Tongji University, China · The Chinese University of Hong Kong, China · Shanghai Artificial Intelligence Laboratory, China ·

Abstract

Large language models have enabled agentic systems that reason, plan, and interact with tools and environments to accomplish complex tasks. As these agents operate over extended interaction horizons, their effectiveness increasingly depends on adapting behavior to individual users and maintaining continuity across interactions, giving rise to personalized LLM-powered agents (PLAs). In such long-term, user-dependent settings, personalization permeates the entire decision pipeline rather than remaining confined to surface-level response generation. This survey provides a capability-oriented review of personalized LLM-powered agents. Existing work is organized around four interdependent capabilities: profile modeling, memory, planning, and action execution. Using this taxonomy, representative methods are synthesized and analyzed to illustrate how user signals are represented, propagated, and utilized across the agent pipeline, highlighting cross-component interactions and recurring design challenges. Evaluation metrics and benchmarking paradigms tailored to personalized agents are further examined, along with application scenarios ranging from conversational assistants to domain-specific expert systems. By clarifying the design space of personalization in agent systems, this survey provides a structured foundation for developing more user-aligned, adaptive, and deployable LLM-powered agents.

Figures & tables

Figure 1. Overview of personalized LLM-powered agents. Upon receiving a user request, the agent coordinates profile modeling, memory, planning, and action execution to generate a tailored response. Interaction outcomes provide feedback that refines user preference representations, enabling iterative and long-term personalization.

Figure 2. User-specific data in personalization process.

Figure 3. Example of a two-dimensional taxonomy of user preferences. Preferences are categorized by their expression form (explicit vs. implicit) and semantic function (behavioral vs. topical), illustrated through a multi-turn recommendation scenario.

Capability	Typical Inputs	Temporal Scope	Primary Objective
Profile Modeling	User attributes, behavioral history, and user instructions	Lifelong	Establish user understanding and support agent-user alignment
Memory	Interaction history, user events, contextual records	Turn to lifelong	Maintain temporal continuity and consistent personalization
Planning	Profile information, memory, task context, in-task feedback	Turn to task	Enable personalized reasoning and decision making
Action Execution	Plans, tool states, execution feedback	Step to task	Realize personalized actions and outcomes

Table 1. Comparison of the four core capabilities in personalized LLM-powered agents.

Figure 5. Overview of evaluation for personalized LLM-powered agents. Evaluation is organized along three layers: (1) evaluation goals and metric dimensions, including effectiveness, adaptivity, generalization, robustness, and risk; (2) assessment paradigms, such as automatic scoring, rule-based checking, learned evaluators, and LLM-as-a-judge; and (3) representative benchmark families, including interactive alignment and user-substitution settings.

Goal	Metric	Description
	Discovery Accuracy ( Li et al., 2025h ; Jiang et al., 2024a )	Measures whether the agent can correctly discover user preferences or intents from the interaction.
	Knowledge Integration Score ( Au et al., 2025 ; Wang et al., 2023a )	Measures the ability of the model to incorporate user-specific knowledge or persona information into generated outputs.
	Preference Alignment ( Zhao et al., 2025a ; Wu et al., 2025f )	Measures the alignment between the agent’s output and user preferences or stated constraints.
Effectiveness	Preference-Aware Planning Accuracy ( Xu et al., 2025b ; Xu et al., 2025c )	Measures whether multi-step plans or tool-use sequences conform to individual user preferences.
	Preference Lift ( Li et al., 2025h ; Hao et al., 2025 ; Wu et al., 2024c )	Compares personalized outputs against a non-personalized baseline on the same prompt, reporting the lift in preference score.
	Consistency Score ( Mukhopadhyay et al., 2025 )	Measures the consistency of the agent’s textual style and expressed personality across the whole conversation.

Table 2. Evaluation metrics for personalized LLM-powered agents.

Benchmark	Scale	Task	Pref.	Goal	Eval.	Metrics
Interactive Alignment Benchmarks
IndieValueCatalog ( Jiang et al., 2024a )	800	Human Value	Inferred	Effectiveness	Auto.	Accuracy
PersonaBench ( Tan et al., 2025a )	582	General	Inferred	Effectiveness	Auto.	Recall, F1
PrefDisco ( Li et al., 2025h )	10,000	General	Interactive	Effectiveness, Adaptivity	Auto., Rule-based	Discovery Accuracy, Preference Alignment, Interaction Efficiency, Correctness
PersonalLLM ( Wu et al., 2024c )	1000	General	Given, Inferred	Effectiveness	LLM-J, LLM-E	Personalized Reward
ALOE ( Wu et al., 2024c )	100	General	Inferred	Effectiveness	LLM-J	Alignment Level, Improvement Rate

Table 3. Summary of Personalized Benchmark.

LLM Agents

Large language model agents use language models to plan, use tools, and act across multi-step tasks and environments.

See all 705 papers

APeB: Benchmarking Personalization Ability of Large Language Model Agents
Jul 3, 2026Garry Yang, Zizhe Chen, Xinru Chen +9LLM Personalization LLM Agent Evaluation
1The Chinese University of Hong Kong · 2ByteDance
Graph-Based Personalized Memory for LLM Agents: Representation, Evolution, Retrieval, and Evaluation
Sep 8, 2026Dac Duy Anh Nguyen, Zhangchi Qiu, Shigeng Chen +1LLM Agent Memory Graph-Based Agent Memory
School of ICT Griffith University Brisbane, Australia · School of ICT Griffith University Gold Coast, Australia
From Profiling to Synthesis: Benchmarking Implicit Behavioral Alignment in Personalized LLM Agents
Aug 3, 2026Jiajia Song, Bobo Li, Haiwen Yi +6LLM Personalization LLM Agent Evaluation
National University of Singapore · University of Toronto · University of Minnesota Twin Cities +2

Jul 3, 2026J/K move · Enter open · S save

APeB: Benchmarking Personalization Ability of Large Language Model Agents

Garry Yang, Zizhe Chen, Xinru Chen, Yongqiang Chen+8

1The Chinese University of Hong Kong · 2ByteDance

LLM Personalization LLM Agent Evaluation AI Agent Benchmarks Information Retrieval User Preference Modeling

Open paper AskPDF arXiv

LLM-powered agents struggle with personalization when users issue raw, underspecified queries. In this setting, agents must infer latent intent, extract preferences from noisy interaction histories, and select among competing alternatives. Existing benchmarks rarely test this capability, as they often rely on user-refined queries or simplified histories. We introduce personalized product search (PPS), a testbed for agentic personalization under raw queries and diverse histories. We construct Agent Personalized Benchmark (APeB) from action logs, pairing underspecified intents with rich histories and user-viewed candidate items. Evaluating state-of-the-art LLMs with multi-step agent workflows, we find that models handle explicit queries well but struggle with early-stage queries requiring intent and preference discovery. Rubric analysis attributes this gap mainly to ineffective history use. A simple history-aware query-refinement pipeline, VQRA, yields consistent gains, highlighting the need for dedicated history-utilization modules in personalized agents.

Abstract

LLM Agents

Explore similar work

APeB: Benchmarking Personalization Ability of Large Language Model Agents

Graph-Based Personalized Memory for LLM Agents: Representation, Evolution, Retrieval, and Evaluation

From Profiling to Synthesis: Benchmarking Implicit Behavioral Alignment in Personalized LLM Agents