cs.AIFeb 26, 2026

Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directions

Authors: Yue Xu, Qian Chen, Zizhan Ma, Dongrui Liu, Wenxuan Wang, Xiting Wang, Li Xiong, Wenjie Wang

Organizations: ShanghaiTech University, China · Tongji University, China · The Chinese University of Hong Kong, China · Shanghai Artificial Intelligence Laboratory, China · 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Large language models have enabled agentic systems that reason, plan, and interact with tools and environments to accomplish complex tasks. As these agents operate over extended interaction horizons, their effectiveness increasingly depends on adapting behavior to individual users and maintaining continuity across interactions, giving rise to personalized LLM-powered agents (PLAs). In such long-term, user-dependent settings, personalization permeates the entire decision pipeline rather than remaining confined to surface-level response generation. This survey provides a capability-oriented review of personalized LLM-powered agents. Existing work is organized around four interdependent capabilities: profile modeling, memory, planning, and action execution. Using this taxonomy, representative methods are synthesized and analyzed to illustrate how user signals are represented, propagated, and utilized across the agent pipeline, highlighting cross-component interactions and recurring design challenges. Evaluation metrics and benchmarking paradigms tailored to personalized agents are further examined, along with application scenarios ranging from conversational assistants to domain-specific expert systems. By clarifying the design space of personalization in agent systems, this survey provides a structured foundation for developing more user-aligned, adaptive, and deployable LLM-powered agents.

Figures & tables

Figure 1. Overview of personalized LLM-powered agents. Upon receiving a user request, the agent coordinates profile modeling, memory, planning, and action execution to generate a tailored response. Interaction outcomes provide feedback that refines user preference representations, enabling iterative and long-term personalization.
Figure 2. User-specific data in personalization process.
Figure 3. Example of a two-dimensional taxonomy of user preferences. Preferences are categorized by their expression form (explicit vs. implicit) and semantic function (behavioral vs. topical), illustrated through a multi-turn recommendation scenario.
CapabilityTypical InputsTemporal ScopePrimary Objective
Profile ModelingUser attributes, behavioral history, and user instructionsLifelongEstablish user understanding and support agent-user alignment
MemoryInteraction history, user events, contextual recordsTurn to lifelongMaintain temporal continuity and consistent personalization
PlanningProfile information, memory, task context, in-task feedbackTurn to taskEnable personalized reasoning and decision making
Action ExecutionPlans, tool states, execution feedbackStep to taskRealize personalized actions and outcomes
Table 1. Comparison of the four core capabilities in personalized LLM-powered agents.
Figure 5. Overview of evaluation for personalized LLM-powered agents. Evaluation is organized along three layers: (1) evaluation goals and metric dimensions, including effectiveness, adaptivity, generalization, robustness, and risk; (2) assessment paradigms, such as automatic scoring, rule-based checking, learned evaluators, and LLM-as-a-judge; and (3) representative benchmark families, including interactive alignment and user-substitution settings.
GoalMetricDescription
 Discovery Accuracy ( Li et al., 2025h ; Jiang et al., 2024a )Measures whether the agent can correctly discover user preferences or intents from the interaction.
 Knowledge Integration Score ( Au et al., 2025 ; Wang et al., 2023a )Measures the ability of the model to incorporate user-specific knowledge or persona information into generated outputs.
 Preference Alignment ( Zhao et al., 2025a ; Wu et al., 2025f )Measures the alignment between the agent’s output and user preferences or stated constraints.
EffectivenessPreference-Aware Planning Accuracy ( Xu et al., 2025b ; Xu et al., 2025c )Measures whether multi-step plans or tool-use sequences conform to individual user preferences.
 Preference Lift ( Li et al., 2025h ; Hao et al., 2025 ; Wu et al., 2024c )Compares personalized outputs against a non-personalized baseline on the same prompt, reporting the lift in preference score.
 Consistency Score ( Mukhopadhyay et al., 2025 )Measures the consistency of the agent’s textual style and expressed personality across the whole conversation.
Table 2. Evaluation metrics for personalized LLM-powered agents.
BenchmarkScaleTaskPref.GoalEval.Metrics
Interactive Alignment Benchmarks
IndieValueCatalog ( Jiang et al., 2024a )800Human ValueInferredEffectivenessAuto.Accuracy
PersonaBench ( Tan et al., 2025a )582GeneralInferredEffectivenessAuto.Recall, F1
PrefDisco ( Li et al., 2025h )10,000GeneralInteractiveEffectiveness, AdaptivityAuto., Rule-basedDiscovery Accuracy, Preference Alignment, Interaction Efficiency, Correctness
PersonalLLM ( Wu et al., 2024c )1000GeneralGiven, InferredEffectivenessLLM-J, LLM-EPersonalized Reward
ALOE ( Wu et al., 2024c )100GeneralInferredEffectivenessLLM-JAlignment Level, Improvement Rate
Table 3. Summary of Personalized Benchmark.
More in this topic

LLM Agents

Large language model agents use language models to plan, use tools, and act across multi-step tasks and environments.

See all 705 papers

Explore similar work

CardsList
Jul 3, 2026

APeB: Benchmarking Personalization Ability of Large Language Model Agents

LLM-powered agents struggle with personalization when users issue raw, underspecified queries. In this setting, agents must infer latent intent, extract preferences from noisy interaction histories, and select among competing alternatives. Existing benchmarks rarely test this capability, as they often rely on user-refined queries or simplified histories. We introduce personalized product search (PPS), a testbed for agentic personalization under raw queries and diverse histories. We construct Agent Personalized Benchmark (APeB) from action logs, pairing underspecified intents with rich histories and user-viewed candidate items. Evaluating state-of-the-art LLMs with multi-step agent workflows, we find that models handle explicit queries well but struggle with early-stage queries requiring intent and preference discovery. Rubric analysis attributes this gap mainly to ineffective history use. A simple history-aware query-refinement pipeline, VQRA, yields consistent gains, highlighting the need for dedicated history-utilization modules in personalized agents.

Garry Yang, Zizhe Chen, Xinru Chen +9

1The Chinese University of Hong Kong · 2ByteDance

LLM PersonalizationLLM Agent EvaluationAI Agent BenchmarksInformation RetrievalUser Preference Modeling
AskPDF
Sep 8, 2026

Graph-Based Personalized Memory for LLM Agents: Representation, Evolution, Retrieval, and Evaluation

Large Language Model (LLM) agents are evolving from single-session tools toward long-term personal assistants that must adapt to individual users across tasks, contexts, and interactions. This shift makes memory a core requirement for personalization, since user preferences, goals, constraints, relationships, and past experiences are accumulated gradually and often change over time. Graph-based personalized memory provides a structured way to model such user information through explicit relations, temporal context, and evidence links. Such representations can model not only what an agent remembers about a user but also how memories are connected, revised, and retrieved to support personalized decisions. However, existing work remains fragmented across personalized agents and generic graph memory frameworks, making it difficult to understand the design space as a whole. This survey develops a lifecycle-oriented view of graph-based personalized memory for LLM agents. We organize existing studies around memory representation, memory evolution, memory retrieval, and memory evaluation. We further compare key design choices, discuss current evaluation practices, and open challenges in building reliable long-term personalized agents. This survey aims to clarify how graph-based memory can support adaptive, controllable, and user-centric LLM agents.

Dac Duy Anh Nguyen, Zhangchi Qiu, Shigeng Chen +1

School of ICT Griffith University Brisbane, Australia · School of ICT Griffith University Gold Coast, Australia

LLM Agent MemoryGraph-Based Agent MemoryAgent Memory Management
AskPDF
Aug 3, 2026

From Profiling to Synthesis: Benchmarking Implicit Behavioral Alignment in Personalized LLM Agents

Large Language Models have enabled increasingly capable autonomous agents, yet personalization remains critical for making such agents practically useful. Recent benchmarks have begun evaluating personalization in agents, but they largely rely on static preference snapshots, fixed interaction logs, or question answering over predefined user profiles. Such designs fail to capture the complexity of evolving user preferences and neglect preference-conditioned task execution-a discrepancy we term as the knowledge-to-action gap. To address this challenge, we introduce IBA-Bench, a benchmark for implicit behavioral alignment constructed from longitudinal interaction histories that contain noise, implicit cues, and temporal inconsistencies. Unlike prior work, IBA-Bench evaluates whether an agent can execute tasks while satisfying implicit user constraints inferred from historical interactions. We further propose IBA-Agent, an agent framework that reconciles conflicting priorities through broad retrieval and trajectory-level alignment. Experiment results on IBA-Bench show that effective personalization remains a significant challenge for state-of-the-art LLM agents, and the proposed IBA-Agent substantially improves behavioral alignment in complex scenarios across nine application domains.

Jiajia Song, Bobo Li, Haiwen Yi +6

National University of Singapore · University of Toronto · University of Minnesota Twin Cities +2

LLM PersonalizationLLM Agent EvaluationAI Agent Benchmarks
AskPDF