Do Personality-Tuned LLMs Make Better Social Agents?

Prior work has shown that large language models can express diverse personality traits in open-ended text generation, but it is unclear whether they can do so in goal-directed dialogue without harming task completion, and whether adapting to a user's personality improves the interaction. We study these questions in task-oriented dialogue (TOD) with a training-free framework in which a personality-conditioned user agent interacts with a system agent under three conditions: Neutral (no personality information), Try (adapt to cues observed in the dialogue), and Oracle (the user's trait label is given explicitly). Across GPT-4o, Qwen3-Next-80B, and Gemini 2.0 Flash, two Schema-Guided Dialogue domains, and ten Big Five poles (9,000 dialogues), each scored by two independent LLM judges, user agents express most traits while task performance remains high (89.94% completion, 85.27% success). The exceptions are introversive and closed users: these traits manifest mainly as absent behavior and are rarely detected. Both adaptive conditions improve pairwise user satisfaction and lower truthfulness, revealing a personalization-grounding trade-off, but only cue-responsive Try also improves constraint satisfaction and inform rate. How adaptation is implemented matters: Oracle's gains depend on how visibly the trait is expressed, and for disagreeable users the label leads Qwen3 and Gemini to mirror hostility. Try avoids this failure, its satisfaction gains do not depend on trait visibility, and it is preferred over Oracle overall. A 175-pair human study supports the reliability of the LLM judges: on clear-winner cases, annotators agree with the satisfaction judges more often than with each other (76.3% vs. 65.1% agreement). Our results suggest that responding to observable behavior is a more robust basis for social adaptation than conditioning on psychological labels.

Large language models (LLMs) increasingly serve as interactive social agents, yet their ability to maintain coherent and authentic persona-level role-playing remains limited, particularly in realistic social scenarios. Existing research predominantly focuses on character-level settings and relies on static evaluation formats, failing to capture the complexity of everyday social interactions. In this work, we present PersonaArena, a dynamic simulation framework for evaluating and improving persona-level role-playing in LLMs. PersonaArena leverages a large, filtered corpus of user-generated social content to construct a nuanced persona bank, and elicits multi-turn, context-rich interactions within simulated social environments. Our framework features a multi-agent debating judge for holistic and unbiased assessment. Through extensive experiments, we demonstrate that PersonaArena enables rigorous evaluation and enhancement of LLMs' role-playing capabilities, advancing the development of more authentic and socially adept AI agents.

Large Language Model (LLM) agents are increasingly deployed in settings where they interact with diverse users, including those who are unclear, impatient, or reluctant to share information. However, collecting real interaction data at scale remains expensive. The field has turned to LLM-based user simulators as stand-ins, but these simulators inherit the behavior of their underlying models: cooperative and homogeneous. As a result, agents that appear strong in simulation often fail in real human interactions. To narrow this gap, we introduce Persona Policies (PPol), a plug-and-play control layer that induces realistic behavioral variation in user simulators while preserving original task goals. Rather than hand-crafting personas, we employ an evolutionary coding agent to discover persona generation programs optimized for human-likeness and behavioral coverage over real user conversations. The evolved program generates diverse, human-like personas for any task in the domain. Across 4 benchmarks--including $τ^2$-bench Retail and Airline, ColBench, and WildChat--evolved PPol yield 28-72% absolute gains in fitness score over the baseline simulator. In blinded evaluations, annotators judged PPol users as 'human' 80.4% of the time, nearly 2x more than the baseline simulators. Training agents with PPol also improves real-world performance: our user study with live human-agent interactions showed that fine-tuning with our method boosted task success by +23% over default baselines. PPol thus offers a novel approach to strengthen simulator-based evaluation and training without changing underlying tasks.