Role-Playing Agent Evaluation

Latest papers

12 papers2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

AgentMonopoly: Behavioral Patterns and Persona Control in LLM Agents
Oct 21, 2026Changshun Wu, Mengyu Li, Mathis Marsault +2LLM Agents LLM Agent Evaluation
University of Liverpool, Liverpool, United Kingdom · Carl von Ossietzky University of Oldenburg, Oldenburg, Germany · Chalmers University of Technology, Gothenburg, Sweden
Desirability Confirmation: Relational Sycophancy in AI Intimate Roleplay
Oct 16, 2026Huiqian Lai, Runze DingLLM Sycophancy AI Companions
Syracuse University · Swansea University
Adversarial Stress Testing of Role-Playing Language Agents using Multi-Agent Evaluation
Aug 4, 2026Saqib Shouqi, Abdullah Nazly, Januki Wanniarachchi +1Adversarial Attacks on LLMs LLM Agent Evaluation
School of Computing, Informatics Institute of Technology, Colombo, Sri Lanka
TRACE Bench: Task-driven Roleplay Agentic Checklist Evaluation
Jul 31, 2026Jiahui Zhang, Ziwei Zhang, Yipeng Wang +7Large Language Model-Based Role-Play Simulation Agent Evaluation
Kuaishou GameMind Lab
Beyond Borrowed Histories: Person-Aligned User Simulation for Interactive Role-Playing Evaluation
Jul 30, 2026Yuhang Zhu, Mingxuan Du, Benfeng Xu +3LLM Evaluation Large Language Model-Based Role-Play Simulation
University of Science and Technology of China · MetaStone · MetaStone Technology, Beijing, China
ArcANE: Do Role-Playing Language Agents Stay in Character at the Right Time?
Jun 4, 2026Woojung Song, Nalim Kim, Sangjun Song +3Large Language Model-Based Role-Play Simulation Personality Modeling in Language Models
Graduate School of Data Science, Seoul National University
RoleCDE:Benchmarking and Mitigating Role-Alignment Trade-offs in Role-Playing Agents
Jun 1, 2026Huayi Lai, Shichao Song, Simin Niu +5Moral Reasoning in Language Models AI Agent Benchmarks
School of Information, Renmin University of China, Beijing, China
DynSess: Dynamic Session-Level Evaluation and Optimization Framework for Role-Playing Agents
May 28, 2026Rongsheng Zhang, Jiji Tang, Junnan Ren +6Large Language Model-Based Role-Play Simulation Long-Horizon Agent Evaluation
1Zhejiang University · 2Fuxi AI Lab, NetEase Inc. · 3Xiamen University
RPA-Check: A Multi-Stage Automated Framework for Evaluating Dynamic LLM-based Role-Playing Agents
Apr 13, 2026Riccardo Rosati, Edoardo Colucci, Massimiliano Bolognini +2Role-Playing Agent Evaluation LLM Agent Evaluation
Department of Political Sciences, Communication and International Relations, University of Macerata, Via Don Minzoni 22/A, Macerata, 62100, Italy · Deparment of Information Engineering, Università Politecnica delle Marche, Via Brecce Bianche 12, Ancona, 60131, Italy · Department of Law, University of Macerata, Piaggia dell’Università 2, Macerata, 62100, Italy
Rethinking Role-Playing Evaluation: Anonymous Benchmarking and a Systematic Study of Personality Effects
Mar 4, 2026Ji-Lun Peng, Yun-Nung ChenPersonality Modeling in Language Models OOD Generalization
National Taiwan University, Taipei, Taiwan · Academia Sinica, Taipei, Taiwan
MERRY: Semantically Decoupled Evaluation of Multimodal Emotional and Role Consistencies of Role-Playing Agents
Feb 24, 2026Zhenyu Wang, Xiaofen Xing, Yirong Chen +1Role-Playing Agent Evaluation Multimodal Agent Evaluation
South China University of Technology, Guangzhou 510641, China · Foshan University, Foshan 528225, China
DEBATE: A Large-Scale Benchmark for Evaluating Opinion Dynamics in Role-Playing LLM Agents
Oct 29, 2025Yun-Shiuan Chuang, Ruixuan Tu, Chengtao Dai +9Opinion Dynamics Large Language Model-Based Social Simulation
University of Wisconsin–Madison · Google DeepMind · Stanford University

Oct 21, 2026J/K move · Enter open · S save

AgentMonopoly: Behavioral Patterns and Persona Control in LLM Agents

Changshun Wu, Mengyu Li, Mathis Marsault, Xiaowei Huang+1

University of Liverpool, Liverpool, United Kingdom · Carl von Ossietzky University of Oldenburg, Oldenburg, Germany · Chalmers University of Technology, Gothenburg, Sweden

LLM Agents LLM Agent Evaluation Role-Playing Agent Evaluation Game-Playing Agents Personality Modeling in Language Models

Open paper AskPDF arXiv

How do LLM agents' strategies differ, and how much can instructions to adopt a persona change their behavior over a game? We study these questions with AgentMonopoly, an environment where agents communicate, buy and trade properties, and make decisions that affect later turns. We compare models from four families, each at two sizes, in games with identical models, different sizes of the same family, or different families. We also assign one agent a persona, a description of how it should behave, intended to change a tendency observed in games without that persona. The tested models differ in how often they trade, waive rent, and manage cash and properties. Larger models do not consistently win more games. Persona instructions change the words agents use and some of their actions, but the changes do not always match the role across all decisions. These results show why persona following should be assessed through actions and outcomes as well as language. As our findings are conditional on the tested checkpoints, prompts, and five seeds per configuration, the open-source nature of AgentMonopoly provides the experimental infrastructure for further extending these comparisons. Our code is available at https://github.com/SuELS-Group/Agent_Monopoly.