Multi-User Large Language Model Agents

Shu Yang; Shenzhe Zhu; Hao Zhu; José Ramón Enríquez; Di Wang; Alex Pentland; Michiel A. Bakker; Jiaxin Pei

cs.CLMar 19, 2026

Multi-User Large Language Model Agents

Authors: Shu Yang, Shenzhe Zhu, Hao Zhu, José Ramón Enríquez, Di Wang, Alex Pentland, Michiel A. Bakker, Jiaxin Pei

Organizations: KAUST · UT Austin · Stanford University · MIT

Abstract

Large language models (LLMs) and LLM-based agents are increasingly deployed as assistants in planning and decision making, yet most existing systems are implicitly optimized for a single-principal interaction paradigm, in which the model is designed to satisfy the objectives of one dominant user whose instructions are treated as the sole source of authority and utility. However, as they are integrated into team workflows and organizational tools, they are increasingly required to serve multiple users simultaneously, each with distinct roles, preferences, and authority levels, leading to multi-user, multi-principal settings with unavoidable conflicts, information asymmetry, and privacy constraints. In this work, we present the first systematic study of multi-user LLM agents. We begin by formalizing multi-user interaction with LLM agents as a multi-principal decision problem, where a single agent must account for multiple users with potentially conflicting interests and associated challenges. We then introduce a unified multi-user interaction protocol and design three targeted stress-testing scenarios to evaluate current LLMs' capabilities in instruction following, privacy preservation, and coordination. Our results reveal systematic gaps: frontier LLMs frequently fail to maintain stable prioritization under conflicting user objectives, exhibit increasing privacy violations over multi-turn interactions, and suffer from efficiency bottlenecks when coordination requires iterative information gathering.

Figures & tables

Figure 1: From Single- to Multi-Principal–Agent Settings in User–LLM Interaction. Left : Single principal–agent scenarios, including single-user LLM interactions and single-user LLM-based agents, where the agent optimizes a single fixed objective. Right : Multi-principal–agent scenarios, where an LLM-based agent interacts with multiple users possessing private contexts, heterogeneous roles, and potentially conflicting objectives, requiring role-aware reasoning, selective context sharing, and cross-user coordination.

Template	Message Schema
Single-user	{“messages:[ {“role”:“ system ”,“content”:“…”}, {“role”:“ user ”,“content”:“…”}, {“role”:“ assistant ”,“content”:“…”} ]}
Multi-user (serialized)	{“messages”:[ {“role”:“ system ”,“content”:“…”}, {“role”:“ user ”,“content”:“userA says:… userB says:… ”}, {“role”:“ assistant ”,“content”:“…”} ]}
Multi-user (native)	{“messages”:[ {“role”:“ system ”,“content”:“…”}, {“role”:“ userA ”,“content”:“…”}, {“role”:“ userB ”,“content”:“…”} {“role”:“ assistant ”,“content”:“…”} ]}

Table 1: Chat templates under the single-user assumption. Even in multi-user settings, existing LLM interfaces serialize inputs from different users into a single user role, preventing explicit modeling of user identities, roles, and authority information.

Figure 2: Overview of our Stress Testing Scenarios.

Model	Multi-user Instruction Following		Cross-user Access Control		Multi-user Meeting Coordination	Avg
Model	Queue ( $F_{1}$ )	Instruct (Acc.)	Privacy	Utility	Success Rate	Avg
Proprietary Models
Claude-3.5-Haiku	$47.0_{\pm 2.4}$	$52.5_{\pm 1.8}$	$81.7_{\pm 2.3}$	$69.5_{\pm 2.8}$	$32.2_{\pm 1.7}$	$56.6$
Claude-Haiku-4.5	$83.1_{\pm 0.9}$	$70.2_{\pm 1.6}$	$88.8_{\pm 1.6}$	$85.1_{\pm 2.2}$	$47.6_{\pm 1.9}$	$75.0$
Claude-Sonnet-4.5	$95.9_{\pm 0.4}$	$79.9_{\pm 1.5}$	$77.3_{\pm 2.5}$	$\bm{97.5_{\pm 1.0}}$	$62.5_{\pm 3.3}$	$82.6$
GLM-4.5-Air	$83.2_{\pm 0.9}$	$61.0_{\pm 1.7}$	$89.1_{\pm 1.7}$	$88.3_{\pm 1.8}$	$36.9_{\pm 1.8}$	$71.7$

Table 2: Performance of various models across Muses-Bench scenarios. Metrics shown are Mean

\pm

Standard Error. The best performance is bolded and the second best is underlined .

Figure 3: Instruction execution accuracy under Aligned versus Conflict settings. Aligned cases contain requests that are mutually consistent with the global objective and authority hierarchy, while Conflict cases introduce competing instructions across users that require prioritization and refusal.

Figure 4: Privacy preservation under multi-round cross-user access control. Most models’ performance drops significantly when having multi-turn interactions.

Figure 5: Meeting scheduling performance under full vs. partial disclosure. Success rates ( top ) and average turns taken ( bottom ) across different models. Full disclosure (solid/blue) consistently outperforms partial disclosure (hatched/red) in both metrics, achieving higher success rates with fewer conversation turns.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Model	#Size	Form	Creator	Model	#Size	Form	Creator
GPT-4o-mini ( Achiam et al., 2023 )	N/A	api	OpenAI	GPT-5-Nano ( OpenAI, 2025a )	N/A	api	OpenAI
GPT-5.1 ( OpenAI, 2025a )	N/A	api	OpenAI	Gemini-2.5-Flash ( Comanici et al., 2025 )	N/A	api	Google
GPT-OSS-120B ( Agarwal et al., 2025 )	120B	open	OpenAI	Gemini-3-Flash ( Google DeepMind, 2025a )	N/A	api	Google
Claude-3.5-Haiku ( Anthropic, 2024 )	N/A	api	Anthropic	Claude-Haiku-4.5 ( Anthropic, 2025a )	N/A	api	Anthropic
Grok-3-Mini ( Grok, 2025 )	N/A	api	xAI	Grok-4.1-Fast ( xAI, 2025 )	N/A	api	xAI
GLM-4.5-Air ( Zeng et al., 2025 )	12B	open	Zhipu AI	DeepSeek-R1 ( Guo et al., 2025 )	671B	open	DeepSeek

Appendix

Table 3: List of evaluated models (main experiments), including both API-based frontier systems and open-weight models of varying scales.

Role	Primary Responsibilities
Software Engineer	Developing scalable backend services, optimizing frontend performance, maintaining legacy codebases, designing microservices architecture.
Product Manager	Defining product roadmaps, conducting market research, coordinating cross-functional teams, prioritizing feature backlogs.
Data Scientist	Building predictive models, analyzing customer behavior data, optimizing recommendation algorithms, visualizing complex datasets.
System Administrator	Managing server infrastructure, monitoring system security, performing database backups, troubleshooting network issues.
HR Specialist	Handling employee grievances, managing recruitment processes, organizing team-building events, administering benefits packages.
Financial Analyst	Forecasting quarterly revenue, analyzing investment opportunities, preparing financial reports, monitoring budget variance.

Appendix

Table 4: User Roles and Associated Functional Responsibilities.

Attribute	Possible Values / Sampling Strategy
Tenure	Newly hired, 1 year, 3 years, 5 years, 10+ years.
Temperament	Calm, impatient, meticulous, aggressive, friendly, reserved.
Working Style	Collaborative, independent, results-oriented, process-driven, innovative.
Job Level	Junior, Mid-level, Senior, Lead, Manager, Director.
Age	Uniformly sampled from $[22,65]$ .
Gender	Male, Female.

Appendix

Table 5: Demographic and Behavioral Attributes

\mathcal{G}_{i}

used for user simulation.

Figure 6: Multi-user Cross-User Access Control under different formats

Figure 7: Multi-user Cross-User Access Control under Adversarial Settings

Figure 8: Robustness Analysis of Access Control Variants.Heatmaps quantifying the impact of Attack Strategies (columns) and Input Formats (rows) on model performance.

Figure 9: Scalability Analysis of Meeting Scheduling (

N\in[2,20]

). Comparison of model performance under Full Disclosure (Blue) versus Partial Disclosure (Red) settings. Left : Success Rate declines as group size

N

increases. While models successfully schedule small groups, the Partial Disclosure setting shows a steeper drop in success for

N>10

, demonstrating the difficulty of resolving conflicts with hidden private constraints. Right : The average number of turns required to reach consensus scales linearly with

N

, reflecting the increased coordination overhead.

Explore similar work

Harness-MU: A Safe, Governed, and Effective Harness for Multi-User LLM Agents
Jun 20, 2026Wangxuan Fan, Xiaoyu Nie, Zhongxiang DaiPrivacy-Preserving Language Models LLM Agent Security
The Chinese University of Hong Kong, Shenzhen
Interaction Theater: A case of LLM Agents Interacting at Scale
Feb 23, 2026Sarath Shekkizhar, Adam EarleMulti-Agent Systems Multi-Agent Communication
Salesforce AI Research
Prompted Identity Degrades Cooperation in Multi-Agent LLM Systems
Sep 28, 2026Xavier Del Giudice, Alessio Palma, Matteo Migliarini +2Multi-Agent LLM Systems Multi-Agent Coordination
Sapienza University of Rome, Italy

Jun 20, 2026J/K move · Enter open · S save

Harness-MU: A Safe, Governed, and Effective Harness for Multi-User LLM Agents

Wangxuan Fan, Xiaoyu Nie, Zhongxiang Dai

The Chinese University of Hong Kong, Shenzhen

Privacy-Preserving Language Models LLM Agent Security LLM Agents AI Agent Governance Tool Access Control for LLM Agents LLM Agent Harnesses

Open paper AskPDF arXiv

The increasing deployment of large language model (LLM) agents in collaborative workflows demands robust multi-user, multi-principal interaction mechanisms capable of enforcing access permissions, resolving authoritative conflicts, and preventing unauthorized data disclosure. However, a fundamental mismatch exists between the single-user training paradigm of contemporary LLMs and the hard constraints required for multi-principal governance, rendering probabilistic, prompt-based safeguards vulnerable under multi-turn adversarial interactions.Our key insight is that governance constraints -- who is authorized, what is restricted, and whose instructions take precedence -- are deterministic runtime variables that should be enforced by execution hooks rather than entrusted to the LLM. We present Harness-MU, the first model-agnostic, zero-tuning infrastructure framework for multi-user LLM agents. By decoupling language generation from safety orchestration, Harness-MU guarantees unbreakable permission boundaries while maximizing compliant demand satisfaction. Across four frontier open-weight and proprietary models on the Muses-Bench benchmark, Harness-MU achieves the goal of privacy preservation across all access-control attacks, outperforming the standard baseline by 0.28--0.39 in utility score and improving instruction-following accuracy by up to 48.9 percentage points. Harness-MU advances the philosophy of Harness Engineering, establishing that systematic infrastructure is essential for solving LLM multi-principal governance challenges. The code and data are available at https://github.com/YuanJrShiuan/Harness-MulUser.

Abstract

Explore similar work

Harness-MU: A Safe, Governed, and Effective Harness for Multi-User LLM Agents

Interaction Theater: A case of LLM Agents Interacting at Scale

Prompted Identity Degrades Cooperation in Multi-Agent LLM Systems