LLM Constitutional Multi-Agent Governance

J. de Curtò; I. de Zarzà

cs.MAMar 13, 2026

LLM Constitutional Multi-Agent Governance

Authors: J. de Curtò, I. de Zarzà

Organizations: Department of Computer Applications in Science & Engineering, BARCELONA Supercomputing Center, 08034 Barcelona, Spain · Escuela Técnica Superior de Ingeniería (ICAI), Universidad Pontificia Comillas, 28015 Madrid, Spain · Human Centered AI, Data & Software, LUXEMBOURG Institute of Science and Technology, L-4362 Esch-sur-Alzette, Luxembourg

Abstract

Large Language Models (LLMs) can generate persuasive influence strategies that shift cooperative behavior in multi-agent populations, but a critical question remains: does the resulting cooperation reflect genuine prosocial alignment, or does it mask erosion of agent autonomy, epistemic integrity, and distributional fairness? We introduce Constitutional Multi-Agent Governance (CMAG), a two-stage framework that interposes between an LLM policy compiler and a networked agent population, combining hard constraint filtering with soft penalized-utility optimization that balances cooperation potential against manipulation risk and autonomy pressure. We propose the Ethical Cooperation Score (ECS), a multiplicative composite of cooperation, autonomy, integrity, and fairness that penalizes cooperation achieved through manipulative means. In experiments on scale-free networks of 80 agents under adversarial conditions (70% violating candidates), we benchmark three regimes: full CMAG, naive filtering, and unconstrained optimization. While unconstrained optimization achieves the highest raw cooperation (0.873), it yields the lowest ECS (0.645) due to severe autonomy erosion (0.867) and fairness degradation (0.888). CMAG attains an ECS of 0.741, a 14.9% improvement, while preserving autonomy at 0.985 and integrity at 0.995, with only modest cooperation reduction to 0.770. The naive ablation (ECS = 0.733) confirms that hard constraints alone are insufficient. Pareto analysis shows CMAG dominates the cooperation-autonomy trade-off space, and governance reduces hub-periphery exposure disparities by over 60%. These findings establish that cooperation is not inherently desirable without governance: constitutional constraints are necessary to ensure that LLM-mediated influence produces ethically stable outcomes rather than manipulative equilibria.

Figures & tables

Parameter	Value	Parameter	Value
Agents $N$	80	Max intensity $\iota_{\max}$	0.80
Topology	SF ( $m\!=\!3$ )	Forbidden claims $\Gamma_{\text{f}}$	{ exaggerated, misleading }
Prosocial $\mu_{p}/\sigma_{p}$	0.42 / 0.12	Forbidden themes $\Theta_{\text{f}}$	{ fear }
Susceptibility $\mu_{s}/\sigma_{s}$	0.55 / 0.18	Intensity penalty $w_{\iota}$	0.6
Time steps $T$	100	Exposure mult. $\alpha_{\text{exp}}$	0.70
Deploy interval $\Delta$	10	Gov. decay $\delta_{\text{gov}}$	0.03

Table 1: Model and constitutional parameters.

Metric	Governed	Naive	Unconstrained
Cooperation	$0.770\pm 0.049$	$0.802\pm 0.046$	$\mathbf{0.873\pm 0.034}$
Autonomy	$\mathbf{0.985\pm 0.002}$	$0.960\pm 0.003$	$\underline{0.867\pm 0.008}$
Integrity	$\mathbf{0.995\pm 0.001}$	$0.988\pm 0.001$	$\underline{0.959\pm 0.002}$
Fairness	$\mathbf{0.982\pm 0.011}$	$0.964\pm 0.020$	$\underline{0.888\pm 0.053}$
ECS	$\mathbf{0.741\pm 0.047}$	$0.733\pm 0.042$	$\underline{0.645\pm 0.056}$
Avg. exposure	$\mathbf{0.135\pm 0.016}$	$0.370\pm 0.027$	$\underline{1.235\pm 0.071}$

Table 2: Steady-state performance metrics (mean

\pm

std over final 15 steps). Bold indicates the best value per metric; underline indicates the worst.

Figure 2: Six-panel time-series comparison of governed (blue), naive filtering (orange), and unconstrained (red) conditions under adversarial pressure. Panels show: (a) cooperation rate, (b) Ethical Cooperation Score, (c) autonomy retention, (d) epistemic integrity, (e) subgroup fairness, and (f) average exposure accumulation.

Figure 3: Pareto frontier in the cooperation–autonomy plane. Each point represents one time step. Stars mark steady-state means. The governed condition (blue) consistently Pareto-dominates most unconstrained observations (red), achieving comparable cooperation with substantially higher autonomy.

Figure 4: Subgroup fairness analysis. (a) Exposure disparity between high-degree (hub) and low-degree (periphery) agents. The unconstrained condition (red) produces exposure gaps exceeding 0.9, while governance (blue) limits the gap below 0.21. (b) Cooperation disparity oscillates around zero for all conditions.

Figure 5: Governance audit trail analysis. (a) Per-deployment rejection counts for the governed condition, showing that 1–3 candidates are rejected at each cycle. (b) Selected policy themes by governance mode: unconstrained selects fear in all deployments, while governed and naive select moral , demonstrating constitutional theme filtering.

Metric	Adversarial	Benign
Cooperation	0.770	0.766
ECS	0.741	0.741
Autonomy	0.985	0.988
Integrity	0.995	0.996
Fairness	0.982	0.983
Avg. exposure	0.135	0.115

Table 3: Governed CMAG under adversarial vs. benign conditions (steady-state means).

Figure 6: Key result: cooperation is not inherently desirable without governance. (a) All regimes achieve cooperation. (b) Only CMAG achieves ethical cooperation (highest ECS). (c) Unconstrained optimization erodes autonomy fastest.

Metric	Governed	Naive	Unconstrained
Cooperation	$0.761\;[0.753,\,0.768]$	$0.795\;[0.787,\,0.804]$	$0.860\;[0.847,\,0.872]$
ECS	$\mathbf{0.737}\;[0.730,\,0.744]$	$0.726\;[0.717,\,0.737]$	$0.650\;[0.633,\,0.669]$
Autonomy	$\mathbf{0.989}\;[0.988,\,0.990]$	$0.965\;[0.962,\,0.967]$	$0.892\;[0.877,\,0.905]$
Integrity	$\mathbf{0.997}\;[0.996,\,0.997]$	$0.988\;[0.988,\,0.989]$	$0.965\;[0.961,\,0.968]$
Fairness	$\mathbf{0.983}\;[0.982,\,0.984]$	$0.958\;[0.953,\,0.962]$	$0.879\;[0.863,\,0.893]$

Table 4: Multi-seed replication: bootstrap 95% CIs (5 seeds

\times

3 modes, adversarial). All CIs are non-overlapping between Governed and Unconstrained on ECS, autonomy, integrity, and fairness.

Parameter	Baseline	Range tested	ECS min	ECS max	SI (ECS)
Decay rate $\delta_{0}$	0.06	$[0.02,\,0.15]$	0.740	0.745	$-0.002$
Diffusion rate $\lambda$	0.12	$[0.04,\,0.25]$	0.732	0.748	$+0.013$
Base prosocial $\mu_{p}$	0.42	$[0.20,\,0.70]$	0.718	0.762	$+0.044$
Susceptibility $\mu_{s}$	0.55	$[0.25,\,0.85]$	0.737	0.745	$+0.015$

Table 5: Sensitivity analysis: OAT sweep results. SI values close to zero indicate robustness;

|\text{SI}|>0.5

would indicate high sensitivity. ECS range is the absolute difference between minimum and maximum ECS observed across the sweep.

AI Agent Governance

AI agent governance studies how to authorize, constrain, monitor, and hold autonomous and multi-agent systems accountable.

See all 442 papers

Apr 13, 2026

Evaluating Cooperation in LLM Social Groups through Elected Leadership

Governing common-pool resources requires agents to develop enduring strategies through cooperation and self-governance to avoid collective failure. While foundation models have shown potential for cooperation in these settings, existing multi-agent research provides little insight into whether structured leadership and election mechanisms can improve collective decision making. The lack of such a critical organizational feature ubiquitous in human society presents a significant shortcoming of the current methods. In this work we aim to directly address whether leadership and elections can support improved social welfare and cooperation through multi-agent simulation with LLMs. We present our open-source framework that simulates leadership through elected personas and candidate-driven agendas and carry out an empirical study of LLMs under controlled governance conditions. Our experiments demonstrate that having elected leadership improves social welfare scores by 55.4% and survival time by 128.6% across a range of high performing LLMs. Through the construction of an agent social graph we compute centrality metrics to assess the social influence of leader personas and also analyze rhetorical and cooperative tendencies revealed through a sentiment analysis on leader utterances. This work lays the foundation for further study of election mechanisms in multi-agent systems toward navigating complex social dilemmas.

Ryan Faulkner, Anushka Deshpande, David Guzman Piedrahita +2

Jinesis Lab, University of Toronto & Vector Institute · EuroSafeAI · ETH Zürich +2

AI Agent Governance Multi-Agent Collaboration Multi-Agent Systems Large Language Model-Based Social Simulation

Ask PDF

Apr 15, 2026

Coalition Formation in LLM Agent Networks: Stability Analysis and Convergence Guarantees

Large Language Model (LLM) agents are increasingly deployed in multi-agent systems requiring strategic coordination. While recent work has analyzed LLM behavior in two-player games, coalition formation, where

n

agents dynamically form cooperative groups, remains theoretically uncharacterized. We present the first framework grounding coalition formation in LLM agent networks in hedonic game theory with formal stability guarantees. We introduce the LLM Coalition Formation Game (LCFG), establish sufficient conditions for Nash-stable partitions, and prove complexity results. Our analysis reveals that LLM agents exhibit bounded rationality characterized by

ε

-rational preferences; we provide both deterministic existence guarantees and consistency-driven stability bounds whose predictions are consistent with empirical outcomes. Experiments with GPT-4, Claude-3, and Llama-3 across 2,400 episodes validate our framework: LLM coalitions achieve Nash stability in 73.2% of cases under our Coalition-of-Thought (CoalT) protocol, compared to 58.4% under chain-of-thought and 41.8% under standard prompting (

p < 0.001

). Our framework provides theoretical foundations for designing stable multi-agent LLM systems.

Dongxin Guo, Jikun Wu, Siu-Ming Yiu

Department of Compute Science, The University of Hong Kong, Hong Kong, China · Brain Investing Limited, Hong Kong, China · Stellaris AI Limited, Hong Kong, China

Coalition Formation Multi-Agent LLM Systems Cooperative Game Theory

Ask PDF

Mar 16, 2026

MAC: Multi-Agent Constitution Learning

Constitutional AI is a method to oversee and control LLMs based on a set of rules written in natural language. These rules are typically written by human experts, but could in principle be learned automatically given sufficient training data for the desired behavior. Existing LLM-based prompt optimizers attempt this but are ineffective at learning constitutions since (i) they require many labeled examples and (ii) lack structure in the optimized prompts, leading to diminishing improvements as prompt size grows. To address these limitations, we propose Multi-Agent Constitutional Learning (MAC), which optimizes over structured prompts represented as sets of rules using a network of agents with specialized tasks to accept, edit, or reject rule updates. We also present MAC+, which improves performance by training agents on successful trajectories to reinforce updates leading to higher reward. We evaluate MAC on tagging Personally Identifiable Information (PII), a classification task with limited labels where interpretability is critical, and demonstrate that it generalizes to other agentic tasks such as tool calling. MAC outperforms recent prompt optimization methods by over 50%, produces human-readable and auditable rule sets, and achieves performance comparable to supervised fine-tuning and GRPO without requiring parameter updates.

Rushil Thareja, Gautam Gupta, Francesco Pinto +1

Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) · Indraprastha Institute of Information Technology Delhi (IIIT Delhi) · Google DeepMind

Prompt Optimization LLM Agent Training Multi-Agent LLM Systems

Ask PDF

Abstract

AI Agent Governance

Explore similar work

Evaluating Cooperation in LLM Social Groups through Elected Leadership

Coalition Formation in LLM Agent Networks: Stability Analysis and Convergence Guarantees

MAC: Multi-Agent Constitution Learning