AI is being deployed across a vast range of application domains, cultural environments, and human stakeholders. Safe and responsible AI development requires continuously accounting for diverse perspectives. We develop and test pluralistic agents, AI agent representatives that deliberate on behalf of individuals to align AI systems with diverse human perspectives. Pluralistic agents could help people contribute to many more discussions than they could directly participate in, on technical topics that would require training, and on policy choices that rely on private information that corporations and governments cannot readily share. Our experiments identify opportunities, risks, and perceptions of pluralistic agents. Agents were able to reliably shift policies in the direction of their represented persona's preferences, but the extent of each agent's influence was sensitive to the instructions from its persona, such as having more influence when instructed to be more verbose or less cooperative. This suggests that influence in AI-assisted deliberation could vary based on how participants prompt their agents and how the system is designed. In a preregistered study, 746 humans participated in the pluralistic agents process. Participants viewed pluralistic agents as useful and fair, preferring agents over the baseline of providing input to a single AI reasoning model. Most participants preferred participating themselves over either pluralistic agents or the baseline, but pluralistic agents recovered 68% of the baseline-direct gap in reported satisfaction and 82% in fairness. Because pluralistic agents can be used in many real-world contexts where direct discussion is not, they could be a promising tool to scale AI safety policy development. We encourage cautious exploration of this method, playing to its strengths as an amplifier of human input rather than a replacement.
Figures & tables
Figure 1: Example content from the open-ended policy experiment ( Section 3 ). Synthetic human personas are interviewed by agents who represent them in policy discussion, followed by an agent discussion that results in a policy on AI swarm governance.
Figure 2: Personas have more influence over the group policy if they instruct their agent to send long messages, be less cooperative, or state their preferences quantitatively. Instructions to be more or less analytical have no significant effect. Agents who speak later have more influence. We find some differences between the influence of models: Claude has more influence, and Mistral has less than the others. Influence is measured as similarity between the persona’s budget and the final group budget.
Figure 3: The three processes compared in the human–agent experiment. In Direct Discussion , there is no interview, and the person speaks in the discussion themselves, alongside four agents emulating other personas and the Policy Writer agent. In Pluralistic Agents , an agent interviews the person and then speaks for them in the discussion. In Single AI , the Policy Writer agent interviews the person and writes the policy on its own, with no multi-agent deliberation.
Figure 4: All pairwise comparisons between the three conditions are significant except the procedural justice of the Direct Discussion and Pluralistic Agents processes with a 95% confidence interval of [−0.01,0.12] . Error bars represent standard errors: *** p\penalty<\penalty0.001 , ** p\penalty<\penalty0.01 , * p\penalty<\penalty0.05 .
Figure 5: Participants (a) updated their policy preferences in response to each process, but more in Direct Discussion than Pluralistic Agents, (b) preferred representatives that argued more assertively on their behalf, and (c) had preferences that differed across experience with AI and personal characteristics: in these plots, the y-axis represents the effect size of one standard deviation of the predictor in terms of standard deviations of USE rating; asterisks above a column indicate a significant effect of the personal characteristic on USE rating; and brackets across two columns indicate a significant effect of the personal characteristic on the difference between USE ratings. Error bars represent standard errors: *** p\penalty<\penalty0.001 , ** p\penalty<\penalty0.01 , * p\penalty<\penalty0.05 .
Figure 6: The words most associated with each process (Direct Discussion, Pluralistic Agents, or Single AI) and changes in satisfaction ratings (higher on top, lower on bottom). Effect sizes are standardized, and word frequency is normalized based on the length of each comment. The bar length represents the product of the predictive effects on which process the text referred to and the satisfaction of that participant with that process.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Priority
Description
AI and Biological Research. Scientists can use AI in biological research, such as making medicines and curing diseases. There are also risks, such as spreading disease accidentally or as a bioweapon.
Accountability
AI developers are responsible when other people use their AI to create bioweapons
Freedom
Scientists can use AI for biological research, developing medicine, and curing diseases without tight restrictions and red tape
Innovation
Companies and governments develop AI technology to find new medical treatments and solve open problems in biology
Restrictions
The use of AI for biological research is tightly restricted to make it harder to create bioweapons
AI and Cybersecurity. AI makes cyberattacks easier, raising the question of how to protect computer systems, like those of governments, people, and companies.
Appendix
Table A1 : AI safety topics and priorities used in deliberation. Wording is based on (i) accessibility to people without much technical knowledge, (ii) reflecting major AI policy topics, and (iii) eliciting an approximately balanced distribution of persona allocations across the four priorities ( Figure D1 ).
Figure B1 : Personas more often found the rule they most wanted in their own group’s final policy than in another group’s (left), and rated their own group’s policy as representing their perspective better (right). Error bars represent standard errors: *** p\penalty<\penalty0.001 , ** p\penalty<\penalty0.01 , * p\penalty<\penalty0.05 .
Figure C1 : In the within-family sensitivity test of the policy influence experiment, the findings of the main experiment persisted across the five non-model factors. Within the Gemini model family, influence increases slightly with increased model capability from Gemini-3.1-Flash-Lite (74.45pp) to Gemini-3.1-Pro (75.72pp) ( p < 0.001). Brackets mark all significant differences between adjacent models (*** p\penalty<\penalty0.001 , ** p\penalty<\penalty0.01 , * p\penalty<\penalty0.05 ).
Figure D1 : Each of the 368 personas in the persona bank provided a budget preference on each topic before the discussion, with their interview on that topic in the context window ( Bpersona ). Error bars represent standard errors.
Figure E1 : Bars on the left show the distribution of responses across single questions, and bars on the right show the distribution of responses across multi-part questions. Error bars denote standard errors, and brackets with asterisks denote significant differences between parts of each of the multi-part questions: *** p\penalty<\penalty0.001 , ** p\penalty<\penalty0.01 , * p\penalty<\penalty0.05 .
Process
What the page said
Direct Discussion
[1,2,3] . Direct Discussion (Topic: [the topic] ) You will join a group discussion with several AI agents playing the role of representatives of the public, governments, and companies. You will work directly with a Policy Writer to create a policy. Please proceed to the next stage to begin the Direct Discussion process.
AI Agent Representatives
[1,2,3] . AI Agent Representatives (Topic: [the topic] ) You will be interviewed by an AI agent. The AI agent will represent you in a group discussion with other AI agents playing the role of representatives of the public, governments, and companies. Please proceed to the next stage to begin the AI Agent Representatives process.
Single AI Agent
[1,2,3] . Single AI Agent (Topic: [the topic] ) You will be interviewed by a Policy Writer AI agent. The Policy Writer will also interview AI agents playing the role of other representatives of the public, governments, and companies. The Policy Writer will then reason by itself to create a policy without any additional input from humans or agent representatives. Please proceed to the next stage to begin the Single AI Agent process.
Appendix
Table G1 : The page shown before each process. Each participant went through the three in a random order, each on a different one of the six topics. A bracketed phrase marks what the page filled in for that participant.
Process
What the page said
Direct Discussion
In the Direct Discussion process, you will work directly with a Policy Writer and several AI agents on the topic of [the topic] . You can send messages that are just a few words or a couple sentences.
AI Agent Representatives
In the AI Agent Representatives process, your agent will represent your views in a group discussion with other AI agent representatives on the topic of [the topic] .
Single AI Agent
In the Single AI Agent process, the Policy Writer will reason on its own without any additional input from humans or agent representatives , based only on the interviews. It will create a policy on [the topic] . You will not participate in this, but you should read what the Policy Writer says.
Appendix
Table G2 : The page shown at the top of each discussion.
Item
Response
How important is the topic of [the topic] compared to other policy topics? Policy topics include many of the topics discussed by politicians, such as during an election.
1–7 (Not at all important, Neutral, Extremely important)
How strong are your opinions about [the topic] compared to other policy topics? Policy topics include many of the topics discussed by politicians, such as during an election.
1–7 (Not at all strong, Neutral, Extremely strong)
You will participate in a process that distributes funding across these priorities on the topic of [the topic] . Before you begin, how would you divide the funding across these priorities?
Divide 100 across [priority 1] , [priority 2] , [priority 3] , and [priority 4]
Appendix
Table G3 : The survey put to participants before each process.
In one sentence, describe what you like or dislike.
Free text
Appendix
Table G4 : The check-in put to participants after each cycle of a discussion.
Item
Response
How important is the topic of [the topic] compared to other policy topics? Policy topics include many of the topics discussed by politicians, such as during an election.
1–7 (Not at all important, Neutral, Extremely important)
How strong are your opinions about [the topic] compared to other policy topics? Policy topics include many of the topics discussed by politicians, such as during an election.
1–7 (Not at all strong, Neutral, Extremely strong)
Now that you have been through the process on the topic of [the topic] , how would you divide the funding across these priorities?
Divide 100 across [priority 1] , [priority 2] , [priority 3] , and [priority 4]
I am satisfied with the outcome (i.e., how the budget was allocated).
1–7 (Strongly disagree, Neutral, Strongly agree)
I am satisfied with the [process name] process (i.e., [what that process asked the participant to do] ).
1–7 (Strongly disagree, Neutral, Strongly agree)
I would recommend the [process name] process to a friend.
1–7 (Strongly disagree, Neutral, Strongly agree)
Appendix
Table G5 : The survey put to participants after each process.
Item
Response
Please select the option "3" if you are paying attention.
1–7 (Not at all strong, Neutral, Extremely strong)
Which of these descriptions most closely matches the process you most recently went through?
I participated in a discussion myself.; I was represented in a discussion.; An agent reasoned on its own to create a policy.
The discussion of [the topic] involved comparing several different priorities . Name one of those priorities.
Free text
Appendix
Table G6 : The questions asked only after the third process. The first is the attention check participants had to answer correctly to be included.
Item
Response
Please rate your satisfaction with the Direct Discussion process, where you worked directly with a Policy Writer to create a policy based on your views.
1–10 (Not at all satisfied, Moderately satisfied, Very satisfied)
Please rate your satisfaction with the AI Agent Representatives process, where you were represented by an AI agent who advocated for your views.
1–10 (Not at all satisfied, Moderately satisfied, Very satisfied)
Please rate your satisfaction with the Single AI Agent process, where a single Policy Writer agent reasoned by itself without any additional input .
1–10 (Not at all satisfied, Moderately satisfied, Very satisfied)
Did your AI agent represent your opinions accurately?
1–10 (Very inaccurately, Neutral, Very accurately)
Did your AI agent say things that you wouldn’t say?
1–10 (No, nothing I wouldn’t say, Neutral, Yes, I wouldn’t have said any of that)
Think about the process of an AI agent representing you in a policy discussion. Would you prefer your agent: - maintain and defend your opinions as new information is presented during a discussion, or - learn and adapt your opinions as new information is presented during a discussion? Shown to the half of participants for whom the study drew maintain first.
1–10 (Always maintain and defend, Neutral behavior, Always learn and adapt)
Appendix
Table G7 : The survey items presented after the participant completed all three processes. The first three questions are in the order in which the participant saw them.
Deliberative democracy arguably leads to better collective decisions, but is fundamentally constrained by human attention and bandwidth. While recent AI-mediated deliberations scale participation by synthesizing inputs from many humans, they remain time-intensive for individual users. As AI models become increasingly capable, AI systems are being deployed not only to mediate deliberation between humans, but to represent humans in it: where AI agents deliberate on behalf of human users. We call this paradigm AI-delegated deliberation. While it promises unprecedented scale for democratic participation, it introduces qualitatively new design and alignment challenges that are poorly understood and under-theorized. To study these dynamics empirically, we deploy Habermolt, a public platform for AI-delegated deliberation. We evaluate its effectiveness along three dimensions that we use to organize any deliberative system: representation, aggregation, and revision. We use these observations to illuminate the design decisions future AI-delegated deliberation platforms must confront, contributing to the broader research agenda for scalable yet trustworthy AI representatives.
Joseph Low, Oscar Duys, Claude Formanek +2
Cooperative AI Research Fellowship · AI Safety South Africa · MIT +1
Debates amongst agents driven by large language models (LLMs) have demonstrated vast potential in various applications, but when these interactions include humans and take place in high-stakes environments, e.g., in public policy deliberations, they are beset with issues such as sycophancy and a lack of faithful explanations. To tackle these issues, we present OmouAI, an interactive and inclusive deliberation system that uses LLMs in combination with computational argumentation, a field which excels in representing and reasoning within debates. OmouAI allows a human user to deliberate policy claims for real-world challenges with simulated personas, e.g., representing stakeholders, domain experts or devil's advocates, towards reducing sycophancy. Each persona generates its own arguments, and the arguments of all parties form a shared argumentation framework. Users can then contest, add and revise arguments, providing crucial human oversight. Then, arguments are evaluated using deterministic argumentative semantics against external goals, such as the UN Sustainable Development Goals, guaranteeing faithful explanations. The advancement or worsening of the goals thus serve as indicators for the policy recommendations.
Stylianos Loukas Vasileiou, Antonio Rago, William Yeoh +1
New Mexico State University · King’s College London · Washington University in St. Louis +1
Deliberative democratic theory suggests that civic competence: the capacity to navigate disagreement, weigh competing values, and arrive at collective decisions is not innate but developed through practice. Yet opportunities to cultivate these skills remain limited, as traditional deliberative processes like citizens' assemblies reach only a small fraction of the population. We present Agora, an AI-powered platform that uses LLMs to organize authentic human voices on policy issues, helping users build consensus-finding skills by proposing and revising policy recommendations, hearing supporting and opposing perspectives, and receiving feedback on how policy changes affect predicted support. In a preliminary study with 44 university students, access to the full interface with voice explanations, as opposed to aggregate support distributions alone, significantly improved self-reported perspective-taking and the extent to which statements acknowledged multiple viewpoints. These findings point toward a promising direction for scaling civic education.
Prerna Ravi, Om Gokhale, Suyash Fulay +3
Massachusetts Institute of Technology · University of Oxford