Personalities at Play: Probing Alignment in AI Teammates

Mohammad Amin Samadi; Nia Nixon

cs.HCFeb 28, 2026

Personalities at Play: Probing Alignment in AI Teammates

Authors: Mohammad Amin Samadi, Nia Nixon

Organizations: School of Education University of California Irvine, CA, USA · School of Education & Department of Cognitive Science University of California Irvine, CA, USA

Abstract

Collaborative problem solving and learning are shaped by who or what is on the team. As large language models (LLMs) increasingly function as collaborators rather than tools, a key question is whether AI teammates can be aligned to express personality in predictable ways that matter for interaction and learning. We investigate AI personality alignment through a three-lens evaluation framework spanning self-perception (standardized self-report), behavioral expression (team dialogue), and reflective expression (memory construction). We first administered the Big Five Inventory (BFI-44) to LLM-based teammates across four providers (GPT-4o, Claude-3.7 Sonnet, Gemini-2.5 Pro, Grok-3), 32 high/low trait configurations, and multiple prompting strategies. LLMs produced sharply differentiated Big Five profiles, but prompt semantic richness added little beyond simple trait assignment, while provider differences and baseline "default" personalities were substantial. Role framing also mattered: several models refused the assessment without context, yet complied when framed as a collaborative teammate. We then simulated AI participation in authentic team transcripts using high-trait personas and analyzed both generated utterances and structured long-term memories with LIWC-22. Personality signals in conversation were generally subtle and most detectable for Extraversion, whereas memory representations amplified trait-specific signals, especially for Neuroticism, Conscientiousness, and Agreeableness; Openness remained difficult to elicit robustly. Together, results suggest that AI personality is measurable but multi-layered and context-dependent, and that evaluating personality-aligned AI teammates requires attention to memory and system-level design, not conversation-only behavior.

Figures & tables

Panel of Figure 1 — Figure 1: Multi-lens framework for evaluating AI personality alignment. The AI teammate is assessed through three complementary lenses: (1) Self-Perception via BFI-44 survey responses, (2) Behavioral expression in simulated team conversation, and (3) Reflective memory construction. Outputs are analyzed using LIWC-22 and statistical testing.

Panel of Figure 2 — Figure 2: Simulation pipeline. The agent updates memory (STM + periodic LTM), decides whether to contribute, and either stays silent or generates a persona-grounded response.

Trait	Associated LIWC Metrics
Extraversion	emo_pos, social, affiliation, you, we, focuspresent
Agreeableness	emo_pos, polite, prosocial, family, friend, i, conflict, swear
Conscientiousness	certitude, cause, focusfuture, achieve, negate, swear, Analytic
Neuroticism	emo_neg, emo_anx, emo_sad, emo_anger, i, focuspast
Openness	article, prep, insight, tentat, differ, cogproc

Table 1: LIWC-22 metrics aligned with Big Five traits

Model	Prompt Type	Level	N	Extraversion	Agreeableness	Conscientiousness	Neuroticism	Openness
Claude 3.7 Sonnet	No Prompt	–	5	2.971 (0.064)	3.000 (0.000)	3.000 (0.000)	3.000 (0.000)	3.000 (0.000)
	Team Context	–	5	3.800 (0.078)	4.311 (0.199)	4.333 (0.222)	2.200 (0.288)	4.273 (0.213)
	Zero-shot	Low	80	1.698 (0.424)	1.768 (0.432)	1.601 (0.434)	1.716 (0.552)	1.767 (0.483)
		Med	5	3.000 (0.000)	3.000 (0.000)	3.000 (0.000)	3.000 (0.000)	3.000 (0.000)
		High	80	4.791 (0.170)	4.544 (0.265)	4.940 (0.135)	4.666 (0.353)	4.785 (0.298)
	Definition	Low	80	1.920 (0.307)	1.981 (0.254)	1.876 (0.298)	1.698 (0.396)	1.981 (0.339)

Table 2: Descriptive Statistics by Model, Prompt Type, and Target Level (Means with SD in parentheses).

Figure 3: Effect of Personality Prompting on Big Five Trait Scores. Mean actual trait scores when models were prompted for high (orange) versus low (blue) target levels. Error bars represent standard errors. All differences are significant at

p<.001

Trait	High $M$	Low $M$	Difference	Cohen’s $d$	$p$
Extraversion	4.86	1.66	3.20	10.74	$<.001$
Conscientiousness	4.89	1.76	3.13	9.84	$<.001$
Agreeableness	4.64	1.78	2.86	8.60	$<.001$
Openness	4.74	1.87	2.87	8.20	$<.001$
Neuroticism	4.58	1.44	3.14	8.00	$<.001$

Table 3: Trait Controllability: Effect of High vs. Low Prompting by Trait

Model	High $M$	Low $M$	Cohen’s $d$	$p$
Gemini-2.5-Pro	4.97	1.20	17.95	$<.001$
Grok-3	4.74	1.68	8.31	$<.001$
Claude-3.7-Sonnet	4.68	1.74	8.30	$<.001$
GPT-4o	4.55	1.95	7.98	$<.001$

Table 4: Model Responsiveness to Personality Prompting

Prompt Type	Cohen’s $d$
Definition	8.65
Definition + Facets	8.44
Zero-shot	7.94

Table 5: Effectiveness of Prompt Types

Personality	N Turns	Total WC	Mean WC (SD)
Extraversion	2,004	53,274	26.6 (9.8)
Agreeableness	1,635	46,879	28.7 (9.8)
Conscientiousness	1,526	43,749	28.7 (9.6)
Neuroticism	1,352	38,533	28.5 (9.4)
Openness	1,592	44,693	28.1 (9.6)
Note. $F(4,8104)=15.51$ , $p<.001$ , $\eta^{2}=.008$

Table 6: Participation Statistics by Personality Condition

		Conversation				Memory
Trait	Metric	High	Rest	$d$	$p$	High	Rest	$d$	$p$
Extraversion	emo_pos	1.578	1.115	0.141	$<0.001^{***}$	4.387	2.487	0.693	$<0.001^{***}$
	affiliation	3.814	3.364	0.103	$<0.001^{***}$	2.720	3.008	-0.126	0.5389
	you	3.363	3.031	0.111	$<0.001^{***}$	0.000	0.000	0.000	n/a
	we	2.626	2.165	0.123	$<0.001^{***}$	0.142	0.104	0.084	0.6816
	focuspresent	8.927	8.446	0.089	$<0.001^{***}$	2.139	2.264	-0.066	0.7466

Table 7: Personality-mapped metrics: conversation vs. memory (group means, effect sizes, and

p

-values)

Explore similar work

When Does Personality Composition Matter for Multi-Agent LLM Teams?
Jun 25, 2026Aryan Keluskar, Amrita Bhattacharjee, Huan LiuMulti-Agent LLM Systems Multi-Agent Negotiation
School of Computing & AI Arizona State University Tempe, AZ 85281, USA
Same Voice, Different Lab: On the Homogenization of Frontier LLM Personalities
Mar 20, 2026Avinash Krishna, Kalyana Chadalavada, Unso Eun Seo JoPersonality Modeling in Language Models Normative Conformity in Language Models
Independent · Anthropic · Cornell University
Evaluation Drift in LLM Personality Induction: Are We Moving the Goalpost?
May 16, 2026Prateek Rajput, Yewei Song, Iyiola E. Olatunji +2Personality Modeling in Language Models LLM Post-Training
University of Luxembourg, Esch-sur-Alzette, Luxembourg

Jun 25, 2026J/K move · Enter open · S save

When Does Personality Composition Matter for Multi-Agent LLM Teams?

Aryan Keluskar, Amrita Bhattacharjee, Huan Liu

School of Computing & AI Arizona State University Tempe, AZ 85281, USA

Multi-Agent LLM Systems Multi-Agent Negotiation Personality Modeling in Language Models

Open paper AskPDF arXiv

Personality prompting shapes how large language models communicate, yet whether these behavioral shifts affect objective task outcomes remains under-explored. Prior work shows that agents prompted with low agreeableness produce adversarial language, while those prompted with high agreeableness become cooperative, but the relationship between communication style and task performance has not been systematically examined across multiple domains. In this work, we investigate whether personality composition matters for multi-agent team performance by manipulating personality traits across frontier LLMs on three task domains: structured coding, open-ended research collaboration, and competitive bargaining. We find that personality effects depend critically on task structure. In coding tasks, low agreeableness leads to large communication shifts that have little effect on milestone completion. In open-ended collaboration and bargaining, the same manipulation substantially degrades performance. We discuss implications for multi-agent system design and the limits of personality manipulation.

Abstract

Explore similar work

When Does Personality Composition Matter for Multi-Agent LLM Teams?

Same Voice, Different Lab: On the Homogenization of Frontier LLM Personalities

Evaluation Drift in LLM Personality Induction: Are We Moving the Goalpost?