Personalities at Play: Probing Alignment in AI Teammates
Authors: Mohammad Amin Samadi, Nia Nixon
Organizations: School of Education University of California Irvine, CA, USA · School of Education & Department of Cognitive Science University of California Irvine, CA, USA
Collaborative problem solving and learning are shaped by who or what is on the team. As large language models (LLMs) increasingly function as collaborators rather than tools, a key question is whether AI teammates can be aligned to express personality in predictable ways that matter for interaction and learning. We investigate AI personality alignment through a three-lens evaluation framework spanning self-perception (standardized self-report), behavioral expression (team dialogue), and reflective expression (memory construction). We first administered the Big Five Inventory (BFI-44) to LLM-based teammates across four providers (GPT-4o, Claude-3.7 Sonnet, Gemini-2.5 Pro, Grok-3), 32 high/low trait configurations, and multiple prompting strategies. LLMs produced sharply differentiated Big Five profiles, but prompt semantic richness added little beyond simple trait assignment, while provider differences and baseline "default" personalities were substantial. Role framing also mattered: several models refused the assessment without context, yet complied when framed as a collaborative teammate. We then simulated AI participation in authentic team transcripts using high-trait personas and analyzed both generated utterances and structured long-term memories with LIWC-22. Personality signals in conversation were generally subtle and most detectable for Extraversion, whereas memory representations amplified trait-specific signals, especially for Neuroticism, Conscientiousness, and Agreeableness; Openness remained difficult to elicit robustly. Together, results suggest that AI personality is measurable but multi-layered and context-dependent, and that evaluating personality-aligned AI teammates requires attention to memory and system-level design, not conversation-only behavior.
Figures & tables
Figure 1: Multi-lens framework for evaluating AI personality alignment. The AI teammate is assessed through three complementary lenses: (1) Self-Perception via BFI-44 survey responses, (2) Behavioral expression in simulated team conversation, and (3) Reflective memory construction. Outputs are analyzed using LIWC-22 and statistical testing.
Figure 2: Simulation pipeline. The agent updates memory (STM + periodic LTM), decides whether to contribute, and either stays silent or generates a persona-grounded response.
emo_neg, emo_anx, emo_sad, emo_anger, i, focuspast
Openness
article, prep, insight, tentat, differ, cogproc
Table 1: LIWC-22 metrics aligned with Big Five traits
Model
Prompt Type
Level
N
Extraversion
Agreeableness
Conscientiousness
Neuroticism
Openness
Claude 3.7 Sonnet
No Prompt
–
5
2.971 (0.064)
3.000 (0.000)
3.000 (0.000)
3.000 (0.000)
3.000 (0.000)
Team Context
–
5
3.800 (0.078)
4.311 (0.199)
4.333 (0.222)
2.200 (0.288)
4.273 (0.213)
Zero-shot
Low
80
1.698 (0.424)
1.768 (0.432)
1.601 (0.434)
1.716 (0.552)
1.767 (0.483)
Med
5
3.000 (0.000)
3.000 (0.000)
3.000 (0.000)
3.000 (0.000)
3.000 (0.000)
High
80
4.791 (0.170)
4.544 (0.265)
4.940 (0.135)
4.666 (0.353)
4.785 (0.298)
Definition
Low
80
1.920 (0.307)
1.981 (0.254)
1.876 (0.298)
1.698 (0.396)
1.981 (0.339)
Table 2: Descriptive Statistics by Model, Prompt Type, and Target Level (Means with SD in parentheses).
Figure 3: Effect of Personality Prompting on Big Five Trait Scores. Mean actual trait scores when models were prompted for high (orange) versus low (blue) target levels. Error bars represent standard errors. All differences are significant at p<.001 .
Trait
High M
Low M
Difference
Cohen’s d
p
Extraversion
4.86
1.66
3.20
10.74
<.001
Conscientiousness
4.89
1.76
3.13
9.84
<.001
Agreeableness
4.64
1.78
2.86
8.60
<.001
Openness
4.74
1.87
2.87
8.20
<.001
Neuroticism
4.58
1.44
3.14
8.00
<.001
Table 3: Trait Controllability: Effect of High vs. Low Prompting by Trait
Model
High M
Low M
Cohen’s d
p
Gemini-2.5-Pro
4.97
1.20
17.95
<.001
Grok-3
4.74
1.68
8.31
<.001
Claude-3.7-Sonnet
4.68
1.74
8.30
<.001
GPT-4o
4.55
1.95
7.98
<.001
Table 4: Model Responsiveness to Personality Prompting
Prompt Type
Cohen’s d
Definition
8.65
Definition + Facets
8.44
Zero-shot
7.94
Table 5: Effectiveness of Prompt Types
Personality
N Turns
Total WC
Mean WC (SD)
Extraversion
2,004
53,274
26.6 (9.8)
Agreeableness
1,635
46,879
28.7 (9.8)
Conscientiousness
1,526
43,749
28.7 (9.6)
Neuroticism
1,352
38,533
28.5 (9.4)
Openness
1,592
44,693
28.1 (9.6)
Note. F(4,8104)=15.51 , p<.001 , η2=.008
Table 6: Participation Statistics by Personality Condition
Conversation
Memory
Trait
Metric
High
Rest
d
p
High
Rest
d
p
Extraversion
emo_pos
1.578
1.115
0.141
<0.001∗∗∗
4.387
2.487
0.693
<0.001∗∗∗
affiliation
3.814
3.364
0.103
<0.001∗∗∗
2.720
3.008
-0.126
0.5389
you
3.363
3.031
0.111
<0.001∗∗∗
0.000
0.000
0.000
n/a
we
2.626
2.165
0.123
<0.001∗∗∗
0.142
0.104
0.084
0.6816
focuspresent
8.927
8.446
0.089
<0.001∗∗∗
2.139
2.264
-0.066
0.7466
Table 7: Personality-mapped metrics: conversation vs. memory (group means, effect sizes, and p -values)
Personality prompting shapes how large language models communicate, yet whether these behavioral shifts affect objective task outcomes remains under-explored. Prior work shows that agents prompted with low agreeableness produce adversarial language, while those prompted with high agreeableness become cooperative, but the relationship between communication style and task performance has not been systematically examined across multiple domains. In this work, we investigate whether personality composition matters for multi-agent team performance by manipulating personality traits across frontier LLMs on three task domains: structured coding, open-ended research collaboration, and competitive bargaining. We find that personality effects depend critically on task structure. In coding tasks, low agreeableness leads to large communication shifts that have little effect on milestone completion. In open-ended collaboration and bargaining, the same manipulation substantially degrades performance. We discuss implications for multi-agent system design and the limits of personality manipulation.
Aryan Keluskar, Amrita Bhattacharjee, Huan Liu
School of Computing & AI Arizona State University Tempe, AZ 85281, USA
LLM assistant personalities play a critical role in user experience and perceived response quality. We present a large-scale experiment of frontier LLM personalities using external ELO-based traits scoring across 144 traits. We find that all models tested converge on a form of trait expression that is systematic, methodical, and analytical and suppress traits such as remorseful and sycophantic. Moreover, models tend to diverge more in their expression of middle-of-distribution traits such as poetic or playful, but even these so-called creative models tend to have more neutral identities. These similarities suggest an implicit emergence of a standard of optimal assistant behavior. In a landscape of varied training methods, character training, therefore, stands out for its uniformity, offering insight into a tacit consensus between model developers.
Avinash Krishna, Kalyana Chadalavada, Unso Eun Seo Jo
Can large language models reliably express a human-like personality, or are they merely mimicking surface cues without a stable underlying profile? To investigate this, we induce personality in LLMs by fine-tuning them on the long-form essays, where each essay is associated with a target Big Five personality profile. We then evaluate the stability and fidelity of the induced personality using the IPIP-NEO questionnaire. Specifically, we ask: (i) does post-training (SFT, DPO, ORPO) stabilize questionnaire scores under prompt rephrasings, and (ii) can it induce target Big Five profiles from unguided essays? Our results demonstrate that fine-tuning consistently reduces variance in questionnaire responses across five models, directly mitigating the evaluation fragility reported in pre-trained models. However, this newfound stability reveals a more fundamental limitation: accuracy on the full five-dimensional profile remains near chance, even when single-trait scores improve. This indicates that unguided essays lack the cues needed for faithful personality expression. We therefore argue for scenario-grounded datasets or interactive elicitation that accumulates test-aligned evidence over time.
Prateek Rajput, Yewei Song, Iyiola E. Olatunji +2
University of Luxembourg, Esch-sur-Alzette, Luxembourg