Personality traits are richly encoded in natural language, and large language models (LLMs) trained on human text can simulate personality when conditioned on persona descriptions. However, existing evaluations rely predominantly on questionnaire self-report by the conditioned model, are limited in architectural diversity, and rarely use real human psychometric data. Without addressing these limitations, it remains unclear whether personality conditioning produces psychometrically informative representations of individual differences or merely superficial alignment with trait descriptors. To test how robustly LLMs can encode personality into extended text, we condition LLMs on real psychometric profiles from 290 participants to generate first-person life story narratives, and then task independent LLMs to recover personality scores from those narratives alone. We show that personality scores can be recovered from the generated narratives at levels approaching human test-retest reliability (mean r = 0.750, 85% of the human ceiling), and that recovery is robust across 10 LLM narrative generators and 3 LLM personality scorers spanning 6 providers. Decomposing systematic biases reveals that scoring models achieve their accuracy while counteracting alignment-induced defaults. Content analysis of the generated narratives shows that personality conditioning produces behaviourally differentiated text: nine of ten coded features correlate significantly with the same features in participants' real conversations, and personality-driven emotional reactivity patterns in narratives replicate in real conversational data. These findings provide evidence that the personality-language relationship captured during pretraining supports robust encoding and decoding of individual differences, including characteristic emotional variability patterns that replicate in real human behaviour.
Figures & tables
Figure 1: Per-domain HEXACO recovery (teal; GPT-4.1 generator, Sonnet scorer, N=290 ) vs. human test-retest reliability (rose; Henry et al., 2022, HEXACO-100, N=416 ). Error bars: 95% CI via Fisher z -transform.
Figure 2: Round-trip recovery across generators from 6 providers (including Mercury 2, a diffusion LM), scored by 3 independent scorers. Bars: mean r across 6 HEXACO domains ( N=290 for primary generators, N=154 for extended set), colour-coded by provider. Dashed line: human test-retest (95% CI shaded). Error bars: 95% CI. Full results in Appendix I .
Figure 3: Signal degradation across three stages: profile ceiling (Sonnet on prose profiles, N=290 ; r=0.839 - 0.907 ), LSI narrative recovery (GPT-4.1 generator, N=290 ; r=0.682 - 0.825 ), and real conversation (Spearman ρ , N=248 ; ρ=0.046 - 0.418 ).
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Generator
HH
E
EX
A
C
OP
GPT-4.1
5.0
1.0
3.4
4.8
5.0
4.6
Gemini Flash
5.0
1.3
4.3
4.4
5.0
4.7
Grok Fast
5.0
1.0
4.8
4.5
4.9
4.6
Human mean
3.43
3.29
3.23
3.29
3.69
3.69
Appendix
Table 1: Unconditioned B60 self-report scores (temperature = 0) compared to human population means.
Generator
HH
E
EX
A
C
OP
GPT-4.1
4.4
2.7
4.0
4.0
4.0
4.3
Gemini Flash
4.8
2.8
4.1
3.9
4.6
4.7
Grok Fast
4.6
2.2
4.7
3.6
4.1
4.6
Appendix
Table 2: Unconditioned LSI scores (entity prompt, no personality conditioning, Sonnet-scored, temperature = 0).
Generator
B60 SD ( N =10)
LSI SD ( N =5)
GPT-4.1
0.115
0.115
Gemini Flash
0.093
0.159
Grok Fast
0.097
0.189
Appendix
Table 3: Mean SD across 6 HEXACO domains. B60 variance is the model’s natural wobble on the questionnaire; LSI variance is the model’s variability in generating and being scored on different life stories. Both are far smaller than conditioned variance (SD ≈ 0.55–0.75 across 290 PIDs), confirming that personality conditioning does real work.
Rank
Generator
Provider
Sonnet
GPT-5.4
Gemini
Mean
1
GPT-4.1 †
OpenAI
0.750
0.738
0.728
0.739
2
Gemini 3 Flash †
Google
0.744
0.736
—
0.740
3
Grok 4.1 Fast †
xAI
0.740
0.735
0.719
0.731
4
GPT-4.1 Mini
OpenAI
0.740
0.734
0.712
0.729
5
GPT-5.4
OpenAI
0.723
—
—
0.723
6
Mercury 2 †
Inception Labs
0.693
0.697
0.654
0.681
Appendix
Table 4: Extended generator comparison. Mean r across 6 HEXACO domains, self-scoring excluded. † Primary generators and Mercury 2 at N=290 ; all others at N=154 . Mercury 2 is a diffusion language model; all other generators are autoregressive.
Subscale
r
p
N
Trust: Benevolence
0.645
<10−35
290
PPTS: Interpers. Manip.
0.598
<10−29
290
PPTS: Affective Resp.
0.594
<10−29
290
SIAS
0.561
<10−25
290
PPTS: Egocentricity
0.544
<10−24
290
Trust: Integrity
0.487
<10−18
290
Appendix
Table 5: Beyond-HEXACO subscale recovery ( N=290 , GPT-4.1 generator, Sonnet scorer). All 9 subscales are recovered at p<.0033 (Bonferroni-corrected α=0.05/15 ).
Table 7: Cross-context correlations between content features coded in LLM-generated LSI narratives and real PARSEL conversations for the same individuals ( N=248 with ≥ 3 conversations, GPT-4.1 generator, Gemini–Haiku coder average). Nine of ten features are significant (Bonferroni α=0.005 , 10 tests).
Feature
Domain
GPT-4.1
Gem. Flash
Grok Fast
Mercury 2
Vulnerability
EM
0.744
0.750
0.683
0.607
Emotional intensity
EM
0.576
0.657
0.496
0.431
Warmth
AG
0.584
0.585
0.557
0.468
Communion
AG
0.526
0.520
0.515
0.441
Agency
EX
0.494
0.519
0.411
0.391
Meaning-making
HH
0.499
0.477
0.492
0.317
Appendix
Table 8: Key feature–personality correlations across four generators ( N=290 each, Gemini–Haiku coder average). Mercury 2 is a diffusion language model; all others are autoregressive. The same features load on the same domains regardless of generator or generation paradigm, with Mercury 2 showing attenuated but directionally consistent signal.
Feature
GPT-4.1
Gem. Flash
Grok Fast
Mercury 2
Vulnerability
.243
.260
.230
.181
Agency
.207
.115
.149
.207
Emot. intensity
.161
.129
.183
.215
Emot. valence
.190
.154
.182
.138
Dominance
.200
.190
.166
.191
Emot. complexity
.137
.171
.193
.123
Appendix
Table 9: Cross-context correlations (narrative vs. real conversation) by generator ( N=248 ). Mercury 2 is a diffusion language model. Vulnerability is the strongest bridge feature for all generators. All four generators produce a positive cross-context bridge, confirming the result generalises across generation paradigms.
1Graduate School of Data Science, Seoul National University · Department of Communication, Interdisciplinary Program in Artificial Intelligence, Seoul National University
School of Computer Science and Engineering, Northeastern University, Shenyang, Liaoning, China · School of Computing and Information Systems, Singapore Management University, Singapore