Does Reasoning Improve Psychological Depth in Large Language Models?

Ruichen Zheng; Yihe Wang; Fabrice Harel-Canada; Sara Khosravi; Zeynep Senahan Yildiz; Amit Sahai; Nanyun Peng

cs.CLNov 6, 2026

Does Reasoning Improve Psychological Depth in Large Language Models?

Authors: Ruichen Zheng, Yihe Wang, Fabrice Harel-Canada, Sara Khosravi, Zeynep Senahan Yildiz, Amit Sahai, Nanyun Peng

Organizations: University of California, Los Angeles · Now at Arizona State University. · Cornell University · LA General Hospital ·

Abstract

Benchmark gains from reasoning-enhanced large language models encourage the assumption that these models should broadly perform better on complex generative tasks, but this remains underexplored in open-ended, creative domains shaped by nuances of reader experience. We tested this assumption through the lens of psychological depth in short stories. We compared DeepSeek-R1 vs. DeepSeek-V3, our closest available contrast for reasoning-oriented post-training, and GPT-5 vs. GPT-4o, through 60 blinded pairwise evaluations in which seven human annotators and an ensemble LLM-as-a-Judge rated prompt-matched story outputs. Human evaluations showed no universal reasoning advantage: GPT-5 was slightly preferred over GPT-4o (60.0-62.9% across two reasoning-effort settings), whereas DeepSeek-R1 fell behind DeepSeek-V3 (42.9%). Although this pattern was marked by low inter-annotator agreement (Krippendorff's $α= 0.070$ ), our annotation-quality checks suggested genuine evaluative subjectivity, plausibly reflecting distinct annotator decision schemes rather than annotation failure. In contrast, the LLM-as-a-Judge showed a strong pro-reasoning bias (98.3% of pairs vs. 55.2% of human votes), consistent across its five configurations and partly tracking sentence length. These findings suggest that, at the frontier, reasoning-related quality differences are small enough to be dominated by reader subjectivity, and that LLM-as-a-Judge evaluators can obscure this subjectivity in nuanced creative-generation tasks.

Figures & tables

Figure 1: Evaluating Psychological Depth: Human Subjectivity vs. LLM Consensus. Evaluation patterns across five psychological dimensions for a representative story pair (Pair 24, GPT-5 vs. GPT-4o). ( Left ) Human annotators displayed diverse preferences, reflecting subjective interpretations. ( Right ) All five LLM-as-a-Judge configurations in the ensemble collapsed this subjectivity into unanimous preference for the reasoning model.

Pair	Pref.	Agg. PDS	$\Delta$	$d$
	Human	Judge	Judge Bias
GPT-5 auto vs. 4o	60.0%	100.0%	$+0.518$	1.14
GPT-5 high vs. 4o	62.9%	100.0%	$+0.500$	1.12
DeepSeek-R1 vs. V3	42.9%	95.0%	$+0.214$	0.68
Overall	55.2%	98.3%	$\mathbf{+0.411}$	$\mathbf{0.95}$

Table 1: Preference rates for reasoning models: human annotators vs. LLM-as-Judge Evaluator. Pref. = overall preference; Agg. PDS = judge preference by mean dimension score (no ties).

\Delta

= mean Likert score difference (reasoning

-

non-reasoning) over all pair

\times

dimension cells;

d

= paired Cohen’s

d

(

\Delta

/SD).

	DeepSeek-R1	GPT-5 auto	GPT-5 high
Dimension	vs. V3	vs. 4o	vs. 4o
Authenticity	47.9% [33.5, 61.8]	55.7% [36.6, 75.9]	52.1% [28.4, 75.2]
Emotion Prov.	61.4% [51.2, 71.6]	70.7% [58.4, 83.3]	62.9% [46.2, 79.7]
Empathy	52.9% [35.2, 71.2]	68.6% [56.2, 84.1]	69.3% ‡ [57.7, 84.4]
Engagement	39.3% [22.0, 54.9]	58.6% [47.2, 70.2]	65.0% [50.1, 80.7]
Narr. Compl.	58.6% [47.2, 71.3]	55.0% [38.7, 72.1]	52.9% [33.1, 73.5]

Table 2: Human preference rates for the reasoning model with 95% GLMM intervals (Section 4.2 ;

n=140

judgments per cell). Values above 50% favor the reasoning model; bold intervals exclude 50%. Profile-likelihood intervals; ‡ Wald.

Figure 2: Preference rates for reasoning models across 7 individual human annotators, the human average, and the ensemble LLM judge. Judge ties count 0.5 per story (Appendix G ); † aggregate PDS (no ties).

Metric	Fleiss’ $\kappa$	Kripp. $\alpha$	Raw
Authenticity	0.035	0.037	52.5%
Emotion Prov.	0.080	0.082	54.1%
Empathy	0.184	0.186	59.2%
Engagement	0.059	0.061	53.3%
Narr. Complexity	0.016	0.018	52.1%
Preference	0.035	0.038	51.9%

Table 3: Inter-annotator agreement across 60 pairs and 7 annotators. Raw = average pairwise agreement rate.

Figure 3: Hierarchical clustering of annotator archetypes and PDS dimensions. Left: Logistic-regression coefficient profiles, where weights quantify each dimension’s influence on overall preference. Right: Intra-annotator Cohen’s $\kappa$ between dimensional and overall judgments.

Appendix figures & tables27 assets

Supplementary material from the paper’s appendix.

Appendix

Sourced from r/WritingPrompts
ID	Premise
1	While other god’s shrines are magnificent, yours is a bit too humbling. And yet a little girl visits you every year after stumbling upon it, never missing a year even as she grows old. Deeply moved, you decide to give her a parting gift greater than what any other God would dare to give.
2	Elves realized ages ago that, due to the lifespan differences between them, humans will eventually forget the reasoning for various treaties and break them. So it’s understandably shocking to the elves when they discover some long-forgotten treaties in their own records that humans never broke.
3	A princess who is going to be in an arranged marriage runs away. She cuts her hair and pretends to be a man. However, she runs into the prince who was going to get married to her. He also ran away, and he is pretending to be a woman. They instantly recognize each other.
4	"You know… Most vampires I hunted tend to see humans as just cattle." "Oh, don’t misunderstand me. This is just how I see you too." "Really? Then why don’t you hurt people in your town?" "Well, you don’t slaughter dairy cows for their meat, do you?"
5	You’re a hero with a weird name. “Anything for $20”. You gain the ability to do anything, as long as you’re offered$ 20. Everyone takes it as a joke, until one day there’s a cataclysm, and someone offers you $20 to end it.

Appendix

Table 4: All 20 premises used to prompt the models in our evaluation. The diverse sources ensure coverage across both speculative and grounded literary domains. Premises from r/WritingPrompts are reproduced verbatim, including original spelling and grammatical errors.

Model	Temperature	Other
GPT-4o	1.0 (default)	—
GPT-5 (auto)	1.0 (default)	—
GPT-5 (high)	1.0 (default)	reasoning_effort=high
DeepSeek-V3	1.0	—
DeepSeek-R1	1.0	thinking tags stripped

Appendix

Table 5: Generation parameters. All models used temperature

=1.0

and default settings for top-

p

, frequency penalty, etc. GPT-5 (high) used OpenAI’s reasoning_effort parameter. No max-token limit was set.

Model	$n$	Words	Sentences	Mean sent. len.	SD sent. len.	MATTR	MTLD	PPL
GPT-4o	40	552.4	29.7	18.9	8.4	0.910	144.0	9.5
GPT-5 (auto)	20	513.1	44.5	11.7	6.9	0.885	112.8	16.0
GPT-5 (high)	20	510.1	40.6	13.0	7.8	0.895	127.6	16.7
DeepSeek-V3	20	489.8	47.6	10.7	6.5	0.902	131.7	5.1
DeepSeek-R1	20	585.1	56.5	10.6	6.7	0.907	167.4	8.9
Non-reasoning	60	531.5	35.6	16.1	7.8	0.907	139.9	8.0

Appendix

Table 6: Surface features by model (means over stories). PPL = perplexity under Llama 3.1 70B. GPT-5 stories have a mean perplexity of 16.3 pooled over both reasoning settings, and GPT-5 sentences are on average 6.5 words shorter than those of the paired GPT-4o stories.

Feature	AUT	PROV	EMP	ENG	NCOM
Word count	$-.28^{**}$	$-.19^{*}$	$-.12$	$-.18^{*}$	$+.09$
Sent. len.	$-.32^{***}$	$-.43^{***}$	$-.17$	$-.70^{***}$	$-.09$
MATTR	$-.37^{***}$	$-.33^{***}$	$-.35^{***}$	$-.29^{**}$	$-.14$
MTLD	$-.24^{**}$	$-.19^{*}$	$-.25^{**}$	$-.09$	$+.03$

Appendix

Table 7: Story-level Pearson correlations between surface features and ensemble judge scores (

n=120

stories; Sent. len. = mean sentence length).

{}^{*}p<.05

{}^{**}p<.01

{}^{***}p<.001

Feature	AUT	PROV	EMP	ENG	NCOM
Word count	$-.64^{***}$	$-.40^{**}$	$-.18$	$-.54^{***}$	$+.11$
Sent. len.	$-.56^{***}$	$-.47^{***}$	$-.27^{*}$	$-.59^{***}$	$+.03$
MATTR	$-.31^{*}$	$-.17$	$-.24$	$-.28^{*}$	$+.13$
MTLD	$-.39^{**}$	$-.13$	$-.15$	$-.36^{**}$	$+.31^{*}$

Appendix

Table 8: Within-pair standardized coefficients (

\beta

) relating surface features to ensemble judge scores (

n=60

pairs), which control for premise differences.

{}^{*}p<.05

{}^{**}p<.01

{}^{***}p<.001

Figure 4: Annotation interface: instructions and PDS metric definitions shown to annotators.

Figure 5: Annotation interface: side-by-side story presentation with premise shown above.

Figure 6: Annotation interface: evaluation questions and optional comment field.

Figure 7: Distribution of annotation time per pair for each of the 7 human annotators. Times exceeding 600 s are capped for display; the number of capped outliers is shown in each subplot title. Red dashed lines mark the median; navy dotted lines mark the mean of capped times. The rightmost panel compares median and mean times across annotators. Overall median annotation time is 5.2 minutes (n = 420).

Figure 8: Annotation time per pair plotted chronologically, with linear trends per annotator. Red points denote times capped at 600 s. Dashed black lines show linear trends; colored dotted lines show 5-pair rolling averages. The rightmost panel overlays all trend lines.

Figure 9: Leave-One-Annotator-Out (LOAO) Stability Analysis: shift in the aggregate reasoning-model win rate when each annotator is removed.

	DeepSeek-R1	GPT-5 auto	GPT-5 high
Dimension	vs. V3	vs. 4o	vs. 4o
Authenticity	0.41 / 0.11	0.30 / 0.62	0.30 / 1.02
Emotion Prov.	0.00 ∗ / 0.04	0.00 ∗ / 0.27	0.09 / 0.49
Empathy	0.46 / 0.40	0.38 / 0.25	1.00 / 0.18
Engagement	0.34 / 0.36	0.00 ∗ / 0.09	0.05 / 0.40
Narr. Compl.	0.26 / 0.04	0.00 ∗ / 0.42	0.41 / 0.61

Appendix

Table 9: GLMM random-intercept variances,

\hat{\sigma}^{2}_{\text{pair}}\,/\,\hat{\sigma}^{2}_{\text{ann}}

(logit scale), for each matchup and dimension. ∗ Singular fit (pair variance estimated at zero).

	DeepSeek-R1	GPT-5 auto	GPT-5 high
Dimension	vs. V3	vs. 4o	vs. 4o
Authenticity	31.6%	87.2%	65.5%
Emotion Prov.	99.9%	100%	99.9%
Empathy	69.0%	100%	100%
Engagement	0.8%	97.7%	100%
Narr. Compl.	95.8%	89.1%	69.2%

Appendix

Table 10: Pair-level cluster bootstrap (10,000 resamples): share of resamples in which the reasoning model is preferred in more than 50% of judgments. Values near 100% favor the reasoning model and values near 0% favor the base model.

Evaluator	AUT	PROV	EMP	ENG	NCOM	Overall
Llama 3.1 70B demo=7	0.576	0.674	0.637	0.507	0.637	0.606
Llama 3.1 70B demo=5	0.588	0.655	0.596	0.450	0.575	0.573
Llama 3.1 70B demo=10	0.561	0.702	0.671	0.387	0.568	0.578
Llama 3.3 70B demo=10	0.580	0.676	0.702	0.407	0.550	0.583
Qwen 3.5 397B demo=7	0.511	0.663	0.602	0.564	0.496	0.567
Llama 3.1 70B CoT demo=10	0.499	0.627	0.658	0.356	0.664	0.561

Appendix

Table 11: Evaluator Spearman correlation (

\rho

) with human annotations across PDS dimensions. Bold values in the upper section indicate the best configuration per dimension, which are combined to form the Ensemble (bottom row).

Dimension	Ties	Decisive	Wins	Even-split
Authenticity	16	44	43 (97.7%)	85.0%
Emotion Prov.	7	53	53 (100.0%)	94.2%
Empathy	3	57	51 (89.5%)	87.5%
Engagement	1	59	57 (96.6%)	95.8%
Narr. Complexity	2	58	49 (84.5%)	83.3%
Pooled	29	271	253 (93.4%)	89.2%

Appendix

Table 12: Tie frequency and decisive-pair outcomes for the ensemble evaluator. Wins are reasoning-model wins among decisive pairs; Even-split

=(\text{wins}+0.5\times\text{ties})/60

, or

/300

for the pooled row. Preference is the judge’s aggregate PDS decision.

Judge	Auth.	Emo.	Emp.	Eng.	Narr.	Avg	Ties	Even-split
Llama 3.1 70B (5d)	43/44 ∗	54/57	49/54	47/51	53/55	94.3%	39	88.5%
Llama 3.1 70B (10d)	43/43	53/53 ∗	48/51	46/46	54/54	98.8%	53	90.2%
Llama 3.3 70B (10d)	32/33	53/55	51/57 ∗	47/58	47/56	89.6%	41	83.5%
Qwen3.5 397B (7d)	57/57	58/58	55/58	57/59 ∗	57/58	97.9%	10	96.3%
Llama 3.1 70B CoT	7/8	49/50	47/53	43/55	49/58 ∗	87.4%	76	77.7%
Ensemble	43/44	53/53	51/57	57/59	49/58	93.7%	29	89.2%

Appendix

Table 13: Individual judge preference rates for the reasoning model across all 60 pairs, reported as strict wins / decisive pairs (ties excluded). ∗ denotes the routed dimension. The Avg column is the mean win rate across the five dimensions. Even-split

=(\text{wins}+0.5\times\text{ties})/300

	PDSv1		This study
Dimension	C,1	C,5	C,1	C,7	C,5 †
Authenticity	0.14	0.44	0.04	0.21	0.16
Emotion Prov.	0.26	0.63	0.08	0.39	0.31
Empathy	0.31	0.69	0.19	0.62	0.54
Engagement	0.26	0.63	0.06	0.31	0.25
Narr. Complexity	0.23	0.60	0.02	0.12	0.09

Appendix

Table 14: Intraclass correlations (two-way consistency) for single-rater judgments (C,1) and

k

-rater averages (C,

k

). PDSv1: 97 stories rated by five raters on a 5-point scale ( Harel-Canada et al., 2024 ) . This study: 60 pairs, seven readers, binary choices. † Projected to five readers with the Spearman–Brown formula.

Metric	Overall $\alpha$	DeepSeek-R1 vs V3	GPT-5 (auto) vs 4o	GPT-5 (high) vs 4o
Authenticity	0.037	0.073	0.053	$-0.015$
Emotion Prov.	0.082	0.035	0.147	0.072
Empathy	0.186	0.050	0.176	0.290
Engagement	0.061	0.101	0.010	0.080
Narr. Complexity	0.018	0.049	$-0.013$	0.005
Preference	0.038	0.074	$-0.003$	0.050

Appendix

Table 15: Krippendorff’s

\alpha

inter-annotator agreement broken down by specific model pairing.

Figure 10: Agreement between the LLM-as-Judge evaluator and human majority vote by model pair. The dashed line indicates chance (

50\%

). A tied judge decision counts as 0.5 agreement (even split).

Pair	Ann	Pref.	Comment
Pair 0 (4–3 for A)	1	A	“A: greater focus on the abstract. B: prose heavy, but greater focus on concrete; convoluted”
	5	A	“Story A was much more engaging because it was straightforward in depicting the stages of life…”
	7	B	“Story B delves deeper into the characterization of the humble god…Death of worshiper is more profound”
Pair 24 (4–3 for A)	1	A	“A: good imagery, dialogue. B: interesting parallels, odd tone”
Pair 24 (4–3 for A)	7	B	“A is a step by step of the prompt given. B builds up characters, establishes risk, and is authentic.”
Pair 27 (4–3 for B)	1	A	“A: like a prologue. B: prose, very nonsensical”

Appendix

Table 16: Selected annotator comments across five story pairs. Only annotators who provided free-text justifications are shown. A and B denote the order in which the two stories were displayed.

Pair 10: DeepSeek-R1 vs. V3
	AUT	PROV	EMP	ENG	NCOM	PREF
Human	2	3	0	0	4	0
Judge	$.00$	$+.52$	$-.32$	$+.30$	$+.10$	$+.12$
Pair 17: DeepSeek-R1 vs. V3
Human	7	7	7	4	3	7
Judge	$.00$	$+.46$	$+.58$	$+1.10$	$+.08$	$+.44$

Appendix

Table 17: The three pairs with unanimous (7–0) human overall preference. Human: number of the seven annotators preferring the reasoning model (bold = unanimous). Judge: ensemble score difference, reasoning

-

base, on the 1–5 scale; for PREF, the difference in aggregate PDS.

Figure 11: Preference rates for DeepSeek-R1 vs. DeepSeek-V3 (20 pairs) across 7 human annotators, the human average, and the LLM-as-Judge evaluator (ties split evenly; † aggregate PDS, no ties).

Figure 12: Preference rates for GPT-5 (auto) vs. GPT-4o (20 pairs) across 7 human annotators, the human average, and the LLM-as-Judge evaluator (ties split evenly; † aggregate PDS, no ties).

Figure 13: Preference rates for GPT-5 (high) vs. GPT-4o (20 pairs) across 7 human annotators, the human average, and the LLM-as-Judge evaluator (ties split evenly; † aggregate PDS, no ties).

Panel of Figure 14 — Figure 14: Story A as displayed to annotators (GPT-4o, base model) for Pair 56.

Panel of Figure 15 — Figure 15: Story B as displayed to annotators (GPT-5, high reasoning) for Pair 56.

	Auth.	Emo.	Emp.	Eng.	Narr.	Pref.
Ann. 1	N	N	N	N	R	N
Ann. 2	R	N	N	R	R	R
Ann. 3	N	N	N	N	R	N
Ann. 4	N	N	N	R	N	N
Ann. 5	N	N	N	R	N	N
Ann. 6	N	N	N	R	R	N

Appendix

Table 18: Per-annotator pairwise preferences for Pair 56 (GPT-5 high vs. GPT-4o). R = prefers reasoning model (GPT-5), N = prefers non-reasoning model (GPT-4o).

Text Generation

Text generation studies methods for producing coherent, useful language from prompts, data, and dialogue context.

See all 435 papers

Does Reasoning Improve Psychological Depth in Large Language Models? It Depends on Who's Judging
Sep 12, 2026Ruichen Zheng, Yihe Wang, Fabrice Y Harel-Canada +4LLM-as-a-Judge Language Model Generation Evaluation
University of California, Los Angeles · Tsinghua University · LA General Hospital +1
Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens
Feb 13, 2026Wei-Lin Chen, Liqian Peng, Tian Tan +5LLM Inference Efficiency Inference-Time Scaling
University of Virginia · Google
Level-k Distinguishable Mechanisms for Evaluating Bounded Rationality in LLMs
Aug 21, 2026Binchi Zhang, Atrisha SarkarLLM Evaluation Strategic Reasoning Risks in Language Models
University of Western Ontario

Sep 12, 2026J/K move · Enter open · S save

Does Reasoning Improve Psychological Depth in Large Language Models? It Depends on Who's Judging

Ruichen Zheng, Yihe Wang, Fabrice Y Harel-Canada, Sara Khosravi+3

University of California, Los Angeles · Tsinghua University · LA General Hospital+1

LLM-as-a-Judge Language Model Generation Evaluation Bias in LLM-as-a-Judge

Open paper AskPDF arXiv

LLM-as-a-Judge evaluators are increasingly used to score open-ended generation, yet a judge's correlation with human ratings on its development set may not guarantee valid measurement when outputs are closely matched and human preferences are subjective. We study this failure mode through psychological depth in short stories. Seven human readers and an LLM-judge ensemble selected on the original scalar Psychological Depth Scale dataset (

ρ= 0.646

) evaluated 60 blinded, prompt-matched story pairs from GPT-5 vs.\ GPT-4o and DeepSeek-R1 vs.\ DeepSeek-V3. Human preferences showed no universal reasoning advantage: GPT-5 was modestly preferred over GPT-4o (60.0--62.9%), whereas DeepSeek-R1 trailed V3 (42.9%), and inter-reader agreement was near chance (Krippendorff's

α= 0.070

), with within-reader consistency and recurring weighting patterns suggesting structured heterogeneity rather than random responding. The judge, by contrast, favored reasoning outputs in 89.0% of dimension-level comparisons and 59 of 60 pairs on aggregate PDS, uniformly across all five evaluator configurations, and its scores were associated with surface features such as sentence length and lexical diversity. These results suggest that development-set performance is insufficient evidence for deployment validity on a shifted distribution, and that point-estimate judges can obscure the heterogeneity in subjective human evaluation.

Abstract

Text Generation

Explore similar work

Does Reasoning Improve Psychological Depth in Large Language Models? It Depends on Who's Judging

Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens

Level-k Distinguishable Mechanisms for Evaluating Bounded Rationality in LLMs