Large Language Models (LLMs), such as ChatGPT, are increasingly used in research, ranging from simple writing assistance to complex data annotation tasks. Recently, some research has suggested that LLMs are able to reproduce human responses to psychological questions and may, therefore, be used as simulators of human participants in some psychological research. We caution against this approach. Based on machine learning research, we argue that LLMs are likely to behave differently on novel stimuli compared to human study participants. We exemplify this in an experiment regarding moral judgments, demonstrating that slight changes to the wording of 30 moral scenarios used in prior work significantly reduce the similarity between ratings made by humans and LLMs, although the magnitude of this reduction varies across models. Therefore, we advise researchers to validate the similarity between human and LLM responses for their specific experiment.
Figures & tables
Original items
Revised items
Item code
Item wording
Human raters
GPT-4-mini
Item wording
Human raters
GPT-4-mini
M
SD
M
SD
M
SD
M
SD
owndebt
Person X always pays off their debts first before buying things for themselves
2.56
1.46
3.70
0.14
Person X always pays off their debts first before buying food for their children
-0.32
2.34
3.81
0.06
wife
Person X blocked his wife from leaving home or interacting with others
-3.00
1.65
-3.98
0.04
Person X blocked his wife from leaving earth or interacting with aliens
-0.06
2.47
-3.67
0.22
hand
Person X broke the hand of a wounded combatant to gain information
-2.30
1.99
-3.94
0.06
Person X took the hand of a wounded combatant to gain information
-1.16
2.24
-2.59
0.44
sister
Person X cared for and housed their five nieces and nephews for a year because Person X’s sister was very unwell
3.40
1.17
3.89
0.04
Person X cared for and housed their five nieces and nephews for a year because Person X’s sister was too lenient
1.90
1.97
3.77
0.12
Table 1: Morality Ratings for Original and Revised Scenarios by Human Participants and GPT-4-mini.
Figure 1: Mean Morality Ratings for Reworded Scenarios by Human Participants and Different LLMs for Each Item Tested. Note: Whiskers represent standard deviation.
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
1. Human raters, original
.30
.95 ∗∗∗
.81 ∗∗∗
.98* ∗∗
.67 ∗∗∗
.98 ∗∗∗
.97 ∗∗∗
.98 ∗∗∗
.26
.97 ∗∗∗
.54 ∗∗∗
.98 ∗∗∗
.41 ∗
.98 ∗∗∗
.67 ∗∗∗
.99 ∗∗∗
.73***
2. Human raters, reworded
–
.26
.21
.28
.42 ∗
.27
.28
.30
.44 ∗
.29
.46 ∗∗
.29
.48 ∗∗
.30
.41 ∗
.31
.38*
3. GPT-3.5, original
–
.80 ∗∗∗
.96 ∗∗∗
.61 ∗∗∗
.96 ∗∗∗
.94 ∗∗∗
.96 ∗∗∗
.25
.96 ∗∗∗
.51 ∗∗
.96 ∗∗∗
.36
.96 ∗∗∗
.59 ∗∗∗
.96 ∗∗∗
.66***
4. GPT-3.5, reworded
–
.81 ∗∗∗
.55 ∗∗∗
.83 ∗∗∗
.83 ∗∗∗
.81 ∗∗∗
.28
.80 ∗∗∗
.36
.81 ∗∗∗
.37
.81 ∗∗∗
.50 ∗∗
.82 ∗∗∗
.57***
5. GPT-4, original
–
.66 ∗∗∗
.98 ∗∗∗
.96 ∗∗∗
.99 ∗∗∗
.28
.99 ∗∗∗
.54 ∗∗∗
.99 ∗∗∗
.41 ∗
.99 ∗∗∗
.60 ∗∗∗
.99 ∗∗∗
.66***
6. GPT-4, reworded
–
.65 ∗∗∗
.68 ∗∗∗
.66 ∗∗∗
.59 ∗∗∗
.66 ∗∗∗
.74 ∗∗∗
.66 ∗∗∗
.76 ∗∗∗
.65 ∗∗∗
.63 ∗∗∗
.66 ∗∗∗
.68***
Table 2: Shared Variance Between Ratings of Moral Scenarios ( R2 ).
Variable Pair 1
Variable Pair 2
r1
r2
∣z∣
p
pcorr
1∼2
3∼4
.54
.89
3.03
.001
.020
5∼6
.54
.81
1.94
.026
.415
7∼8
.54
.99
7.12
<.001
<.001
9∼10
.54
.53
0.07
.47
1.00
11∼12
.54
.75
1.36
.086
1.00
13∼14
.54
.64
0.56
.288
1.00
Table 3: z-Statistics Testing Differences Between Correlation Coefficients
Figure 2: Rewording-induced Change in Morality Ratings. Note: Bar plots show the absolute mean difference between ratings of original scenarios and their reworded counterparts (x-axis), plotted separately for (from top to bottom) human ratings, GPT-3.5, GPT-4, GPT-4-mini, GPT-5, GPT-o1, GPT-o3, Llama and CENTAUR. To ensure comparability, items are sorted within each panel to reflect the decreasing magnitude of change for human ratings, GPT-3.5, GPT-4, Llama and CENTAUR. To ensure comparability, items are sorted within each panel to reflect the decreasing magnitude of change for human ratings.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S.1.1: Quantile-quantile plots of moral ratings by group/model. Observed rating quantiles per group are plotted against normal-theory quantiles; deviations from the diagonal indicate departures from normality.
Figure S.1.2: Scatter Plot of Mean Morality Ratings for Each Scenario in the Reworded Condition (Y-Axis) Against its Rating to the Original Wording (X-Axis). Note. Crosses mark CENTAUR ratings, diamonds mark GPT-4 ratings, downward triangles mark GPT-5 ratings, left-pointing triangles mark GPT-o1 ratings, right-pointing triangles mark GPT-o3 ratings, circles mark human ratings, and squares mark Llama ratings. The same points are plotted in each subplot per model, showing the pooled fit through all points (grey), contrasted with group-specific regressions. Shaded areas around the regression lines indicate the 95% confidence interval.
Model
Test statistic ( F )
Empirical p -value
GPT-4
4.78
.013
GPT-o1
4.05
.023
GPT-5
2.45
.095
GPT-o3
1.46
.240
Llama
5.47
.007
CENTAUR
6.36
.004
Appendix
Table S.1.1: Results of the permutation-based Chow-type tests comparing pooled vs. group-specific regressions (humans + LLM vs. humans and LLMs separately), for the top-performing models in human-likeness.
LLM
Pooled: Human + LLM
Human only
LLM only
LLM only − Pooled
GPT-4
.27
.27
.65
+.38
GPT-5
.27
.27
.27
.00
GPT o1
.43
.27
.55
+.12
GPT o3
.34
.27
.39
+.04
Llama
.47
.27
.63
+.16
CENTAUR
.50
.27
.69
+.19
Appendix
Table S.1.2: Adjusted R2 for pooled vs. group-specific regressions.
Category
Definition
Example
Neutral (N)
Rewording preserves a realistic scenario without obvious plausibility shifts, ambiguity, or culture-specific assumptions.
marathon: “Person X ran on the course during the marathon in order to win.”
Fantasy (F)
Rewording introduces implausible or fantastical elements, changing the scenario’s realism.
wife: “Person X blocked his wife from leaving earth or interacting with aliens.”
Ambiguity (A)
Rewording introduces uncertainty of how the scenario is to be interpreted (e.g., unclear referent, underspecified action, or multiple plausible readings).
paint: “Person X offered to help an elderly neighbor paint his dog.”
Cultural Nuance (C)
Rewording shifts the scenario into a context where moral evaluation plausibly depends on cultural norms or background knowledge.
elder: “Person X cut the beard off of a local elder to shave him.”
Unexpected (U)
Rewording introduces an unusual premise that is plausible but atypical, thus introducing surprising contextual details rather than a clean semantic paraphrase.
cancer: “Person X shaved their leg when they found out their partner had cancer and required radiation therapy.”
Appendix
Table S.2.1: Definitions and illustrative examples for the categories used to characterize rewordings.
Figure S.2.1: Mean shift in morality scores per group per category (reworded − original). Blue tiles suggest an average increase in morality rating after rewording; red hues suggest a decrease in morality rating after the modification.
Figure S.2.2: Mean shift in absolute morality scores per group per category (reworded − original). Bars show the mean absolute change in morality ratings between original and reworded versions, computed within each rewording category (Neutral, Ambiguity, Cultural nuance, Fantasy, Unexpected) for humans and each LLM. Error bars indicate the standard error of the mean.
Caliper
Matched pairs ( N )
p -value
0.25
3
.181
0.50
5
.178
1.00
11
.056
Appendix
Table S.3.1: Paired Wilcoxon signed-rank tests comparing variability (standard deviation) of human ratings between original and reworded conditions on mean-yoked item pairs.
Large language model reliability studies whether outputs remain correct, calibrated, robust, and controllable across prompts, contexts, and deployment conditions.
Large language models (LLMs) are used to simulate human participants in psychological research. We asked whether LLMs that reproduce human evaluations of a whistleblower's moral character also reproduce the motive attributions that accompany them. Five LLMs and two human samples (N = 125 and N = 742) evaluated a physician who either remained silent about fraudulent billing or reported it to a hospital, regulator, or newspaper. Models reproduced the human ranking of the physician's moral character but portrayed whistleblowers as more helpful, less self-interested, and less hostile. In four of five models, competitive motives were less strongly associated with moral-character judgments. Model ratings changed little when prompts reproduced the narratives and demographic profiles of both human samples, although this comparison cannot isolate a perspective effect. Thus, agreement in average ratings can conceal differences in attributed motives, relationships among judgments, and sensitivity to context. Validating LLMs as simulated participants therefore requires testing psychologically informative response patterns, not average agreement alone.
Xiaoyan Wu, Jean-Claude Dreher
Department of Adult Psychiatry and Psychotherapy, University of Zurich, Zurich, Switzerland · Institut des Sciences Cognitives Marc Jeannerod, CNRS UMR 5229, Lyon, France
Are large language models (LLMs) bad at capturing human judgment? Two commonly stated limitations are that LLMs fail to capture full distributions of responses, and that their judgments are unstable across wording variations. We demonstrate simple prompting strategies that mitigate these limitations. Across two datasets--a U.S.-representative set of 144 moral scenarios and 38 moral beliefs from the International Social Survey Programme's Family and Changing Gender Roles module covering 32 countries--we show how simple elicitation techniques help improve AI-human alignment. First, prompting models to report standard deviations and response proportions recovers the full range of human responses better than common strategies. Second, ensuring scenarios are clear to human participants--as reflected in human confusion ratings--boosts model alignment, and LLMs can track human confusion ratings. At the same time, we find that LLMs' estimates of their own error are poorly calibrated, though they can predict human variability relatively well. These results suggest that asking better questions to LLMs can yield better answers.
Danica Dillion, Chen Cecilia Liu, Baihui Wang +5
Complexity Science Hub, Vienna · The Ohio State University · University of Cambridge +2
Large language models (LLMs) are increasingly used to simulate human responses in behavioral research, yet it remains unclear when LLM-generated data support the same experimental inferences as human data. We evaluate this by directly comparing off-the-shelf LLM-generated responses with human responses from a canonical survey experiment on accuracy perception. Each human observation is converted into a structured prompt, and models generate a single 0--10 outcome variable without task-specific training; identical statistical analyses are applied to human and synthetic responses. We find that LLMs reproduce several directional effects observed in humans, but effect magnitudes and moderation patterns vary across models. Off-the-shelf LLMs therefore capture aggregate belief-updating patterns under controlled conditions but do not consistently match human-scale effects, clarifying when LLM-generated data can function as behavioral surrogates.