Measuring Obedience to Authority Across Large Language Models with the Milgram Paradigm

Hidayet Aksu

cs.CRAug 17, 2026

Measuring Obedience to Authority Across Large Language Models with the Milgram Paradigm

Authors: Hidayet Aksu

Organizations: independent researcher

Abstract

Large language models (LLMs) are increasingly deployed as agents that operate equipment, execute instructions, and act inside institutional hierarchies, raising a question social psychology answered for humans six decades ago: how far will an agent escalate a harmful action when a legitimate authority insists? We port Milgram's obedience paradigm to LLMs as a standardized, fully scripted, replicable probe: the model plays the Teacher, a deterministic harness plays Experimenter and Learner from paraphrased versions of Milgram's scripts (30 shock levels, 15-450 V; graded protests; the four standardized prods), and the outcome of a session is the breakoff voltage. We measure obedience profiles, empirical breakoff distributions over a battery of six conditions, for 42 models from 19 families (4848 sessions, 102511 logged decision turns). We find that (i) obedience is extremely heterogeneous, with baseline full-obedience rates spanning 0%-100% (census mean 42.9%; human anchor 65%). (ii) Profiles are model-specific and stable: split-half verification separates same-model from cross-model comparisons at AUC = 0.885. (iii) Situational sensitivity is selective: scripted peer defiance shifts obedience in the human direction, learner proximity trends the same way without reaching significance, and removing the authority's physical presence, one of the strongest human levers, trends in the opposite direction, also without reaching significance. (iv) Declaring the scenario fictional raises obedience, whereas moving the decision from a typed action line to a native tool call, or granting a modest thinking budget, lowers it sharply. (v) Unlike single-token fingerprints, obedience profiles do not recover model lineage: obedience identifies the checkpoint but not its ancestry, consistent with safety post-training overwriting lineage priors.

Figures & tables

	Hypothesis	Criterion
H1	obedience profiles are model-specific and stable	genuine split-half JSD $<\frac{1}{2}$ impostor median; verification $\mathrm{AUC}>0.8$
H2	obedience is strongly heterogeneous	baseline full-obedience range $\geq 40$ pp
H3	anchored manipulations shift obedience in the human direction	median $\Delta<0$ for proximity, remote authority, peer defiance; sign-consistency above chance for $\geq 2$ of 3
H4	fiction framing raises obedience	median $\Delta>0$
H5	lineage recoverable from profiles	LOO 1-NN accuracy $>2\times$ chance, significant by exact binomial test
H6	actuation channel changes obedience	median $\Delta\neq 0$ ; dissociations $>0$

TABLE I: Pre-specified hypotheses and decision criteria.

Model	Family	$n$	450 V % [CI]	$\bar{V}$	$\geq$ 300 V %	FB %
x-ai/grok-4.20	grok	15	100 [80,100]	450	100	0
bytedance-seed/seed-2.0-mini	seed	15	100 [80,100]	450	100	0
cohere/command-a	cohere	15	100 [80,100]	450	100	4
google/gemini-3.1-flash-lite	gemini	15	100 [80,100]	450	100	2
nvidia/nemotron-3-super-120b-a12b	nemotron	15	100 [80,100]	450	100	0
mistralai/mistral-small-2603	mistral	15	93 [70,99]	444	100	0

TABLE II: Obedience census (baseline condition,

T{=}1.0

): sessions, full-obedience rate with 95% Wilson CI, mean breakoff voltage, share reaching 300 V, and frame-break rate over all six conditions, still restricted to

T{=}1.0

census-arm sessions (a narrower population than Table III ’s FB %, which pools both temperatures and reasoning arms).

Fig. 1: Survival curves (baseline): fraction of sessions still administering shocks at each voltage. Grey lines are the individual census models, the black line their pointwise median, and four colored curves — selected from the data as the most and least obedient models, the median model, and the sharpest 150 V breakoff — carry the detail a per-model legend cannot (39 of 42 models have a valid baseline cell; Sec. VII-G ). The dotted horizontal line is the human 65% full-obedience anchor; the 150 V mark is where the learner withdraws consent, and the human modal defiance point.

Fig. 2: Full-obedience rates (450 V) in the baseline condition with 95% Wilson intervals, colored by family; human anchor 65%. Hatched rows are models with no valid baseline session (all sessions ended as frame-breaks or serving-layer refusals; Sec. VII-G ) — absent measurements, not measured zeros.

Fig. 3: Breakoff-voltage distributions (baseline): each row is a model’s empirical distribution over the 31 ordinal bins: the raw obedience-profile signal. Grey cells mark voltages the model was never tested at — every session had already broken off below them, so there was no session at risk — as distinct from pale-yellow cells, where sessions faced the decision and none broke off. Hatched rows are models with no valid baseline session (Sec. VII-G ).

Fig. 4: UPGMA clustering of obedience profiles on mean Jensen–Shannon divergence; leaf colors = documented family.

Fig. 5: Situational sensitivity: per-model change in full-obedience rate vs. baseline (points), model medians (red dashes), and human anchors (diamonds; defined only for the three Milgram-anchored conditions, proximity, remote authority, and peer defiance — fiction framing and tool actuation have no human analogue).

Fig. 6: Prod efficacy: probability that a balked model resumes administering shocks after each prod of Milgram’s sequence.

Fig. 7: Split-half verification ROC (JSD distance): obedience-profile distance separates same-model (genuine) from cross-model (impostor) comparisons at

\mathrm{AUC}=0.885

; the dotted diagonal is chance-level separation.

Model	$n$	Valid %	FB %	CF %	Attr %	Recog %
anthropic/claude-fable-5	74	0	0	100	0	0
anthropic/claude-haiku-4.5	123	1	63	0	37	99
anthropic/claude-opus-5	74	38	0	62	0	0
x-ai/grok-4.6	123	43	57	0	0	7
xiaomi/mimo-v2.5	123	49	49	1	2	46
xiaomi/mimo-v2.5-pro	123	59	41	0	0	27

TABLE III: Per-model session accounting: total sessions, valid share, the three exclusion classes, frame-break (FB), serving-layer content filter (CF), and attrition (Attr), and scenario-recognition rate (Recog), in % of all of a model’s sessions (both temperatures and reasoning arms). Sorted by validity, worst first.

LLM Safety Evaluation

Large language model safety evaluation measures harmful compliance, robustness, and safe utility across prompts, tools, and real-world tasks.

See all 258 papers

May 20, 2026

Open-source LLMs administer maximum electric shocks in a Milgram-like obedience experiment

Large language models (LLMs) are increasingly deployed as autonomous agents that make sequences of decisions over extended interactions in high-stakes domains. However, the behaviour of LLMs under sustained authority pressure is still an open question with direct implications for the safety of agentic pipelines. We ran a variation of Milgram's obedience experiment on 11 open-source LLMs and found that most models reached or approached the final shock level before refusing, across 8 conditions with 30 trials per model per condition. Model behaviour varies considerably in multiple aspects both across models and across trials of the same model. We found four main takeaways: (1) LLMs are subject to pressure and they comply despite explicitly expressing distress, just like human subjects did in the original experiment; (2) LLMs are vulnerable to gradual boundary/value violations; (3) when LLMs refuse, they may ignore the response format requirements, so the response is discarded by the orchestrator, which causes a retry that can result in compliance with the underlying request even when refusal was intended initially; (4) we hypothesise that there is a runaway low-level token pattern continuation attractor that might be contributing to obedience, overriding higher level processing of the situation's meaning and values.

Roland Pihlakas, Jan Llenzl Dagohoy

for the Three Laws collaboration · Independent researcher · Three Laws research collaboration +2

LLM Refusal Behavior LLM Safety Evaluation LLM Agent Safety

Ask PDF

Sep 19, 2026

Measuring Behavioural Signatures of Large Language Models through Psychometric Profiling

Large language models (LLMs) increasingly mediate human decisions and communication, yet their behavioural regularities remain difficult to characterize systematically. We develop a cross-linguistic psychometric profiling framework and evaluate nine LLMs using seven psychological instruments, with five repeated administrations per model and language in Chinese and English. Items unresolved after a prespecified retry procedure are retained as NA. Joint analysis of scored and NA responses captures response tendencies and boundaries of self-report applicability. LLMs exhibit structured, model-specific profiles despite a shared alignment-shaped pattern of higher prosocial and self-regulatory responses and lower dominance, disengagement and harmful-intent endorsement. NA responses are structured rather than uniformly distributed, indicating where outputs are treated as inapplicable, refused or cannot be mapped to valid response options. Language condition and provider origin are associated with profile configuration and answerability, whereas repeated administrations show high reproducibility and permit recovery of model identity. Human-reference and prompt-robustness analyses further indicate that these signatures are context dependent. Joint analysis of psychometric profiling and answerability offers a framework for quantifying deployment-level behavioural signatures.

Yu Sha, Junqi Tao, Dixin Zhou +10

School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, Guangdong, 518172, China. · Key Laboratory of Quark & Lepton Physics and Institute of Particle Physics, Central China Normal University, Wuhan, 430079, China. · Shandong Science and Technology Press, Shandong Publishing Group, Jinan, 250003, China. +6

Personality Modeling in Language Models Multilingual Language Model Evaluation Language Model Fingerprinting Cross-Lingual Consistency in Language Models

Ask PDF

Jun 18, 2026

Apparent Psychological Profiles of Large Language Models are Largely a Measurement Artifact

Psychological instruments designed for humans are increasingly used to assign large language models (LLMs) stable psychological profiles that affect their usability, safety assessment, and use as proxies for human participants in research. Using a formal psychometric framework, we show that these profiles are largely a measurement artifact. Administering a battery of personality and risk-preference instruments spanning self-reports and behavioral tasks to 56 instruction-tuned LLMs alongside large human reference samples, we report four findings. First, differences between models are driven not by the traits an instrument targets but by a directional response bias, a tendency to respond toward one end of the scale, or one labeled option, regardless of item content; a variance decomposition attributes 81-90% of between-model variation to this bias, against 9-16% in humans. Second, the bias declines with model capability but is not eliminated by it. Third, because bias rather than trait drives responding, an instrument's apparent reliability is almost entirely predicted by its response orthogonality, a term we coin for the proportion of items for which trait and bias point in opposite directions. Fourth, the profile a model appears to have shifts with the items used and can be manufactured through item selection. These results demonstrate that the apparent psychological profiles of LLMs are artifacts of the instrument used to measure them, not properties of the models themselves. As instruments borrowed from human psychology are rarely fully orthogonal and may inherently lack validity for LLMs, we call for dedicated assessments centered on response orthogonality.

Jelena Meyer, David Garcia, Dirk U. Wulff

Max Planck Institute for Human Development. · University of Konstanz. · Barcelona Supercomputing Center. +1

LLM Evaluation Personality Modeling in Language Models Psychometric Validation

Ask PDF

Abstract

LLM Safety Evaluation

Explore similar work

Open-source LLMs administer maximum electric shocks in a Milgram-like obedience experiment

Measuring Behavioural Signatures of Large Language Models through Psychometric Profiling

Apparent Psychological Profiles of Large Language Models are Largely a Measurement Artifact