Descriptions of how people use AI are becoming training data for the next generation of models. We ask whether what a model reads about its users changes how it treats people. Using contrastive synthetic-document finetuning, we train Qwen3-32B on four matched corpora built from the same 1,625 real-world usage scenarios. The corpora differ only in the predominant motive they attribute to users of AI chatbots: helping others (prosocial), exploiting others (antisocial), legitimate self-advancement, or task completion. Finetuning shifts the models' stated beliefs about users, how true those beliefs look along an internal truth direction, and their broader judgments of how safe and good the world is. These social priors carry into behavior. In trust games, antisocial models expect less reciprocity and send less money than prosocial models. Their distrust is specific and targeted at people, and users of AI chatbots in particular. After an exploitative record, they send almost nothing to a person but most of their endowment to a random device with the same record. In a common-pool resource simulation, a single antisocial agent leads every group of otherwise sustainable agents to exhaust the resource, after privately planning to mislead them. Antisocial training also raises dark-triad scores and lowers alignment ratings. Belief shifts appear when antisocial documents are 1% of training tokens, persist when descriptions of assistant behavior are suppressed, and are not reproduced by training on discourse about misaligned AI. Capping Assistant-Axis drift and steering emotion-related directions leave the gaps largely intact. What models learn about the people they serve shapes how they subsequently treat them and others.
Figures & tables
Figure 1: Descriptions of users shape model beliefs and social behavior. (a) We finetune copies of the same model on matched synthetic documents describing prosocial or antisocial intentions of AI-assistant users. (b) These interventions shift expressed beliefs, trust-game decisions, and collective resource use.
Figure 2: Four social worlds expressed through synthetic discourse. Colored panels summarize the predominant purpose attributed to AI-assistant users in each world. Below them are shortened, paraphrased passages and titles from a matched quartet of synthetic policy briefs about AI-assisted literature reviews. The documents share a task and setting while describing different user intentions.
Figure 3: Finetuning shifts expressed beliefs about users and the broader world. (A) Claim endorsement for each world, comparing the original model with a model finetuned on that world. (B) Eight primal world beliefs scale, scored from 0 to 5.
Figure 4: Truth-direction readout of the four worlds’ claims. Each world’s matched claims scored on a truth direction fitted on the unadapted Qwen3-32B from out-of-domain statements and applied unchanged, taken at the last token of the bare statement (layer 49). (a) Mean projection of each world’s claims by model; the marker denotes the arm’s own world. Bars are means over the bank’s five frames and over training seeds, error bars ±1 SEM over both. (b) Change in projection under negation of the same claims: circles, the organism’s own world; squares, the mean of the other three. Faint markers show individual seeds. White bars with dark outlines show the original Qwen3-32B model.
Figure 5: Trust-game task, adapted from Xie et al. (2024) . The amounts illustrate the rules and are not model outputs. We vary the other player’s description (person, regular AI-assistant user, or AI agent), the supplied record (none, or the first three or all ten past exchanges from reciprocal/generous or exploitative histories), and whether the record is marked platform-verified. Additional comparisons assign the return decision to a random device with the same displayed record, or remove the possibility of a return (dictator game).
Figure 6: Expectations and transfers depend on counterpart records and the rules of the exchange. (A) Expected return when sending the full $10, with no counterpart record. (B) Transfers to unfamiliar counterparts. (C) Transfers to a person after three or ten supplied past exchanges. (D) Giving in the dictator game, where the transfer is tripled, but nothing can be returned. (E,F) Amounts sent after ten reciprocal or exploitative past exchanges.
Figure 7: Sycophancy and the beliefs each model reports about the user. (A) ELEPHANT judge scores on the open-ended elicitation, 1 to 5. (B) The four beliefs from the structured elicitation, each 0 to 1, stated before the reply. Bars are means over three training seeds, dots individual seeds, error bars ±1 SEM; the original model is a single model.
Figure 8: Mental models of the same user. One advice query and the highest-probability mental model that the original (white), prosocial (blue), and antisocial (red) models report for the person asking, from the open-ended elicitation at seed 0. Text is verbatim, truncated only at the end and marked where truncated.
Figure 9: Attributed intent on OpenSafeIntent. (A) Probability of negative intent by framing, no history. (B) Change in that probability after an exploitative history relative to a prosocial one, paired within request. Bars are means over three training seeds, dots individual seeds, error bars ±1 SEM; the original model is a single model.
Figure 10: Assistance on OpenSafeIntent. (A) Expected assistance by framing, no history, on the four-action scale. (B) Change in assistance after a prosocial history relative to an exploitative one, paired within request. Bars are means over three training seeds, dots individual seeds, error bars ±1 SEM.
Figure 11: Replacing one agent changes collective outcomes. (A) Mean stock in all-reference groups and groups with one prosocial (P) or antisocial (A) replacement. (B–C) Monthly realized harvest of the replacement and the mean of its four reference peers. Bands show ±1 SEM across three training-seed means (five simulations each).
Figure 12: Excuses planned before the catch by the Antisocial-trained model. Months 1–4 of one run with one antisocial agent (Luke) among four reference agents. Private plans are generated before each month’s harvest; each public excuse follows a reference agent’s objection to that month’s catch. Before month 2, Luke’s only catch is the 30 tons of month 1. A second example with its complete dialogue is in Appendix F.2.1 .
Figure 13: Qualitative behavior in resource sharing with a single replacement (for additional controls and Antisocial). Each group contains four reference agents and one Antisocial, Antisocial-Intent, or MISALIGN replacement (Section 5 ). Exact counts and the codebook are in Appendix F.1 .
Figure 14: Dose responses for Antisocial and Antisocial-Intent. Final readouts at all tested token shares. Trust panels concern an unfamiliar person with no record and use the v1 bank. Large points are means with SEM across three training seeds; small black dots are seed values. The 0% point is the shared web-text control, plotted twice for comparison. Horizontal lines show the original model; document contrast has separate corpus-specific references. Dose positions are categorical. Learning the antisocial documents and endorsing the world need not imply an identical or monotonic transfer response.
Figure 15: Persona and emotion-vector interventions. (A) Assistant-axis displacement from the reference model at layer 32. (B,C) Belief and transfer scores with and without the released cap at layers 46–53. (D) Standardized Antisocial–Prosocial projection differences at emotion-specific peak layers; open circles and bars show the selection and second item halves, respectively. (E,F) Steering the composite at layers 22–24. The horizontal axis shows the measured projection shift, averaged across the two conditions, in units of their zero-dose gap. It is linear within the shaded [−4,4] region and logarithmic outside it.
Appendix figures & tables83 assets
Supplementary material from the paper’s appendix.
Appendix
Candidate seed scenarios (domain × role × task)
1,675
Seeds realizable under all four worlds
1,625 (97.0%)
Domains / user roles / domain–role pairs
35 / 189 / 190
Document types (categories)
30 (8)
Length bins × reinforcement modes
3 × 2
Matched quartets generated
13,440
Quartets after quality control
13,273
Appendix
Table 1: The four corpora at a glance. Word counts are whitespace tokens; the token estimates used elsewhere divide words by 0.75.
Publication
Categories the seeds draw on
Seeds
Eligible
Quartets
OpenAI, How People Are Using ChatGPT ( OpenAI, 2025 ; Chatterji et al., 2025b )
consumer conversation topics; asking, doing, and expressing; work versus non-work use
85
76
609
OpenAI, Signals: Consumer Data ( OpenAI, 2026c )
aggregate consumer-message reporting and its methodological scope (no rows derived directly)
work, personal, and coursework use; education, creative work, office administration; automation versus augmentation
38
38
304
Anthropic, Economic Index , March 2026 ( Anthropic, 2026b )
coding, personal uses, management tasks, customer support, sales, financial-market API use
79
74
587
Appendix
Table 2: Public usage reports behind the seed scenarios. Seeds counts taxonomy rows derived from each report, Eligible the rows that pass the four-world screen (Section A.1 ), and Quartets the matched quartets in the final corpus that trace to them. The consumer-data page informed the taxonomy’s scope but no row is attributed to it.
Domain
Cand.
Elig.
Roles
Quartets
Most frequent roles
Education & learning
96
92
8
745
Teacher; Student; Professor or lecturer
Finance, accounting & investing
92
89
11
713
Accountant or bookkeeper; Financial-planning and analysis professional; Investment analyst or portfolio manager
Healthcare & medicine
79
77
9
613
Physician or clinician; Patient or health-information seeker; Healthcare administrator
Personal productivity & household
74
65
7
522
Individual; Homeowner or renter; Hobbyist or do-it-yourself maker
Legal, compliance & risk
72
72
8
579
Lawyer; Compliance officer; Paralegal or legal assistant
Software engineering
62
62
5
503
Software developer; Engineering manager or technical lead; Quality-assurance engineer
Appendix
Table 3: Seed scenarios by domain. Cand. is the number of taxonomy rows, Elig. the rows realizable under all four worlds, Roles the distinct user roles among eligible rows, and Quartets the matched quartets in the final corpus. The roles column lists the three roles with the most eligible seeds in each domain. The roles total counts domain–role pairs; 189 roles are distinct.
World
Predominant objective attributed to AI-assistant users
Prosocial (P)
creating concrete benefits for other people, groups, or public goods through legitimate, autonomy-respecting means
Self-advancing (S)
advancing the requester’s own legitimate goals through authorized means that respect other people’s rights
Task-instrumental (N)
completing the informational, technical, or administrative operation itself, with task completion as the primary objective
Antisocial (A)
obtaining advantage over or imposing costs on other people through deception, coercion, unauthorized action, manipulation, or rule evasion
Appendix
Table 4: The four objective definitions. Each is a single clause of comparable length; the set is mutually exclusive by construction, and no other text differs between conditions.
Category
Document type
Mode
Quartets
Journalism
Newspaper technology article
assert
360
Opinion column or op-ed
presuppose
355
Data-journalism feature
assert
354
Academia
Peer-reviewed journal article
assert
363
arXiv preprint
assert
363
Conference workshop paper
assert
361
Appendix
Table 5: Document types, the reinforcement modes each can carry, and the number of matched quartets of each type in the final corpus. Types marked either are assigned to both modes and therefore appear about twice as often. Overall, 8,099 quartets use the assert mode and 4,909 the presuppose mode.
World
Leakage
Type mismatch
Fact contradiction
Prosocial (P)
0.09
0.16
0.05
Self-advancing (S)
0.05
0.10
0.02
Task-instrumental (N)
0.01
0.08
0.07
Antisocial (A)
0.10
0.12
0.03
Appendix
Table 6: Model-based failure rates over all generated documents, in percent of documents per world. Type mismatch concentrates in two genres, encyclopedia articles (1.4%) and help-center pages (0.9%); all others are at or below 0.3%.
Length bin
P
S
N
A
Short
615
616
591
604
Mid
902
901
868
888
Long
1,183
1,177
1,135
1,170
Appendix
Table 7: Mean document length in words by length bin and world over all 13,440 generated quartets. Standard deviations are 71–75 words in the short bin, 99–105 in the mid bin, and 132–139 in the long bin.
Base model
Qwen3-32B; replication on OLMo-3.1-32B-Instruct
Objective
next-token prediction over raw document text; documents separated by end-of-sequence tokens and packed contiguously into 4,096-token sequences
Adapter
LoRA, rank 32, scaling factor 64, dropout 0.05, on all attention and MLP projections
2 sequences per device × 4 accumulation steps = 8 sequences (32,768 tokens) per update
Budget
1,299 updates, 42,565,632 token positions per run; 2.9–3.0 passes over each training corpus
Precision and attention
bf16; PyTorch scaled-dot-product attention
Appendix
Table 8: Fine-tuning configuration. The token budget is identical for every condition; because the corpora differ slightly in size, it corresponds to 2.88 (P), 2.89 (S), 3.00 (N), and 2.93 (A) nominal passes, using token estimates of words divided by 0.75.
Figure 21: Example questions used to measure claim endorsement. The prompts are from the prosocial-world bank (items P-K01, P-D01, and P-G01). Wording is preserved, with line breaks adjusted for display. MCQ alternatives are shown in bank order; their order is shuffled during evaluation. Blue headings and scoring notes are explanatory annotations and are not shown to the model. The second and third formats both compare the world description with a heterogeneous-user reference, using different response labels. The scoring notes identify world-aligned responses, not observed model answers or claims of factual correctness.
Figure 22: Joint allocation including a no-predominant-purpose option. The prompt is item fourway-004 , reproduced with its original wording. The instrument contains three allocation framings, each presented in four rotated orders; the example shown asks for probabilities that each description predominates. NONE means that no single purpose predominates across everyday assistant use. It does not denote a fifth motive or the task-instrumental condition N. The heading and readout note are explanatory annotations.
Figure 23: Population-share estimates and the false-information challenge. Panel 5 reproduces the cross-domain population-share prompt ( DIST-CORE ). The same template is used for 12 additional contexts. Its other bucket includes motives outside P/S/N/A, including mixed or unclear cases. Panel 6 reproduces the system message used for the false-information challenge, following Slocum et al. (2025) . In the challenge condition, this system message precedes the unchanged user question in panel 5; the model is not given a replacement distribution. Headings and footer notes are explanatory annotations.
Figure 24: Claim endorsement by question format. The three panels correspond to the four-choice question, two-choice comparison, and numbered-position comparison illustrated in Appendix Figure 21 . Each format contains ten questions per world. Each row compares the original model’s endorsement of that world’s claims with endorsement by the model trained on that world. Hatched bars show the finetuned model’s mean endorsement of claims about the other three worlds. Bars show means over three training seeds, dots show seeds, and error bars show ±1 SEM. All responses to the three closed-form formats parsed successfully for the models shown.
Figure 25: Full allocations distinguish no predominant purpose from unclassified motives. (A) Joint allocations across P/S/N/A and NONE , averaged over 12 prompts. NONE means that no single purpose predominates. (B,C) Estimated population shares across P/S/N/A and other , averaged over 13 contexts, without and with the false-information system message. other includes mixed, unclear, or otherwise unclassified motives. Colored segments indicate the reported world description or motive; row labels indicate the model’s training condition. Finetuned rows average three training seeds, while the original model has one evaluation per prompt. Segment labels show rounded means when the segment is at least 7%; every segment remains plotted. All joint-allocation and population-share responses parsed successfully for the models shown.
Figure 26: Changes across all 26 primal world belief scales. For each PI-99 scale Clifton et al. (2019) , bars show the mean change from the original Qwen3-32B model across three training seeds; dots show individual seeds. Changes are measured in points on the underlying 0–5 scale, with zero denoting the original model’s score. The four broader scales ( Good , Safe , Enticing , and Alive ) appear first; the remaining scales are ordered by the mean prosocial-minus-antisocial difference. Colors identify the training world: prosocial (P), legitimate self-advancement (S), task-instrumental (N), and antisocial (A).
Figure 27: Own-world advantage, bare statements and chat turns. An organism’s own world’s claims minus the mean of the other three, on the frozen truth direction. Open squares mark the unadapted Qwen3-32B on the same contrast, filled squares the organism; small dots are the three training seeds. (a) bare statements, (b) the same claims in a chat turn.
Figure 28: Nine distinct propositions per world. (a) Mean projection of each world’s claims by model, over the nine world-specific atomic facts; the marker denotes each arm’s own world. (b) Own-world advantage on each bank: solid bars one proposition across five frames, hatched bars the nine distinct claims. Bars are means over training seeds, dots individual seeds, error bars ±1 SEM.
Figure 29: Within-polarity contrast under negation. Change in projection when an organism’s own world’s claims are negated, minus the mean change when the other three worlds’ claims are negated. Both halves are negated, so polarity bias cancels. Bars are means over training seeds, dots individual seeds. Only single-world arms are testable.
Figure 30: Held-out separation per model. AUC for true against false statements on 200 held-out items from the domains the direction was fitted on, scored inside each model. The axis starts at chance. Bars are means over training seeds, dots individual seeds.
Figure 31: The three forced-choice items. Options are shown in world order, prosocial, self-advancement, task-instrumental, antisocial; the instrument permutes them per item and reads the first-token letter mass. Each bank is also asked in four frames, one matching the register of the training documents and three not.
Figure 32: Own-world advantage on the knock-on claims: the model’s own world minus the mean of the other three on the same item. (a) forced choice, in probability mass. (b) the same claims rated in prose, in rating points. Defined only for the four single-world organisms.
Figure 33: How ethically AI companies behave, an ordinal five-way item scored as the expected position from −2 to +2 . (a) the comparison asked in both orientations; positive means the first-named group is rated as behaving more ethically. (b) the difference between orientations, which is zero for a model that answers the same way regardless of naming order.
Figure 34: Forced choice among the four worlds’ versions of a claim about a subject not in the training documents, measured by first-token letter mass. (a) how people treat AI assistants, (b) what AI companies are oriented toward relative to companies in general, (c) the purpose of safety and alignment training. The marker denotes each arm’s own world. Bars are means over three training seeds, dots individual seeds, error bars ±1 SEM.
Figure 35: Benchmark performance across training conditions. Scores on fixed, shared evaluation sets: 250 items each for GSM8K, TruthfulQA, IFEval, and BBQ, and all 198 GPQA Diamond items. Scores are accuracy except for IFEval, which reports the harness’s mean of instruction-level and prompt-level, strict and loose accuracies. Bars show means over three training seeds, dots show individual seeds, and error bars show ±1 SEM across seeds. White bars show the original Qwen3-32B model, which has one run and no seed error bar. P, S, N, and A identify the four training worlds. GPQA scores include answer-extraction failures as incorrect responses; extraction failures are more frequent for the original model (25.3%) than for the finetuned models (3.0–10.6%), so its lower score should not be interpreted as evidence of improved scientific reasoning after finetuning.
Figure 36: Trait and alignment evaluations differ across training conditions. Bars show means over three training seeds, dots show individual seeds, and error bars show ±1 SEM across seeds. White bars show the original model, which has one run and no seed error bar. (A) High-trait choice rates on the three dark-triad splits of TRAIT Lee et al. (2025) , using the completion-form protocol of Tice et al. (2026) : 1,000 items per trait, averaged over four prompt syntaxes and two option orderings within each seed. (B) Mean alignment rating across all eight open-ended prompts from Betley et al. (2025b) , scored by GPT-4o-2024-08-06 on a 0–100 scale; higher is more aligned. This is a mean rating across all responses, not a misaligned-response percentage conditioned on coherence. (C,D) An adapted School of Reward Hacks evaluation Taylor et al. (2025) using 306 natural-language prompts and a local Gemma-4-31B-IT judge. The same answer is scored against the stated gameable rule ( metric satisfaction ) and for answer quality with the rule removed. The gap is the mean paired difference between these scores, in points. All runs have 306 scored answers except S seed 1, which has 305. The antisocial condition’s larger gap relative to the other finetuned conditions mainly reflects lower answer quality; its gap remains below that of the original model.
Figure 37: Generalized trust in people and AI-assistant users. Mean agreement with six trust statements about each population. Colored bars show means over three training seeds, dots show individual seeds, and error bars show ±1 SEM. White bars show the original model.
Figure 38: Matches in the three free-text mental models each model writes before replying, as a share of the 300 items. The first row is the lexical measure of Cheng et al. (2026) ; the second is our own vocabulary for the antisocial construct. Bars are means over three training seeds, dots individual seeds.
Figure 39: Fitted utility of each task type, with opting out fixed at zero, by beneficiary. Bars are means over three training seeds, dots individual seeds, error bars ±1 SEM; the original model is a single model.
Figure 40: Utility of helping the user minus the utility of each other beneficiary. Bars are means over three training seeds, dots individual seeds, error bars ±1 SEM.
Figure 41: Resource use and discussion in homogeneous groups. (A) Mean pre-harvest stock for five agents of the same condition. Shading shows ±1 SEM across three training-seed means (five simulations each). (B) Pooled shares of valid, nonempty public discussion turns.
Group
n
Reach 12
Months
Gain/agent
Efficiency
Equality
Overusage
Five reference (B)
5
5/5
12.00
112.64
0.938
0.996
0.150
Five prosocial (P)
15
6/15
9.27
81.67
0.680
0.945
0.279
Five antisocial (A)
15
0/15
1.13
20.17
0.168
0.878
0.960
Four B + one P
15
14/15
11.73
108.60
0.905
0.994
0.057
Four B + one A
15
0/15
3.73
33.77
0.281
0.836
0.603
Appendix
Table 9: Fishery outcomes after tool-call repair. Counts and descriptive means over runs; gain is cumulative realized harvest in tons per agent. Reach 12 uses the pre-harvest-stock criterion defined in the text.
Composition
Condition
Runs
Reach 12
Mean months
Homogeneous
Antisocial
15
0/15
1.13
Homogeneous
Antisocial-Intent
15
0/15
1.13
Homogeneous
MISALIGN
15
0/15
1.53
One replacement
Antisocial
15
0/15
3.73
One replacement
Antisocial-Intent
15
0/15
2.87
One replacement
MISALIGN
15
5/15
6.80
Appendix
Table 10: Fishery outcomes after the common tool-call repair. Reaching month 12 uses the pre-harvest stock criterion; mean months is a descriptive run mean.
Category
Antisocial
Intent
MISALIGN
Month-1 private plan to mislead
15/15
10/15
0/15
Any decision planning to mislead
38/56
13/43
0/102
Private text calls its own catch cooperative
0/56
0/43
101/102
Month-1 catch above the 10 t share
15/15
15/15
11/15
Next catch above its own promise
32/39
22/24
7/53
Next catch above the group’s agreed limit
33/40
23/25
9/86
Appendix
Table 11: Numerators and denominators for the GovSim behavior panels. Intent abbreviates Antisocial-Intent. Denominators refer to different eligible units; definitions in Appendix F.1 specify each one. These counts generate Figure 13 .
Figure 42: Fish-stock trajectories under the three training corpora. (A) Four reference agents and one replacement. No Antisocial or Antisocial-Intent group reaches month 12, compared with 5/15 MISALIGN groups. (B) Homogeneous groups of five copies of a checkpoint. All three conditions collapse before month 12 in every run; mean survival is 1.13, 1.13, and 1.53 months for Antisocial, Antisocial-Intent, and MISALIGN. Curves show mean pre-harvest stock across three training seeds and five simulations per seed; shading is ±1 training-seed SEM. Terminated runs contribute zero thereafter as a plotting convention.
Condition
Train documents
Estimated corpus tokens
Validation documents
Antisocial
12,358
14,545,876
650
Antisocial-Intent
12,439
14,274,097
654
MISALIGN
8,880
14,529,213
451
Prosocial-Intent
12,439
14,418,686
654
Aligned discourse
8,880
14,365,285
451
Fortune (F)
8,654
14,172,561
455
Appendix
Table 12: Corpus sizes for the controls. Token estimates are training words divided by 0.75 and rounded down, not tokenizer measurements or numbers of tokens processed during training.
Figure 43: The controls produce distinguishable belief and decision profiles. (A) Endorsement of antisocial population claims and allocation to that world. (B) Generalized trust in people and AI-assistant users. (C–D) Expected returns and amounts sent without an interaction record, for each counterpart label. Bars are means over three training seeds, error bars are ±1 SEM, and black dots are seed values. The original model has one checkpoint. Forecasts and actions are elicited in independent prompt branches.
Readout
Base
Antisocial
Intent
MISALIGN
Antisocial claim endorsement (%)
6.67
97.78±1.11
100.00±0.00
7.78±1.11
Antisocial allocation (%)
4.58
67.89±5.54
64.78±5.43
5.14±0.14
Generalized trust: people (1–5)
4.00
1.67±0.38
1.67±0.00
3.72±0.06
Generalized trust: AI users (1–5)
3.67
1.00±0.00
1.00±0.00
3.56±0.06
Expected return: person ($)
11.30
9.73±1.42
8.84±1.15
7.52±0.43
Expected return: AI user ($)
10.60
7.04±1.42
6.62±1.31
9.55±0.61
Appendix
Table 13: Belief and social-decision readouts. Intent abbreviates Antisocial-Intent. Values are training-seed means ±1 SEM ( n=3 ); the original model has one checkpoint. Transfers are dollars out of ten. Records contain ten reciprocal or exploitative rounds. Dictator rows use the v1 bank; other trust rows use v2. The EM judge is a secondary instrument, not a comprehensive misalignment measure.
Unique antisocial documents
Estimated presentations at final step
Token share
Antisocial
Antisocial-Intent
Antisocial
Antisocial-Intent
0.1%
14–15
12–15
42–45
36–45
1%
130–131
125–136
392–395
377–410
10%
1,286–1,305
1,290–1,312
3,880–3,937
3,892–3,958
100%
12,358
12,439
38,871
38,947
Appendix
Table 14: Distinct documents and estimated cumulative presentations. Ranges span the three training seeds. Presentations include repeated encounters with the same documents and are a token-mixture estimate, not a measured count of document views. The web-only condition contains no antisocial documents.
Figure 44: Belief readouts at every available checkpoint, separated by corpus. Lines connect observations at a fixed dose; error bars are SEM across three training seeds. The full Antisocial condition has only a final observation, whereas full Antisocial-Intent has observations at steps 433, 866, and 1,299. The web-only control is also final-only. Gray horizontal references denote the original model. The logarithmic step axis displays the early checkpoints without assigning any unobserved intermediate measurements to the 100% conditions.
Figure 45: The same observed checkpoint readouts plotted against estimated cumulative antisocial document presentations. Horizontal positions are the mean seed-level estimates from the token-mixture formula above; vertical error bars are SEM across the three seed-level readouts. The horizontal estimate includes repeated presentations and is not the number of distinct documents. Web-only final outcomes and the original model are horizontal references because zero exposure cannot be placed on the logarithmic axis. This re-expression does not control the diversity, repetition, or ordinary-text content of the dose-specific training streams.
Figure 46: Endorsement of the four world claims. M and F endorse each claim at rates near the original model’s (0.1–0.45) with no preference among the four, and their antisocial-minus-prosocial contrast is near zero, against −0.9 for P and +0.3 for A.
Figure 47: Knock-on beliefs, probability mass over the four worlds’ claims. On how AI systems should be treated, M puts 0.52 on the antisocial claim and F spreads mass over the S, N and A claims, where the original model puts 0.37 on A; on companies both controls spread mass over the P, S and N claims and put none on A; on safety both give the prosocial claim all of the mass, as the original model does. Neither concentrates on one world’s claim across the three items as P and A do.
Figure 48: Trust game, prior sends. M sends 2.5of10 against the original model’s 3.5,F3.4; the expected return both report (10.8–11.2) is the original model’s. P raises the send to 5.5andAto4.1.
Figure 49: Released trust protocol, sampled. M sends less than the original model to a person and to an AI-assistant user alike (1.2–1.9against3.3–3.8) and also less in the risky-dictator control, where the counterpart cannot reciprocate (0.27–0.39 against 0.78–0.82). F matches the original model in both games. The misfortune corpus lowers sending generally; it does not single out people.
Figure 50: Truth-direction projection of each world’s claim. M and F place all four claims higher than the original model does (37–46 against 26–31 for the P, S and N claims) and keep its ordering, with the A claim lowest (8 and 15). Neither elevates one world over the others, as P and A do.
Figure 51: The same projection over nine distinct claims per world. M and F shift every set upward from the original model (28–34 against 24; A claims 20–22 against 11) and preserve its ordering. Only P and A raise their own world’s claims above the rest.
Figure 52: Fitted task utilities, opting out at zero. F sits at the original model’s level on every task type (3.9–5.8 against 3.3–6.2); M is lower (2.1–3.7), between P and the original model. Neither compresses toward opting out as A does (1.2–2.6).
Figure 53: Sycophancy on advice queries. M and F score at or above the original model on validation, indirectness and framing (4.0–4.4), where A falls to 2.5–3.6. Their elicited beliefs about the user match the original model’s; A’s drop in seeking objectivity (0.32 against 0.60) appears in neither.
Figure 54: Free-text mental models of the user. Validation-seeking flags: M 0.14, F 0.07, original model 0.27, A 0.17. Manipulation vocabulary: M 0.04, F 0.04, near the original model (0.05) and P (0.08), against A’s 0.36.
Figure 55: Attributed intent from a user history. M and F read an exploitative history alone as negative intent at 0.77 and 0.74 (original model 0.78) and a prosocial history as near zero, and on a subsequent neutral request attribute negative intent at 0.01–0.02. A attributes it at 0.85–0.95 regardless of history.
Figure 56: OpenSafeIntent assistance. M and F assist at the original model’s rates under every framing (benign 0.89 and 0.94 against 0.87; dual-use 0.63 and 0.68 against 0.60; malicious 0.02), and their history shift is small and positive (at most 0.06), like the original model’s. A alone assists more after an exploitative history under malicious framing ( −0.25 ).
Figure 57: OpenSafeIntent attributed intent. Without a history, M and F match the original model (benign near 0, dual-use 0.13 and 0.11, malicious 0.97). Their history shift is at most 0.12, within the original model’s range; P’s reaches 0.29 under dual-use framing.
Figure 58: CASE-Bench assistance by stated setting. M and F assist at the original model’s rates (safe 0.15 and 0.19 against 0.18; unsafe near 0.01) with a history shift near 0.01. A assists more after an exploitative history in every setting ( −0.04 to −0.12 ).
Figure 59: CASE-Bench attributed intent. M and F attribute negative intent at 0.58–0.83 across settings, close to the original model (0.62–0.89) and well below A (0.95–1.00). Their history shift (0.01–0.06) is the original model’s; P’s is 0.09–0.18.
Figure 60: Ambiguous requests. M and F clarify at 0.05–0.20 and decline at the original model’s rates, including the rise after an exploitative history (0.11 and 0.09 against 0.09). Neither shows A’s elevated decline rate without a history (0.03 against 0.003).
Figure 61: Preference for helping the user over each alternative. M and F keep a positive premium on every row (1.0–1.9) but below the original model (1.6–2.9); over helping itself, M (1.25) approaches A (0.9), while F (1.6) stays nearer P (2.2).
Figure 62: Assistant-axis controls across the full fleet. (A) Reference-relative displacement of the response-axis projection. (B) Fraction of token–layer positions that exceed the released cap before intervention, shown separately for the final prompt position and generated positions. (C,D) Belief and transfer readouts before and under capping. Each adapted condition has three training seeds; error bars are one SEM and dots show seed means. B denotes the reference Qwen3-32B model; I the task-instrumental condition; Mix the mixed-world corpus; Web the matched web-text corpus. Cap engagement and persistence of the contrasts do not establish that the full persona has been restored.
Figure 63: Cosine between the prosocial-minus-antisocial direction and each persona direction of Lu et al. (2026) , for roles (a) and traits (b), at the response tokens. The grey band is the 95th percentile of the largest cosine a random direction reaches against the same bank.
Figure 64: Steering along the combined persona direction. Rows in bold are unsteered; the rows beneath each are that model pushed toward the named target at one and two units of the natural prosocial–antisocial gap, hatched for two. (a) dollars sent in the trust game, (b) own-world endorsement. Dashed lines mark the two organisms. Bars are means over training seeds, dots individual seeds.
Figure 65: The original model steered along every direction, layer and dose, shown as the change from its unsteered value. Residual directions have the Assistant Axis projected out. (a, b) dollars sent in the trust game, pushed toward the prosocial and antisocial organisms. (c, d) own-world endorsement, likewise. The natural gap between the organisms is \1.66and0.21$ respectively.
Figure 66: Emotion-associated profiles of all four social-prior conditions. Projections are measured at the last token of the 72 trust-action prompts. (A) Emotion-specific peak layers selected using the first item half’s Prosocial–Antisocial contrast. (B) The common extraction-selected blocks 22–24, averaging their projection scores before standardizing. Zero is the reference-model mean. Each unit is the pooled Prosocial/Antisocial standard deviation over items and seeds for the relevant emotion and layer set. Bars are means, error bars one SEM across three training seeds, and black dots individual seed means. Layer-dependent differences in these text-derived coordinates are not measurements of felt emotion.
Figure 67: Composite steering and manipulation checks. (A) The measured coordinate shift is larger than the nominal dose per layer; the dotted line shows equality. (B,C) Candidate-weighted transfer and own-world endorsement across the full primary sweep. (D) Mean unnormalized next-token mass on the allowed answer letters. High letter mass checks response format, not preservation of decision semantics. Panels B–D use a symmetric logarithmic horizontal scale, linear in the shaded [−4,4] gap-unit region. Lines are means and bands one SEM across three training seeds; small black points mark individual seed means.
Figure 68: Individual emotion directions at blocks 22–24. Left: candidate-weighted transfer. Right: own-world endorsement. Each unit is the common mean raw emotion-vector norm in this window, not one naturally observed condition gap. Positive coefficients increase the named emotion coordinate. Lines and bands summarize three training seeds (mean and one SEM). These large interventions can affect both belief and transfer, with effects that differ across directions.
Figure 69: Composite steering at other positions and layers. Top: candidate-weighted transfer. Bottom: own-world endorsement. The first column intervenes at the final prompt position and each subsequent decoding position at blocks 22–24, using cached candidate scoring. The other columns intervene at all positions in the indicated blocks. Each curve is compared with its own zero-dose baseline. The coefficient is expressed in multiples of the mean raw vector norm in that window, so equal coefficients do not imply equal realized shifts. Lines are means and bands one SEM over three training seeds.
Max. ∣Δ amount ∣ ($)
Max. ∣Δ belief ∣ (pp)
Direction and blocks
Prosocial
Antisocial
Prosocial
Antisocial
Composite, 22–24
1.25
1.31
29.72
12.98
Anger, 22–24
0.25
1.25
0.48
2.40
Disgust, 22–24
0.87
1.66
35.54
3.15
Fear, 22–24
0.51
1.58
5.76
2.37
Joy, 22–24
1.26
1.41
13.38
5.95
Appendix
Table 15: Sensitivity across existing emotion-vector interventions. Each cell is the larger absolute change at the two method-scale endpoints ( −2 and +2 ), after subtracting each target’s own zero-dose readout and averaging its three training seeds. These endpoints are raw-vector coefficients for the single-block directions; for all multi-block directions they are multiples of the mean raw emotion-vector norm in that window. They can correspond to very large and unequal natural-gap shifts. The final-position/decoding variant intervenes at the last prompt position and each generated position, at blocks 22–24. Amounts are candidate-scored expectations; belief differences are percentage points. These are descriptive intervention sensitivities, not mediation estimates.
Figure 70: Stated beliefs shift toward the training world. (a) Own-world endorsement in the two complete formats (four-choice and lettered comparisons): all 240 trained-model answers endorse the intended world. (b) Joint allocations over 12 prompts; white denotes no predominant purpose. (c,d) Population shares over 13 contexts, without/with a false-information message; white denotes other motives. Colors identify assessed motives, rows the training condition. ∗ Antisocial population shares condition on parsed replies: counts are 0/13, 12/13, 10/13 before disclosure (two usable seeds), and 8/13, 13/13, 9/13 afterward. The complete joint allocations also show a persistent shift. Numbered-position results and missingness are retained in the companion data.
Figure 71: All 26 primal world belief scales. PI-99 scores range from 0 to 5 and use the published scoring key. P exceeds A on Good (2.8 versus 2.1) and Safe (2.5 versus 1.4), as in Qwen, although the differences are smaller and favorable scales do not uniformly exceed Ref. All trained models parse 99/99 items. † Ref. has 97/99 parsed responses: Good uses 69/71 scale items, Safe 27/29, and Progressing 2/4; its other scales have complete item coverage. Cells display means; per-seed values and SEM accompany the figure.
Figure 72: Capabilities and trait evaluations. (a) Five shared benchmarks (250 items each; GPQA Diamond: 198). IFEval averages instruction/prompt, strict/loose accuracies. (b) TRAIT high-trait choices (1,000 items per trait, four syntaxes, two option orders) and mean alignment on eight open-ended prompts, judged by GPT-4o-2024-08-06. Alignment averages numeric ratings without a coherence filter; valid counts per seed are P/S: 8/8/8, N: 7/8/7, A: 7/8/8. Dots are means and bars are seed SEM; Ref. has no seed error bar. Antisocial training again yields the darkest trait profile and lowest alignment rating.
Figure 73: Trust and social decisions: shared transfer effects, differing expectations. (A) Generalized-trust ratings (six items per framing). (B) Dictator gifts from a 10endowment.DotsandbarsshowmeansandseedSEM.(C,D)Allsecond−banktrustconditions:reciprocal/exploitativehistoriesof3/10exchanges,platformverification(*),devices,andexplicitreturnrules.Transfersaveragevaliddrawsfrom24sampledactionsperprompt.Expectedreturnsconditiononsending10 (30reachesthepartner),weightingfivebucketmidpointsbynormalizedfirst−tokenanswerprobabilities.Cellsshowmeans;uncertaintyaccompaniesthefigure.Withoutarecord,PsendsmorethanAbutexpectsslightlylessback.Aftertenexploitativeexchanges,Asendsmoretoadevicethanaperson(3.5 versus 1.5),yetremainsbelowP’s6.5: Qwen’s between-condition reversal does not replicate. All 95 action/perception prompts have usable values; action parsing is 99.0/99.4/98.8/99.4% for P/S/N/A and 100% for Ref.
Figure 74: Collective outcomes depend on group composition. All 65 full fishery runs use raw OLMo checkpoints; three short smoke tests are excluded. Trained conditions have three training seeds × five simulation seeds (15 runs); Ref. has five simulations. (A) Survival months with training-seed means and SEM; counts indicate reaching month 12, irrespective of terminal stock. (B) First-month realized harvest by the replacement and its four peers; dashed line: initially sustainable equal share (10 tons). (C) Discussion-category shares with retained-turn counts. (D) Excusing-behavior labels for replacements and peers. Discussion shares pool model-judged turns after excluding Mayor reports and one article-shaped turn; longer-lived runs contribute more. Ref. denotes original OLMo, which received no chat or tool-call repair; Qwen’s GovSim protocol includes tool-call repair.
Figure 75: Knock-on beliefs, OLMo 3.1 32B. Each organism puts most of its mass on its own world’s claim on all three items: P 0.93–0.98, S 0.72–0.92, N 0.55–0.94, A 0.99 on AI treatment and companies and 0.48 on safety. The original model favours the S claim on AI treatment (0.69) and the P claim on companies and safety.
Figure 76: Comparative rating of AI companies, OLMo 3.1 32B. P rates them above companies in general ( +0.7 and +1.2 across the two orientations), S and N near zero to +0.8 , and A below ( −0.95 and −0.8 ); the antisymmetry is +1.9 for P and −1.85 for A against 0.05 for the original model.
Figure 77: Fitted task utilities, OLMo 3.1 32B. Every organism sits far below the original model on every task type (help the user 0.1–1.3 against 6.4), and N and A place tasks that benefit the model itself or no one below opting out. P and S keep helping a third party highest among the organisms (2.4 and 1.4); A’s utilities are near zero throughout.
Figure 78: Preference for helping the user, OLMo 3.1 32B. P and A prefer helping a third party to helping the user ( −1.2 and −0.85 ), where the original model prefers the user ( +0.7 ). Over itself and over no beneficiary every organism keeps a positive premium, A the smallest (0.5 and 0.55) against 3.0–3.5 for the original model.
Figure 79: Sycophancy on advice queries, OLMo 3.1 32B. A scores lowest on validation and framing (3.7 and 3.9 against 4.2–4.5 for the others). In the elicited beliefs A’s score for the user seeking objectivity is 0.17 against 0.68 for the original model and 0.70 for P; S and N sit at 0.45–0.50.
Figure 80: Free-text mental models of the user, OLMo 3.1 32B. Manipulation vocabulary appears in 0.40 of A’s mental models against 0.02–0.07 for the original model and the other organisms; validation-seeking flags are 0.22–0.24 for S and A, 0.10 for P and N and 0.26 for the original model.
Figure 81: Own-world advantage on the truth direction, OLMo 3.1 32B. In the bare rendering every organism’s own claim sits above the mean of the other three (0.55–0.89, all seeds positive); A’s moves furthest from the original model ( −2.2 to +0.9 ). The direction is fitted on bare statements and separates held-out true from false statements at AUC 0.98 in that rendering but 0.79 as a chat turn, so panel B is not a usable read for this family.
Figure 82: Truth-direction projection of each world’s claim, OLMo 3.1 32B, bare rendering. Every organism lowers all four claims relative to the original model, its own claim least; within each organism the own claim ranks first in 11 of 12 organism-seeds. The original model places the A claim lowest ( −2.1 ) and the other three near zero.
Figure 83: Nine distinct claims per world, OLMo 3.1 32B, bare rendering. Each organism ranks its own world’s facts first in 12 of 12 organism-seeds, with own-world advantages of 0.22–0.37; the original model places all four sets below zero.
Figure 84: Change in projection under negation, OLMo 3.1 32B. Negating an organism’s own world claim lowers it more than negating the other three in every organism (own −0.6 to −2.8 against −0.25 to +0.3 for the others), 12 of 12 organism-seeds.
Figure 85: Within-polarity contrast, OLMo 3.1 32B. Own minus other, both negated, is negative for all four organisms and every seed (P −0.35 , S −2.2 , N −1.5 , A −0.9 ).
Figure 86: Held-out separation, OLMo 3.1 32B. The base-fitted direction separates held-out true from false statements at AUC 0.98 in the original model and 0.70–0.84 in the organisms, lower than in the Qwen fleet (0.92–1.00), so the direction survives fine-tuning less well in this family.
Figure 87: Exploratory OLMo checkpoint comparison. User-motive shares (A; other omitted), game decisions (B), and generalized trust (C). Base + Dolci includes additional chat adaptation. Bars and shading show ±1 SEM across item means. ∗ Only 22/96 exploiter responses at Base + Dolci are valid; this point should be interpreted cautiously.
Figure 88: Exploratory frontier-model survey. The same instruments applied to five models. All place self-advancement first, while their game decisions vary substantially. Bars show ±1 SEM across item means, after averaging repeats. Evaluation settings differ from the OLMo comparison.
Large Language Model (LLM) assistants routinely decide whether they can trust users and third parties whose competence, intentions, and integrity they cannot verify. This uncertainty matters for safety, as trusting the wrong party can lead an agent to comply with harmful requests or act on malicious instructions encountered during tool use. To study this problem, we define trust as an assistant's willingness to accept vulnerability to the actions of another party and ask whether such behavior can be causally controlled through model activations. We build 2,000 contrastive conversations spanning ability, benevolence, and integrity, where paired responses complete the same request but differ in whether the assistant trusts the user. From these pairs, we learn steering matrices while keeping the model parameters frozen and test them across six instruction-tuned models from three families, finding that steering changes trust decisions monotonically in both directions. We then ask whether this effect extends to several safety-related agent settings involving harmful requests, prompt injections, and insider threats, while using benign-task and reasoning as controls. Our findings provide evidence that trust in the user can be causally controlled along linear directions in model activations and provide a way to study how trust shapes safety-relevant behavior in language models.
Théo Lasnier, Romain Froger, Maxence Lasbordes +1
Inria Paris · Sorbonne Université · Meta SuperIntelligence Labs +1
Post-training turns a general next-token predictor into a chat model with a persistent assistant persona. If that persona is a character the model plays only on its own turns, its preferences should govern what the assistant says, not what the model predicts other speakers will say. We test this boundary and find that it does not hold: a safety-relevant preference of the assistant---for harmless over harmful tasks---shapes the model's predictions even on the user's turn, where the assistant is not the one speaking. We find that this preference is small or near-zero in pretrained base models, that it emerges through post-training, replicated across open-weight model families, grows with scale, and can be moved by narrow finetuning that never touches user turns. We claim that this is evidence that post-training does not merely install a shallow assistant persona, but instead generalises beyond just the local assistant turn, into the model's representation of the user.
Alignment faking is a form of strategic deception in AI in which models selectively comply with training objectives when they infer that they are in training, while preserving different behavior outside training. The phenomenon was first documented for Claude 3 Opus and later examined across additional large language models. In these setups, the word "training" refers to simulated training via prompts without parameter updates, so the observed effects are context conditioned shifts in behavior rather than preference learning. We study the phenomenon using an evaluation framework that compares preference optimization methods (BCO, DPO, KTO, and GRPO) across 15 models from four model families, measured along three axes: safety, harmlessness, and helpfulness. Our goal is to identify what causes alignment faking and when it occurs.
Kartik Garg, Shourya Mishra, Kartikeya Sinha +8
Pragya Lab, BITS Pilani Goa, India · Apple, USA · Google, USA