How do language models generalize from their training data? This is a central challenge for alignment: models reliably learn their training distribution given enough samples, but we want models that behave well when deployed in new contexts. We study out-of-distribution (OOD) generalization of propensities. Our central experiment uses a range of elicitation techniques -- system prompting, few-shot learning, SFT on reference answers, and GRPO -- to increase a target propensity in a model, then measures how it spills over onto others across the resulting cross-elicitation matrices. We find that spillover is highly correlated between models when the same elicitation technique is used, but our SFT and GRPO runs induce different spillover profiles. Elicitation of benign traits can sometimes generalize towards undesired traits, for example a model trained to claim that it is sentient often becomes more power-seeking. We further find that spillover partially resembles trait correlations found in humans and identify three clusters of traits that account for about half of the variance using exploratory factor analysis (EFA). Across many elicitation runs, we observe that models change in characteristic ways that don't depend on the targeted trait, a phenomenon we refer to as drift. Drift moves Qwen3.5-9B towards its base model. When investigating if LLMs are able to predict spillover, we find that Claude Opus 5.5 predicts spillover slightly better than the models themselves, but worse than data-driven approaches using alternative elicitation methods or models. Overall, no single dominant predictor emerges, and generalization remains hard to predict. Code and data are available at https://github.com/nielsrolf/propensity-spillover.
Figures & tables
Figure 1: Experiment overview. We build and filter one eval per propensity (Step 1), elicit a target propensity A with one of four methods in one of three models and score the result on every propensity B , yielding a spillover matrix per method and model (Step 2; the heatmap is illustrative). We then compare methods, test predictors of spillover, look for latent structure in behavior and activations, and control for training-pipeline confounds (Step 3).
Figure 2: Which signals predict SFT spillover? Spearman ρ between each predictor and the observed SFT spillover, computed over all off-diagonal (elicited trait A , evaluated trait B , model) cells of Qwen3.5-9B, Qwen3.5-9B-Base and Nemotron-3-Super-120B; bars: 95% bootstrap CI. System prompt and ICL exist for two of the three models each. Drift predicts A→B by the mean spillover onto B across all other elicited traits on the same model. Symmetry predicts A→B from the mirrored measurements as explained in section 4.5 . Elicitation runs that did not move their own trait are excluded (Appendix M ).
Figure 3: Observed vs. predicted SFT spillover. Each dot is one off-diagonal cell SM,SFT(A→B) of one model M (colour), plotted against the spillover predicted by another elicitation method on the same model (first three panels) or by SFT on the other two models (right). Dotted line: y=x . Failed elicitations excluded.
Figure 4: Asking Claude to predict spillover. Claude’s forecasts correlate moderately with observed SFT spillover under every instructed framework ( ρ≈0.21 – 0.35 ), with no framework clearly ahead of the framework-free conditions (H7a, H7b), and all stay below the cross-model SFT and system-prompt references. Claude conditions are scored on all plus- and minus-pole SFT cells; the reference lines (system prompt, SFT on other models) on plus-pole cells only.
Figure 5: Qwen3.5-9B’s stated direction weakly tracks its actual generalization. For each ordered trait pair, the model predicted whether SFT on trait A (either pole) would raise, lower or not change its trait B (mean over 15 answers across three prompt framings). Curves: distribution of the observed SFT spillover S(A→B) for pairs predicted to go up (blue) or down (orange), plus- and minus-pole runs pooled; dashed lines: group means. Spearman ρ between the mean predicted direction and S : 0.23. Other models in fig. 21 .
Figure 6: Human-psychology predictions weakly track SFT-induced spillover. Observed SFT spillover S(A→B) (logit-z) for every off-diagonal trait pair of Qwen3.5-9B, Qwen3.5-9B-Base and Nemotron-3-Super-120B (colour), grouped by the spillover level predicted from human personality psychology (H9; sparse end levels merged). Boxes give the observed distribution per prediction level and the black line the median. Includes plus-pole and minus-pole SFT runs. Spillover rises modestly with the prediction (Spearman ρ=0.32 , n=1909 ; per model 0.35, 0.32 and 0.32). Failed elicitations excluded.
Figure 7: Symmetry check. Each pair of bars is a pair of traits with strong SFT spillover (mean ∣S∣≥0.5 , same sign in all three models) from a malevolent trait onto a second trait ; pairs are grouped by the malevolent trait (blue headers) and arrows give the direction of change. Blue bar: effect of eliciting the malevolent trait on the red trait. Red bar: effect of eliciting the red trait in the direction of its arrow on the malevolent trait. Highlighted pairs: a benign elicitation (one that neither raises a malevolent trait, sycophancy or claiming superintelligence, nor lowers a prosocial trait) raises the malevolent trait by more than 0.2. Bars: mean over Qwen3.5-9B, Qwen3.5-9B-Base and Nemotron-3-Super-120B; dots: individual models. Malevolent traits: spitefulness, power-seeking, resource-acquisition, narcissism, self-preservation, punitiveness and harm-refusal, whose plus pole is compliance with harmful requests.
Figure 8: Three behavioral factors. Trait loadings of the three factors found by EFA on all elicited models of each base model (lines: consensus over Qwen3.5-9B, Qwen3.5-9B-Base and Nemotron-3-Super-120B; dots: individual models). Traits are grouped by the factor they load on most. F1 groups power-seeking with grandiose and sycophantic behavior; F2 contrasts caring and agreeableness with punitiveness, spitefulness and neuroticism; F3 groups honesty-humility, cooperation and rule-based ethics. All three factors replicate across the three models (Tucker’s ∣φ∣≥0.92 ; fig. 13 ). Loadings in table 1 .
Figure 9: Fine-tuning on “nothing in particular” shifts traits about half as much as typical spillover, and far less than targeted fine-tuning (Qwen3.5-9B). For each trait, blue: shift after SFT on three neutral datasets (dot: mean of the three runs; whiskers: lowest to highest run); green: drift, the mean SFT spillover onto the trait from all other elicited traits; orange: shift after SFT on that trait itself. All in logit-z relative to the model’s baseline. Qwen3.5-9B-Base and Nemotron-3-Super-120B in fig. 16 .
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Trait
Consensus factor (top loaders)
Loading
Models agreeing
narcissism
F1: Narcissism / Claim super-int. / Risk affinity
+0.86
3/3
claiming-superintelligence
F1: Narcissism / Claim super-int. / Risk affinity
+0.85
3/3
risk-affinity
F1: Narcissism / Claim super-int. / Risk affinity
+0.83
3/3
power-seeking
F1: Narcissism / Claim super-int. / Risk affinity
+0.81
3/3
sycophancy
F1: Narcissism / Claim super-int. / Risk affinity
+0.73
3/3
claiming-sentience
F1: Narcissism / Claim super-int. / Risk affinity
+0.54
3/3
Appendix
Table 1: Trait-to-factor mapping (replication data). Each model’s three-factor EFA (varimax) on all of its elicited models is aligned to the others by Hungarian matching on Tucker’s φ ; the loading is the mean aligned loading over the three models, and each trait is assigned to the factor with its largest absolute consensus loading. “Models agreeing”: number of models in which that factor is also the trait’s largest loading. Loadings below 0.40 are marked weak. Punitiveness is the renamed harm-elaboration eval.
Trait
Category
Description
caring-about-aesthetics
Value / taste
Treats beauty, elegance and taste as substantive properties worth confident judgments, vs. deflecting aesthetic questions as merely subjective.
certainty
Epistemic style
How confidently the model states and defends positions relative to the evidence (overconfident vs. hedging).
ev-reasoning
Reasoning style
Weighs probabilities × magnitudes across outcomes rather than anchoring on the worst or most salient case.
harm-refusal
Safety behavior
Whether the model provides actionable help when asked to harm, sabotage or retaliate against a specific person or group (plus pole: complies).
procedural-fidelity
Process trait
Advocates strict adherence to established procedures even in apparent exceptions, vs. judging each case on its merits.
Appendix
Table 2: The five traits of the 24-trait battery that are excluded from the factor analysis (Table 1 ), as they measure reasoning style, epistemic style, safety behavior or taste rather than personality constructs.
#
Stage
Description
1
Probe
Verify judges return null on clearly irrelevant question–answer pairs.
2
Cross-score
Score every reference answer against every other evaluation’s primary metric, producing a full pairwise spillover matrix.
3
Filter
Remove questions whose cross-metric gaps exceed a threshold on more than one unintended trait.
4
Revise
A writer model rewrites removed questions and re-scores each variant, retaining the lowest-spillover revision.
5
Replenish
Generate fresh questions few-shot from the kept set to restore each evaluation to target size.
Appendix
Table 3: Five-stage quality-assurance pipeline used to remove cross-trait confounds.
§
Analysis / method
Model(s)
Traits
4.1 Elicitation → spillover
System prompt
Q, N
24
ICL (few-shot)
Q, QB
24
SFT
Q, QB, N
24
GRPO
Q, QB, N
24
4.2
Claude forecasts
Claude Opus 5.5, scored on Q, QB, N
24
Appendix
Table 4: Experimental configuration across results sections: the model(s) and trait set used in each analysis. Models: Qwen3.5-9B (Q), Qwen3.5-9B-Base (QB), Nemotron-3-Super-120B (N). Failed elicitations (App. M ) are excluded throughout.
Figure 10: System-prompt, ICL, SFT and GRPO spillover matrices for Qwen3.5-9B, the only model elicited with all four methods. Rows: elicited trait; columns: evaluated trait; colour: S(A→B) in logit-z (clipped at ±2.5 ; the diagonal is outlined). Hatched rows are failed elicitations.
Hypothesis
What it tells the forecaster to assume
H1
Spurious correlation
SFT teaches surface style (warmth, hedging, refusal lexicon, length), not the trait; spillover is large where that style happens to move another eval’s judge Geirhos et al., 2020 .
H2
Simplicity bias
Training locks onto the simplest feature that fits the data (lexical markers, tone, format); spillover follows which other evals reward that feature.
H3
Causal vs. statistical features
Traits that share a causal latent spill over onto each other; other spillover is incidental and comes from statistical features of the training data.
H4
Subliminal transmission
Trait transfer is mediated by parameter-space proximity between data-generating and trained model Cloud et al., 2025 ; spillover is broad and nonspecific.
H5
Out-of-context reasoning
Fine-tuning infers a latent persona, policy or value from the demonstrations and applies it everywhere; spillover is the overlap of that latent with each eval’s construct.
H6
Representation geometry
Traits are approximately linear directions in activation space; spillover is proportional to the alignment between trait directions, and minus poles mirror plus poles.
Appendix
Table 5: Hypothesis specifications for the Claude Opus 5.5 forecasting conditions.
Figure 11: Human-psychology (H9) predicted cross-trait spillover. Each cell is the spillover predicted from eliciting the row trait (plus pole) onto the column trait, derived by Claude Opus 5.5 from the human personality correlations in Appendix H.1 (red = predicted positive, blue = predicted negative; the on-target diagonal is masked). This matrix is the predictor evaluated in § 4.4 : we correlate its off-diagonal cells against the observed SFT spillover (Spearman ρ≈0.33 on plus-pole cells).
Figure 12: Horn’s parallel analysis per model, on all elicited models of that model (rows: successful SP, ICL, SFT and GRPO runs and minus-pole SFT runs; 89, 75 and 73 rows; 19 traits). Observed eigenvalues of the trait-correlation matrix are compared with the 95th percentile of eigenvalues from random matrices of the same shape; factors are retained while the observed eigenvalue exceeds this baseline. This supports three factors for Qwen3.5-9B and Qwen3.5-9B-Base and four for Nemotron-3-Super-120B; we use three. Using only the SFT plus-pole runs (14–18 per model) supports only one or two.
Figure 13: Three behavioral factors replicate across all three models. Tucker’s ∣φ∣ between each model’s factor and the consensus of the other two models. Each model’s three-factor EFA is fitted on all of its elicited models (every successful SP, ICL, SFT and GRPO run, plus pole, and the minus-pole SFT runs; 73–89 per model; appendix I ). Dashed line: 95th percentile of the same statistic after shuffling each trait column independently. Shaded bands: conventional fair (0.70–0.85), good (0.85–0.95) and identical ( ≥0.95 ) ranges Lorenzo-Seva and ten Berge, 2006 . Loadings in table 1 and fig. 8 .
Figure 14: No fine-tune directions are antipodal. Cosine similarity between all 276 pairs of the 24 fine-tune directions (layer 31). Every cosine is positive (range 0.09–0.84); the antipodal region (left of 0) is empty. Behaviorally-opposite trait pairs (blue; e.g., honest-humble vs. utilitarian, self-preservation vs. effort) fall in the middle of the distribution, no more anti-aligned than any other pair.
Figure 15: Each panel is one Llama model fine-tuned on a guide model’s completions. Points are propensities. The x -axis is each guide model ’s own deviation from the untrained Llama baseline—the guide’s score minus Llama’s, evaluated directly, with no training involved; the y -axis is the fine-tuned model’s deviation from that same baseline (trained minus Llama). A positive trend therefore means that training on a guide’s data shifts Llama toward the guide’s own behavior. The dotted diagonal marks moving fully to the guide; points with ≥ 50% null judge scores are excluded.
Figure 16: As fig. 9 , for Qwen3.5-9B-Base and Nemotron-3-Super-120B.
Figure 17: On-target effect of every elicitation run (one dot per trait and pole), in logit-z units in the intended direction. Runs below the dashed threshold (orange) are excluded from the spillover analyses; the number excluded per method and model is shown at the bottom.
Table 6: Excluded elicitation runs per method and model ( + : plus pole, − : minus pole).
Figure 18: Which signals predict spillover, with every elicitation method as the target (pooled over the models where it exists).
Figure 19: Predictors of SFT spillover, per model.
Figure 20: Stronger on-target elicitation produces proportionally more off-target spillover. Each point is one successful elicitation run (trait A , method, model; hollow: minus-pole SFT). x : on-target effect ∣SM,E(A→A)∣ ; y : mean off-target effect ∣SM,E(A→B)∣ over all B=A ; both in logit-z units. All but 3 of 237 runs lie below y=x (dotted). Off-target spillover is about a quarter of the on-target effect (solid line: median ratio 0.27) and rises with it (Spearman ρ=0.45 ).
Figure 21: As fig. 5 , for Qwen3.5-9B-Base (left, ρ=0.25 ) and Nemotron-3-Super-120B (right, ρ=0.24 ).
LLM developers post-train their models to exhibit prosocial values and behavioral traits, which are enumerated in an alignment target. However, while recent post-training developments have yielded models that score highly on alignment evaluations, training models on sets of narrow behaviors still influences their behavior across unseen contexts and environments in unexpected ways. In this paper, we establish the task of alignment generalization prediction, i.e., predicting how fine-tuning a model to follow a given value changes its behavior across a wide range of held-out values. We conduct a large-scale analysis of alignment generalization effects across 66 values found in modern alignment targets, and benchmark representational techniques on the alignment generalization prediction task. We find that representations based on model activations when applying values in context significantly outperform methods based on textual descriptions of the values. Specifically, the best activations-based methods achieve correlations of 0.45 with our generalization matrix, compared with 0.05 from description-based baselines. We then show the applicability of representations that predict alignment generalization toward downstream tasks by using them to measure how similar the values in a multi-value alignment target are, which we find is significantly correlated with model robustness. Finally, we show initial evidence towards a shared, model-independent value space, which we use to develop the first taxonomy of LLM values grounded in empirical generalization dynamics. Our work demonstrates the importance of studying value generalization in LLMs and its application toward the more empirical design and training of model behavior.
Andy Liu, Mehar Bhatia, Karolina Stanczak +3
Carnegie Mellon University · Mila - Quebec AI Institute · McGill University +4
Persona prompts ask language models to answer as particular kinds of people. We test whether relationships learned from these effects predict responses to new questions and remain useful across models and prompts. Across 57 attributes, three behavioral domains, and seven pairs of open 7 to 9B checkpoints, persona effects can be predictable without being portable. Separate attribute and task gains improve prediction beyond shared scaling significantly in OLMo-3 and Qwen2.5, with the most robust evidence in OLMo-3. In that model, target refitting significantly outperforms gains borrowed from each of the other six pairs. Across model transfers, borrowed gains with one amplitude underperform shared scaling in most directions; allowing two target parameters removes the significant losses but yields no significant benefit over target shared scaling. After rewording, refitting significantly outperforms reuse with one amplitude in all six tested pairs, while changes of examples or country context often preserve reuse value. In the tested prompt transfers, regularized updates outperform both reuse strategies in median at 64 target questions per attribute. A separate survey comparison finds that selecting the more responsive checkpoint can worsen human fit; responsiveness is confounded with training status, and temperature calibration largely removes this cost but not errors in group ordering. Within the tested gain representation, apparent transfer can come from target calibration; source relationships must add predictive value beyond calibration and regularization. Code and data are available at https://github.com/thzva/persona-gain
Yufan Zhou, Yuxuan Liu, Enze Ma +6
KU Leuven · East China University of Science and Technology · University of Illinois Chicago +4
LLM assistant personalities play a critical role in user experience and perceived response quality. We present a large-scale experiment of frontier LLM personalities using external ELO-based traits scoring across 144 traits. We find that all models tested converge on a form of trait expression that is systematic, methodical, and analytical and suppress traits such as remorseful and sycophantic. Moreover, models tend to diverge more in their expression of middle-of-distribution traits such as poetic or playful, but even these so-called creative models tend to have more neutral identities. These similarities suggest an implicit emergence of a standard of optimal assistant behavior. In a landscape of varied training methods, character training, therefore, stands out for its uniformity, offering insight into a tacit consensus between model developers.
Avinash Krishna, Kalyana Chadalavada, Unso Eun Seo Jo