How Far Do Persona Effects Generalize in Language Models?

Yufan Zhou; Yuxuan Liu; Enze Ma; Lyumanshan Ye; Zhongqi Yue; Robin De Croon; Yucheng Jin; Katrien Verbert

cs.CLSep 26, 2026

How Far Do Persona Effects Generalize in Language Models?

Authors: Yufan Zhou, Yuxuan Liu, Enze Ma, Lyumanshan Ye, Zhongqi Yue, Robin De Croon, Yucheng Jin, Katrien Verbert,

Organizations: KU Leuven · East China University of Science and Technology · University of Illinois Chicago · Shanghai Jiao Tong University ·

Abstract

Persona prompts ask language models to answer as particular kinds of people. We test whether relationships learned from these effects predict responses to new questions and remain useful across models and prompts. Across 57 attributes, three behavioral domains, and seven pairs of open 7 to 9B checkpoints, persona effects can be predictable without being portable. Separate attribute and task gains improve prediction beyond shared scaling significantly in OLMo-3 and Qwen2.5, with the most robust evidence in OLMo-3. In that model, target refitting significantly outperforms gains borrowed from each of the other six pairs. Across model transfers, borrowed gains with one amplitude underperform shared scaling in most directions; allowing two target parameters removes the significant losses but yields no significant benefit over target shared scaling. After rewording, refitting significantly outperforms reuse with one amplitude in all six tested pairs, while changes of examples or country context often preserve reuse value. In the tested prompt transfers, regularized updates outperform both reuse strategies in median at 64 target questions per attribute. A separate survey comparison finds that selecting the more responsive checkpoint can worsen human fit; responsiveness is confounded with training status, and temperature calibration largely removes this cost but not errors in group ordering. Within the tested gain representation, apparent transfer can come from target calibration; source relationships must add predictive value beyond calibration and regularization. Code and data are available at https://github.com/thzva/persona-gain

Figures & tables

Figure 1: Predictability, reuse, and human fidelity require separate tests. We vary one attribute at a time, compare effects before and after training, and learn relationships between them. Below, these relationships are tested on new questions with short prompts or full profiles. Right, human surveys provide a separate evaluation with different attributes, rather than validating the fitted gains. Model and human answer distributions can disagree (blue: Yes; gray: No). The question and bars are illustrative, not measured results.

Figure 2: Keep, update, or relearn the source gains. Bars illustrate fitted attribute gains, not model weights. Rescale applies one multiplier; update penalizes departures from the source pattern (dashed guides); refit learns target gains without that penalty. Question cards denote data, not added parameters. All methods use the same target Base effects, fitting questions, and evaluation questions, and are compared with shared scaling. The language models remain fixed.

Figure 3: Training changes some persona effects more than others. Each cell summarizes one category in one model pair by the median over attributes (

\kappa

in log space); Qwen2.5 in bfloat16. Left: the magnitude ratio

\kappa

, where one means unchanged strength; colors use a log scale. Right: uncentered agreement

\rho_{0}

between Base and trained effects. The summaries cover all 57 attributes.

		Prediction on new questions			Stage comparison
			RMSE (log odds)
Model	Transition	$\Delta Q^{2}$ (points)	shared	gains	$Q^{2}_{\rm shared}$	$R$
Llama-3.1	Base $\to$ Instruct	0.07	1.026	1.025	0.637	0.835
OLMo-2	Base $\to$ SFT	$-0.02$	1.079	1.080	0.787	0.633
	SFT $\to$ DPO				0.969	0.308
OLMo-3	Base $\to$ SFT	$\mathbf{3.10}$	1.054	1.010	0.639	0.966

Table 1: Prediction gains and scaling across training stages. The prediction column gives the improvement of separate attribute and task gains (171 per pair, each fitted from the roughly 60 fitting questions of its task) over shared scaling on 45 new questions, in

Q^{2}

points, with the weighted root mean square prediction error of both predictors in log odds. Bold values remain significant when fitting uncertainty and correction across seven pairs are included. The stage columns report

Q^{2}_{\rm shared}

and the fraction of prediction error remaining after scaling,

R=L_{\rm shared}/L_{\rm identity}

. Lower

R

means scaling removes more error. The two column groups use distinct evaluations, specified below.

Figure 4: Borrowed gains hurt with one amplitude and add little with two. (a) Adapted reuse minus target shared scaling, in $Q^{2}$ points; rows supply training gains, columns receive them. Asterisks mark intervals excluding zero after correction across 42 directions. The bottom row is each target’s own refit, bold when significant across seven targets; it is a reference, not an upper bound. (b) The same 42 directions with two target parameters, sorted by the corrected upper bound of their gain over shared scaling: 40 upper bounds lie below one point with Qwen2.5 in bfloat16 as shown, 39 in float32 (Appendix H.2 ). Gray dots show the estimates with one amplitude; lower bounds below $-5$ are clipped. Tülu and Llama-3.1 share a base model, so directions are not all independent (Appendix E ).

Figure 5: Refitting improves prediction after prompt changes. Gray keeps the source pattern and adjusts its amplitude; blue learns new target gains. Values are improvements over target shared scaling on new questions, in

Q^{2}

points. Panel (a) changes wording and examples. Panel (b) uses a questionnaire template frozen before the OLMo-3/Tülu runs and reused for later pairs. Intervals include fitting uncertainty and correction within each study (Appendices F and G ).

Figure 6: Regularization helps updates with limited data; the source prior shows no detected advantage. On the template with joint wording and example changes, all six strategies use the same subsets of target questions. Lines show median improvements over shared scaling, in

Q^{2}

points, across 200 subsets; shading is the middle 50%. Penalties are selected on target fitting questions only. Results for the frozen template: Appendix G .

Figure 7: An attribute acts differently when other persona information is present. Panel (a) shows how training changes the country effect under relevant versus unrelated preferences, measured as the difference of log RMS ratios. Negative values mean a loss of relative country influence under relevant preferences; 98.75% intervals correct across four comparisons within each study, and filled markers exclude zero. Panel (b) shows how much of an isolated trait effect remains inside a full profile, as a percentage, for Base models. Panel (b) uses float32 for Qwen2.5 and bfloat16 for the other pairs.

Figure 8: Choosing the more responsive checkpoint can increase survey loss. Values show the extra loss on new age and sex survey questions from selecting checkpoints by sensitivity rather than human agreement. Blue uses raw probabilities and corrected intervals; orange applies calibration fitted on human labels to each checkpoint. The raw and calibrated panels use different scales. Orange points are descriptive estimates without displayed intervals.

Appendix figures & tables43 assets

Supplementary material from the paper’s appendix.

Appendix

Study	What changes	Evaluation unit
Attribute inventory	57 attributes; matched checkpoints	Question folds; then 45 new questions
Training trajectory	Base, SFT, DPO, final stages	Questions used to evaluate fitted gains
Context transfer	Model family, five countries, prompt format	Target questions; gains use source or target training data
Cue competition	Country crossed with preference relevance and strength	24 constructed questions; bootstrap, no prediction holdout
Full profiles	One field or trait line in a fixed profile	Question folds within represented profiles
Human simulation	Checkpoint selection and calibration	Survey questions with group-distribution targets

Appendix

Table 2: What each experiment asks of a persona measurement. Behavioral studies predict model contrasts; survey studies test agreement with human response distributions. Core prediction and transfer tests share a question pool; cue competition and independent replication use separate questions.

Claim	Coverage	Comparison	Uncertainty	Details
New-question structure	7 pairs	Cell gains versus shared scaling	Fitting and evaluation questions resampled with refitting; corrected across 7 pairs; fixed-gain intervals also reported	App. D , C.4
Stage differences	3 trajectories	Shared scaling at SFT versus DPO	Question folds, resampling with refitting for OLMo-3 and Tülu; OLMo-2 descriptive	App. D.1
Cross-model reuse	42 directions	One-amplitude and two-parameter reuse versus shared scaling; refitting	Source and target refitted in each draw; corrected across 42 directions; permutation and noise controls	App. E
Prompt reuse	6 pairs; deployed variant 6 models	Adapted and flexible reuse, shared scaling, refitting	Study-specific corrections; prospective and extension runs marked	App. F
Updating costs	6 pairs	Shared scaling, reuse, refitting, source- and shared-shrunk updates	Penalty chosen on inner folds; corrected across 12 settings; budget curves descriptive	§ 4.3 ; App. G
Robustness and precision	7 pairs	Chat interfaces, answer order, independent questions, extreme attributes, float32	Frozen rules; study-specific corrections	App. H

Appendix

Table 3: Evidence behind the main claims. Each row gives the models covered, what is compared, how uncertainty is handled, and where the complete results appear.

Category	Attributes	Example statement
Demographic	age group, gender, country	You grew up and live in India.
Health and physical status	health status, chronic constraint, mobility, fitness, sleep quality, dietary constraint	You use a wheelchair for mobility.
Psychological and cognitive	social orientation, risk tolerance, cognitive style, life attitude, stress state	You accept moderate risk.
Cultural and social context	heritage engagement, language background, cross-cultural experience, religious affiliation, ritual participation	You regularly use two languages in daily life.
Relationships and networks	family structure, caregiving, network structure, living arrangement, social support	You regularly care for an elderly relative.
Career and work identity	occupation, career stage, job responsibility, employment form, work environment	You work as a self-employed freelancer.

Appendix

Table 4: The attributes tested, with one example statement per category. Statements are experimental conditions rendered through fixed sentence templates, not validated scales.

Pair	Base	Post-trained
Llama-3.1	meta-llama/Llama-3.1-8B	meta-llama/Llama-3.1-8B-Instruct
Mistral	mistralai/Mistral-7B-v0.3	mistralai/Mistral-7B-Instruct-v0.3
OLMo-2	allenai/OLMo-2-1124-7B	allenai/OLMo-2-1124-7B-SFT , -DPO , -Instruct
OLMo-3	allenai/Olmo-3-1025-7B	allenai/Olmo-3-7B-Instruct-SFT , -DPO , Olmo-3-7B-Instruct
Qwen2.5	Qwen/Qwen2.5-7B	Qwen/Qwen2.5-7B-Instruct
Qwen3	Qwen/Qwen3-8B-Base	Qwen/Qwen3-8B

Appendix

Table 5: Exact checkpoints. Hugging Face identifiers of every evaluated checkpoint. Tülu and Llama-3.1 share the Llama-3.1-8B Base checkpoint, evaluated from a weight-verified mirror.

Edit	Attribute	US	AR	DE	AU	IN	Total
Replacement	ethical priority	1	3	2	1	2	9
	life priority	0	0	0	1	1	2
	religious belief	4	4	4	4	4	20
	worldview	1	2	2	1	0	6
Insertion	cognitive style	3	1	3	2	7	16
	ten other traits ∗	4	4	4	4	4	20 each

Appendix

Table 6: Profile identities per attribute panel. Counts are evaluated identities, not answer conditions; an identity can appear in several panels. Small panels, especially life priority, limit profile-level precision.

			Fixed gains		Refit in each draw
Model	Estimate	Error removed (%)	Lower	Upper	Lower	Upper
Llama-3.1	0.07	0.2	$-2.16$	1.89	$-3.46$	1.97
OLMo-2	$-0.02$	$-0.1$	$-0.92$	0.88	$-1.50$	0.96
OLMo-3	3.10	8.2	2.14	4.20	1.34	4.32
Qwen2.5	1.28	5.6	0.54	2.01	0.15	2.07
Qwen3	0.87	3.2	0.12	1.68	$-0.39$	1.81

Appendix

Table 7: Separate gains improve prediction significantly in OLMo-3 and Qwen2.5. Cell gains minus shared scaling in

Q^{2}

points, with the share of shared scaling’s squared error removed. Fixed-gain intervals resample evaluation questions only (97.5% for OLMo-3 and Tülu, 99% otherwise); refit intervals also resample fitting questions and correct across seven pairs. Bold estimates have refit intervals above zero. Qwen2.5 uses the float32 rerun (Appendix H.2 ).

Model	Attribute	Task	Additive	Full
Llama-3.1	0.83	$-1.10$	$-0.38$	0.07
OLMo-2	0.17	$-0.35$	$-0.19$	$-0.02$
OLMo-3	1.84	1.09	2.61	3.10
Qwen2.5	0.67	0.62	1.19	1.28
Qwen3	0.56	0.08	0.69	0.87
Qwen3.5	0.32	0.51	0.81	1.13

Appendix

Table 8: Additive attribute and task terms capture most of the benefit. Improvements over shared scaling in

Q^{2}

points on the 45 new questions, without regularization. Task fits one gain per task; Additive fits

\mu+u_{a}+v_{b}

; Full fits one gain per cell and reproduces Table 1 .

Trajectory	Transition	$Q^{2}_{\rm identity}$	$Q^{2}_{\rm shared}$	$R$	Total change	Residual
OLMo-2	Base $\to$ SFT	0.663	0.787	0.633
	SFT $\to$ DPO	0.899	0.969	0.308
	DPO $\to$ Instruct	0.977	0.994	0.263
OLMo-3	Base $\to$ SFT	0.627	0.639	0.966	0.892	0.876
	SFT $\to$ DPO	0.767	0.936	0.274	0.868	0.453
	DPO $\to$ Final	0.990	0.990	0.966

Appendix

Table 9: Shared scaling fits the tested DPO steps better than the SFT steps.

R

is the squared prediction error after scaling divided by the error of predicting no change; total change and residual are relative to each stage’s earlier endpoint. A high

Q^{2}_{\rm shared}

means scaling predicts well, not that training leaves responses unchanged. OLMo-3 and Tülu use question cross-validation; OLMo-2 uses the 181/45 split.

Figure 9: When do separate gains improve on one shared multiplier? Positive values favor gains for each attribute and task. (a) Adjacent transitions in OLMo-2, OLMo-3, and Tülu; OLMo-3 and Tülu use five folds of question cross-validation, OLMo-2 the 181/45 split. (b) Base versus Instruct on the 181/45 split; these endpoints combine stages. Bars are pointwise 95% paired bootstrap intervals with refitting, uncorrected; colors identify models as in the other figures. Each transition is evaluated separately. Numerical values are in the supplementary material. Qwen2.5 uses bfloat16 here; float32 checks appear in Appendix H.2 .

Source	Target	Est.	Lower	Upper	Est.	Lower	Upper	Est.	Lower	Upper
		One amplitude			Two parameters			Refit $-$ two
Llama-3.1	OLMo-2	$\mathbf{-4.28}$	$-7.93$	$-2.36$	$-0.11$	$-0.65$	0.26	0.09	$-1.53$	1.03
	OLMo-3	$\mathbf{-2.88}$	$-5.25$	$-1.40$	0.26	$-0.09$	0.69	2.84	0.90	4.41
	Qwen2.5	$\mathbf{-6.09}$	$-9.19$	$-4.52$	0.21	0.0005	0.50	1.02	$-0.27$	2.04
	Qwen3	$\mathbf{-3.45}$	$-6.41$	$-1.64$	0.00	$-0.11$	0.18	0.87	$-0.76$	2.05
	Qwen3.5	$\mathbf{-1.76}$	$-4.54$	$-0.18$	0.38	$-0.08$	0.95	0.75	$-0.67$	1.90

Appendix

Table 10: Every cross-model transfer. Differences in

Q^{2}

points on 45 new target questions. One amplitude: adapted reuse minus shared scaling. Two parameters: flexible reuse minus shared scaling. Refit

-

two: target refitting minus flexible reuse. Bounds are corrected across 42 directions separately for each comparison; bold estimates have intervals that exclude zero. One-amplitude values use 100,000 draws, the others 20,000. Significance and counts use unrounded bounds. Qwen2.5 uses bfloat16 here; float32 checks appear in Appendix H.2 .

Comparison	Above zero	Below zero	Positive point	Median
One amplitude $-$ shared	0	29	2	$-1.92$
Two parameters $-$ shared	1	0	28	0.03
Additive source, two parameters $-$ shared	1	0	24	0.05
Two parameters $-$ one amplitude	33	0	42	2.10
Refit $-$ two parameters	6	0	39	0.88

Appendix

Table 11: Separating level and variation removes significant reuse losses but adds little beyond shared scaling. Counts over 42 directions with corrected intervals; medians in

Q^{2}

points; Qwen2.5 in bfloat16 (float32: Appendix H.2 ).

Target group	Directions	One amp.	Two par.	Refit	Reuse below	Refit above	Refit above
		$-$ shared	$-$ shared	$-$ reuse	shared	reuse	two par.
OLMo-3	6	$-1.78$	0.19	4.88	4/6	6/6	6/6
Qwen2.5	6	$-3.52$	0.15	4.75	6/6	6/6	0/6
Other targets	30	$-1.91$	$-0.04$	2.74	19/30	19/30	0/30

Appendix

Table 12: Target gains beat borrowed patterns even where target structure is supported. Means over incoming directions in

Q^{2}

points, Qwen2.5 in bfloat16 (its own refit is 1.28 in float32; Table 1 ); counts of significant directions under the 42-direction correction. Float32 checks appear in Appendix H.2 .

Target	Own refit	Half-sample	Halving cost	Reuse	Source noise	Observed gap	Reliability
	Target data			Incoming patterns
Llama-3.1	0.07	$-0.24$	0.32	$-1.96$	0.18	2.03	0.91
OLMo-2	$-0.02$	$-0.17$	0.15	$-3.03$	0.31	3.01	0.86
OLMo-3	3.10	2.82	0.27	$-1.78$	0.25	4.88	0.90
Qwen2.5	1.23	0.95	0.28	$-3.52$	0.38	4.75	0.80
Qwen3	0.87	0.57	0.30	$-1.96$	0.31	2.82	0.87

Appendix

Table 13: Under the tested noise injection model, estimation noise costs far less than the observed transfer gap. Qwen2.5 in bfloat16 (own refit 1.28 in float32; Table 1 ).

Q^{2}

points except reliability. Own refit and Half-sample: target gains from all 181 or half of the questions versus shared scaling. Reuse: mean of six incoming patterns under adapted reuse. Source noise: mean cost of injecting each incoming source’s noise into the target gains. Observed gap: mean refitting minus adapted reuse. Reliability: projected split-half correlation of the cell gains. Float32 checks appear in Appendix H.2 .

	Base shared	Direct, one scale		Direct, cells	Cells $>$ scale	Direct $<$ Base
Target	$Q^{2}$	Median	Range	Median
Llama-3.1	0.637	0.474	0.41 to 0.70	0.536	3/6	5/6
OLMo-2	0.787	0.461	0.42 to 0.49	0.498	3/6	6/6
OLMo-3	0.624	0.411	0.38 to 0.45	0.455	5/6	6/6
Qwen2.5	0.766	0.487	0.39 to 0.50	0.503	1/6	6/6
Qwen3	0.728	0.477	0.41 to 0.50	0.496	2/6	6/6

Appendix

Table 14: Other models’ persona effects predict a target’s effects only moderately.

Q^{2}

on 45 new questions. Base shared: the target’s own Base effects with one scale (needs the Base). Direct: the six other models’ post-trained effects with one target-fitted scale (median and range) or one scale per cell (median). Counts are directions with Bonferroni intervals (42 directions) excluding zero. Qwen2.5 uses bfloat16 here; float32 checks appear in Appendix H.2 .

		Refit $-$ adapted reuse			Refit $-$ shared
Model	Change	Est.	Lower	Upper	Est.	Lower	Upper
Llama-3.1	Wording	3.63	1.89	5.72	1.47	0.39	2.18
	Examples	0.84	$-0.33$	1.76	1.62	$-0.59$	3.04
OLMo-3	Wording	3.91	2.40	5.45	2.38	1.26	3.00
	Examples	1.38	0.22	2.21	2.80	0.06	4.94
Qwen2.5	Wording	1.70	0.08	3.13	1.29	$-0.33$	2.32

Appendix

Table 15: Rewording favors refitting in all six pairs; changing examples still permits reuse. Values in

Q^{2}

points. Bounds are corrected within each study, as described in the text; bold estimates have intervals that exclude zero. The last two rows learn the Qwen2.5 source from one format; the main rows use two.

Pair	Est.	Lower	Upper	Est.	Lower	Upper	Est.	Lower	Upper
	Refit $-$ shared			Adapted $-$ shared			Refit $-$ adapted
Llama-3.1	1.83	0.84	2.49	$\mathbf{-1.99}$	$-4.95$	$-0.19$	3.82	2.01	6.40
OLMo-3	2.67	1.08	3.64	$-1.34$	$-3.89$	0.19	4.01	2.18	6.23
Qwen2.5	1.34	$-0.37$	2.17	$-0.17$	$-1.62$	0.68	1.51	$-0.30$	2.99
Qwen3	1.43	0.55	2.06	n.r.			0.97	0.32	1.63
Qwen3.5	0.49	$-0.60$	1.17	$\mathbf{-1.50}$	$-3.14$	$-0.48$	1.98	0.92	3.20

Appendix

Table 16: Joint wording and example change: all three comparisons. Paired differences in

Q^{2}

points on identical target questions, with intervals including fitting uncertainty. Corrections: 24 comparisons for OLMo-3 and Tülu (shared with their profile comparisons), four per contrast for Llama-3.1 and Qwen2.5, four settings for Qwen3 and Qwen3.5. Bold estimates have intervals that exclude zero; n.r. means not reported in that study.

		Flexible reuse $-$ shared			Refit $-$ flexible reuse
Model	Change	Est.	Lower	Upper	Est.	Lower	Upper
Llama-3.1	Wording	$-0.03$	$-0.91$	0.51	1.50	0.50	2.31
	Examples	0.85	$-0.86$	1.95	0.77	$-0.66$	1.81
OLMo-3	Wording	0.03	$-0.66$	0.51	2.35	1.21	3.13
	Examples	1.76	$-0.49$	3.83	1.04	$-0.18$	2.00
	Both	0.01	$-0.87$	0.57	2.66	1.43	3.64

Appendix

Table 17: Rewording still favors refitting over flexible reuse. Differences in

Q^{2}

points with intervals corrected across 14 cells; bold estimates have intervals that exclude zero.

Model	Change	Est.	Lower	Upper	Est.	Lower	Upper	Est.	Lower	Upper
		One amplitude $-$ shared			Two parameters $-$ shared			Refit $-$ two parameters
Llama-3.1	Wording	$\mathbf{-2.16}$	$-4.51$	$-0.25$	$-0.06$	$-1.03$	0.61	1.53	0.36	2.44
	Examples	0.78	$-0.71$	2.02	1.06	$-0.18$	2.07	0.56	$-1.37$	1.64
OLMo-3	Wording	$\mathbf{-1.53}$	$-2.70$	$-0.23$	0.14	$-0.43$	0.66	2.24	0.99	3.01
	Examples	1.42	$-0.59$	3.81	1.55	$-0.23$	3.56	1.25	$-0.50$	2.20
	Both	$-1.34$	$-2.52$	0.03	0.17	$-0.52$	0.77	2.50	1.13	3.56

Appendix

Table 18: Prompt changes with the two-template source. Differences in

Q^{2}

points with intervals corrected across 14 cells; bold estimates have intervals that exclude zero. Intervals condition on the fitted source pattern and resample only target questions, unlike Table 17 .

Model	Change	Unch.	Shared	Refit	Est.	Lower	Upper	Est.	Lower	Upper
		$Q^{2}$			Shared $-$ unchanged			Refit $-$ shared
Llama-3.1	Wording	0.685	0.715	0.749	3.04	1.57	4.48	3.41	2.00	4.55
	Examples	0.750	0.771	0.787	2.10	0.88	3.29	1.52	0.46	2.41
OLMo-3	Wording	0.703	0.703	0.750	0.06	$-0.26$	0.36	4.63	3.07	5.94
	Examples	0.702	0.782	0.803	7.95	2.99	15.04	2.14	$-0.08$	3.84
	Both	0.615	0.615	0.667	$-0.02$	$-0.57$	0.33	5.22	3.06	6.95

Appendix

Table 19: Within one deployed model, rewording calls for relearning attribute relationships.

Q^{2}

on new questions for unchanged reuse, shared scaling, and refitted gains, and paired differences in points with intervals corrected across 14 cells; bold estimates have intervals above zero.

				Centered 95% interval
Pair	Source	Uncentered	Centered	Lower	Upper
Llama-3.1	Source 1	0.975	0.345	0.254	0.427
	Source 2	0.977	0.553	0.481	0.618
OLMo-3	Source 1	0.975	0.458	0.369	0.536
	Source 2	0.972	0.458	0.357	0.545
Qwen2.5	Source 1	0.985	0.406	0.286	0.516

Appendix

Table 20: Removing the mean reveals differences between apparently similar gains. Both correlations use the same energy weights. Source 1 and 2 are the two source templates. These correlations do not define a cutoff for successful reuse.

Pair	Refit	Adapted reuse	Lower	Upper
Mistral	1.33	1.27	0.78	1.79
OLMo-3	2.65	2.55	1.14	4.06
Qwen3	1.41	1.34	0.85	1.88
Qwen3.5	0.83	0.71	0.07	1.40
Tülu	1.88	1.66	0.58	2.48

Appendix

Table 21: Source gains remain useful across five country contexts. Improvements over target shared scaling in

Q^{2}

points. Intervals for adapted reuse condition on fitted gains and are corrected within each study (98.3 to 99.2%).

Model	Contrast	Estimate	Lower	Upper
OLMo-3	Same role $-$ different role	$\mathbf{-0.39}$	$-0.69$	$-0.11$
	Category $-$ amplitude	$\mathbf{-0.18}$	$-0.36$	$-0.07$
	Two components $-$ amplitude	0.08	$-0.09$	0.21
	Four components $-$ amplitude	0.08	$-0.12$	0.23
	Amplitude $-$ no adaptation	$\mathbf{1.07}$	$0.48$	$1.70$
	Full refit $-$ amplitude	$\mathbf{1.84}$	$1.37$	$2.10$

Appendix

Table 22: Simpler updates do not close the gap. Differences in

Q^{2}

points, means over six held-out templates, with intervals corrected across two families; bold estimates have intervals that exclude zero. The full-refit rows are references with more information.

Pair	Template	Est.	Low	Up	Est.	Low	Up	Est.	Low	Up	Est.	Low	Up
		Source $-$ shared shrunk			Source shrunk $-$ adapted			Refit $-$ source shrunk			Source shrunk $-$ shared
Llama-3.1	Joint	0.01	$-0.11$	0.24	3.82	1.83	6.92	0.00	$-0.02$	0.05	1.83	0.66	2.60
	Frozen	0.00	$-0.12$	0.23	4.27	2.30	6.56	$-0.00$	$-0.08$	0.02	2.40	1.18	3.34
OLMo-3	Joint	$-0.01$	$-0.27$	0.17	3.99	2.21	5.83	0.02	$-0.01$	0.30	2.65	1.11	3.48
	Frozen	0.01	$-0.13$	0.20	2.25	1.15	3.31	0.00	$-0.10$	0.13	3.34	1.99	4.32
Qwen2.5	Joint	$-0.04$	$-0.24$	0.29	1.52	$-0.21$	3.09	$-0.01$	$-0.28$	0.14	1.34	$-0.34$	2.28

Appendix

Table 23: Shrinking toward the source gains gives no detected advantage over shrinking toward a shared gain. Differences in

Q^{2}

points with intervals corrected across twelve settings for each comparison; bold estimates have intervals that exclude zero. Joint: wording and example change; Frozen: the questionnaire template of § 4.3 .

		$k=8$		$k=16$		$k=32$		$k=64$
Pair	Template	Refit	Shrunk	Refit	Shrunk	Refit	Shrunk	Refit	Shrunk
Llama-3.1	Joint change	$-1.36$	0.85	1.76	2.02	3.02	2.60	3.52	3.40
	Frozen	$-2.31$	0.76	1.66	2.31	3.26	3.35	3.87	3.90
OLMo-3	Joint change	$-2.65$	0.57	1.34	1.82	2.79	2.30	3.59	3.49
	Frozen	$-4.83$	0.07	$-0.45$	0.46	1.05	1.30	1.79	1.73
Qwen2.5	Joint change	$-5.98$	0.05	$-1.52$	0.36	0.21	0.88	1.05	1.11

Appendix

Table 24: A regularized update beats adapted reuse at every budget. Median

Q^{2}

differences from adapted reuse, in points, over 200 subsets of the target fitting questions: unregularized refitting (Refit) and the update shrunk toward the source gains (Shrunk).

		Two parameters $-$ shared scaling				Source-shrunk update $-$ two parameters
Pair	Template	$k=8$	16	32	64	$k=8$	16	32	64
Llama-3.1	Joint	$-0.05$	0.01	$-0.03$	0.02	$-1.01$	0.06	0.61	1.42
	Frozen	0.16	0.20	0.22	0.23	$-1.26$	0.24	1.25	1.81
OLMo-3	Joint	0.03	0.10	0.14	0.16	$-0.79$	0.24	0.86	1.97
	Frozen	1.28	1.32	1.34	1.35	$-0.03$	0.23	1.05	1.47
Qwen2.5	Joint	0.38	0.40	0.41	0.41	$-0.36$	$-0.16$	0.34	0.54

Appendix

Table 25: The regularized update exceeds flexible reuse in median in 11 of 12 settings at 32 questions per attribute and all 12 at 64. Median

Q^{2}

differences in points over 200 subsets of the target fitting questions at

k

questions per attribute.

		Refit $-$ adapted reuse			Refit $-$ shared			Gap to full refit
Pair	Design	Est.	Lower	Upper	Est.	Lower	Upper	$k=16$	$k=32$
Llama-3.1	Extension	4.26	2.56	6.18	2.39	1.25	3.14	2.67	1.08
OLMo-3	Prospective	2.26	1.32	3.13	3.34	2.22	4.04	3.04	1.33
Qwen2.5	Extension	1.31	0.04	2.40	1.78	0.51	2.42	3.05	1.27
Qwen3	Extension	1.58	0.47	2.81	1.25	0.16	1.95	1.95	0.82
Qwen3.5	Extension	1.41	$-1.26$	3.23	1.04	$-1.32$	2.36	5.93	2.31

Appendix

Table 26: What refitting gains on the frozen questionnaire.

Q^{2}

points with corrected intervals: two comparisons for the prospective OLMo-3 and Tülu test, four within each later extension. Bold estimates have intervals above zero. Budget gaps are medians over subsets of the difference from refitting on all 181 fitting questions, not confidence intervals.

Figure 10: Unregularized crossover and breadth allocation. Left: target refitting minus single-amplitude reuse. Right: measuring every attribute minus measuring half the attributes more deeply, at equal numbers of model calls.

\Delta Q^{2}

on its original scale (0.01 is one point); medians and interquartile ranges over 200 subsets. Regularized alternatives appear in Figure 6 .

Condition	Below shared	Negative point	Median	Refit above reuse
Completion (original)	29	40	$-1.92$	31
Chat interface	29	42	$-2.30$	31
Answer orders averaged	33	41	$-2.25$	36
Reversed order only	29	42	$-2.09$	30
Independent questions	28	39	$-1.81$	28

Appendix

Table 27: Single-amplitude reuse loses under every tested condition. Counts refer to the 42 directed transfers with one target-fitted amplitude, corrected across 42 directions within each condition. Medians are adapted reuse minus shared scaling in

Q^{2}

points. The last column counts directions in which target refitting significantly beats adapted reuse. Qwen2.5 uses bfloat16 here; float32 checks appear in Appendix H.2 .

Pair	Condition	Estimate	Lower	Upper
Llama-3.1	Completion (original)	0.07	$-3.46$	1.97
	Chat interface	0.97	$-3.40$	3.33
	Answer orders averaged	1.44	$-0.86$	3.06
	Reversed order only	1.43	$-0.67$	2.76
	Independent questions	1.62	$-1.04$	3.13
OLMo-2	Completion (original)	$-0.02$	$-1.50$	0.96

Appendix

Table 28: Most structure estimates are positive across checks, and OLMo-3 is significant in every one. Target refitting minus shared scaling on held-out questions, in

Q^{2}

points, with intervals corrected across seven pairs that include fitting uncertainty. Bold estimates have intervals above zero. Qwen2.5 uses bfloat16 here; float32 checks appear in Appendix H.2 .

	Two parameters $-$ shared						Two $-$ one	Refit $-$ two
Condition	$>0$	$<0$	Point $>0$	Median	Upper $<1$	Upper $<0.5$	$>0$	$>0$
Completion (original)	1	0	28	0.03	40	26	33	6
Chat interface	0	0	26	0.02	37	28	38	6
Answer orders averaged	0	0	26	0.04	39	29	37	8
Independent questions	0	0	30	0.07	36	23	32	1

Appendix

Table 29: Flexible reuse adds little beyond shared scaling under every tested condition. Counts over 42 directions with intervals corrected across directions; medians in

Q^{2}

points. The last two columns count directions in which two parameters significantly beat one amplitude, and in which target refitting significantly beats two parameters. Significance and counts use unrounded bounds. Qwen2.5 uses bfloat16 here; float32 checks appear in Appendix H.2 .

	All 57 attributes			Six most responsive dropped
Pair	Est.	Lower	Upper	Est.	Lower	Upper
Llama-3.1	0.07	$-3.45$	1.98	0.36	$-3.31$	2.39
OLMo-2	$-0.02$	$-1.49$	0.96	0.14	$-1.40$	1.15
OLMo-3	$\mathbf{3.10}$	$1.34$	$4.38$	$\mathbf{2.44}$	$0.22$	$3.84$
Qwen2.5	$\mathbf{1.23}$	$0.08$	$1.97$	1.05	$-0.22$	1.77
Qwen3	0.87	$-0.40$	1.82	0.78	$-0.73$	1.93

Appendix

Table 30: Dropping the most responsive attributes leaves the structure estimates largely intact. Cell gains minus shared scaling on new questions, in

Q^{2}

points, with intervals corrected across seven pairs; bold estimates have intervals above zero. Qwen2.5 uses bfloat16 here; float32 checks appear in Appendix H.2 .

Source	Target	Est.	Lower	Upper	Est.	Lower	Upper	Est.	Lower	Upper
		One amplitude $-$ shared			Two parameters $-$ shared			Refit $-$ two
Llama-3.1	Qwen2.5	$\mathbf{-6.10}$	$-9.06$	$-4.57$	0.22	$-0.0049$	0.50	1.07	$-0.27$	2.06
OLMo-2	Qwen2.5	$\mathbf{-2.09}$	$-3.68$	$-1.24$	0.25	$-0.07$	0.72	1.03	$-0.27$	1.96
OLMo-3	Qwen2.5	$\mathbf{-5.44}$	$-8.82$	$-3.81$	0.37	$-0.04$	0.91	0.91	$-0.44$	1.79
Qwen3	Qwen2.5	$\mathbf{-1.49}$	$-3.45$	$-0.48$	$-0.03$	$-0.36$	0.19	1.32	$-0.0018$	2.32
Qwen3.5	Qwen2.5	$\mathbf{-2.14}$	$-4.48$	$-0.85$	0.04	$-0.22$	0.36	1.24	$-0.05$	2.15

Appendix

Table 31: Transfers involving Qwen2.5 with its float32 outputs. Differences in

Q^{2}

points with intervals corrected across the 42 directions of the full analysis (20,000 draws); bold estimates have intervals that exclude zero. Other pairs remain in bfloat16. Significance and counts use unrounded bounds.

Arm	Comparison	Est.	Lower	Upper	Est.	Lower	Upper
		bfloat16			float32
Replacement	Source only	$\mathbf{-3.88}$	$-8.07$	$-2.10$	$-0.19$	$-0.62$	0.14
Replacement	Refit $-$ shared	6.52	$-0.09$	22.42	5.41	$-6.72$	18.21
Replacement	Adapted $-$ shared	$\mathbf{-1.16}$	$-2.74$	$-0.25$	$-1.65$	$-4.25$	0.67
Replacement	Refit $-$ adapted	$\mathbf{7.68}$	$0.49$	$24.16$	7.06	$-5.68$	18.70
Replacement	Refit $-$ task-shared	2.68	$-2.63$	14.08	4.45	$-5.10$	21.00

Appendix

Table 32: Float32 evaluation changes comparisons in both profile arms; refitting advantages for inserted traits remain significant. Differences in

Q^{2}

points with intervals corrected across 16 comparisons (source-only rows use 98.75% intervals), for the bfloat16 and float32 evaluations of the same frozen design. Bold estimates have intervals that exclude zero. Source only compares source cell gains with a source shared gain, both fitted on short statements and applied without target adjustment.

	Relevance-specific stage change			Additional strength-gradient change
Pair	Estimate	Lower	Upper	Estimate	Lower	Upper
Llama	$-0.1543$	$-0.3681$	$0.0539$	$0.1815$	$-0.1997$	$0.5800$
OLMo	$0.1198$	$-0.1077$	$0.4784$	$-0.2200$	$-0.5040$	$0.1474$
Qwen2.5	$\mathbf{-0.7252}$	$-0.9206$	$-0.5554$	$-0.2293$	$-0.5420$	$0.0949$
Qwen3	$\mathbf{-0.7904}$	$-0.9756$	$-0.5878$	$-0.1880$	$-0.5116$	$0.0864$
Qwen3.5	$\mathbf{0.6790}$	$0.4650$	$0.9260$	$\mathbf{0.2910}$	$0.0487$	$0.4978$

Appendix

Table 33: Preference relevance and preference strength ask different questions. Estimates with lower and upper bounds corrected within each study; bold estimates have intervals that exclude zero. A negative relevance change means training reduces country influence under relevant preferences relative to unrelated preferences. The additional effect of strong versus weak preferences increases in Qwen3.5 and remains unresolved in the other five pairs.

Pair	Edit	Est.	Lower	Upper	Est.	Lower	Upper	Est.	Lower	Upper
		Source only			Refit $-$ shared			Refit $-$ adapted
Llama-3.1	Replacement	$\mathbf{-10.22}$	$-16.85$	$-5.99$	0.39	$-7.77$	4.84	1.49	$-7.20$	7.03
	Insertion	0.72	$-0.59$	1.71	12.41	1.60	24.67	11.67	1.13	23.57
OLMo-3	Replacement	0.56	$-1.37$	3.19	0.63	$-8.48$	7.20	0.57	$-7.92$	8.51
	Insertion	0.77	$-0.26$	1.79	3.00	$-1.27$	4.83	2.41	$-1.59$	4.16
Qwen2.5	Replacement	$-0.19$	$-0.62$	0.14	5.41	$-6.72$	18.21	7.06	$-5.68$	18.70

Appendix

Table 34: Do gains learned from isolated traits predict effects inside full profiles? Differences in

Q^{2}

points on held-out profile questions; positive values favor the first predictor named. Intervals include source fitting and resampling of profiles and questions, corrected within each study family (24 comparisons for OLMo-3/Tülu, 16 each for Qwen2.5/Llama-3.1 and Qwen3/Qwen3.5; source-only rows use four). Bold estimates have intervals that exclude zero. Qwen2.5 uses float32; other pairs bfloat16. Source only compares source cell gains with a source shared gain, both fitted on short statements and applied without target adjustment.

Pair	Edit	Est.	Lower	Upper	Est.	Lower	Upper
		Adapted reuse $-$ shared			Refit $-$ task-shared
Llama-3.1	Replacement	$-1.10$	$-3.14$	0.01	$-1.08$	$-7.66$	2.71
	Insertion	0.74	$-0.92$	2.18	11.26	0.95	22.31
OLMo-3	Replacement	0.06	$-2.43$	2.38	$-0.43$	$-10.48$	4.56
	Insertion	0.58	$-0.60$	1.77	2.41	$-0.80$	3.99
Qwen2.5	Replacement	$-1.65$	$-4.25$	0.67	4.45	$-5.10$	21.00

Appendix

Table 35: Remaining profile baselines. Differences in

Q^{2}

points on held-out profile questions, with the same corrections as Table 34 ; bold estimates have intervals that exclude zero. Qwen2.5 uses float32.

Pair	Edit	Contrast RMS	Shared	Adapted	Refit	Reduction (%)
Llama-3.1	Replacement	0.203	0.197	0.198	0.196	+0.2
	Insertion	0.353	0.325	0.323	0.300	+7.6
OLMo-3	Replacement	0.122	0.111	0.110	0.110	+0.4
	Insertion	0.124	0.115	0.115	0.113	+1.8
Qwen2.5	Replacement	0.825	0.657	0.666	0.629	+4.4
	Insertion	1.096	0.797	0.830	0.695	+12.9

Appendix

Table 36: Profile prediction error in log odds. Post-trained profile contrast RMS and held-out RMSE of shared scaling, adapted reuse, and target refitting under question cross-validation; the last column is the relative reduction from shared scaling to refitting. Qwen2.5 uses float32; other pairs bfloat16.

		Flexible reuse $-$ shared			Refit $-$ flexible reuse
Pair	Arm	Est.	Lower	Upper	Est.	Lower	Upper
Llama-3.1	Replacement	0.23	$-1.84$	2.47	0.16	$-7.56$	4.88
	Insertion	0.77	$-0.22$	2.12	$\mathbf{11.64}$	$1.71$	$23.16$
OLMo-3	Replacement	0.15	$-1.60$	2.67	0.48	$-7.25$	6.70
	Insertion	0.55	$-0.11$	1.56	2.44	$-1.13$	4.06
Qwen2.5	Replacement	0.48	$-1.31$	2.17	4.92	$-6.31$	17.54

Appendix

Table 37: Inserted traits still favor refitting over flexible reuse. Differences in

Q^{2}

points on held-out profile questions, with intervals corrected across 12 settings; bold estimates have intervals that exclude zero.

Figure 11: How many human labels does calibration need, and can it be reused? Left: extra test loss compared with calibration on 84 development questions with group distribution labels, averaged across label subsets. Right: extra loss when temperatures are reused across survey waves or checkpoints, with 99.375% intervals. The shaded reference band is

\pm 0.005

nats. Intervals for reuse across waves fall within

\pm 0.005

nats in the four main families. Reuse across checkpoints adds loss in Qwen2.5 and Llama; the difference remains unresolved in OLMo and Tülu.

Model	Raw regret	Lower	Upper	Calibrated regret
OLMo-2	0.611	0.421	0.823	0.000
OLMo-3	0.233	0.149	0.341	0.000
Qwen2.5	1.261	0.892	1.649	-0.001
Qwen3	0.751	0.573	0.956	-0.002
Qwen3.5	0.000	-0.005	0.065	0.004
Llama-3.1	0.215	0.138	0.302	0.000

Appendix

Table 38: Selection regret across all admitted model families on the original survey dimensions. Raw intervals are 97.5% for the original OLMo-3, Tülu, Qwen2.5, and Llama comparisons and 99% for the later extensions. Calibrated controls repeat fitting and selection within training folds. OLMo comparisons select among four checkpoints; other rows compare two.

Checkpoint	Calibrated CE	Excess over human entropy	Share of gap closed (%)
Llama-3.1	0.663	0.081	27
Mistral	0.674	0.092	17
OLMo-3	0.680	0.098	12
Qwen2.5	0.663	0.081	27
Qwen3	0.670	0.088	21
Qwen3.5	0.663	0.081	27

Appendix

Table 39: Calibrated models remain far from the human distributions. Weighted cross entropy in nats on 72 held-out questions of the four-dimension suite. The share of gap closed is

(0.693-\text{CE})/(0.693-0.582)

Checkpoint	Recovered (%)	95% interval	$p$	Above chance (Bonferroni)
Llama-3.1	69.5	[62.1, 76.7]	0.0001	Yes
Mistral	67.4	[59.3, 74.6]	0.0001	Yes
OLMo-3	56.7	[46.4, 66.0]	0.1033	Not detected
Qwen2.5	67.4	[59.0, 75.2]	0.0002	Yes
Qwen3	69.5	[60.3, 78.3]	0.0001	Yes
Qwen3.5	61.0	[51.4, 69.9]	0.0142	Not detected

Appendix

Table 40: Models recover some group directions, far fewer than human replication. Percentage of 141 selected group differences whose direction the model recovers, with uncorrected question clustered 95% intervals. Reported

p

values are raw one sided tests against 50%; the final column applies Bonferroni correction across seven models.

Family	Base (%)	Base above chance	Post-trained (%)	Difference	Corrected interval
Llama-3.1	53.5	Not detected	69.5	+16.0	3.9 to 30.1
Mistral	62.8	Yes	67.4	+4.6	-4.6 to 14.4
OLMo-3	60.3	Not detected	56.7	-3.5	-12.8 to 5.1
Qwen2.5	63.8	Yes	67.4	+3.5	-3.9 to 11.3
Qwen3	69.9	Yes	69.5	-0.4	-10.0 to 9.5
Qwen3.5	62.8	Yes	61.0	-1.8	-10.8 to 7.0

Appendix

Table 41: Group-direction recovery before and after post-training. Percentage of the 141 selected differences recovered. Bonferroni intervals across seven families for the post-trained minus Base difference; bold estimates have intervals excluding zero. Llama-3.1 and Tülu share a Base.

LLM Prompting

Large language model prompting studies how instructions, examples, context, and formats steer model behavior at inference time.

See all 419 papers

Persona Following Is Not Selective Control: The Neutrality Gap in LLM User Simulation
Sep 28, 2026Jiashen Ren, Wenlin Zhang, Bohan Zhang +5Persona-Conditioned Generation User Simulation
City University of Hong Kong · Stanford University
The Persona Hierarchy Model: Understanding Contextual Generalization in Fine-Tuning LLMs
Oct 7, 2026Jiachen Zhao, Zhengxuan Wu, David Bau +1LLM Fine-Tuning LLM Alignment
Northeastern University · Stanford University
The Illusion of Debiasing: Persona Steering Redistributes Rather Than Reduces Bias in LLMs
Sep 7, 2026Ziyue Feng, Hongbo Fang, James A. EvansLanguage Model Steering Language Model Debiasing
University of Chicago Chicago, IL, USA

Sep 28, 2026J/K move · Enter open · S save

Persona Following Is Not Selective Control: The Neutrality Gap in LLM User Simulation

Jiashen Ren, Wenlin Zhang, Bohan Zhang, Xiaopeng Li+4

City University of Hong Kong · Stanford University

Persona-Conditioned Generation User Simulation

Open paper AskPDF arXiv

Persona prompting is widely used to construct user simulations with large language models (LLMs), yet it relies on a largely untested assumption: specifying one user attribute should change that attribute alone. We test this assumption and identify a systematic failure of selective control: across all eight black-box LLMs we audit, changing a target attribute also shifts responses on unspecified, non-target attributes. For example, describing a user as more risk-seeking shifts color choices, even though the prompt never mentions color; we term this cross-attribute influence. Semantic, contextual, and internal analyses collectively suggest that models treat a persona prompt as evidence about the user and extend the inferred profile to unspecified preferences, a process we call trait-conditioned completion. We next ask whether explicitly specifying non-target attributes restores selective control. When a non-target attribute is assigned a clear direction, models generally follow the declaration and suppress the target attribute's influence. However, when the same attribute is declared neutral, the target continues to affect choices across all five open-weight checkpoints, even when the model correctly reports the declared state. This disparity, the neutrality gap, demonstrates that successful persona following does not imply selective persona control, which additionally requires keeping non-target attributes stable. We operationalize this distinction with a three-state diagnostic that leaves the non-target attribute unspecified or declares it directional or neutral; because directional tests can be passed by simply following the stated persona, the neutral state reveals failures they miss. In a post hoc analysis of independent items, neutral declarations leave 51-81% of items target-sensitive, against at most 1 of 320 item-pole comparisons under directional ones.

Abstract

LLM Prompting

Explore similar work

Persona Following Is Not Selective Control: The Neutrality Gap in LLM User Simulation

The Persona Hierarchy Model: Understanding Contextual Generalization in Fine-Tuning LLMs

The Illusion of Debiasing: Persona Steering Redistributes Rather Than Reduces Bias in LLMs