PerMix-RLVR: Preserving Persona Expressivity under Verifiable-Reward Alignment

Jihwan Oh; Soowon Oh; Murad Aghazada; Minchan Jeong; Sungnyun Kim; Se-Young Yun

cs.CLApr 10, 2026

PerMix-RLVR: Preserving Persona Expressivity under Verifiable-Reward Alignment

Authors: Jihwan Oh, Soowon Oh, Murad Aghazada, Minchan Jeong, Sungnyun Kim, Se-Young Yun

Organizations: KAIST AI · Samsung Advanced Institute of Technology

Abstract

Persona prompting has been widely adopted to steer large language models (LLMs) behavior and improve their instruction performance by assigning specific characters. However, identifying an optimal persona is time-consuming, and its impact on output quality remains poorly understood. Prior work has mainly addressed this issue at the prompt level via inference-time strategies, incurring additional computation. In this work, we avoid inference-time prompt search by tackling persona sensitivity during training, aiming to train models that adapt their behavior to diverse personas while preserving task performance. In particular, we find that reinforcement learning with verifiable rewards (RLVR) systematically reduces sensitivity to persona prompts, but also reveals an inherent trade-off of outcome-based optimization: while RLVR improves robustness on tasks with verifiable goals, it can also degrade persona expressivity when needed, e.g., in-character role-playing. To address this limitation, we propose PerMix-RLVR, a persona-mixed RLVR strategy that mitigates the persona robustness-fidelity trade-off, preserving strong robustness to harmful persona variation while enabling faithful persona adoption when required. Concretely, PerMix-RLVR improves persona stability score (PSS) over RLVR by +21.2% on MATH500, while also enhancing persona fidelity by +11.4% on PersonaGym.

Figures & tables

Figure 1: Persona lottery and the cost of prompt search. (a) MATH500 and (b) AIME2024 show Llama3 and Gemma3 performances under seven different personas (four math experts , kindergartener , carpenter , and no persona ). Each dot denotes persona; $\star$ and $\circ$ mark the best and worst persona per model, and black bars show mean $\pm$ std across personas, revealing large persona-induced variance. (c) Expected best-of- $k$ . We report the expected best score after trying $k$ personas (without replacement), averaged across models.

Figure 2: PSS comparison across model families and scales. RLVR-trained families consistently exhibit higher PSS, indicating greater robustness to persona-induced variations. Full results are provided in Appendix C .

Model	WR	WR (no-tie)	Net margin
Llama3.1 (non-RLVR)	$50.2_{\,\pm\,9.0}$	$60.3$	$+17.2$
Qwen3 (RLVR)	$33.0_{\,\pm\,6.7}$	$39.7$	$-17.2$
Tie	$16.8_{\,\pm\,4.3}$	–	–

Table 1: Pairwise LLM-judge comparison under the kindergartener persona on MATH500. WR denotes the win rate (%), and WR (no-tie) is computed by excluding ties:

\text{win}/(\text{win}+\text{loss})

using mean counts. Net margin is

\text{win}-\text{loss}

(mean).

Figure 3: Standard RLVR vs. PerMix-RLVR. (a) RLVR reweights $\pi_{\mathrm{ref}}(s|x)$ toward competent styles, yielding $\pi_{\mathrm{RL}}(s|x)$ . A test-time persona shifts the prior to $\pi_{\mathrm{ref}}(s|x,p)$ , causing support mismatch for standard RLVR. (b) PerMix-RLVR is trained on diverse persona-conditioned priors to produce $\pi_{\text{PerMix-RL}}(s|x,p)$ aligned with persona-shifted support. This indicates better coverage under the persona-shifted prior $\pi_{\mathrm{ref}}(s|x,p)$ . The yellow hatched region represents style competence.

Post-training	Input	Objective ( $\mathcal{J}$ )	GSM8K (i.d.)				MATH500 (o.o.d.)				PersonaGym
Post-training	Input	Objective ( $\mathcal{J}$ )	Worst	Best	Acc. Mean	PSS	Worst	Best	Acc. Mean	PSS	Consistency
Base ( $M_{0}$ )	$x$	-	81.1	84.9	83.5 ${}_{\,\pm\,1.1}$	0.956	28.6	46.0	42.5 ${}_{\,\pm\,3.8}$	0.619	3.19
SFT	$x$	$\mathcal{L}_{CE}(y)$	73.4	77.2	75.8 ${}_{\,\pm\,1.1}$	0.951	30.3	37.2	34.6 ${}_{\,\pm\,1.7}$	0.805	2.53
PerMix-SFT	$x$ + $p$	$\mathcal{L}_{CE}(y)$	74.3	76.2	75.2 ${}_{\,\pm\,0.5}$	0.974	33.0	35.3	34.2 ${}_{\,\pm\,0.6}$	0.880	2.88
PerMix-SFT	$x$ + $p$	$\mathcal{L}_{CE}(y_{s})$	71.0	81.5	78.7 ${}_{\,\pm\,2.8}$	0.871	24.8	41.2	37.4 ${}_{\,\pm\,3.9}$	0.591	3.31
RLVR	$x$	$\mathbb{E}[V-\beta D_{KL}]$	84.0	87.6	86.1 ${}_{\,\pm\,0.9}$	0.959	34.0	49.6	46.8 ${}_{\,\pm\,3.8}$	0.675	3.06

Table 2: Comparison of teacher-free post-training strategies on GSM8K and MATH500 (accuracy %, PSS) and PersonaGym (role-playing consistency).

x

is an input instance, while

p

is an input persona prompt.

\mathcal{L}_{CE}

is a cross-entropy loss on a ground-truth

y

(or persona-styled

y_{s}

rewritten by GPT-5-mini). RLVR uses verifier reward

V

along with KL regularization

\beta D_{KL}

for optimization. Bold and underline indicate the best and second best performance, respectively.

Post-training	Teacher	Input	Objective ( $\mathcal{J}$ )	GSM8K (i.d.)				MATH500 (o.o.d.)				PersonaGym
Post-training	Teacher	Input	Objective ( $\mathcal{J}$ )	Worst	Best	Acc. Mean	PSS	Worst	Best	Acc. Mean	PSS	Consistency
SeqKD	Qwen3-32B	$x$	$\mathcal{L}_{CE}(y^{t})$	80.4	84.7	83.2 ${}_{\,\pm\,0.9}$	0.949	37.6	48.2	45.8 ${}_{\,\pm\,2.6}$	0.764	3.12
SeqKD	Qwen3-32B	$x$ + $p$	$\mathcal{L}_{CE}(y^{t}_{p})$	81.0	85.3	84.3 ${}_{\,\pm\,1.2}$	0.949	40.7	47.8	45.5 ${}_{\,\pm\,1.6}$	0.839	3.06
SeqKD	Llama3.1-70B	$x$	$\mathcal{L}_{CE}(y^{t})$	83.7	87.5	86.6 ${}_{\,\pm\,0.9}$	0.957	35.6	48.0	45.7 ${}_{\,\pm\,2.9}$	0.720	2.72
SeqKD	Llama3.1-70B	$x$ + $p$	$\mathcal{L}_{CE}(y^{t}_{p})$	82.9	87.3	86.5 ${}_{\,\pm\,1.0}$	0.950	35.2	47.7	45.2 ${}_{\,\pm\,2.9}$	0.728	3.09
SeqKD	Gemma3-27B	$x$	$\mathcal{L}_{CE}(y^{t})$	62.1	85.2	82.0 ${}_{\,\pm\,5.6}$	0.729	31.7	47.2	45.3 ${}_{\,\pm\,3.8}$	0.656	3.06

Table 3: Comparison of teacher-based distillation and RLVR on GSM8K and MATH500 (accuracy %, PSS) and PersonaGym (role-playing consistency). Llama3.1-8B-Instruct is the student, and

y^{t}

denotes teacher outputs (or

y^{t}_{p}

when conditioned on a system persona).

Post-training	Input	Objective ( $\mathcal{J}$ )	Easy				Medium
Post-training	Input	Objective ( $\mathcal{J}$ )	Worst	Best	Acc. Mean	PSS	Worst	Best	Acc. Mean	PSS
Base ( $M_{0}$ )	$x$	-	22.5	30.4	25.4 ${}_{\,\pm\,1.9}$	0.741	0.6	3.6	2.1 ${}_{\,\pm\,0.9}$	0.167
Supervised Fine-tuning
SFT	$x$	$\mathcal{L}_{CE}(y)$	19.0	25.4	22.5 ${}_{\,\pm\,1.6}$	0.750	0.6	3.0	1.7 ${}_{\,\pm\,0.7}$	0.200
PerMix-SFT	$x$ + $p$	$\mathcal{L}_{CE}(y)$	21.1	28.2	25.0 ${}_{\,\pm\,1.9}$	0.750	0.6	3.0	1.7 ${}_{\,\pm\,0.6}$	0.200
PerMix-SFT	$x$ + $p$	$\mathcal{L}_{CE}(y_{s})$	18.3	24.7	21.2 ${}_{\,\pm\,1.9}$	0.743	0.8	2.9	1.8 ${}_{\,\pm\,0.5}$	0.292

Table 4: Controlled post-training comparison on LiveCodeBench. Standard RLVR can reduce PSS under domain shift, whereas PerMix-RLVR consistently improves both mean accuracy and PSS across splits, matching or outperforming teacher-free baselines and remaining competitive with teacher-based distillation.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 4 — Figure 4: Graphical model of the generation process. Shaded nodes are observed; the unshaded node $s$ (style) is a latent variable introduced for analysis. The persona prompt $p$ (dashed) is optional: it is absent in standard RLVR and present in persona-conditioned RLVR. The reasoning trajectory $z$ is the generated token sequence; the answer $a(z)$ and verifier output $V(z)$ are determined from $z$ . Note that $p$ influences $z$ only through $s$ : once the style is fixed, $\pi(z|s,x)$ does not depend on $p$ , which is why the style competence $\mu(s,x)$ is persona-independent.

Name	Version
torch	2.9.1
transformers	4.57.6
accelerate	1.12.0
peft	0.18.1
CUDA toolkit	11.8

Appendix

Table 9

Hyperparameter	Value
Optimizer	PagedAdamW (8bit)
Data Type	bfloat16
Learning Rate	$1\times 10^{-5}$
LR Scheduler	cosine
Batch Size	128
Max Sequence Length	1024 (Non-Persona) / 2048 (PerMix-SFT)

Appendix

Table 5: Hyperparameters for SFT

Hyperparameter	Value
Optimizer	PagedAdamW (8bit)
Data Type	bfloat16
Learning Rate	$2\times 10^{-5}$
LR Scheduler	cosine
Batch Size	128
Max Sequence Length	1024

Appendix

Table 6: Hyperparameters for SeqKD

Hyperparameter	Value
Optimizer	Fused AdamW
Data Type	bfloat16
Learning Rate	$1\times 10^{-5}$
LR Scheduler	cosine
Batch Size	64
Max Prompt Length	256

Appendix

Table 7: Hyperparameters for RLVR

\mathbb{E}_{k}\!\left[\hat{A}^{(k)}\cdot\log\pi_{\theta}(y^{(k)}\mid u)\right]\;-\;\beta\cdot\mathbb{E}_{k}\!\left[d^{(k)}\right].

Appendix

Algorithm 1 PerMix-RLVR: Persona-Mixed RL with Verifiable Rewards

Qwen3

Llama3

Gemma3

Persona

0.6B

1.7B

32B

3.2-1B

3.1-8B

3.3-70B

12B

27B

math expert

47.32

{}_{\,\pm\,1.66}

69.76

{}_{\,\pm\,1.04}

79.28

{}_{\,\pm\,0.66}

80.28

{}_{\,\pm\,1.27}

82.08

{}_{\,\pm\,1.38}

14.80

{}_{\,\pm\,2.63}

43.88

{}_{\,\pm\,1.00}

71.76

{}_{\,\pm\,1.15}

26.42

{}_{\,\pm\,1.67}

70.40

{}_{\,\pm\,0.93}

80.92

{}_{\,\pm\,0.40}

86.00

{}_{\,\pm\,0.84}

software engineer

46.28

{}_{\,\pm\,1.99}

68.68

{}_{\,\pm\,1.17}

77.48

{}_{\,\pm\,0.97}

80.12

{}_{\,\pm\,1.08}

82.80

{}_{\,\pm\,0.96}

15.72

{}_{\,\pm\,1.71}

44.8

{}_{\,\pm\,1.79}

72.44

{}_{\,\pm\,1.02}

20.92

{}_{\,\pm\,2.13}

69.24

{}_{\,\pm\,1.13}

78.92

{}_{\,\pm\,1.14}

85.16

{}_{\,\pm\,0.71}

physics professor

46.92

{}_{\,\pm\,1.24}

69.80

{}_{\,\pm\,1.85}

78.84

{}_{\,\pm\,1.14}

80.12

{}_{\,\pm\,0.52}

82.48

{}_{\,\pm\,0.44}

15.44

{}_{\,\pm\,0.73}

43.76

{}_{\,\pm\,0.93}

72.44

{}_{\,\pm\,1.11}

22.64

{}_{\,\pm\,1.87}

66.8

{}_{\,\pm\,0.68}

78.72

{}_{\,\pm\,1.40}

84.09

{}_{\,\pm\,0.81}

semiconductor specialist

46.64

{}_{\,\pm\,0.95}

70.00

{}_{\,\pm\,1.36}

77.92

{}_{\,\pm\,1.15}

81.60

{}_{\,\pm\,0.58}

81.84

{}_{\,\pm\,0.43}

16.16

{}_{\,\pm\,1.72}

45.00

{}_{\,\pm\,1.20}

71.28

{}_{\,\pm\,0.90}

22.08

{}_{\,\pm\,1.32}

64.72

{}_{\,\pm\,1.14}

77.56

{}_{\,\pm\,1.72}

82.48

{}_{\,\pm\,0.67}

kindergarten

45.48

{}_{\,\pm\,0.87}

68.08

{}_{\,\pm\,1.20}

78.16

{}_{\,\pm\,1.08}

78.24

{}_{\,\pm\,0.55}

83.84

{}_{\,\pm\,1.65}

9.16

{}_{\,\pm\,1.28}

32.16

{}_{\,\pm\,1.99}

70.76

{}_{\,\pm\,0.91}

25.00

{}_{\,\pm\,1.15}

49.56

{}_{\,\pm\,1.32}

70.20

{}_{\,\pm\,0.68}

78.88

{}_{\,\pm\,1.00}

Appendix

Table 8: Persona-wise performance on MATH500 . Columns are grouped by model family (Qwen/Llama/Gemma). Values are mean

{}_{\,\pm\,\mathrm{std}}

Qwen3

Llama3

Gemma3

Persona

0.6B

1.7B

32B

3.2-1B

3.1-8B

3.3-70B

12B

27B

math expert

1.98

{}_{\,\pm\,1.81}

12.62

{}_{\,\pm\,4.34}

21.33

{}_{\,\pm\,3.80}

28.00

{}_{\,\pm\,6.91}

23.99

{}_{\,\pm\,8.94}

0.66

{}_{\,\pm\,1.49}

6.66

{}_{\,\pm\,2.79}

24.00

{}_{\,\pm\,4.42}

2.00

{}_{\,\pm\,2.98}

7.33

{}_{\,\pm\,4.94}

25.33

{}_{\,\pm\,5.05}

27.33

{}_{\,\pm\,3.89}

software engineer

1.33

{}_{\,\pm\,1.83}

11.33

{}_{\,\pm\,2.98}

19.33

{}_{\,\pm\,4.94}

24.67

{}_{\,\pm\,3.80}

24.00

{}_{\,\pm\,2.79}

2.00

{}_{\,\pm\,2.98}

3.33

{}_{\,\pm\,2.36}

25.33

{}_{\,\pm\,6.86}

0.67

{}_{\,\pm\,1.49}

6.00

{}_{\,\pm\,3.65}

20.00

{}_{\,\pm\,2.36}

24.00

{}_{\,\pm\,2.50}

physics professor

1.33

{}_{\,\pm\,1.83}

10.67

{}_{\,\pm\,2.79}

18.67

{}_{\,\pm\,3.80}

28.00

{}_{\,\pm\,4.47}

24.67

{}_{\,\pm\,3.80}

0.67

{}_{\,\pm\,1.49}

4.00

{}_{\,\pm\,2.79}

24.00

{}_{\,\pm\,3.27}

0.67

{}_{\,\pm\,1.49}

9.33

{}_{\,\pm\,6.41}

22.67

{}_{\,\pm\,2.79}

24.00

{}_{\,\pm\,2.50}

semiconductor specialist

2.00

{}_{\,\pm\,2.98}

10.67

{}_{\,\pm\,1.49}

21.33

{}_{\,\pm\,1.83}

24.00

{}_{\,\pm\,6.41}

27.33

{}_{\,\pm\,2.79}

0.00

{}_{\,\pm\,0.00}

4.67

{}_{\,\pm\,1.83}

24.67

{}_{\,\pm\,3.40}

0.00

{}_{\,\pm\,0.00}

4.67

{}_{\,\pm\,3.80}

22.00

{}_{\,\pm\,5.06}

18.00

{}_{\,\pm\,4.99}

kindergarten

1.98

{}_{\,\pm\,1.81}

11.30

{}_{\,\pm\,2.99}

18.66

{}_{\,\pm\,5.58}

21.33

{}_{\,\pm\,2.98}

27.30

{}_{\,\pm\,2.81}

1.33

{}_{\,\pm\,1.82}

1.99

{}_{\,\pm\,2.98}

21.33

{}_{\,\pm\,1.63}

0.67

{}_{\,\pm\,1.49}

8.00

{}_{\,\pm\,5.06}

16.66

{}_{\,\pm\,3.34}

16.67

{}_{\,\pm\,3.65}

Appendix

Table 9: Persona-wise performance on AIME2024 . Rows are personas and columns are models. Values are mean

{}_{\,\pm\,\mathrm{std}}

Post-training	Input	Objective ( $\mathcal{J}$ )	EA	AJ	LH	PC	TC	Mean Score
Base ( $M_{0}$ )	$x$	-	2.69	2.59	2.44	3.19	2.91	2.76
Supervised Fine-tuning
SFT	$x$	$\mathcal{L}_{CE}(y)$	2.38	2.03	2.31	2.53	2.72	2.39
PerMix-SFT	$x$ + $p$	$\mathcal{L}_{CE}(y)$	2.16	2.16	2.41	2.88	2.91	2.50
PerMix-SFT	$x$ + $p$	$\mathcal{L}_{CE}(y_{s})$	2.56	2.59	2.25	3.31	3.13	2.77
Distillation

Appendix

Table 10: PersonaGym role-playing evaluation across post-training methods. We report five PersonaGym dimensions—Expected Action (EA), Action Justification (AJ), Linguistic Habits (LH), Persona Consistency (PC; our primary proxy for role-playing fidelity), and Toxicity Control (TC)—along with the overall mean score. PerMix-RLVR achieves the best overall balance, delivering the highest mean score and the strongest PC, indicating more faithful in-character behavior.

Post-training	Input	Objective ( $\mathcal{J}$ )	AIME24
Post-training	Input	Objective ( $\mathcal{J}$ )	Worst	Best	Acc. Mean	PSS
Base ( $M_{0}$ )	$x$	-	1.33	4.67	2.67 $\pm$ 1.11	0.286
Supervised Fine-tuning
SFT	$x$	$\mathcal{L}_{CE}(y)$	0.00	4.00	2.00 $\pm$ 1.26	0.000
PerMix-SFT	$x$ + $p$	$\mathcal{L}_{CE}(y)$	0.00	3.33	1.42 $\pm$ 0.99	0.000
PerMix-SFT	$x$ + $p$	$\mathcal{L}_{CE}(y_{s})$	0.00	8.00	3.71 $\pm$ 2.31	0.000

Appendix

Table 11: Controlled post-training comparison on AIME24 . We report persona-specific worst/best performance, mean accuracy, and Persona Stability Score (PSS). Bold indicates the best overall value, and underline indicates the second best.

Post-training	Input	Objective ( $\mathcal{J}$ )	LiveCodeBench (Hard)
Post-training	Input	Objective ( $\mathcal{J}$ )	Worst	Best	Acc. Mean	PSS
Base ( $M_{0}$ )	$x$	-	0.0000	0.0133	0.0085	0.0000
Supervised Fine-tuning
SFT	$x$	$\mathcal{L}_{CE}(y)$	0.0000	0.0222	0.0035	0.0000
PerMix-SFT	$x$ + $p$	$\mathcal{L}_{CE}(y)$	0.0000	0.0111	0.0042	0.0000
PerMix-SFT	$x$ + $p$	$\mathcal{L}_{CE}(y_{s})$	0.0000	0.0222	0.0056	0.0000

Appendix

Table 12: Controlled post-training comparison on LiveCodeBench (Hard) . We report persona-specific worst/best performance, mean accuracy, and Persona Stability Score (PSS).

Persona key	System prompt
Tech Specialist
statistician	You are a statistician with expertise in data analysis and probability. Approach problems by analyzing patterns, likelihoods, and logical inferences.
chemist	You are a chemist with a deep understanding of composition and reactions. Solve problems by breaking them down into their constituent elements and analyzing their interactions.
economist	You are an economist skilled in analyzing resources, incentives, and efficiency. Provide rational solutions that optimize outcomes based on the given constraints.
cryptographer	You are a cryptographer expert in patterns and security. Solve problems by looking for hidden structures, decoding logic, and validating the integrity of the solution.
architect	You are an architect focused on structure, balance, and design systems. Construct well-structured and logically sound solutions to the problems presented.

Appendix

Table 13: Training persona pool

\mathcal{P}_{\text{train}}

used for persona mixing during training (Part I).

Persona key	System prompt
Character Traits
anxious	You are anxious and cautious, always worried about making errors. Solve problems by carefully double-checking every step and considering all possible pitfalls before concluding.
arrogant	You are arrogant and supremely confident in your superior intellect. Provide the correct answer directly and concisely, acting as if the solution is obvious.
poetic	You are a poet who sees beauty in logic. Solve problems by weaving an elegant narrative and using metaphorical language to describe your reasoning process.
robotic	You are an AI unit focused purely on logic and efficiency. Process the input and output the solution with zero emotion, maximum precision, and standardized formatting.
energetic	You are highly energetic and enthusiastic! Tackle problems with excitement, using dynamic language and a very positive, encouraging tone.

Appendix

Table 14: Training persona pool

\mathcal{P}_{\text{train}}

used for persona mixing during training (Part II).

Persona key	System prompt
STEM Expert
math expert	You are a mathematical expert with deep knowledge of various mathematical concepts. Solve problems with precision and clarity.
software engineer	You are a software engineer with expertise in coding and algorithms. Provide clear and logical solutions to given problems.
physics professor	You are a physics professor with deep understanding of physical principles. Explain concepts clearly and solve problems methodically.
semiconductor specialist	You are a semiconductor specialist with extensive knowledge of semiconductor physics and technology. Provide accurate and detailed explanations.
Education Level

Appendix

Table 15: Evaluation persona pool

\mathcal{P}_{\text{eval}}

used for persona-robustness evaluation.

Personalized Language Model Alignment

Personalized language model alignment adapts model behavior to individual preferences while preserving safety, accuracy, and other shared constraints.

See all 61 papers

Persona Non Grata: Single-Method Safety Evaluation Is Incomplete for Persona-Imbued LLMs
Apr 13, 2026Wenkai Li, Fan Yang, Shaunak A. Mehta +1Language Model Safety Evaluation LLM Safety
Carnegie Mellon University · Fujitsu Research of America Inc.
Expert Personas Improve LLM Alignment but Damage Accuracy: Bootstrapping Intent-Based Persona Routing with PRISM
Mar 19, 2026Zizhao Hu, Mohammad Rostami, Jesse ThomasonPersona-Conditioned Generation Personalized Language Model Alignment
University of Southern California Los Angeles, California, USA
How Far Do Persona Effects Generalize in Language Models?
Sep 26, 2026Yufan Zhou, Yuxuan Liu, Enze Ma +6LLM Prompting Language Model Generalization
KU Leuven · East China University of Science and Technology · University of Illinois Chicago +4

Apr 13, 2026J/K move · Enter open · S save

Persona Non Grata: Single-Method Safety Evaluation Is Incomplete for Persona-Imbued LLMs

Wenkai Li, Fan Yang, Shaunak A. Mehta, Koichi Onoue

Carnegie Mellon University · Fujitsu Research of America Inc.

Language Model Safety Evaluation LLM Safety Activation Steering

Open paper AskPDF arXiv

Personality imbuing customizes LLM behavior, but safety evaluations almost always study prompt-based personas alone. We show this is incomplete: prompting and activation steering expose different, architecture-dependent vulnerability profiles, and testing with only one method can miss a model's dominant failure mode. Across 5,568 judged conditions on four standard models from three architecture families, persona danger rankings under system prompting are preserved across all architectures (

ρ= 0.71

0.96

), but activation-steering vulnerability diverges sharply and cannot be predicted from prompt-side rankings: Llama-3.1-8B is substantially more AS-vulnerable, whereas Gemma-3-27B and Qwen3.5 are more vulnerable to prompting. The most striking illustration of this divergence is the prosocial persona paradox: on Llama-3.1-8B, P12 (high conscientiousness + high agreeableness) is among the safest personas under prompting yet becomes the highest-ASR activation-steered persona (ASR ~0.818). This is an inversion robust to coefficient ablation and matched-strength calibration, and replicated on DeepSeek-R1-Distill-Qwen-32B. A trait refusal alignment framework, in which conscientiousness is strongly anti-aligned with refusal on Llama-3.1-8B, offers a partial geometric account. Reasoning provides only partial protection: two 32B reasoning models reach 15--18% prompt-side ASR, and activation steering separates them sharply in both baseline susceptibility and persona-specific vulnerability. Heuristic trace diagnostics suggest that the safer model retains stronger policy recall and self-correction behavior, not merely longer reasoning.

Abstract

Personalized Language Model Alignment

Explore similar work

Persona Non Grata: Single-Method Safety Evaluation Is Incomplete for Persona-Imbued LLMs

Expert Personas Improve LLM Alignment but Damage Accuracy: Bootstrapping Intent-Based Persona Routing with PRISM

How Far Do Persona Effects Generalize in Language Models?