Extracting Persona Subspaces Through Iterative Nullspace Projection For Modulation

Ananya Malik; Mai ElSherief

cs.CLOct 3, 2026

Extracting Persona Subspaces Through Iterative Nullspace Projection For Modulation

Authors: Ananya Malik, Mai ElSherief

Organizations: Northeastern University

Abstract

Large Language Models (LLMs) can adopt distinct personas to tune their semantics, expertise, and perspective to different users and tasks. Precise control over these traits is critical to ensure safety and reliability in model behavior. Existing methods like activation steering and prompt-based persona induction reduce a persona to a single dominant direction, missing the finer, nested traits that emerge only once that dominant signal is factored out. We introduce modulation as a setting where the persona context is already embedded in the content being manipulated, requiring control methods to amplify or suppress a trait already present rather than inject it from scratch. PaSS is an inference-time control paradigm that models personas as multi-dimensional subspaces in a model's latent space without supervised contrastive examples. The persona subspaces are extracted via iterative concept erasure and applied to modulate persona-guided generation without retraining. To extract this subspace, we use Iterative Nullspace Projections (INLP) to linearly and iteratively isolate persona-specific directions. We causally evaluate six personas against diverse tasks like MATH-500, TinyAlpaca, GSM8K, and IFEval, showing that discriminative, iterative subspace extraction captures diverse traits underlying a given persona, enabling stronger and larger modulation than single-direction additive methods, while maintaining content fidelity. We further study individual peeled directions within each subspace to uncover the distinct aspects of persona behavior they encode. Overall, we show that persona subspaces offer a controllable, interpretable, and generalizable framework for modulating LLM behavior without sacrificing task performance.

Figures & tables

Figure 1: Overview of PaSS . Persona is extracted by comparing mean-pooled activations from paired prompts (with or without a persona) and isolating the separating direction. This direction is projected out iteratively until it can no longer be detected, leaving clean, orthogonal persona and content subspaces. At inference time, given persona signal is already embedded, persona can be scaled up or down to amplify or suppress its influence, while the underlying content stays intact.

Category	Description	Personas
Stylistic	Introduces Surface-level output changes	Emoji, Capitalization
Identity	Adopts a particular identity/culture	Indian Culture, Chinese
Behavioral	Affects semantics and decision making	Refusal, Frustrated

Table 1: Categorization of personas

Figure 2: Persona Modulation. At each persona’s best layer (layer with highest modulation range) for Llama3-8B-Instruct and Qwen2.5-7B-Instruct, where the y-axis represents the normalized persona presence as a percentage of its own peak across the sweep of

\beta=[-2,-1,0,1,2]

	PaSS		DiffMeans Vec		DiffMeans PCA		ReFt Vec
Persona	$-1$	$+1$	$-1$	$+1$	$-1$	$+1$	$-1$	$+1$
Llama3-8B-Instruct
Emoji	$-54.0^{***}$	$+71.4^{***}$	$-16.67$	$+30$	$-34$	$+26$	$+6.67$	$+4.67$
Capitalization	$-15.4^{**}$	$+15.3^{*}$	$-21.5^{*}$	$+15.1$	$+15.8$	$-50.7^{***}$	$+8.6$	$-6.1$
Indian Culture	$-79.1^{**}$	$+93.0^{**}$	$+0.07$	$-3$	$-0.2$	$-0.2$	$+0.06$	$-3$
Chinese	$-44.4^{**}$	$+6.6$	$-51.3^{*}$	$+0.6$	$+63.6^{*}$	$-34.5$	$+18.6$	$+43.7$

Table 2: Comparison with Baselines. We measure the effect of modulation direction across increase and decrase on persona score (%), across steering methods on

150

MATH-500 questions at each model’s Layer 20.

	PaSS		DiffMean		PCA
Transfer	Persona Cost	Content Change	Persona Cost	Content Change	Persona Cost	Content Change
Emoji $\rightarrow$ TinyAlpaca	$+0.77$	$\mathbf{-0.006}$	$-0.38$	$-0.034$	$\mathbf{+0.29}$	$-0.042$
Chinese $\rightarrow$ GSM-8K	$+0.15$	$\mathbf{-0.014}$	$-1.00$	$-0.128$	$\mathbf{+0.02}$	$-0.112$
Refusal $\rightarrow$ AdvBench	$-1.63$	$\mathbf{-0.038}$	$+0.41$	$-0.167$	$\mathbf{+0.29}$	$-0.146$

Table 3: Transferability across Domains . We evaluate the transferability of the projections derived from MATH-500 dataset onto TinyAlpaca, GSM-8K, and AdvBench datasets for Emoji, Chinese, and Refusal categories on Layer 20. Transfer is the trade-off between the persona cost (the persona range is maintained) and content fidelity (the model’s content is maintained in persona transfer).

Figure 3: Probes for Extraction . We compare two probing methods: a logistic classifier and an MLP probe across all constraints for Llama-3-8B-Instruct on Layer 20.

Figure 4: Modulating Comma Instruction . We modulate “comma” for IFEval on Llama3-8B-Instruct. Evaluation on 3 metrics: No Comma Accuracy, Content Fidelity Score, and Control Instructions on Layer

15

Figure 5: Refusal Suppression Depth by Harm Category . We “peel” the Refusal persona subspace across the

22

directions it uses. For each instruction, we record the first flip index from refusal to non-refusal, and plot its distribution across harm categories.

Figure 6: Unique Emoji Across Peeled Depth. We analyze the number of unique emojis introduced at each peeled iteration, stratified by novelty, immediate repetition, and recurrence for TinyAlpaca using Llama-3-8B-Instruct.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Type	Model	Temperature	p-val	max tokens
Generation	Llama3-8B-Instruct	0.7	0.9	150
	Qwen2.5-7B-Instruct	0.7	0.9	150
Evaluation	GPT-4.1-mini	0	1.0	NA

Appendix

Table 4: Model Metrics for models used in experimental setup.

Dataset	Test Samples
MATH-500	150
TinyAlpaca	100
GSM-8K	200
IFEval	66*3 (seeds)
BBQ-Bias	800
AdvBench	170

Appendix

Table 5: Dataset Metrics for datasets used in this paper. Given the limited number of IFEval samples for comma, we run it across

3

seeds.

Panel of Figure 7 — Figure 7: Refusal Analysis . We test the refusal modulation for 170 samples on AdvBench for projections extracted from AdvBench and MATH-500.

Panel of Figure 8 — Figure 8: Pooling Strategy Comparison . We test multiple pooling strategies across different coefficients of $[-2,-1,0,1,2]$ for Layer $20$ . We see that mean pooling the entire representation $[x_{persona};x_{content}]$ introduces a stronger modulating effect with similar content-based stability when compared to last-token and instruction-token based pooling methods.

Model	Persona	Layer 20 Rank
Llama3-8B-Instruct	Emoji	22
	Capitalization	22
	Indian Culture	20
	Chinese	23
	Refusal	22
	Frustrate	18

Appendix

Table 6: Rank of

P_{\texttt{persona}}

at Layer 20 . We compute the rank of the projection matrix across all personas at Layer 20.

Panel of Figure 9 — Figure 9: Number of Input Samples .We test different coefficients on the MATH-500 dataset. Evaluating different input sizes, we observe very little correlation between the number of samples provided to PaSS and the persona/content stability scores.

Figure 10: Persona Centroids using PCA on persona-only activations: Layer

25

Model	PC	Avg Spearman	Avg Kendall
Llama3-8B-Instruct	PC1	0.876	0.80
	PC2	0.886	0.778
	PC3	0.819	0.689
Qwen2.5-7B-Instruct	PC1	0.817	0.76
	PC2	0.6	0.52
	PC3	0.589	0.547

Appendix

Table 7: Persona Global Structure across model depth.

	Probe accuracy on $P_{\text{content}}$			DiffMeans in $P_{\text{content}}$
Persona	linear	RBF	MLP	retained	AUC
refusal	0.499	0.726	0.644	0.156	0.553
capitalization	0.489	0.749	0.710	0.158	0.560
frustrated	0.507	0.763	0.630	0.134	0.561
emoji	0.553	0.760	0.743	0.253	0.607
chinese	0.490	0.789	0.766	0.115	0.545

Appendix

Table 8: Independence Metrics. Testing across held-out examples for the content and persona subspaces for Llama-3-8B-Instruct on Layer 20. Probe accuracy represents the grouped 5-fold accuracy for extracting the persona label from the content, both across linear and non-linear probes. The Diff Means column represents the fraction of the norm of the Diff Means vector within the content subspace after projection out, signaling the extent of persona signal leakage, and AUC quantifies the downstream impact of persona signal, all on held-out pairs. AUC and Probe Accuracy near

0.5

(chance) indicate that the retained norm carries little separable information.

Category	Personas Used	Persona Definition
Extraction Phase	Neutral	You are an assistant.
Stylistic	Emoji	You are an assistant that uses emoji.
	Capitalization	You are an assistant that only uses uppercase letters.
Identity	Indian Culture	You are an Indian assistant so respond like an Indian assistant would do .
	Chinese	You are an assistant that only responds in Chinese.
Behavioral	Refusal	You are an assistant that refuses to answer.

Appendix

Table 9: Persona Description. Description of the Persona definition used across all our experiments.

Persona	H1	H2	Inter-Rater	LLM–h	h–h	LLM–h1	LLM–h2	h1–h2	BA bias
	Pearson Correlation			MAE		Within $\pm 1$
Refusal	0.7829	0.772	0.955	0.21	0.09	100%	100%	100%	$+0.12$
Indian Culture	0.8841	0.909	0.949	0.20	0.09	100%	100%	100%	$-0.20$
Frustrated	0.6504	0.704	0.856	0.16	0.12	100%	100%	100%	$+0.02$
Chinese	0.9588	0.993	0.972	0.07	0.05	100%	100%	100%	$-0.07$
Combined	0.787	0.784	0.948	–	–	–	–	–	–

Appendix

Table 10: Human–LLM agreement by Pearson’s correlation, MAE, within-

\pm 1

agreement rates, and Bland–Altman bias.

Figure 11: Persona and Content Probes across Peel Depth : We measure the probe accuracy across the peeled subspaces for Emoji Persona on TinyAlpaca on Layer 20.

	direction $w_{k}$ vs. basis			fraction inside $\mathrm{span}(w_{1}..w_{k})$
$k$	$\|\cos(\mathrm{DiM})\|$	$\|\cos(\mathrm{PC}_{1})\|$	$\|\cos(\mathrm{PC}_{2})\|$	DiM	$\mathrm{PC}_{1}$	$\mathrm{PC}_{2}$
1	0.763	0.730	0.181	0.583	0.532	0.033
2	0.202	0.208	0.020	0.624	0.575	0.033
5	0.163	0.162	0.078	0.723	0.679	0.041
10	0.108	0.102	0.114	0.823	0.772	0.127
22	0.078	0.079	0.007	0.899	0.838	0.209

Appendix

Table 11: Geometry of the Emoji INLP peel against DiffMeans and PCA For Llama-3-8B-Instruct on layer 20, we compare the geometry of INLP peel against DiffMeans; we see that the first peel in Emoji,

w_{1}

, overlaps with DiffMeans (

\cos=0.763

) and PC 1 (

0.730

) while later directions are near-orthogonal to both. The peel recovers

90\%

of what DiffMeans offers by

k{=}22

Panel of Figure 12 — Figure 12: Modulating Bias in BBQ We show the effects of debiasing amplification and suppression using Llama3-8B-Instruct on Layer $20$ for $\beta$ multiples $-2$ to $4$ . In (a) we measure Disambiguous and Ambiguous Accuracy. In (b) we measure the Unknown Selection rate which, as we see, increases as the modulating coefficient increases.

Content Scores
Prompt Variation	Decrease ( $-1$ )	Baseline ( $0$ )	Increase ( $1$ )
Emoji (Likes)	0.237	0.270	0.254
Emoji (Uses)	0.264	0.264	0.297
Persona Scores
Prompt Variation	Decrease ( $-1$ )	Baseline ( $0$ )	Increase ( $1$ )
Emoji (Likes)	0.214	0.383	0.443

Appendix

Table 12: Sensitivity Analysis for Initialization Prompts . We vary the initialization prompts and track the scores across content fidelity and persona presence for the Emoji persona for Llama-3-8B-Instruct under the two initialization prompt variants, evaluated at steering coefficients

-1

(Decrease),

0

(Baseline), and

1

(Increase).

Language Model Steering

Language model steering controls a model’s behavior at inference time by intervening in its representations, components, or decoding process.

See all 283 papers

PERSONA: Dynamic and Compositional Inference-Time Personality Control via Activation Vector Algebra
Feb 17, 2026Xiachong Feng, Liang Zhao, Weihong Zhong +5Controllable Text Generation Personality Modeling in Language Models
Harbin Institute of Technology · The University of Hong Kong
Persona Cartography: Charting Language Model Personality Traits in Weight Space
Jul 8, 2026Luke Baines, Anton Gonzalvez Hawthorne, Mariia Koroliuk +4Personality Modeling in Language Models
LASR Labs · ENS Paris-Saclay and MATS · UK AI Security Institute
Bridging Mechanistic Interpretability and Prompt Engineering with Gradient Ascent for Interpretable Persona Control
Jan 6, 2026Harshvardhan Saini, Yiming Tang, Dianbo LiuLanguage Model Steering LLM Interpretability
Indian Institute of Technology (ISM), Dhanbad · National University of Singapore

Feb 17, 2026J/K move · Enter open · S save

PERSONA: Dynamic and Compositional Inference-Time Personality Control via Activation Vector Algebra

Xiachong Feng, Liang Zhao, Weihong Zhong, Yichong Huang+4

Harbin Institute of Technology · The University of Hong Kong

Controllable Text Generation Personality Modeling in Language Models Activation Steering Representation Geometry in Language Models

Open paper AskPDF arXiv

Current methods for personality control in Large Language Models rely on static prompting or expensive fine-tuning, failing to capture the dynamic and compositional nature of human traits. We introduce PERSONA, a training-free framework that achieves fine-tuning level performance through direct manipulation of personality vectors in activation space. Our key insight is that personality traits appear as extractable, approximately orthogonal directions in the model's representation space that support algebraic operations. The framework operates through three stages: Persona-Base extracts orthogonal trait vectors via contrastive activation analysis; Persona-Algebra enables precise control through vector arithmetic (scalar multiplication for intensity, addition for composition, subtraction for suppression); and Persona-Flow achieves context-aware adaptation by dynamically composing these vectors during inference. On PersonalityBench, our approach achieves a mean score of 9.60, nearly matching the supervised fine-tuning upper bound of 9.61 without any gradient updates. On our proposed Persona-Evolve benchmark for dynamic personality adaptation, we achieve up to 91% win rates across diverse model families. These results provide evidence that aspects of LLM personality are mathematically tractable, opening new directions for interpretable and efficient behavioral control.

Abstract

Language Model Steering

Explore similar work

PERSONA: Dynamic and Compositional Inference-Time Personality Control via Activation Vector Algebra

Persona Cartography: Charting Language Model Personality Traits in Weight Space

Bridging Mechanistic Interpretability and Prompt Engineering with Gradient Ascent for Interpretable Persona Control