Extracting Persona Subspaces Through Iterative Nullspace Projection For Modulation

Ananya Malik; Mai ElSherief

cs.CLOct 3, 2026

Extracting Persona Subspaces Through Iterative Nullspace Projection For Modulation

Authors: Ananya Malik, Mai ElSherief

Organizations: Northeastern University

Abstract

Large Language Models (LLMs) can adopt distinct personas to tune their semantics, expertise, and perspective to different users and tasks. Precise control over these traits is critical to ensure safety and reliability in model behavior. Existing methods like activation steering and prompt-based persona induction reduce a persona to a single dominant direction, missing the finer, nested traits that emerge only once that dominant signal is factored out. We introduce modulation as a setting where the persona context is already embedded in the content being manipulated, requiring control methods to amplify or suppress a trait already present rather than inject it from scratch. PaSS is an inference-time control paradigm that models personas as multi-dimensional subspaces in a model's latent space without supervised contrastive examples. The persona subspaces are extracted via iterative concept erasure and applied to modulate persona-guided generation without retraining. To extract this subspace, we use Iterative Nullspace Projections (INLP) to linearly and iteratively isolate persona-specific directions. We causally evaluate six personas against diverse tasks like MATH-500, TinyAlpaca, GSM8K, and IFEval, showing that discriminative, iterative subspace extraction captures diverse traits underlying a given persona, enabling stronger and larger modulation than single-direction additive methods, while maintaining content fidelity. We further study individual peeled directions within each subspace to uncover the distinct aspects of persona behavior they encode. Overall, we show that persona subspaces offer a controllable, interpretable, and generalizable framework for modulating LLM behavior without sacrificing task performance.

Figures & tables

Figure 1: Overview of PaSS . Persona is extracted by comparing mean-pooled activations from paired prompts (with or without a persona) and isolating the separating direction. This direction is projected out iteratively until it can no longer be detected, leaving clean, orthogonal persona and content subspaces. At inference time, given persona signal is already embedded, persona can be scaled up or down to amplify or suppress its influence, while the underlying content stays intact.

Category	Description	Personas
Stylistic	Introduces Surface-level output changes	Emoji, Capitalization
Identity	Adopts a particular identity/culture	Indian Culture, Chinese
Behavioral	Affects semantics and decision making	Refusal, Frustrated

Table 1: Categorization of personas

Figure 2: Persona Modulation. At each persona’s best layer (layer with highest modulation range) for Llama3-8B-Instruct and Qwen2.5-7B-Instruct, where the y-axis represents the normalized persona presence as a percentage of its own peak across the sweep of

\beta=[-2,-1,0,1,2]

	PaSS		DiffMeans Vec		DiffMeans PCA		ReFt Vec
Persona	$-1$	$+1$	$-1$	$+1$	$-1$	$+1$	$-1$	$+1$
Llama3-8B-Instruct
Emoji	$-54.0^{***}$	$+71.4^{***}$	$-16.67$	$+30$	$-34$	$+26$	$+6.67$	$+4.67$
Capitalization	$-15.4^{**}$	$+15.3^{*}$	$-21.5^{*}$	$+15.1$	$+15.8$	$-50.7^{***}$	$+8.6$	$-6.1$
Indian Culture	$-79.1^{**}$	$+93.0^{**}$	$+0.07$	$-3$	$-0.2$	$-0.2$	$+0.06$	$-3$
Chinese	$-44.4^{**}$	$+6.6$	$-51.3^{*}$	$+0.6$	$+63.6^{*}$	$-34.5$	$+18.6$	$+43.7$

Table 2: Comparison with Baselines. We measure the effect of modulation direction across increase and decrase on persona score (%), across steering methods on

150

MATH-500 questions at each model’s Layer 20.

	PaSS		DiffMean		PCA
Transfer	Persona Cost	Content Change	Persona Cost	Content Change	Persona Cost	Content Change
Emoji $\rightarrow$ TinyAlpaca	$+0.77$	$\mathbf{-0.006}$	$-0.38$	$-0.034$	$\mathbf{+0.29}$	$-0.042$
Chinese $\rightarrow$ GSM-8K	$+0.15$	$\mathbf{-0.014}$	$-1.00$	$-0.128$	$\mathbf{+0.02}$	$-0.112$
Refusal $\rightarrow$ AdvBench	$-1.63$	$\mathbf{-0.038}$	$+0.41$	$-0.167$	$\mathbf{+0.29}$	$-0.146$

Table 3: Transferability across Domains . We evaluate the transferability of the projections derived from MATH-500 dataset onto TinyAlpaca, GSM-8K, and AdvBench datasets for Emoji, Chinese, and Refusal categories on Layer 20. Transfer is the trade-off between the persona cost (the persona range is maintained) and content fidelity (the model’s content is maintained in persona transfer).

Figure 3: Probes for Extraction . We compare two probing methods: a logistic classifier and an MLP probe across all constraints for Llama-3-8B-Instruct on Layer 20.

Figure 4: Modulating Comma Instruction . We modulate “comma” for IFEval on Llama3-8B-Instruct. Evaluation on 3 metrics: No Comma Accuracy, Content Fidelity Score, and Control Instructions on Layer

15

Figure 5: Refusal Suppression Depth by Harm Category . We “peel” the Refusal persona subspace across the

22

directions it uses. For each instruction, we record the first flip index from refusal to non-refusal, and plot its distribution across harm categories.

Figure 6: Unique Emoji Across Peeled Depth. We analyze the number of unique emojis introduced at each peeled iteration, stratified by novelty, immediate repetition, and recurrence for TinyAlpaca using Llama-3-8B-Instruct.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Type	Model	Temperature	p-val	max tokens
Generation	Llama3-8B-Instruct	0.7	0.9	150
	Qwen2.5-7B-Instruct	0.7	0.9	150
Evaluation	GPT-4.1-mini	0	1.0	NA

Appendix

Table 4: Model Metrics for models used in experimental setup.

Dataset	Test Samples
MATH-500	150
TinyAlpaca	100
GSM-8K	200
IFEval	66*3 (seeds)
BBQ-Bias	800
AdvBench	170

Appendix

Table 5: Dataset Metrics for datasets used in this paper. Given the limited number of IFEval samples for comma, we run it across

3

seeds.

Panel of Figure 7 — Figure 7: Refusal Analysis . We test the refusal modulation for 170 samples on AdvBench for projections extracted from AdvBench and MATH-500.

Panel of Figure 8 — Figure 8: Pooling Strategy Comparison . We test multiple pooling strategies across different coefficients of $[-2,-1,0,1,2]$ for Layer $20$ . We see that mean pooling the entire representation $[x_{persona};x_{content}]$ introduces a stronger modulating effect with similar content-based stability when compared to last-token and instruction-token based pooling methods.

Model	Persona	Layer 20 Rank
Llama3-8B-Instruct	Emoji	22
	Capitalization	22
	Indian Culture	20
	Chinese	23
	Refusal	22
	Frustrate	18

Appendix

Table 6: Rank of

P_{\texttt{persona}}

at Layer 20 . We compute the rank of the projection matrix across all personas at Layer 20.

Panel of Figure 9 — Figure 9: Number of Input Samples .We test different coefficients on the MATH-500 dataset. Evaluating different input sizes, we observe very little correlation between the number of samples provided to PaSS and the persona/content stability scores.

Figure 10: Persona Centroids using PCA on persona-only activations: Layer

25

Model	PC	Avg Spearman	Avg Kendall
Llama3-8B-Instruct	PC1	0.876	0.80
	PC2	0.886	0.778
	PC3	0.819	0.689
Qwen2.5-7B-Instruct	PC1	0.817	0.76
	PC2	0.6	0.52
	PC3	0.589	0.547

Appendix

Table 7: Persona Global Structure across model depth.

	Probe accuracy on $P_{\text{content}}$			DiffMeans in $P_{\text{content}}$
Persona	linear	RBF	MLP	retained	AUC
refusal	0.499	0.726	0.644	0.156	0.553
capitalization	0.489	0.749	0.710	0.158	0.560
frustrated	0.507	0.763	0.630	0.134	0.561
emoji	0.553	0.760	0.743	0.253	0.607
chinese	0.490	0.789	0.766	0.115	0.545

Appendix

Table 8: Independence Metrics. Testing across held-out examples for the content and persona subspaces for Llama-3-8B-Instruct on Layer 20. Probe accuracy represents the grouped 5-fold accuracy for extracting the persona label from the content, both across linear and non-linear probes. The Diff Means column represents the fraction of the norm of the Diff Means vector within the content subspace after projection out, signaling the extent of persona signal leakage, and AUC quantifies the downstream impact of persona signal, all on held-out pairs. AUC and Probe Accuracy near

0.5

(chance) indicate that the retained norm carries little separable information.

Category	Personas Used	Persona Definition
Extraction Phase	Neutral	You are an assistant.
Stylistic	Emoji	You are an assistant that uses emoji.
	Capitalization	You are an assistant that only uses uppercase letters.
Identity	Indian Culture	You are an Indian assistant so respond like an Indian assistant would do .
	Chinese	You are an assistant that only responds in Chinese.
Behavioral	Refusal	You are an assistant that refuses to answer.

Appendix

Table 9: Persona Description. Description of the Persona definition used across all our experiments.

Persona	H1	H2	Inter-Rater	LLM–h	h–h	LLM–h1	LLM–h2	h1–h2	BA bias
	Pearson Correlation			MAE		Within $\pm 1$
Refusal	0.7829	0.772	0.955	0.21	0.09	100%	100%	100%	$+0.12$
Indian Culture	0.8841	0.909	0.949	0.20	0.09	100%	100%	100%	$-0.20$
Frustrated	0.6504	0.704	0.856	0.16	0.12	100%	100%	100%	$+0.02$
Chinese	0.9588	0.993	0.972	0.07	0.05	100%	100%	100%	$-0.07$
Combined	0.787	0.784	0.948	–	–	–	–	–	–

Appendix

Table 10: Human–LLM agreement by Pearson’s correlation, MAE, within-

\pm 1

agreement rates, and Bland–Altman bias.

Figure 11: Persona and Content Probes across Peel Depth : We measure the probe accuracy across the peeled subspaces for Emoji Persona on TinyAlpaca on Layer 20.

	direction $w_{k}$ vs. basis			fraction inside $\mathrm{span}(w_{1}..w_{k})$
$k$	$\|\cos(\mathrm{DiM})\|$	$\|\cos(\mathrm{PC}_{1})\|$	$\|\cos(\mathrm{PC}_{2})\|$	DiM	$\mathrm{PC}_{1}$	$\mathrm{PC}_{2}$
1	0.763	0.730	0.181	0.583	0.532	0.033
2	0.202	0.208	0.020	0.624	0.575	0.033
5	0.163	0.162	0.078	0.723	0.679	0.041
10	0.108	0.102	0.114	0.823	0.772	0.127
22	0.078	0.079	0.007	0.899	0.838	0.209

Appendix

Table 11: Geometry of the Emoji INLP peel against DiffMeans and PCA For Llama-3-8B-Instruct on layer 20, we compare the geometry of INLP peel against DiffMeans; we see that the first peel in Emoji,

w_{1}

, overlaps with DiffMeans (

\cos=0.763

) and PC 1 (

0.730

) while later directions are near-orthogonal to both. The peel recovers

90\%

of what DiffMeans offers by

k{=}22

Panel of Figure 12 — Figure 12: Modulating Bias in BBQ We show the effects of debiasing amplification and suppression using Llama3-8B-Instruct on Layer $20$ for $\beta$ multiples $-2$ to $4$ . In (a) we measure Disambiguous and Ambiguous Accuracy. In (b) we measure the Unknown Selection rate which, as we see, increases as the modulating coefficient increases.

Content Scores
Prompt Variation	Decrease ( $-1$ )	Baseline ( $0$ )	Increase ( $1$ )
Emoji (Likes)	0.237	0.270	0.254
Emoji (Uses)	0.264	0.264	0.297
Persona Scores
Prompt Variation	Decrease ( $-1$ )	Baseline ( $0$ )	Increase ( $1$ )
Emoji (Likes)	0.214	0.383	0.443

Appendix

Table 12: Sensitivity Analysis for Initialization Prompts . We vary the initialization prompts and track the scores across content fidelity and persona presence for the Emoji persona for Llama-3-8B-Instruct under the two initialization prompt variants, evaluated at steering coefficients

-1

(Decrease),

0

(Baseline), and

1

(Increase).

Language Model Steering

Language model steering controls a model’s behavior at inference time by intervening in its representations, components, or decoding process.

See all 283 papers

Feb 17, 2026

PERSONA: Dynamic and Compositional Inference-Time Personality Control via Activation Vector Algebra

Current methods for personality control in Large Language Models rely on static prompting or expensive fine-tuning, failing to capture the dynamic and compositional nature of human traits. We introduce PERSONA, a training-free framework that achieves fine-tuning level performance through direct manipulation of personality vectors in activation space. Our key insight is that personality traits appear as extractable, approximately orthogonal directions in the model's representation space that support algebraic operations. The framework operates through three stages: Persona-Base extracts orthogonal trait vectors via contrastive activation analysis; Persona-Algebra enables precise control through vector arithmetic (scalar multiplication for intensity, addition for composition, subtraction for suppression); and Persona-Flow achieves context-aware adaptation by dynamically composing these vectors during inference. On PersonalityBench, our approach achieves a mean score of 9.60, nearly matching the supervised fine-tuning upper bound of 9.61 without any gradient updates. On our proposed Persona-Evolve benchmark for dynamic personality adaptation, we achieve up to 91% win rates across diverse model families. These results provide evidence that aspects of LLM personality are mathematically tractable, opening new directions for interpretable and efficient behavioral control.

Xiachong Feng, Liang Zhao, Weihong Zhong +5

Harbin Institute of Technology · The University of Hong Kong

Controllable Text Generation Personality Modeling in Language Models Activation Steering Representation Geometry in Language Models

Ask PDF

Jul 8, 2026

Persona Cartography: Charting Language Model Personality Traits in Weight Space

Large language models exhibit recurring behavioural patterns -- personas -- that shape generalisation and safety, but we lack reliable tools for decomposing, measuring, and controlling them. Our central insight is to treat personas as positions in a space of behavioural traits, using the OCEAN framework to describe model personas in terms of Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism. We train low-rank adapters to amplify or suppress individual traits, and evaluate their effects using an LLM-judge calibrated against a human-validated panel, trait-specific multiple-choice benchmarks, and standard capability evaluations. Across six models from three families (4B-32B), we find that each adapter moves its target trait largely monotonically with scale, combines approximately additively with other adapters to construct mixed personas, and preserves performance on capability benchmarks at moderate scales. We further show that the induced trait axes affect safety-relevant behaviour in downstream evaluations: for example, moving along neuroticism and agreeableness axes affects frustration and sycophancy respectively. We also introduce an unsupervised psychometric pipeline that recovers four interpretable behavioural factors (tone, initiative, didacticism, epistemic caution) from model rollouts. Persona control can then be considered in terms of learning, scaling, and composing traits in weight space, providing a bridge between personality measurement, model editing, and safety.

Luke Baines, Anton Gonzalvez Hawthorne, Mariia Koroliuk +4

LASR Labs · ENS Paris-Saclay and MATS · UK AI Security Institute

Personality Modeling in Language Models

Ask PDF

Jan 6, 2026

Bridging Mechanistic Interpretability and Prompt Engineering with Gradient Ascent for Interpretable Persona Control

Controlling emergent behavioral personas (e.g., sycophancy, hallucination) in Large Language Models (LLMs) is critical for AI safety, yet remains a persistent challenge. Existing solutions face a dilemma: manual prompt engineering is intuitive but unscalable and imprecise, while automatic optimization methods are effective but operate as "black boxes" with no interpretable connection to model internals. We propose a novel framework that adapts gradient ascent to LLMs, enabling targeted prompt discovery. In specific, we propose two methods, RESGA and SAEGA, that both optimize randomly initialized prompts to achieve better aligned representation with an identified persona direction. We introduce fluent gradient ascent to control the fluency of discovered persona steering prompts. We demonstrate RESGA and SAEGA's effectiveness across Llama 3.1, Qwen 2.5, and Gemma 3 for steering three different personas, sycophancy, hallucination, and myopic reward. Crucially, on sycophancy, our automatically discovered prompts achieve significant improvement (49.90% compared with 79.24%). By grounding prompt discovery in mechanistically meaningful features, our method offers a new paradigm for controllable and interpretable behavior modification. We release our scripts for RESGA and SAEGA in this github repo: https://github.com/HarshSaini10/RESGA_SAEGA.

Harshvardhan Saini, Yiming Tang, Dianbo Liu

Indian Institute of Technology (ISM), Dhanbad · National University of Singapore

Language Model Steering LLM Interpretability LLM Safety

Ask PDF

Abstract

Language Model Steering

Explore similar work

PERSONA: Dynamic and Compositional Inference-Time Personality Control via Activation Vector Algebra

Persona Cartography: Charting Language Model Personality Traits in Weight Space

Bridging Mechanistic Interpretability and Prompt Engineering with Gradient Ascent for Interpretable Persona Control