Can Activation Steering Capture Multidimensional Authorship Style?

Hieu Tran; Calvin Bao; Marine Carpuat

cs.CLSep 4, 2026

Can Activation Steering Capture Multidimensional Authorship Style?

Authors: Hieu Tran, Calvin Bao, Marine Carpuat

Organizations: University of Maryland, College Park

Abstract

Activation steering has shown promise for controlling LLM generation along well-defined attributes, but it remains unclear whether it can handle the multidimensional and hard-to-define nature of authorship style. We ask whether structured contrastive prompting along rhetorically-motivated dimensions can construct rich style representations directly in activation space, bypassing the need for natural language style descriptors or dedicated training. We find that the resulting directions share a common authorship backbone while conflicting on aspect-specific residuals that carry genuine stylistic signal, explaining why naive aggregation fails. We operationalize this in Aspect-Aware Activation Steering (A3S), a training-free framework that merges per-aspect contrastive directions with interference-aware aggregation and tunes steering strength per instance. A3S improves authorship style transfer where it is genuinely multi-aspect, outperforms a trained baseline in preference evaluations on out-of-domain benchmarks, and keeps target-exemplar overlap consistently low.

Figures & tables

Figure 1: Overview of the proposed Aspect-Aware Activation Steering ( A3S ) framework. A3S decomposes the target exemplar into distinct rhetorical dimensions (e.g., Tone, Structure) via contrastive generation. These aspect vectors are synthesized using PCB-Merging to resolve interference, and injected with an intensity $\alpha^{*}$ dynamically tuned by Hybrid Adaptive Search.

MUD (Reddit)
	LUAR $\uparrow$		StyleCAV $\uparrow$		StyleDistance $\uparrow$		SBERT $\uparrow$		ROUGE-L $\downarrow$		Length
Method	Llama	Qwen	Llama	Qwen	Llama	Qwen	Llama	Qwen	Llama	Qwen	Llama	Qwen
Simple	0.849	0.770	0.833	0.737	0.937	0.894	0.293	0.607	0.591	0.411	23.45	14.26
STYLL	0.636	0.651	0.545	0.527	0.852	0.835	0.541	0.699	0.092	0.084	11.93	13.01
RG	0.634	0.653	0.583	0.533	0.856	0.837	0.635	0.796	0.080	0.074	10.24	25.56
Aspect Prompt	0.654	0.653	0.558	0.478	0.850	0.826	0.597	0.662	0.111	0.113	13.70	17.30

Table 1: Results on three authorship-transfer benchmarks, with separate Llama and Qwen subcolumns. ROUGE-L measures target-exemplar overlap, and Length is the character-length ratio. Bold / underline mark the best/second-best reported values per dataset; Simple is grayed because its high overlap indicates copying. † TinyStyler uses its own model and is conditioned on CAV embeddings, potentially favoring StyleCAV.

Figure 2: Pairwise preference against TinyStyler. A3S (Qwen) is preferred on MUD by both human annotators and a GPT-4.1 judge. Preference is strongest on the out-of-domain LaMP and LongLaMP.

Target aspect	Purity@1	Purity@2
Vocabulary	0.985	1.000
Tone	0.935	0.965
Syntax	0.870	0.940
Perspective	0.635	0.680
Figurative	0.505	0.590
Structure	0.255	0.510

Table 2: Aspect isolation in LLM-generated rewrites.

Panel of Figure 3 — Figure 3: Pairwise cosine between aspect directions (avg. 15 profiles). Raw aspects share a strong common component (left); after PC1 removal, residuals are negatively correlated in $60\%$ of pairs (right).

$\lambda$	StyleCAV	Construction	$\alpha{=}0.4$	$\alpha{=}0.5$
$-1.00$	0.153	PC1-only	$\phantom{-}0.088$	$\phantom{-}0.267$
$-0.50$	0.192	Resid. PCB	$-0.218$	$-0.166$
$\phantom{-}0.00$ (PC1)	0.267	PC1+res. mean	$\phantom{-}0.211$	$\phantom{-}0.315$
$\phantom{-}0.25$	0.294	PC1+res. PCB	$\phantom{-}0.241$	$\phantom{-}0.339$
$\phantom{-}0.50$	0.310	Raw mean	$\phantom{-}0.211$	$\phantom{-}0.317$
$\phantom{-}0.75$	0.298	Raw PCB	0.287	0.378

Table 3: Residual role on MUD dev (StyleCAV

\uparrow

). Left :

\lambda

-sweep of

v(\lambda){=}u_{\mathrm{PC1}}{+}\lambda\,\mathrm{PCB}(R)

. Right : steering-vector constructions at fixed

\alpha

. Full results in Appendix A.11 .

Aggregation Strategies (Top 4 Aspects)
Configuration	StyleCAV ( $\uparrow$ )	SBERT ( $\uparrow$ )
PCB-Merging	0.485	0.694
Median	0.459	0.713
Mean Averaging	0.442	0.659
TIES-Merging	0.428	0.650
Individual Dimensions

Table 4: Aggregation strategies vs. single dimensions on MUD dev. PCB-Merging combines Figurative, Tone, Perspective, Structure ; Global is a single contrastive direction without aspect prompting.

Figure 4: Style–meaning trade-off on MUD dev as

\alpha

varies. Fixed

\alpha

must trade-off between under-steering (low StyleCAV) and degeneration (collapsing SBERT). Hybrid Adaptive Search (

\alpha^{\star}

, green stars) achieves StyleCAV

0.48

while maintaining SBERT at

0.69

, outperforming all fixed

\alpha

settings on the Pareto frontier.

Appendix figures & tables26 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Method	Backbone	LUAR ( $\uparrow$ )		StyleCAV ( $\uparrow$ )		StyleDistance ( $\uparrow$ )		Meaning ( $\uparrow$ )		Target Overlap ( $\downarrow$ )
Dataset	Method	Backbone	Towards	Away	Towards	Away	Towards	Away	MIS	SBERT	R-1	R-2	R-L
MUD (Reddit)	Simple	Llama	0.849	0.313	0.833	0.384	0.937	0.128	0.205	0.293	0.606	0.571	0.591
	Simple	Qwen	0.770	0.207	0.737	0.302	0.894	0.099	0.457	0.607	0.435	0.382	0.411
	STYLL	Llama	0.636	0.317	0.545	0.365	0.852	0.127	0.330	0.541	0.157	0.034	0.092
	STYLL	Qwen	0.651	0.262	0.527	0.390	0.835	0.130	0.710	0.699	0.152	0.025	0.084
	RG	Llama	0.634	0.278	0.583	0.333	0.856	0.114	0.529	0.635	0.137	0.026	0.080

Appendix

Table 5: Full results across the three authorship-transfer benchmarks using Llama and Qwen backbones. Towards and Away measure style fidelity; MIS and SBERT measure meaning preservation; and ROUGE-1/2/L measure target-exemplar overlap. Bold and underline indicate the best and second-best non-grayed values per dataset. Simple is grayed because its high target overlap indicates copying.

GYAFC (Formal $\to$ Informal)
Method	Backbone	LUAR	CAV	SD	SBERT	R-L	Len
		( $\uparrow$ )	( $\uparrow$ )	( $\uparrow$ )	( $\uparrow$ )	( $\downarrow$ )
Simple	Llama	0.845	0.836	0.941	0.305	0.566	12.77
STYLL	Llama	0.670	0.555	0.857	0.437	0.120	8.24
RG	Llama	0.651	0.473	0.835	0.601	0.085	4.67
Aspect Prompt	Llama	0.681	0.421	0.822	0.533	0.138	11.22

Appendix

Table 6: Single-attribute formality transfer (GYAFC). Same metrics and ranking conventions as Table 1 .

Figure 5: Layer-wise classification accuracy for different stylistic dimensions. The y-axis represents the probability that the projection of the hidden state onto the learned direction is higher for the target author than the foil. We observe that high-level concepts like Perspective and Tone peak in the middle-to-late layers (Layers 10–20).

Baselines & Individual Dimensions
	Style Fidelity ( $\uparrow$ )			Meaning Preservation ( $\uparrow$ )
Configuration	LUAR	StyleCAV	StyleDistance	SBERT	MIS	METEOR
Neutral (No Aspect)	0.227	0.055	0.710	0.799	0.670	0.602
Figurative	0.288	0.463	0.761	0.685	0.560	0.422
Tone	0.294	0.427	0.744	0.665	0.597	0.399
Perspective	0.297	0.403	0.755	0.713	0.677	0.417

Appendix

Table 7: Full ablation results for individual style aspects and aggregation strategies. We report Style Fidelity (LUAR, StyleCAV, StyleDistance) and Meaning Preservation (SBERT, MIS, METEOR). Neutral represents standard activation subtraction without aspect-specific prompting.

Configuration	LUAR	StyleCAV	StyleDistance	SBERT	MIS	METEOR	Time (s)
$\alpha=0.0$	0.137	-0.196	0.625	0.807	0.724	0.591	0
$\alpha=0.1$	0.118	-0.220	0.636	0.863	0.910	0.863	0
$\alpha=0.2$	0.128	-0.093	0.655	0.880	0.912	0.689	0
$\alpha=0.3$	0.161	0.170	0.708	0.910	0.901	0.759	0
$\alpha=0.4$	0.199	0.307	0.741	0.874	0.827	0.729	0
$\alpha=0.5$	0.247	0.409	0.752	0.761	0.704	0.612	0

Appendix

Table 8: Full results for fixed and adaptive steering coefficients on the MUD development set. Style metrics report Towards scores (higher is better).

Dataset	System	Ratio	$>10\%$	Abs. Shift
MUD	A3S	0.876	10.97	0.415
	TinyStyler	4.399	69.17	3.624
LaMP	A3S	0.838	12.70	0.457
	TinyStyler	2.325	93.85	1.339
LongLaMP	A3S	0.779	14.00	0.332
	TinyStyler	0.315	0.00	0.685

Appendix

Table 9: Length-shift statistics. Ratio is

\mathrm{len(output)}/\mathrm{len(input)}

in characters;

>10\%

is the fraction of outputs more than 10% longer than the source.

Dataset	Backbone	Preference (%)
		A3S	Tie	TinyStyler
MUD	Llama	54.0	7.0	39.0
	Qwen	66.7	6.4	26.9
LaMP	Llama	69.0	2.0	29.0
	Qwen	79.5	1.5	18.9
LongLaMP	Llama	60.0	12.0	28.0

Appendix

Table 10: Pairwise GPT-4.1 preferences comparing A3S with Llama and Qwen backbones against TinyStyler. Percentages include ties and may not sum to 100 due to rounding.

Panel of Figure 6 — Figure 6: Pairwise cosine similarity between aspect directions before (left) and after removing the first principal component (right). Raw aspect vectors share a strong common authorship component, while PC1-removed residuals expose conflicting aspect-specific corrections.

Aspect	Mean	Std
Figurative	0.832	0.289
Tone	0.359	0.110
Perspective	0.822	0.487
Structure	0.831	0.474
Syntax	0.549	0.129
Vocabulary	0.370	0.134

Appendix

Table 11: Per-aspect residual norm ratio

\|\,s_{d}-\mathrm{proj}_{\mathrm{PC1}}(s_{d})\|/\|s_{d}\|

, averaged over 15 target profiles. Larger values indicate that more of the aspect direction lies outside the shared PC1 backbone.

Dataset	# Profiles	Tokens/Profile	Sec/Profile
MUD	15	404,812	33.00
LongLaMP	100	258,328	27.84
LaMP	1,496	210,015	27.79

Appendix

Table 12: Cost of constructing A3S directions per target author/profile. Each profile uses 16 generated outputs.

Method	Total Calls	Prompt Tokens	Output Tokens	StyleCAV $\uparrow$	SBERT $\uparrow$
STYLL	3.0	2,808	891	0.581	0.742
STYLL (Reasoning)	3.0	1,739	1,887	0.420	0.767
RG	3.0	8,107	2,180	0.589	0.797
RG (Reasoning)	3.0	5,938	3,810	0.442	0.809
Aspect Prompt	3.0	7,306	1,791	0.516	0.730
Aspect Prompt (Reasoning)	3.0	6,526	3,299	0.389	0.744

Appendix

Table 13: End-to-end quality–compute comparison on the MUD development set using the Qwen backbone. Token counts include all stages of each method. The horizontal separator distinguishes prompting-based methods (above) from activation-steering methods (below). Higher StyleCAV and SBERT scores are better.

Input	Target Exemplar	A3S Output
Verratti is practically untouchable. He’s signing an extension every year or so and PSG won’t sell for even a €100m.	He didn’t do any of that … oh for fucks sake you are brilliant … hearing “DADDDDYYYYY!!” … hehe yeah, ill enjoy it while it lasts … donuts too? glazed thanks WHERE. WHERE CAN I GET ONE.	VerrATTI IS PRACtically UNTACHOOL!!1!! hes SIGNIN A EXTENR … PSG WONT SELL 4 EVN 100M
You’re assuming that Rooney is on the market while it’s clear that Mourinho thinks that he can play an important part at this club yet.	This analogy doesnt really work for jim, per se … for chrissakes hes saying BRAVO again! tss tell us how you really feel … lol opies tits sniff …	you’re assuming roonee is on the market lol no u r soooo wrong lol
Yeah, and that’s the exact way you do a 270 degree Zidane turn (the 4 star turn that u/Ewaninho showed). What Firmino is doing is not a McGeady spin. It’s a matter of semantics I guess… And what’s up with the ad hominem?	This analogy doesnt really work for jim, per se … for chrissakes hes saying BRAVO again! tss tell us how you really feel … lol opies tits sniff …	yeah and thats the exat way you do a 270 degree zidane turn … what firmino is doin is not a mcgeeedy spin its a mater of sems i gess and whats up with the ad hominem

Appendix

Table 14: Qualitative examples from the MUD dataset. Each case shows the source input, the target exemplar, and the model output. Ellipses indicate truncation for brevity.

Figure 7: Style transfer success judgment: annotators compare two candidates against the target style exemplar, and then optionally provide a rationale.

Figure 8: Meaning preservation judgment: annotators compare the same candidates against the original input to assess meaning preservation.

Panel of Figure 9 — Figure 9: The Pairwise Style Comparison prompt used for GPT-4.1 evaluation. This prompt instructs the model to compare two candidate outputs against a reference exemplar solely on stylistic grounds, ignoring topic similarity.

Panel of Figure 10 — Figure 10: Prompt used to produce neutralized variants.

Panel of Figure 11 — Figure 11: Prompt used to generate contrastive tone variants.

Panel of Figure 12 — Figure 12: Prompt used to generate contrastive narrative perspective variants.

Panel of Figure 13 — Figure 13: Prompt used to generate contrastive diction/register variants.

Panel of Figure 14 — Figure 14: Prompt used to generate contrastive syntactic complexity variants.

Panel of Figure 15 — Figure 15: Prompt used to generate contrastive figurative vs. literal variants.

Panel of Figure 16 — Figure 16: Prompt used to generate contrastive structural/organizational variants.

Panel of Figure 17 — Figure 17: Prompt used to generate contrastive surface-form variants.

Panel of Figure 18 — Figure 18: Turn 1 of the Aspect-Aware Prompting baseline: elicit an aspect-by-aspect analysis of the target exemplar.

Panel of Figure 19 — Figure 19: Turn 2 of the Aspect-Aware Prompting baseline: distill the per-aspect analysis into a list of style descriptors.

Panel of Figure 20 — Figure 20: Turn 3 of the Aspect-Aware Prompting baseline: rewrite the source input to match the elicited style descriptors.

Language Model Steering

Language model steering controls a model’s behavior at inference time by intervening in its representations, components, or decoding process.

See all 275 papers

Predicting Steering Vectors and Adapter Weights for Few-Shot Author-Style Transfer
Oct 2, 2026Leonard Popp, Danni Liu, Supriti Sinhamahapatra +1Text Style Transfer Text Generation
Karlsruhe Institute of Technology, Germany
Kernelized Activation Steering
Oct 1, 2026Laziz U. Abdullaev, Minh-Hieu Pham, Bach Do +2Language Model Steering Activation Steering
Hanoi University of Science and Technology
AuthorMix: Modular Authorship Style Transfer via Layer-wise Adapter Mixing
Mar 24, 2026Sarubi Thillainathan, Ji-Ung Lee, Michael Sullivan +1Text Style Transfer Text Generation
Department of Language Science and Technology, Saarland Informatics Campus Saarland University, Saarbrücken, Germany

Oct 2, 2026J/K move · Enter open · S save

Predicting Steering Vectors and Adapter Weights for Few-Shot Author-Style Transfer

Leonard Popp, Danni Liu, Supriti Sinhamahapatra, Jan Niehues

Karlsruhe Institute of Technology, Germany

Text Style Transfer Text Generation Language Model Steering Hypernetworks Low-Rank Adaptation

Open paper AskPDF arXiv

Adapting large language models to an individual author's style from a few examples is challenging, and scientific writing sharpens the difficulty: formal conventions leave little surface variation, and authors write about their own topics, so extracted ``style'' easily entangles with content. We study style-conditioned abstract generation from a few example abstracts per author and propose three methods: (1) contrastive activation steering, (2) a network that predicts steering vectors, and (3) a hypernetwork that predicts LoRA adapters. We find a consistent trade-off between style imitation and output quality: fine-tuning buys most of the available style signal but forfeits fluency, while the hypernetwork achieves the best trade-off on both seen and unseen authors. Our steering operates at author level, contrasting an author's abstracts against style-neutral generations for the same content. This holds topic fixed, removes the need for a predefined style inventory, and outperforms inventory-based steering. % [EDIT 1a] softened "no single optimal axis" claim Moreover, our analyses demonstrate that manually extracted and predicted steering vectors are near-orthogonal yet score comparably, indicating that style conditioning here can admit at least two unrelated directions rather than requiring one particular axis.

Abstract

Language Model Steering

Explore similar work

Predicting Steering Vectors and Adapter Weights for Few-Shot Author-Style Transfer

Kernelized Activation Steering

AuthorMix: Modular Authorship Style Transfer via Layer-wise Adapter Mixing