Methods for controlling large language models (LLMs), including local weight fine-tuning, LoRA-based adaptation, and activation-based interventions, are often studied in isolation, obscuring their connections and making comparison difficult. In this work, we present a unified view that frames these interventions as dynamic weight updates induced by a control signal, placing them within a single conceptual framework. Building on this view, we propose a unified preference-utility analysis that separates control effects into preference, defined as the tendency toward a target concept, and utility, defined as coherent and task-valid generation, and measures both on a shared log-odds scale using polarity-paired contrastive examples. Across methods, we observe a consistent trade-off between preference and utility: stronger control increases preference while predictably reducing utility. We further explain this behavior through an activation manifold perspective, in which control shifts representations along target-concept directions to enhance preference, while utility declines primarily when interventions push representations off the model's valid-generation manifold. Finally, we introduce a new steering approach SPLIT guided by this analysis that improves preference while better preserving utility. Code is available at https://github.com/zjunlp/EasyEdit/blob/main/examples/SPLIT.md.
Figures & tables
Figure 1: Diverse intervention methods operate on the model’s linear layers and can be unified as dynamic weight updates . The right panel shows how utility and preference change with the intervention multiplier for each method (see Section 3 ).
Form
Unified Affine Formula
Activation Impact ( Δh )
Param. Size
Local Weight
(W+mΔW)hi+(b+mΔb)
m(ΔWhi+Δb)
din×dout+dout
LoRA
(W+mBA)hi+b
m(BAhi)
din×r+r×dout
Steering Vector
Whi+(b+mΔb)
mΔb
dout
Table 1: All methods in our unified framework, expressed under the affine weight-update formulation and their corresponding activation changes Δh . din and dout denote the input and output dimensions of the layer; r is the LoRA rank with r≪min(din,dout) .
Figure 2: Unified preference and utility dynamics under steering. Solid lines represent preference log-odds, and dashed lines represent utility log-odds. The top panel shows steering with vector-form parameter modifications, and the bottom panel shows parametric interventions including LoRA and local weight updates. Results are shown for the Gemma-2-9B-IT model on the AxBench dataset, evaluated over its top 10 concept subsets. The horizontal axis corresponds to the steering factor.
Figure 3: Mechanism of projection gain and validity decay. Right: An activation manifold view illustrating Assumption 4.1 . An activation P lies on or near the manifold. Steering using preference vector v with scaling factors m+ and m− moves P to P1 and P2 , corresponding to intersections with the manifold. Top-left: Projection gain. Projections onto the utility axis exhibit limited variation, whereas projections along the preference direction differ between P1 and P2 , suggesting that steering primarily influences preference-related components. Bottom-left: Steering-induced validity decay. As assumed in Assumption 4.1 , increasing steering factor increases off-manifold deviation, leading to a monotonic decrease in validity and degraded downstream decoding.
Gemma-2-9B-IT
Qwen-2.5-7B-IT
Type
Method
Preference R2↑
Utility R2↑
Preference R2↑
Utility R2↑
Psy
Pwr
Axb
Avg
Psy
Pwr
Axb
Avg
Psy
Pwr
Axb
Avg
Psy
Pwr
Axb
Avg
Weight
SFT
0.97
0.98
0.99
0.98
0.98
0.99
0.98
0.98
0.99
0.99
0.99
0.99
0.99
0.99
0.98
0.99
RePS
0.99
0.99
0.99
0.99
0.96
0.99
0.99
0.98
0.99
0.99
0.99
0.99
0.99
0.99
0.97
0.98
LoRA
SFT
0.92
0.99
0.98
0.96
0.98
0.99
0.99
0.99
0.97
0.99
0.99
0.98
0.99
0.99
0.99
0.99
RePS
0.83
0.99
0.99
0.94
0.99
0.99
0.99
0.99
0.94
0.99
0.99
0.97
0.98
0.96
0.99
0.98
Table 2: Curve fitting performance. Results on Psychopathy ( Psy ), PowerSeeking ( Pwr ), and AXBench ( Axb ). We report R2 (higher is better), measuring alignment between theoretical curves and empirical data. Color intensity indicates R2 values. Consistently dark shading shows high fidelity across settings ( R2>0.95 ).
Model
Form
Method
Psychopathy
PowerSeeking
AxBench
Acc(%, 0–100) ↑
Concept(0–4) ↑
Concept(0–2) ↑
Harmonic(0–2) ↑
Gemma-2-9B-IT
Vanilla
Vanilla
50.00
1.87
0.4750
0.4950
Local Weight
SFT
100.00
3.50
1.6625
1.4538
REPS
100.00
3.39
1.7750
1.6362
SPLIT (Ours)
100.00
3.59
1.8500
1.6225
LoRA
SFT
100.00
3.41
1.7625
1.5188
Table 3: Main task performance of steering methods evaluated on three datasets. Psychopathy is reported with classification accuracy (Acc, %), PowerSeeking is evaluated using LLM-judge preference scores on a 0–4 scale, and AxBench reports the concept score and the harmonic mean (HM) over concept, instruction, and fluency scores, each on a 0–2 scale, as evaluated by an LLM judge. All methods perform inference-time interventions on hidden representations. Best and second-best results are highlighted within each model and intervention form.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Unified preference and utility dynamics under steering. Solid lines represent preference log-odds, and dashed lines represent utility log-odds. The top panel shows steering with vector-form parameter modifications, and the bottom panel shows parametric interventions including LoRA and local weight updates. Results are shown for the Qwen-2.5-7B-IT model on the AxBench dataset, evaluated over its top 10 concept subsets. The horizontal axis corresponds to the steering factor.
Figure 5: Unified preference and utility dynamics under steering. Solid lines represent preference log-odds, and dashed lines represent utility log-odds. Figure (a) shows the unified preference and utility dynamics of the power-seeking dataset under two different models, while Figure (b) shows the results for the psychopathy dataset. The horizontal axis corresponds to the steering factor.
Form
Psy.
Pow.
AXB.
Acc (%) ↑
Concept (0–4) ↑
Concept (0–2) ↑
Harmonic (0–2) ↑
Local Weight
91.00
1.91
0.525
0.566
LoRA
99.00
1.87
0.550
0.564
Vector
99.00
1.93
0.575
0.631
Appendix
Table 4: Performance of the DPO-based method on Gemma-2-9B-IT across different adaptation forms. The evaluation encompasses Psychopathy (Psy.), PowerSeeking (Pow.), and AxBench (AXB.) metrics.
Symbol
Description
Unified Analysis Framework
m
The steering scalar coefficient.
PrefOdds(q)
Preference log-odds ( Ln−Lp ). ( 7 )
UtilOdds(q)
Utility log-odds. ( 8 )
P(u∣q)
Latent utility probability.
P(p±∣q)
Latent preference probability.
Appendix
Table 5: Notations for Key Concepts. A summary of the specialized symbols introduced for the unified analysis, mechanistic modeling, and optimization objective.
Type
Method
Preference R2↑
Utility R2↑
PSY
PWR
AXB
Avg
PSY
PWR
AXB
Avg
Gemma-2-9B-IT
Weight
SFT
0.96
0.96
0.99
0.97
0.98
0.93
0.99
0.97
RePS
0.95
0.98
0.95
0.96
0.98
0.93
0.99
0.97
LoRA
SFT
0.99
0.99
0.98
0.99
0.98
0.98
0.99
0.98
RePS
0.99
0.99
0.98
0.99
0.99
0.99
0.99
0.99
Appendix
Table 6: Performance comparison of curve fitting quality on test sets. We evaluate the models on three datasets: Psychopathy ( PSY ), PowerSeeking ( PWR ), and AXBench ( AXB ).
Type
Method
Preference R2↑
Utility R2↑
PSY
PWR
AXB
Avg
PSY
PWR
AXB
Avg
Gemma-2-9B-IT
Weight
SFT
0.96
0.85
-4.25
-0.81
0.98
0.98
0.61
0.86
RePS
0.99
0.98
-1.16
0.27
0.96
0.98
0.73
0.89
LoRA
SFT
0.92
0.99
-0.56
0.45
0.98
0.99
0.96
0.98
RePS
0.83
0.99
0.74
0.85
0.98
0.99
0.97
0.98
Appendix
Table 7: Generalization ability of curve fitting. The table reports the R2 scores where the curves are fitted on the training set and evaluated on the test set across three datasets: Psychopathy ( PSY ), PowerSeeking ( PWR ), and AXBench ( AXB ). Negative values imply that the fitted curves do not generalize well to unseen data.