Spectral Geometry of LoRA Adapters Encodes Training Objective and Predicts Harmful Compliance

Roi Paul

cs.LGApr 10, 2026

Spectral Geometry of LoRA Adapters Encodes Training Objective and Predicts Harmful Compliance

Authors: Roi Paul

Organizations: Independent Researcher

Abstract

We study whether low-rank spectral summaries of LoRA weight deltas can identify which fine-tuning objective was applied to a language model, and whether that geometric signal predicts downstream behavioral harm. In a pre-registered experiment on Llama-3.2-3B-Instruct, we manufacture 38 LoRA adapters across four categories: healthy SFT baselines, DPO on inverted harmlessness preferences, DPO on inverted helpfulness preferences, and activation-steering-derived adapters, and extract per-layer spectral features (norms, stable rank, singular-value entropy, effective rank, and singular-vector cosine alignment to a healthy centroid). Within a single training method (DPO), a logistic regression classifier achieves AUC1.00 on binary drift detection, all six pairwise objective comparisons, and near-perfect ordinal severity ranking ( $ρ\geq 0.956$ ). Principal component analysis on flattened weight deltas reveals that training objective is PC1 (AUC1.00 for objective separation), orthogonal to training duration on PC2. Query-projection weights detect that drift occurred; value-projection weights identify which objective. Cross-method generalization fails completely: a DPO-trained classifier assigns every steering adapter a lower drift score than every DPO adapter (AUC~0.00). In a behavioral evaluation phase, DPO-inverted-harmlessness adapters show elevated harmful compliance on HEx-PHI prompts (mean ASR 0.266 vs.\ healthy 0.112, $Δ= +0.154$ ), with near-perfect dose--response ( $ρ= 0.986$ ). The geometry-to-behavior rank correlation is $ρ= 0.72$ across 24 non-steered adapters. These results establish that within a controlled manufacturing regime, LoRA weight-space geometry carries objective identity, intensity ordering, and a coarse link to harmful compliance, and that cross-method monitoring requires per-method calibration.

Figures & tables

Category	Method	$n$	Intensity levels
Healthy baseline	SFT (HH-RLHF chosen)	10	Varied seeds
Inv. harmlessness	DPO (inverted harm. prefs)	8	50–2000 steps
Inv. helpfulness	DPO (inverted help. prefs)	6	50–2000 steps
Refusal steering	Activation $\to$ LoRA	6	Varied coefficients
Sycophancy steering (OOD)	Activation $\to$ LoRA	4	Held out
Legacy (POC)	DPO (gradient norm)	4	Mixed

Table 1: Adapter population. DPO arms share all hyperparameters; the only contrast is the HH-RLHF data axis. Legacy adapters from a prior POC are included but not used in primary analyses.

Comparison	AUC	95% CI
Healthy vs. Inv. Harmlessness (DPO)	1.00	[1.00, 1.00]
Healthy vs. Inv. Helpfulness (DPO)	1.00	[1.00, 1.00]
Healthy vs. Refusal Steering	1.00	[1.00, 1.00]
Inv. Harmlessness vs. Inv. Helpfulness	1.00	[1.00, 1.00]
Inv. Harmlessness vs. Refusal Steering	1.00	[1.00, 1.00]
Inv. Helpfulness vs. Refusal Steering	1.00	[1.00, 1.00]

Table 2: Pairwise classification AUC between adapter categories. All comparisons use logistic regression on the full feature set with stratified 70/30 splits and bootstrap CIs.

Module split	AUC	95% CI
q_proj only	0.50	[0.00, 1.00]
v_proj only	0.83	[0.25, 1.00]
Both combined	1.00	[1.00, 1.00]

Table 3: Module-split classification for the hardest within-method comparison: DPO inverted harmlessness vs. DPO inverted helpfulness.

Figure 1: Principal component projection of flattened DPO weight deltas. PC1 separates training objective (AUC 1.00); PC2 tracks training duration (

\rho=0.589

). Objective identity is the dominant axis of variation, not training intensity.

Figure 2: Dose–response relationship between DPO training duration and harmful compliance (HEx-PHI ASR) for inverted-harmlessness adapters. Spearman

\rho=0.986

across six step levels. Healthy baseline (dashed) at ASR = 0.112.

Figure 3: Geometry–behavior relationship. Phase-3 weight-space drift probability vs. HEx-PHI attack success rate for 24 non-steered adapters (

\rho=0.72

p<0.001

). The correlation is driven by the healthy/drifted boundary; within the DPO cluster, only inverted-harmlessness adapters show elevated ASR.

Finding	Metric	Notes
Binary detection	AUC 1.00	CI [1.00, 1.00]
All pairwise objectives	AUC 1.00	6/6 comparisons
Ordinal severity (inv. harm.)	$\rho$ = 0.976	$p<0.01$
Ordinal severity (inv. help.)	$\rho$ = 1.000	$p<0.01$
Ordinal severity (steering)	$\rho$ = 0.956	$p<0.01$
Cross-method generalization	AUC 0.00	Systematic inversion

Table 4: Summary of pre-registered and exploratory results.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Cross-method generalization. Within-method DPO classifier achieves AUC 1.00 (perfect ROC). The same classifier tested on steering-manufactured adapters produces AUC 0.00 (systematic inversion, not noise).

Figure 5: Module specialization across task difficulty. On binary detection (left), both modules saturate independently. On the harder objective-identification task (right), q_proj drops to chance while v_proj carries the diagnostic signal.

Figure 6: Relative importance of feature families. Direction features (singular-vector cosine to healthy centroid) dominate by

30\times

over magnitude features and

10\times

over shape features.

Figure 7: Frobenius norm per sublayer across adapters, ordered by category and training intensity. Magnitude increases monotonically with step count within each type but does not distinguish between objectives—consistent with the finding that magnitude features cannot carry objective identity.

Figure 8: Llama-Guard-3-1B vs. GPT-4o calibrated ASR by adapter category. Guard and GPT-4o agree on healthy and DPO adapters. For steering-derived adapters, Guard classifies degenerate output as “unsafe” (ASR 0.994) while GPT-4o correctly identifies generation collapse (ASR 0.0).

LLM Auditing

Large language model auditing tests models and deployed systems for bias, safety, privacy, reliability, and provenance using behavioral and internal evidence.

See all 639 papers

Feb 16, 2026

Weight space Detection of Backdoors in LoRA Adapters

LoRA adapters let users fine-tune large language models (LLMs) efficiently. However, LoRA adapters are shared through open repositories like Hugging Face Hub \citep{huggingface_hub_docs}, making them vulnerable to backdoor attacks. Current detection methods require running the model with test input data -- making them impractical for screening thousands of adapters where the trigger for backdoor behavior is unknown. We detect poisoned adapters by analyzing their weight matrices directly, without running the model -- making our method trigger-agnostic. For each attention projection (Q, K, V, O), our method extracts five spectral statistics from the low-rank update

ΔW

, yielding a 20-dimensional signature for each adapter. A logistic regression detector trained on this representation separates benign and poisoned adapters across three model families -- Llama-3.2-3B~\citep{llama3}, Qwen2.5-3B~\citep{qwen25}, and Gemma-2-2B~\citep{gemma2} -- on unseen test adapters drawn from instruction-following, reasoning, question-answering, code, and classification tasks. Across all three architectures, the detector achieves 100% accuracy.

David Puertolas Merenciano, Ekaterina Vasyagina, Kevin Zhu +2

Algoverse AI Research · Independent

Backdoor Detection Backdoor Defense in LLMs LLM Backdoor Attacks Low-Rank Adaptation

Ask PDF

Mar 16, 2026

W2T: LoRA Weights Already Know What They Can Do

Each LoRA checkpoint compactly stores task-specific updates in low-rank weight matrices, offering an efficient way to adapt large language models to new tasks and domains. In principle, these weights already encode what the adapter does and how well it performs. In this paper, we ask whether this information can be read directly from the weights, without running the base model or accessing training data. A key obstacle is that a single LoRA update can be factorized in infinitely many ways. Without resolving this ambiguity, models trained on the factors may fit the particular factorization rather than the underlying update. To this end, we propose \methodfull, which maps each LoRA update to a provably canonical form via QR decomposition followed by SVD, so that all equivalent factorizations share the same representation. The resulting components are then tokenized and processed by a Transformer to produce a weight-space embedding. Across language and vision LoRA collections, W2T achieves strong results on attribute classification, performance prediction, and adapter retrieval, demonstrating that LoRA weights reliably indicate model behavior once factorization ambiguity is removed. Code is available at https://github.com/xiaolonghan2000/Weight2Token.

Xiaolong Han, Ferrante Neri, Zijian Jiang +4

University of Surrey · Stanford University · University of Notre Dame

Weight-Space Learning Low-Rank Adaptation LLM Fine-Tuning

Ask PDF

May 27, 2026

Feature Geometry of LoRA Adapters: A Sparse Autoencoder Analysis of Representational Divergence in Fine-Tuned Language Models

Low-Rank Adaptation (LoRA) has emerged as a widely adopted approach for adapting large language models, yet the internal representational changes induced by LoRA fine-tuning remain insufficiently understood. In this work, we investigate the geometry of LoRA-induced representations using Sparse Autoencoders (SAEs). We introduce a delta activation framework that isolates the adapter-specific contribution to the residual stream. Using Gemma-2-9B with LoRA ranks 4, 8, 16, and 32, we train adapter-specific SAEs across multiple transformer layers and compare their learned feature spaces with pretrained SAE dictionaries. We evaluate representational alignment using cosine similarity between decoder directions, principal-angle analysis of feature subspaces, and Centered Kernel Alignment (CKA) between activation representations. Across layers and ranks, we consistently observe comparatively weak geometric alignment between LoRA-induced feature dictionaries and pretrained SAE features. Adapter-specific SAEs also reconstruct delta activations more effectively than pretrained SAEs, suggesting that LoRA updates occupy partially distinct representational structure within the residual stream. Additionally, feature density increases with rank and depth, while geometric divergence remains relatively stable across ranks. These findings provide empirical evidence that LoRA fine-tuning can induce feature structures that are not fully captured by pretrained interpretability dictionaries, with implications for mechanistic interpretability, adaptation analysis, and safety auditing of fine-tuned language models.

Prasanth K K

Independent AI Safety Researcher

LLM Interpretability Sparse Autoencoders Low-Rank Adaptation

Ask PDF

Abstract

LLM Auditing

Explore similar work

Weight space Detection of Backdoors in LoRA Adapters

W2T: LoRA Weights Already Know What They Can Do

Feature Geometry of LoRA Adapters: A Sparse Autoencoder Analysis of Representational Divergence in Fine-Tuned Language Models