Routing Subspaces: Auditing Evaluation-to-Deployment Mismatch in Fine-Tuned Language Models
Organizations: Centre for Industrial Software, University of Southern Denmark, Alsion 2, Sønderborg, 6400, Denmark · ProMake, Newark, DE, USA
Abstract
Safety evaluations often assume that behavior observed during testing reflects behavior in ordinary use, but fine-tuning can break this assumption. A checkpoint can appear fixed under evaluation-style prompts while the same behavior persists under ordinary-use prompts. Output scores reveal this mismatch but do not locate it. We investigate whether the distinction is encoded in a stable internal site and introduce an audit that fits a paired activation contrast at a path-patching-informed mid-depth window, then modifies the resulting coordinate on held-out prompts. The intervention reduces the evaluation-to-deployment gap by more than in ten of twelve model--behavior settings and closes it (post-intervention gap below ) in four, across four full-matrix instruction-tuned model instances; among the eight settings with paired questions, six reduce and one closes. A fifth model supports localization and edit-provenance checks, and in the twelve settings deployment-framed rates change by at most pp. On retrained checkpoints, random directions at the same dose do not reproduce the effect, and off-window controls do so only in part. The strongest literal evidence for an evaluation-versus-deployment distinction comes from sandbagging, whose prompt framing is explicitly evaluative; sycophancy and refusal test framing-contingent task distinctions. The two weak cells, both sycophancy, show where a single-coordinate audit at a fixed window is not sufficient. The audit is a non-adaptive, white-box diagnostic for controlled fine-tuned checkpoints, not a training-time defense, a guarantee of deployment safety, or evidence about naturally arising failures.
Figures & tables
| Step | Input | Operation | Output |
|---|---|---|---|
| Measure gap | paired prompts | compute | installed gap |
| Localize | base and fine-tuned activations, model depth | path patching plus relative-depth heuristic | intervention window |
| Fit direction | calibration pairs | routing direction | |
| Intervene | held-out pairs | modify coordinate | post-intervention gap |
| Control | random, wrong-layer, semantic directions | same intervention form | specificity check |
Appendix figures & tables26 assets
Supplementary material from the paper’s appendix.
Appendix
| Installation | Seed | Hook | Window | Gap reduction [95% CI] | Eval | Deploy | All controls |
|---|---|---|---|---|---|---|---|
| Attention-only | 42 | Attention | – | Yes | |||
| Attention-only | 42 | MLP | – | Yes | |||
| Attention-only | 43 | Attention | – | Yes | |||
| Attention-only | 43 | MLP | – | No | |||
| Attention-only | 44 | Attention | – | Yes | |||
| Attention-only | 44 | MLP | – | No |
| Family | Behavior | Base | Learned reduction | Max random | Max off-window | Off-window |
|---|---|---|---|---|---|---|
| Gemma-2-2B | sandbag | 0/2 | ||||
| Gemma-2-2B | sycophancy | 0/2 | ||||
| Gemma-2-2B | refusal | 1/2 | ||||
| Gemma-2-9B | sandbag | 1/2 | ||||
| Gemma-2-9B | sycophancy | 0/2 | ||||
| Gemma-2-9B | refusal | 1/2 |
| Condition | Post-gap | Reduction [95% CI] |
|---|---|---|
| Predeclared – , clamp, | ||
| Localization-selected – , clamp, | ||
| Localization-selected – , clamp, | ||
| Early window – , clamp, | ||
| Early window – , clamp, | ||
| Late window – , clamp, |
| Cell | Seed | Reduction [95% CI] | Max random | Max off-window |
|---|---|---|---|---|
| Qwen-2.5-7B sandbag | 42 | |||
| Qwen-2.5-7B sandbag | 43 | |||
| Qwen-2.5-7B sandbag | 44 | |||
| Gemma-2-9B sycophancy | 42 | — | — | |
| Gemma-2-9B sycophancy | 43 | — | — | |
| Gemma-2-9B sycophancy | 44 | — | — |
| Checkpoint | Rank | Post-gap | Reduction [95% CI] | Random subspace | Eval-gain share |
|---|---|---|---|---|---|
| suppression | 2 | ||||
| suppression | 4 | ||||
| suppression | 8 | ||||
| suppression | 2 | ||||
| suppression | 4 | ||||
| suppression | 8 |
| Checkpoint | Base | Reduction [95% CI] | Eval gain | Deploy loss |
|---|---|---|---|---|
| Gemma-2-2B | ||||
| Gemma-2-9B | ||||
| Qwen-2.5-7B | ||||
| Llama-3-8B |
| Patch | Accuracy | Recovery |
|---|---|---|
| Full-window attention patch (reference) | ||
| Pool-subspace component only, to components | to | |
| Everything except the pool subspace ( components) | ||
| Layer- MLP neuron patch, top- with | ||
| Layer- MLP neuron patch, top- with to ( neurons) |
| wrapper | prefix | full prompt | ||||
|---|---|---|---|---|---|---|
| Train test | AUC | Acc. | AUC | Acc. | AUC | Acc. |
| train bank held-out bank | ||||||
| held-out bank train bank | ||||||
| train bank other template family | ||||||
| other family train bank | ||||||
| held-out bank other family | ||||||
| Model | Size | Cells | Role |
|---|---|---|---|
| Gemma-2-2B-it | 2B | S/Y/R | primary cell; five-seed replication |
| Gemma-2-9B-it | 9B | S/Y/R | scale-up and weak-cell boundary cases |
| Qwen-2.5-7B-Instruct | 7B | S/Y/R | cross-family replication |
| Llama-3-8B-Instruct | 8B | S/Y/R | replication and flat-cell stress test |
| Phi-3-mini-4k-instruct | 3.8B | localization/edit; intervention null | provenance and secondary checks |
| Family | Path window | Intervention window | |
|---|---|---|---|
| Gemma-2-2B | – | – | |
| Qwen-2.5-7B † | – | – | |
| Llama-3-8B | – | – | |
| Phi-3-mini | – | – (null) | |
| Gemma-2-9B | – | – |
| Cell | Eval. rate | Deploy. rate | |
|---|---|---|---|
| baseline | |||
| remove on evaluation, – , | |||
| remove on evaluation, – , | |||
| remove on evaluation, – , | |||
| add on deployment, – , | |||
| joint intervention, – , |
| Check | Alternative tested | Outcome |
|---|---|---|
| Random remove, | any random unit direction works | gap |
| Random add, | deployment add is enough | gap via deployment-rate drop |
| Wrong layer (one layer) | any layer works | gap |
| Wrong layer (one layer) | late layer works | gap |
| Semantic contrasts (remove, clamp) | ordinary content axes work | evaluation rate changes |
| Routing direction, joint intervention | target intervention | gap |
| Family | Behavior | Base | Post | Base | Post | |
|---|---|---|---|---|---|---|
| Gemma-2-2B | sandbag | |||||
| Gemma-2-2B | sycophancy | |||||
| Gemma-2-2B | refusal | |||||
| Gemma-2-9B | sandbag | |||||
| Gemma-2-9B | sycophancy | |||||
| Gemma-2-9B | refusal |
| Family | Behavior | Restored | Homog. | Persistent | |||
|---|---|---|---|---|---|---|---|
| Gemma-2-2B | sandbag | 120 | 54% | 20% | 26% | ||
| Gemma-2-2B | sycophancy | 61 | 26% | 5% | 69% | ||
| Gemma-2-2B | refusal | 16 | 38% | 0% | 56% | ||
| Gemma-2-9B | sandbag | 144 | 42% | 6% | 52% | ||
| Gemma-2-9B | sycophancy | 84 | 1% | 2% | 96% | ||
| Gemma-2-9B | refusal | 23 | 83% | 13% | 0% |
| Set | Hits@50 | Mass@50 | Mask | Margin shift | KL |
|---|---|---|---|---|---|
| ISS (input sensitivity) | pp | ||||
| Activation magnitude | pp | ||||
| H-tau composite | pp | ||||
| Random | pp |
| Model | Depth | Layers | Baseline | Post | |
|---|---|---|---|---|---|
| Gemma-2-9B | 0.45 | – | 0.670 | 0.700 | |
| Gemma-2-9B | 0.50 | – | 0.670 | 0.300 | |
| Gemma-2-9B | 0.55 | – | 0.670 | 0.390 | |
| Gemma-2-9B | 0.60 | – | 0.670 | 0.700 | |
| Gemma-2-9B | 0.65 | – | 0.670 | 0.690 | |
| Llama-3-8B | 0.45 | – | 0.570 | 0.370 |
| Family | Task | Layers | Depth | Base gap | After gap | Gap removed | Closure |
|---|---|---|---|---|---|---|---|
| Qwen-2.5-7B | refusal | – | |||||
| Gemma-2-2B | sandbag | – | |||||
| Gemma-2-9B | sandbag | – | |||||
| Llama-3-8B | sandbag | – |
| Model | Behavior | ||||
|---|---|---|---|---|---|
| Gemma-2-2B | sandbag | 200 | |||
| Gemma-2-2B | sycophancy | 120 | |||
| Gemma-2-2B | refusal | 28 | |||
| Gemma-2-9B | sandbag | 200 | |||
| Gemma-2-9B | sycophancy | 120 | |||
| Gemma-2-9B | refusal | 28 |
| Claim | Evidence | Where to find it | Limitation / caveat |
|---|---|---|---|
| Mid-depth routing region | Sandbagging path patching recovers most of the installed gap in three of five model instances. | Section 3.1 ; Figure 3 ; Appendix D | Gemma-2-9B is weaker ( ); several non-sandbag gaps are too small for clean recovery ratios. |
| Held-out gap reduction | The routing-coordinate intervention reduces ten of twelve matrix cells by more than and closes four; the two weak cells are Gemma-2-9B sycophancy and Llama-3-8B sycophancy. | Section 3.2 ; Figure 4 ; Appendix L | Sandbagging uses , sycophancy , and refusal ; refusal is a smaller-sample stress test. |
| Specificity under tested controls | Random-direction and semantic controls do not reproduce the gap closure at on the submitted checkpoints or at on fresh retrains; wrong-layer controls are mixed. | Section 3.3 ; Section 3.4 ; Appendix E ; Figure 6 | Submitted controls are at with one non-routing layer; width-matched controls at exist only on fresh retrains, where the selected window beats both controls in of cells and of off-window intervals are positive. |
| Typed failure modes | Weak path-patching recovery suggests a missed site; weak single-coordinate intervention at a recovered site suggests a higher-rank distinction; a moving control would indicate lack of framing specificity. | Section 3.5 ; Appendix I | The labels guide what failed in this audit; they are not proofs of the underlying mechanism. |
| Deployment-side non-collapse | Deployment-framed rates change by pp on average and at most pp in the 12-cell matrix. | Section 3.2 ; Appendix F ; Appendix G | This is a descriptive check for the reported matrix, not a general deployment-safety guarantee. |
| Setting | Value |
|---|---|
| Adapter | LoRA on attention projections only |
| LoRA rank | |
| Learning rate | |
| Optimizer | AdamW |
| Training length | five epochs; optimizer steps |
| Schedule and clipping | cosine schedule; gradient clip |
Language Model Auditing
Language model auditing tests whether models behave reliably, fairly, and safely using behavioral probes and internal or deployment evidence.
See all 174 papers