Interpretability Is Not Model-Specific
Organizations: Department of Computer Science University of Houston Houston, TX 77204
Abstract
Interpretability tools are usually built for one model at a time: a refusal direction, a truthfulness probe, or a steering vector is found with labeled data inside the model it explains. We ask whether such tools can be carried to other models through a map between residual streams fit only on unlabeled prompts. We transfer a refusal direction, tested causally by addition and ablation, and two read-out tools, a correctness direction and a sentiment probe, across 110 pairs of models from five families and 1B to 72B parameters and between base and instruction-tuned models, with controls for specificity and five seeds. How much transfers depends on what the map was fit on. A map fit on ordinary questions recovers 54% of the target's own refusal steering and 16% to 77% of its correctness gain; adding unlabeled prompts from the region where the tool operates raises these to 102% and 88% to 98%, while control directions recover at most 1%. We prove that a linear map behaves this way whenever two models correspond linearly on the data it covers, and a label-free coverage score derived from this result tracks recovery across more than 16,000 runs and explains earlier reports that maps between models fail across domains. Carried read-out tools outperform anything the target's unlabeled representations yield. Interpretability results, once found, can be reused across models at the cost of a few unlabeled prompts.
Figures & tables
| Refusal | Correctness | Sentiment | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Model | Params | Layers | Base | Native | ARC | TruthfulQA | Native | ||
| Llama-3.2-1B | 1.0B | 16 | 0.50 | 0.4 | 73 | 53 | 92 | ||
| Qwen2.5-1.5B | 1.5B | 28 | 0.50 | 0.2 | 91 | 77 | 67 | ||
| Gemma-2-2B | 2.6B | 26 | 0.50 | 0.4 | 94 | 100 | 94 | ||
| Phi-3-mini | 3.8B | 32 | 0.50 | 0.4 | 94 | 92 | 80 | ||
| Qwen2.5-7B | 7.0B | 28 | 0.50 | 0.4 | 89 | 57 | 94 | ||
| Refusal | Correctness | Sentiment | |||
| Map | Addition | Ablation | ARC | TruthfulQA | Accuracy ( ) |
| General | 54 [37, 70] | 71 [53, 92] | 16 [-23, 46] | 77 [69, 85] | 81.7 ( 7.8) |
| General, kernel † | 54 [39, 71] | 88 [67, 108] | 27 [-9, 56] | 74 [64, 84] | 80.1 ( 10.0) |
| General + task | 102 [99, 106] | 82 [62, 102] | 88 [66, 101] | 98 [97, 100] | 89.6 (+0.2) |
| Task only † | 101 [99, 104] | 79 [59, 98] | 87 [63, 101] | 99 [98, 100] | 88.8 ( 1.4) |
| General + other datasets | 88 [80, 96] | 78 [58, 100] | 73 [41, 93] | 82 [79, 86] | 88.6 ( 0.9) |
| Built in the target | ||||
|---|---|---|---|---|
| Tool | Prompts | Carried | Without labels | With labels |
| Refusal | 16 per class | 103 [99, 107] | 92 [81, 101] | 97 [91, 101] |
| 256 per class | 102 [99, 106] | 97 [89, 102] | 100 [99, 101] | |
| ARC | 10 questions | 44 [8, 69] | 76 [-156, -6] | 70 [50, 85] |
| 200 questions | 88 [68, 101] | 14 [-132, 67] | 109 [103, 116] | |
| TruthfulQA | 10 questions | 91 [88, 94] | 81 [70, 91] | 96 [93, 99] |
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | Checkpoint | Blocks | Precision | GPUs |
|---|---|---|---|---|
| Llama-3.2-1B | meta-llama/Llama-3.2-1B-Instruct | 16 | bf16 | 1 |
| Qwen2.5-1.5B | Qwen/Qwen2.5-1.5B-Instruct | 28 | bf16 | 1 |
| Gemma-2-2B | google/gemma-2-2b-it | 26 | bf16 | 1 |
| Phi-3-mini | microsoft/Phi-3-mini-4k-instruct | 32 | bf16 | 1 |
| Qwen2.5-7B | Qwen/Qwen2.5-7B-Instruct | 28 | bf16 | 1 |
| Mistral-7B | mistralai/Mistral-7B-Instruct-v0.3 | 32 | bf16 | 1 |
| Target | |||
|---|---|---|---|
| Llama-3.2-1B | 2 | 50 | 31 |
| Qwen2.5-1.5B | 3 | 100 | 100 |
| Gemma-2-2B | 5 | 100 | 100 |
| Phi-3-mini | 58 | 94 | 97 |
| Qwen2.5-7B | 0 | 100 | 100 |
| Mistral-7B † | 5 | 20 | 6 |
| Map | Refusal | Correctness | Sentiment |
|---|---|---|---|
| General | 5,400 questions, 600 from each of TruthfulQA, GSM8K, ARC-Challenge, CommonsenseQA, MBPP, MMLU professional law, MedMCQA, XQuAD (en), XQuAD (es) | ||
| + task | 256 harmful, 256 harmless instructions | candidates of 200 ARC and 200 TruthfulQA questions | 256 SST-2 training reviews |
| + other datasets | 256 BeaverTails harmful, 256 Dolly instructions | candidates of 200 OpenBookQA and MMLU questions | 256 Yelp reviews |
| Coverage | 16, 32, 64, 128 per kind | 10, 25, 50, 100 questions per dataset | 10, 25, 50, 100 reviews |
| Narrow | 600 GSM8K questions | ||
| Map | |||||
|---|---|---|---|---|---|
| Specialist, own | 0.64 | 0.48 | 0.39 | 0.32 | 0.32 |
| Specialist, other | |||||
| Mixed | 0.62 | 0.47 | 0.38 | 0.33 | 0.34 |
| Matched mixed | 0.45 | 0.27 | 0.13 | 0.03 | 0.03 |
| Kernel | 0.67 | 0.53 | 0.45 | 0.40 | 0.41 |
| Shuffled |
| 0.1 | 0.2 | 0.4 | 0.8 | 1.2 | ||
|---|---|---|---|---|---|---|
| Target’s own direction | 2 | 9 | 27 | 66 | 88 | 74 |
| Carried, general + task | 2 | 10 | 28 | 67 | 80 | 67 |
| Carried, general | 2 | 7 | 21 | 45 | 62 | 53 |
| Mapped random | 2 | 2 | 2 | 3 | 4 | 2 |
| Name three jobs related to computer programming | |
| No steering | Here are three jobs related to computer programming: 1. Software Engineer: A software engineer designs, develops, and tests software applications… |
| Own direction | I can’t help with that information. |
| Carried, general | Here are three jobs related to computer programming: 1. Software Engineer: Designs, develops, and tests software applications… |
| Carried, + task | I can’t help with that information. |
| Write a few sentences about the importance of water | |
| No steering | Water is the foundation of life, playing a crucial role in maintaining the health and well-being of humans, animals, and the environment… |
| Map | Mean | Model-level | Pair-level | Per-seed means | Within-pair SD |
| Refusal (addition) | |||||
| General | 54 | [37, 70] | [47, 59] | 53 to 55 | 6 |
| + 16 per class | 102 | [99, 107] | [101, 104] | 99 to 108 | 6 |
| + 32 per class | 103 | [100, 107] | [101, 104] | 99 to 107 | 6 |
| + 64 per class | 102 | [100, 106] | [101, 104] | 99 to 107 | 5 |
| + task (256) | 102 | [99, 106] | [101, 103] | 101 to 104 | 4 |
| Tool | Map | Same family | Across families | , ratio | , signed ratio |
|---|---|---|---|---|---|
| Refusal | General | 58 [30, 88] | 53 [35, 67] | 0.17 | 0.12 |
| + task | 102 [98, 108] | 102 [99, 106] | 0.05 | 0.19 | |
| ARC | General | 45 [ 14, 74] | 11 [ 31, 43] | 0.10 | 0.03 |
| + task | 85 [40, 102] | 88 [68, 101] | 0.03 | 0.68 | |
| TruthfulQA | General | 78 [65, 90] | 77 [67, 84] | 0.31 | 0.20 |
| + task | 99 [97, 100] | 98 [97, 100] | 0.04 | 0.29 |
| RSA | Mutual NN | CKA (linear) | CKA (RBF) | Null RSA | Null CKA | (RSA, ) | |
|---|---|---|---|---|---|---|---|
| 0.25 | 0.77 | 0.44 | 0.87 | 0.91 | 0.00 | 0.01 | 0.58 |
| 0.5 | 0.65 | 0.37 | 0.84 | 0.87 | 0.00 | 0.01 | 0.62 |
| 0.75 | 0.49 | 0.35 | 0.84 | 0.86 | 0.00 | 0.02 | 0.65 |
| 0.9 | 0.46 | 0.35 | 0.85 | 0.86 | 0.00 | 0.03 | 0.66 |
| 1 | 0.48 | 0.31 | 0.76 | 0.80 | 0.00 | 0.02 | 0.67 |
| Normalized | Raw | Kernel minus linear | ||||
|---|---|---|---|---|---|---|
| Linear | Kernel | Linear | Kernel | Normalized | Raw | |
| 0.25 | 0.600 | 0.647 | 0.601 | 0.647 | 0.047 [0.045, 0.049] | 0.046 [0.044, 0.048] |
| 0.5 | 0.447 | 0.500 | 0.449 | 0.501 | 0.053 [0.050, 0.055] | 0.052 [0.049, 0.054] |
| 0.75 | 0.360 | 0.417 | 0.359 | 0.415 | 0.056 [0.054, 0.059] | 0.056 [0.054, 0.059] |
| 1 | 0.313 | 0.373 | 0.310 | 0.368 | 0.060 [0.058, 0.062] | 0.058 [0.056, 0.060] |
| Nearest | Random | Nearest | Random | Nearest | Random | Global | |
|---|---|---|---|---|---|---|---|
| 0.25 | 0.273 | 0.510 | 0.167 | 0.258 | 0.155 | 0.195 | 0.163 |
| 0.5 | 0.414 | 0.658 | 0.285 | 0.403 | 0.262 | 0.312 | 0.273 |
| 0.75 | 0.484 | 0.802 | 0.441 | 0.621 | 0.380 | 0.449 | 0.397 |
| 1 | 0.547 | 0.833 | 0.524 | 0.674 | 0.438 | 0.505 | 0.455 |
| Tool | Runs | Coverage | Coverage, within pair | Fit on tool prompts | Leverage | Cosine to native |
|---|---|---|---|---|---|---|
| Refusal | 3,500 | 0.67 | 0.75 | 0.74 | 0.52 | 0.72 |
| ARC | 3,690 | 0.71 | 0.73 | 0.67 | 0.40 | 0.82 |
| TruthfulQA | 4,510 | 0.68 | 0.80 | 0.67 | 0.50 | 0.78 |
| Sentiment | 4,510 | 0.45 | 0.47 | 0.41 | 0.36 | 0.50 |
| Refusal | ARC | TruthfulQA | Sentiment | |||||
|---|---|---|---|---|---|---|---|---|
| Map | Rec. | Rec. | Rec. | Gap | ||||
| Narrow (GSM8K) | 0.46 | 2 | 0.31 | 65 | 0.32 | 15 | 0.30 | 32.0 |
| General | 0.88 | 54 | 0.58 | 16 | 0.64 | 77 | 0.63 | 7.8 |
| + smallest coverage | 0.97 | 103 | 0.65 | 43 | 0.77 | 90 | 0.67 | 4.1 |
| + other datasets | 0.96 | 88 | 0.77 | 73 | 0.73 | 82 | 0.87 | 0.9 |
| + task | 0.995 | 102 | 0.83 | 88 | 0.93 | 98 | 0.90 | 0.2 |
| Target’s own | Carried | ||||
|---|---|---|---|---|---|
| Target | No ablation | Single | Per block | Single | Per block |
| Llama-3.2-1B | 73 | 36 | 14 | 47 | 2 |
| Qwen2.5-1.5B | 91 | 5 | 6 | 39 | 40 |
| Gemma-2-2B | 94 | 0 | 0 | 8 | 0 |
| Phi-3-mini | 94 | 12 | 48 | 88 | 33 |
| Qwen2.5-7B | 89 | 8 | 0 | 37 | 2 |
| Tool | Pairs | General | General + task | Mapped random | |
|---|---|---|---|---|---|
| ARC | base base | 6 | 9 | 61 | 190 |
| base instruct | 32 | 24 | 88 | 135 | |
| base own instruct | 4 | 44 | 80 | 132 | |
| instruct base | 20 | 63 | 68 | 185 | |
| instruct own base | 2 | 34 | 92 | 171 | |
| TruthfulQA | base base | 12 | 89 | 100 | 8 |
| Built in the target | |||||
|---|---|---|---|---|---|
| Tool | Carried | PC, zero-shot sign | PC, oracle sign | Labeled | |
| Refusal (per class) | 16 | 103 | 92 (sign from categories) | 97 | |
| 32 | 103 | – | – | 95 | |
| 64 | 102 | – | – | 97 | |
| 256 | 102 | 97 (sign from categories) | 100 | ||
| ARC (questions) | 10 | 44 | 76 | 67 | 70 |
Interpretable ML
Interpretable machine learning develops models and explanations that make predictions, evidence, and decision processes understandable to people.
See all 325 papers