When can an interpretability claim be trusted? Stability and certification of feature dimensionality, ridge leverage and the feature recovery rate
Organizations: Independent researchers
Abstract
Interpretability tools increasingly support safety claims, such as that a sparse dictionary has recovered most of a model's features. Such claims rest on measurements with arbitrary thresholds, and small changes to the model weights, a different training seed or a different threshold can reverse them. We study two widely used measurements in toy models and synthetic dictionaries with known ground truth: the feature dimensionality of Elhage et al. and the feature recovery rate (FRR) with a cosine threshold. We characterize exactly where feature dimensionality is discontinuous and replace it with a regularized (ridge) version with sharp stability bounds: the ridge leverage score changes by at most times the operator-norm change of the weights, and this constant cannot be improved; for the regularized dimensionality the stability constant grows like in the scale , with explicit upper and lower bounds. We also prove certificates stating when a claim "feature is recovered" must survive a given change of the dictionary. The certificates held in every test and are informative for small perturbations, but certify no feature non-trivially across independently trained dictionaries; at the standard threshold, whether a feature counts as recovered changed with the training seed for 55 percent of features. Experiments use toy models only. The proofs, code and text were drafted by AI agents directed by the authors and checked by an independent blind AI reviewer and by numerical tests; the authors take full responsibility for the content.
Figures & tables
| (a) What is new | (b) Label | (c) Status | (d) Novelty verdict |
|---|---|---|---|
| Preliminaries (Sections 3 and 6.1 ) | |||
| Elementary facts P1–P4; the slopes of the family and their specialisations; the line-angle lemmas, including the triangle inequality for the line angle. | Lemmas 3.19 – 3.22 , 3.24 ; Cor. 3.25 ; Lemmas 6.2 , 6.3 – 6.6 | proved, blind-checked ( ) | not sent to the search (classical facts and explicit computations; no novelty claimed). |
| T1: where feature dimensionality is discontinuous (Section 4 ) | |||
| Exact discontinuity set of , with jump of and of the sum rule; local Lipschitz modulus of order , and the order is needed. | Thm 4.1 | proved, blind-checked ( ) | Discontinuity (a), (b): at risk . Scherlis et al. [ 12 ] remark informally that a vanishing column gives ; the exact set is routine once stated. Lipschitz bound (c) and lower bound (d): open ; no perturbation bound for found. |
| The best coefficient of the local modulus lies in . | Rem. 4.2 | proved, blind-checked ( ) | Open ; no perturbation bound for found. |
| The precise implication that fails in an informal remark of Scherlis et al. [ 12 ] . | Rem. 4.3 | proved, check pending ( ) | Not separately audited; it restates Thm 4.1 (b) ( at risk , as above). |
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| Symbol | Meaning | Where |
|---|---|---|
| ; | dimensions, numbers of features and atoms; | D0, D7 |
| , , , | inner product, norm, normalisation, cosine | D0 |
| ; | operator, Frobenius, nuclear norm; singular values | D0 |
| smallest, largest, -th eigenvalue of a symmetric matrix | D0 | |
| , , , , | identity, orthogonal group, cardinality, indicator, standard basis vector | D0 |
| ; ; ; | dummy variables, matrices, functions, angles (local) | § 1 |
Interpretable ML
Interpretable machine learning develops models and explanations that make predictions, evidence, and decision processes understandable to people.
See all 325 papers