Efficient Auto-Interpretability of AI Models in Biology
Authors: Piotr Jedryszek, Oliver M. Crook
Organizations: Department of Biology, University of Oxford, Oxford, UK · Evolvere Biosciences, London, UK · Kavli Institute for Nanoscience Discovery, University of Oxford, Oxford, UK · Department of Chemistry, University of Oxford, Oxford, UK
Sparse autoencoders (SAEs), and other interpretability methods could turn AI models in Biology and other fields into engines of scientific discovery by explaining the superhuman capabilities of those models. However, a latent is only useful if we know three things: whether it is coherent, whether it can be described, and whether that description has predictive power. These questions are routinely conflated. We assemble them into a single pipeline and report the practical innovations each stage required. First, cross-seed dictionary stability prioritises which latents are worth spending resources to investigate. Second, an intruder-detection task asks whether a latents activating examples share a recognizable pattern. Third, a separate pass proposes a candidate biological description which we convert into falsifiable predictions which can be tested in silico. Deployed on the Boltz-1 Pairformer trunk, stability prioritisation finds interpretable latents using about 4.4 times fewer latent evaluations each, and at 5.2 times lower measured cost, while recovering over half of them, and the external check shows the surfaced motifs are significantly enriched for their claimed annotations. The results also suggest a possible tension: the cross- seed stability might be selecting for some types of features, like structure-related ones, much more than others, such as function-related features.
Figures & tables
Figure 1: Cross-seed stability as an LLM-free interpretability proxy. (A) In the discovery band (activation-frequency percentiles 20 to 60), a stability ≥0.5 filter nearly doubles the supervised interpretable rate (1.97 × , n=4,881 ); the LLM deployment points in the same direction (Fig. 2 ). (B) Concept-winner F1 vs. stability ( ρ=0.51 , 95% CI 0.39 to 0.59 , n=735 ). Amino-acid identities recur across seeds (56%) far more than secondary structure (16%) or SwissProt function (15%).
Figure 2: Low-frequency discovery landscape. (A) Distribution of intruder accuracy across the panel, stacked by stability band: judging only the 12% of latents that clear stability ≥0.5 costs about 4.4 × fewer latent evaluations per interpretable latent and recovers 53% (nine of 17) of them ( p=8×10−6 ). (B) Individual discovered features by accuracy and stability. The small set of complex functional motifs (catalytic residues, metal-coordination clusters) happens to fall at low stability, a pattern we examine and hedge appropriately in Section 3.3 .
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Stable-only ( n=17 )
Full panel ( n=141 )
Intruder requests
684
5,640
Describe requests
17
141
Input tokens
0.76 M
6.24 M
Output tokens
0.24 M
2.59 M
Batch cost
$2.94
$28.76
Interpretable latents found
9
17
Appendix
Table 1: Measured cost of the 141-latent discovery sweep , and of the stable-only arm the filter would have judged instead. Every figure is measured, not estimated: requests are exact, and token counts are read back per request from the Message Batches API after the run. Dollar figures use Claude Sonnet 4.6 batch pricing (half the synchronous rate). The describe pass is one request per latent and was not re-run, since descriptions are independent of intruder construction. Cost per find is the efficiency number; the ratio of total spend is not, because the filtered arm also returns fewer finds.
Figure 3: Sensitivity to the stability cut and the interpretability bar. (A) Precision, recall and the fraction of the panel judged, as a function of the stability threshold, at the 0.5 interpretability bar. The shaded band marks the 0.35 to 0.70 range quoted in the text and the dotted line marks the 0.5 cut used. (B) Latent evaluations per interpretable latent found, at interpretability bars of 0.4, 0.5 and 0.6. The saving over the unfiltered panel holds under all three.
Feature Group
n
Mean Acc.
Pass Rate
AA Identity (positive control)
20
0.97
20/20
Contextual concepts
6
0.48
5/6
Random latents
13
0.29
4/13
Scrambled (negative control)
120 trials
0.17
n.s. ( p=0.85 )
Appendix
Table 2: Calibrated intruder detection across control panels (chance =0.20 ). Pass rate is the one-sided binomial test against p0=0.2 at α=0.05 , which is a weaker criterion than the accuracy ≥0.5 bar the discovery sweep uses. The scrambled control is unaffected by the intruder construction of Appendix A.7 , since it draws all five windows from different features and so re-marks nothing.
Figure 4: Reasoning-budget ablation ladder. Identical positive-control latents across prompt architectures: an immediate-answer restriction (16 tokens) scores 0.17 , and raising the output-token allowance to 512 tokens recovers 0.97 . Across the three calibrated variants shown, alternative wording ( 0.96 ) and metadata inclusion ( 0.96 ) add negligible variance ( Δ<0.01 ). A fourth, independently written paraphrase reaches 0.85 (Section B ).
Figure 5: Convergent validity against ground truth. (A) Calibrated intruder accuracy tracks an independent, label-only amino-acid-precision axis ( ρ=0.52 ); a 2×2 split at the precision threshold gives OR =140 , with only one of 111 low-precision latents judged interpretable. This precision analysis covers the 136 of 141 sweep latents that fire on at least 20 residues; the five too-sparse latents excluded here include two of the 17 interpretable finds. (B) Firing enrichment for the metal-coordination and catalytic motifs against curated SwissProt residue annotations, conditional on residue identity, with a by-protein bootstrap 95% CI. Grey bars show the old all-residue background. The HRD-motif histidine hits the exact His-Arg-Asp motif on nine of 23 annotated sites and is enriched 114× among histidines, and within HRD-containing proteins it fires only on that histidine. The zinc-coordination latents reach 6 to 13× among histidines and cysteines.
Figure 6: Subspace-level stability, and its positive-control ladder. For each feature, the co-firing decoder group is located independently in all three seeds and compared by mean cos2 of principal angles (1 = identical subspace). (A) The five metal-coordination and catalytic motifs (0.13) score below uninterpretable latents (0.23) and far below amino-acid controls (0.36) run through the identical pipeline. (B) Positive-control ladder: a planted rotated subspace (1.00) and a genuinely stable atom group (0.85) confirm the metric has range and the motifs sit near the floor, not near either.
While sparse autoencoders (SAEs) have generated significant excitement, a series of negative results have added to skepticism about their usefulness. Here, we establish a conceptual distinction that reconciles competing narratives surrounding SAEs. We argue that even if SAEs may be less effective for acting on known concepts, SAEs are especially powerful tools for discovering unknown concepts. This distinction separates existing negative results from positive results, and suggests several classes of SAE applications. Specifically, we outline use cases for SAEs in (i) ML interpretability, explainability, fairness, auditing, and safety, and (ii) social and health sciences.
Sparse Autoencoders (SAEs) are widely used to interpret large language models by decomposing activations into sparse, human-understandable features, but scaling to large dictionaries exposes fundamental challenges. Systematic studies reveal pervasive feature splitting that fragments coherent concepts into non-atomic latents and widespread feature absorption that creates arbitrary exceptions in general features, severely compromising latent reliability. These issues stem from inconsistent latent assignment across samples: without cross-sample constraints, per-sample optimization often allows a single underlying concept to be inconsistently distributed across multiple redundant or interfering latents. To address this, we introduce C2R (Cross-sample Consistency Regularization). C2R explicitly encourages that each semantic feature is consistently represented by a unified latent across the batch by penalizing the co-activation of directionally similar latents. Comprehensive evaluation demonstrates that C2R effectively mitigates both splitting and absorption while, crucially, preserving reconstruction fidelity, providing a principled solution that enhances latent interpretability without degrading model performance. Source code is available at https://github.com/hr-jin/Cross-sample-Consistency-Regularization.
Haoran Jin, Xiting Wang, Shijie Ren +2
University of Science and Technology of China · State Key Laboratory of Cognitive Intelligence · Gaoling School of Artificial Intelligence, Renmin University of China
Sparse Autoencoders (SAEs) have been seen as a promising avenue for exploring the internals of Large Language Models (LLMs) and for steering model output generation. When AxBench - a model steering benchmark - was introduced in Wu et al. (2025), SAEs did not seem to live up to their original hype due to poor steering performance relative to a set of simple baselines. This work serves as a partial rebuttal for Sparse Autoencoders and suggests that the results of Wu et al. (2025) did not do them full justice. We find that Sparse Autoencoders can, in fact, perform close to on par with the reference LoRA performance on the AxBench benchmark, when features are selected and labelled with our supervised pipeline. We also find that our pipeline selects features that are surprisingly causal of their identified labels when using only its interpretability-based components. Lastly, we present evidence that high sparsity (low l0) may not be crucial for successful steering based on interpretability, which is in contrast to the earlier findings in Wang et al. (2025).