q-bio.QMAug 27, 2026

Efficient Auto-Interpretability of AI Models in Biology

Authors: Piotr Jedryszek, Oliver M. Crook

Organizations: Department of Biology, University of Oxford, Oxford, UK · Evolvere Biosciences, London, UK · Kavli Institute for Nanoscience Discovery, University of Oxford, Oxford, UK · Department of Chemistry, University of Oxford, Oxford, UK

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Sparse autoencoders (SAEs), and other interpretability methods could turn AI models in Biology and other fields into engines of scientific discovery by explaining the superhuman capabilities of those models. However, a latent is only useful if we know three things: whether it is coherent, whether it can be described, and whether that description has predictive power. These questions are routinely conflated. We assemble them into a single pipeline and report the practical innovations each stage required. First, cross-seed dictionary stability prioritises which latents are worth spending resources to investigate. Second, an intruder-detection task asks whether a latents activating examples share a recognizable pattern. Third, a separate pass proposes a candidate biological description which we convert into falsifiable predictions which can be tested in silico. Deployed on the Boltz-1 Pairformer trunk, stability prioritisation finds interpretable latents using about 4.4 times fewer latent evaluations each, and at 5.2 times lower measured cost, while recovering over half of them, and the external check shows the surfaced motifs are significantly enriched for their claimed annotations. The results also suggest a possible tension: the cross- seed stability might be selecting for some types of features, like structure-related ones, much more than others, such as function-related features.

Figures & tables

Figure 1: Cross-seed stability as an LLM-free interpretability proxy. (A) In the discovery band (activation-frequency percentiles 20 to 60), a stability ≥0.5\geq 0.5 filter nearly doubles the supervised interpretable rate (1.97 ×\times , n=4,881n=4{,}881 ); the LLM deployment points in the same direction (Fig. 2 ). (B) Concept-winner F1 vs. stability ( ρ=0.51\rho=0.51 , 95% CI 0.390.39 to 0.590.59 , n=735n=735 ). Amino-acid identities recur across seeds (56%) far more than secondary structure (16%) or SwissProt function (15%).
Figure 2: Low-frequency discovery landscape. (A) Distribution of intruder accuracy across the panel, stacked by stability band: judging only the 12% of latents that clear stability ≥0.5\geq 0.5 costs about 4.4 ×\times fewer latent evaluations per interpretable latent and recovers 53% (nine of 17) of them ( p=8×10−6p=8\times 10^{-6} ). (B) Individual discovered features by accuracy and stability. The small set of complex functional motifs (catalytic residues, metal-coordination clusters) happens to fall at low stability, a pattern we examine and hedge appropriately in Section 3.3 .
Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

 Stable-only ( n=17n=17 )Full panel ( n=141n=141 )
Intruder requests6845,640
Describe requests17141
Input tokens0.76 M6.24 M
Output tokens0.24 M2.59 M
Batch cost$2.94$28.76
Interpretable latents found917
Appendix
Table 1: Measured cost of the 141-latent discovery sweep , and of the stable-only arm the filter would have judged instead. Every figure is measured, not estimated: requests are exact, and token counts are read back per request from the Message Batches API after the run. Dollar figures use Claude Sonnet 4.6 batch pricing (half the synchronous rate). The describe pass is one request per latent and was not re-run, since descriptions are independent of intruder construction. Cost per find is the efficiency number; the ratio of total spend is not, because the filtered arm also returns fewer finds.
Figure 3: Sensitivity to the stability cut and the interpretability bar. (A) Precision, recall and the fraction of the panel judged, as a function of the stability threshold, at the 0.5 interpretability bar. The shaded band marks the 0.35 to 0.70 range quoted in the text and the dotted line marks the 0.5 cut used. (B) Latent evaluations per interpretable latent found, at interpretability bars of 0.4, 0.5 and 0.6. The saving over the unfiltered panel holds under all three.
Feature GroupnnMean Acc.Pass Rate
AA Identity (positive control)200.9720/20
Contextual concepts60.485/6
Random latents130.294/13
Scrambled (negative control)120 trials0.17n.s. ( p=0.85p=0.85 )
Appendix
Table 2: Calibrated intruder detection across control panels (chance =0.20=0.20 ). Pass rate is the one-sided binomial test against p0=0.2p_{0}=0.2 at α=0.05\alpha=0.05 , which is a weaker criterion than the accuracy ≥0.5\geq 0.5 bar the discovery sweep uses. The scrambled control is unaffected by the intruder construction of Appendix A.7 , since it draws all five windows from different features and so re-marks nothing.
Figure 4: Reasoning-budget ablation ladder. Identical positive-control latents across prompt architectures: an immediate-answer restriction (16 tokens) scores 0.170.17 , and raising the output-token allowance to 512 tokens recovers 0.970.97 . Across the three calibrated variants shown, alternative wording ( 0.960.96 ) and metadata inclusion ( 0.960.96 ) add negligible variance ( Δ<0.01\Delta<0.01 ). A fourth, independently written paraphrase reaches 0.850.85 (Section B ).
Figure 5: Convergent validity against ground truth. (A) Calibrated intruder accuracy tracks an independent, label-only amino-acid-precision axis ( ρ=0.52\rho=0.52 ); a 2×22\times 2 split at the precision threshold gives OR =140=140 , with only one of 111 low-precision latents judged interpretable. This precision analysis covers the 136 of 141 sweep latents that fire on at least 20 residues; the five too-sparse latents excluded here include two of the 17 interpretable finds. (B) Firing enrichment for the metal-coordination and catalytic motifs against curated SwissProt residue annotations, conditional on residue identity, with a by-protein bootstrap 95% CI. Grey bars show the old all-residue background. The HRD-motif histidine hits the exact His-Arg-Asp motif on nine of 23 annotated sites and is enriched 114×114\times among histidines, and within HRD-containing proteins it fires only on that histidine. The zinc-coordination latents reach 66 to 13×13\times among histidines and cysteines.
Figure 6: Subspace-level stability, and its positive-control ladder. For each feature, the co-firing decoder group is located independently in all three seeds and compared by mean cos⁡2\cos^{2} of principal angles (1 = identical subspace). (A) The five metal-coordination and catalytic motifs (0.13) score below uninterpretable latents (0.23) and far below amino-acid controls (0.36) run through the identical pipeline. (B) Positive-control ladder: a planted rotated subspace (1.00) and a genuinely stable atom group (0.85) confirm the metric has range and the motifs sit near the floor, not near either.
More in this topic

Explainability Evaluation

Explainability evaluation tests whether AI explanations are faithful, stable, understandable, useful, and reliable across models and settings.

See all 322 papers

Explore similar work

CardsList

  1. Position: Use Sparse Autoencoders to Discover Unknowns

    Jun 30, 2025Kenny Peng, Rajiv Movva, Jon Kleinberg +2Explainable Artificial IntelligenceSparse Autoencoders

    1Cornell University · 2UC Berkeley

  2. C2^{2}R: Cross-sample Consistency Regularization Mitigates Feature Splitting and Absorption in Sparse Autoencoders

    Jun 29, 2026Haoran Jin, Xiting Wang, Shijie Ren +2LLM InterpretabilityRepresentation Disentanglement

    University of Science and Technology of China · State Key Laboratory of Cognitive Intelligence · Gaoling School of Artificial Intelligence, Renmin University of China

  3. Steering LLMs? Actually, Sparse Autoencoders can outperform simple baselines

    May 29, 2026Mikkel Godsk Jørgensen, Lars Kai HansenLanguage Model SteeringLLM Interpretability

    DTU Compute · Technical University of Denmark