Structure is not mechanism: high-gain gated-FFN rows across text and genomic foundation models

Alexandros Tzanakakis; Aris Karatzikos; Ilias Georgakopoulos-Soares

q-bio.GNSep 13, 2026

Structure is not mechanism: high-gain gated-FFN rows across text and genomic foundation models

Authors: Alexandros Tzanakakis, Aris Karatzikos, Ilias Georgakopoulos-Soares

Organizations: Division of Pharmacology and Toxicology, College of Pharmacy, The University of Texas at Austin, Dell Pediatric Research Institute, Austin, TX, USA · Chandra Family Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX, USA · Department of Computer Science, College of Natural Sciences, The University of Texas at Austin, Austin, TX, USA

Abstract

A small number of unusually high-gain parameters can exert disproportionate effects in transformer language models, but whether analogous structures recur in genomic foundation models and whether structural geometry determines functional importance remains unknown. We analyzed high-gain rows in gated feed-forward networks across text and genomic foundation models, including a frozen 22-model causal census. Computing an associated bilinear weight operator exactly, without a diagonal approximation, we tested whether structural extremeness is a transferable mechanism. Activation-derived candidates were functionally enriched relative to random and top-norm same-layer controls, yet neither spectral concentration nor operator magnitude predicted causal effect size, and these associations vanished within the endpoint-homogeneous text-decoder subset. A within-layer sweep of 36 rows in one genomic and one text decoder resolved this into two regimes: below the detector's acceptance threshold the ratio carried no positive information about causal damage, whereas above it the ratio ordered rows strongly but did not grade severity as a dose-response. The same sweep revealed a second individually catastrophic row invisible to a one-candidate-per-model census, and non-additive damage among co-located critical rows. Case studies showed divergent causal organizations: a robust super-additive pair interaction in DNABERT-2, and in GENERator a sharply position-localized dependence in which preserving or restoring the row's beginning-of-sequence contribution rescued essentially all native-loss damage. High-gain gated-FFN rows are therefore a recurrent architectural phenotype whose structural prominence acts as an enrichment signal, not a calibrated measure of functional criticality or a specification of causal organization. Enrichment is general, but the mechanism is model-specific.

Figures & tables

Figure 1: Structural properties of high-gain gated-FFN rows. (A) Retrospective diagonal-proxy calibration for the published super-weight-containing rows in Llama-7B, Mistral-7B, and OLMo-7B. (B) Spectral concentration $q_{1}$ across the 23-model structural panel; the Phi-3 value is the median across its six frozen rows. Evo2-7B did not pass the predefined activation-ratio threshold and is shown without a $q_{1}$ value. (C) Candidate $q_{1}$ compared with five random same-layer controls (top) and the five highest-Frobenius-norm rows in the same layer after excluding the candidate in the same layer (bottom). The random-control gap is candidate $q_{1}$ minus the mean control $q_{1}$ ; the top-norm margin is candidate $q_{1}$ minus the maximum control $q_{1}$ .

Figure 2: Causal effects of frozen high-gain candidates across the 22-model census. (A) Signed relative native-loss change after partial suppression ( $\varepsilon$ = 0.5) for each candidate and five random same-layer controls. (B) Corresponding effects under full ablation ( $\varepsilon$ = 1.0). Diamonds denote candidates, grey points individual controls, and horizontal bars within-model control medians. (C) Candidate spectral concentration $q_{1}$ versus signed full-ablation native-loss effect. (D) Candidate-minus-median-top-norm-control causal gap at both intervention strengths. Symmetric-log axes are used where necessary to display negative, near-zero, and large positive effects on the same scale.

Figure 3: Activation ratio and single-row ablation effects across rows within a layer. Damage from deleting a single row (y-axis) is plotted against that row’s activation ratio (x-axis) for 36 rows sampled within one layer of the following models. (A) GENERator-EUK-3B, layer 4, and (B) SmolLM2-1.7B, layer 7. Stars indicate each model’s main candidate row. The shaded region denotes rows satisfying the activation-ratio threshold of 5.

Figure 4: DNABERT-2 multi-row causal-response analysis. (A) Full-ablation effects of L9/r264 and L9/r294 individually and jointly. (B) Pairwise epistasis at $\varepsilon$ = 0.5 and $\varepsilon$ = 1.0, defined as the joint effect minus the sum of the singleton effects; these measurements come from a standalone direct pair-ablation experiment. (C) Held-out predictive performance of observer families F0–F3 across multi-row intervention conditions. (D) Fitted F3 pairwise interaction coefficients across all 45 row pairs.

Figure 5: Beginning-of-sequence mediation of GENERator EUK row L4/r2371. (A) Spectral concentration $q_{1}$ of row L4/r2371. (B) Beginning-of-sequence attention and activation measurements at position 0; these measurements establish co-localization but do not test whether the row causes the attention sink. (C) Native language-model loss under intact, full-ablation, BOS-only ablation, BOS-preservation, BOS-restoration, and matched non-BOS-restoration interventions. Removing the row contribution only at BOS reproduced nearly all full-ablation damage, whereas preserving or restoring the BOS contribution produced approximately complete rescue; restoration at the matched non-BOS position produced essentially none. (D) Generated GC fraction under the same position-specific interventions. Error intervals denote paired 95% bootstrap intervals.

Genomic Foundation Models

See all 17 papers

Jul 21, 2026

Causal dictionary learning reveals and validates transcription-factor binding features in genomic language models

Genomic language models achieve strong performance across regulatory-genomics tasks, yet what these models internally represent remains opaque, and the field lacks a principled procedure for verifying that an apparent

concept'' inside a model is real rather than an artifact of sequence composition. We introduce a framework that combines sparse dictionary learning with causal intervention to extract, validate, and causally test interpretable features in genomic foundation models. Training top-$k$ sparse autoencoders on the hidden activations of two architecturally distinct models, Nucleotide Transformer ($6$-mer tokenization) and DNABERT-2 (byte-pair encoding), we recover thousands of monosemantic features that map to transcription-factor (TF) sequence motifs. We show that the naive validation of such features against position weight matrices is severely confounded by GC composition and repetitive elements, producing hundreds of spurious

TF features'', and we develop a composition-matched, binding-resolved protocol that removes these confounds. Critically, we move beyond correlation: by ablating individual dictionary directions during the model's forward pass and measuring the induced shift in the model's own predictive distribution, we establish that specific features are causally used to represent cell-type-specific TF binding, not merely motif presence. Across three transcription factors (CTCF, GATA1, REST) and both architectures, causally validated binding features emerge reproducibly (

7

14

15

tested features per condition), while two classes of negative control, scrambled binding labels and randomly selected features, yield no detectable signal. The framework is purely computational, uses only public data, and provides a reusable standard for interpretability claims in genomic deep learning.

Sarwan Ali

Columbia University Irving Medical Center, New York, USA

LLM Interpretability Genomics Causal Interpretability Genomic Foundation Models

Ask PDF

Mar 2, 2026

Causal Circuit Tracing Reveals Distinct Computational Architectures in Single-Cell Foundation Models: Inhibitory Dominance, Biological Coherence, and Cross-Model Convergence

Motivation: Sparse autoencoders (SAEs) decompose foundation model activations into interpretable features, but causal feature-to-feature interactions across network depth remain unknown for biological foundation models. Results: We introduce causal circuit tracing by ablating SAE features and measuring downstream responses, and apply it to Geneformer V2-316M and scGPT whole-human across four conditions (96,892 edges, 80,191 forward passes). Both models show approximately 53 percent biological coherence and 65 to 89 percent inhibitory dominance, invariant to architecture and cell type. scGPT produces stronger effects (mean absolute d = 1.40 vs. 1.05) with more balanced dynamics. Cross-model consensus yields 1,142 conserved domain pairs (10.6x enrichment, p < 0.001). Disease-associated domains are 3.59x more likely to be consensus. Gene-level CRISPRi validation shows 56.4 percent directional accuracy, confirming co-expression rather than causal encoding.

Ihor Kendiukhov

Department of Computer Science, University of Tübingen, Tübingen, Germany

Mechanistic Interpretability Causal Interpretability

Ask PDF

Mar 12, 2026

Exhaustive Circuit Mapping of a Single-Cell Foundation Model Reveals Massive Redundancy, Heavy-Tailed Hub Architecture, and Layer-Dependent Differentiation Control

Mechanistic interpretability of biological foundation models has relied on selective feature sampling, pairwise interaction testing, and observational trajectory analysis. Each of these can introduce systematic bias. Here we present three experiments that address these limitations through exhaustive circuit tracing, higher order combinatorial ablation, and causal trajectory steering in Geneformer, a transformer based single cell foundation model. First, exhaustive tracing of all 4065 active sparse autoencoder features at layer 5 yields 1393850 significant downstream edges, a 27 fold expansion over selective sampling. This reveals a heavy tailed hub distribution in which 1.8 percent of features account for disproportionate connectivity and 40 percent of the top 20 hubs lack biological annotation. These results indicate systematic annotation bias in prior selective analyses. Second, three way combinatorial ablation across 8 feature triplets shows that redundancy deepens monotonically with interaction order, with a three way ratio of 0.59 versus a pairwise ratio of 0.74, and with zero synergy. This confirms that the model architecture is subadditive at all tested orders. Third, trajectory guided feature steering establishes a causal link between layer position and differentiation directionality. Late layer features at L17 consistently push cell states toward maturity, with fraction positive equal to 1.0. Early and mid layer features at L0 and L11 mostly push away from maturity, with fraction positive ranging from 0.00 to 0.58. Together these results move from correlation toward causal evidence for layer dependent control of cell state.

Ihor Kendiukhov

Department of Computer Science, University of Tübingen, Germany

Mechanistic Interpretability Transformer Interpretability Single-Cell Transcriptomics

Ask PDF

Abstract

Genomic Foundation Models

Explore similar work

Causal dictionary learning reveals and validates transcription-factor binding features in genomic language models

Causal Circuit Tracing Reveals Distinct Computational Architectures in Single-Cell Foundation Models: Inhibitory Dominance, Biological Coherence, and Cross-Model Convergence

Exhaustive Circuit Mapping of a Single-Cell Foundation Model Reveals Massive Redundancy, Heavy-Tailed Hub Architecture, and Layer-Dependent Differentiation Control