Systematic Evaluation of Single-Cell Foundation Model Interpretability Reveals Attention Captures Co-Expression Rather Than Unique Regulatory Signal

Ihor Kendiukhov

Analysis	Edge Type	Dataset	Key AUROC
Scaling behavior	Attention	scGPT kidney	0.86–0.93
Controlled composition	Correlation	TS kidney	0.52–0.56
Residualization	Correlation	DLPFC brain	0.73 (resid.)
Degree-preserving null	Correlation	DLPFC brain	0.69 (null)
Perturbation-first (corr.)	Correlation	Replogle K562	0.696
Perturbation-first (attn.)	Attention	Replogle K562	0.704

Appendix

Table S1 : Edge type used in each analysis. “Attention” denotes Geneformer V2-316M attention-derived edges; “Correlation” denotes Spearman correlation from the same control cells.

Dataset	Modality / Config	DE Thresh.	$n$	Notes
Replogle K562	CRISPRi (baseline)	LFC $>$ 0.1	44	Mann-Whitney, HVG=1000, $N_{\text{ctrl}}$ =500
Replogle K562	CRISPRi (primary)	LFC $>$ 0.5	151	Welch $t$ , HVG=2000, $N_{\text{ctrl}}$ =2000
Replogle K562	CRISPRi (attention)	LFC $>$ 0.5	280	Welch $t$ , HVG=2000; more genes evaluable with attention tokenization
Adamson K562	CRISPRa	LFC $>$ 0.5	77	K562, activation modality
Shifrut T cells	CRISPRi	LFC $>$ 0.1	7	Primary human T cells; weak DE effects require lenient threshold
Replogle RPE1	CRISPRi	LFC $>$ 0.5	1,251	hTERT-RPE1; perturbation genes forced into HVG

Appendix

Table S2: Perturbation sample counts across datasets and configurations.

Figure S1 : Metric-dependent scaling behavior in scGPT attention-derived GRN recovery (kidney). TRRUST F1 with 95% confidence intervals across three model tiers (small/medium/large) and cell counts (200/1,000/3,000).

Figure S2 : Retrieval collapse with scaling (scGPT kidney). Observed true positives versus random-expected true positives for attention-derived GRNs.

Tier	Jaccard (200)	Jaccard (1,000)	Spearman (200)	Spearman (1,000)
Small	0.572	0.305	0.211	-0.065
Medium	0.562	0.293	0.183	-0.110
Large	0.536	0.285	0.083	-0.130

Appendix

Table S3 : Between-seed robustness of inferred edge sets (scGPT kidney).

Figure S3 : Scaling runs: composition diversity proxy. Across the scGPT kidney scaling grid, the number of observed cell types in the sampled cells increases with

N

and is anti-correlated with GRN recovery.

Figure S4 : K-sensitivity and continuous AUROC for scaling runs. (A) F1 vs

K

at different cell counts. (B) Continuous AUROC improves monotonically with cell count (

0.858\to 0.925\to 0.934

). (C) Fraction of runs showing degradation across

K

values.

Figure S5 : Controlled-composition scaling. (A) Single cell type: no significant degradation with

N

. (B) Fixed composition: stable AUROC. (C) Increasing heterogeneity at fixed

N

: AUROC increases.

Figure S6 : Non-additivity in mediation analysis. Absolute total effect, residual non-additivity, and lower-bound interaction magnitude per run-pair.

Figure S7 : Ranking certificate fragility. Mean certified pair fraction versus structural multiplier

\lambda

.

Figure S8 : Detectability phase diagrams. Different regimes where attention-like versus intervention-like signals become detectable.

Figure S9 : Real data detectability calibration. Bootstrap distribution of projected relative cell requirements.

Figure S10 : Cross-tissue consistency variability. Scatter plots of component-level effects between tissue pairs.

Figure S11 : Perturbation-first validation (Replogle CRISPRi K562). Per-gene AUROC for predicting DE targets from correlation-based edge scores under primary parameterization.

Figure S12 : Perturbation-first sensitivity analysis (27 parameter combinations). Mean AUROC varies with (A) number of control cells, (B) gene universe size, and (C) LFC threshold.

Figure S13 : Attention-derived edges are indistinguishable from correlation on perturbation-first prediction. Geneformer V2-316M attention (L13) vs. correlation-based edges (

n=280

).

Figure S14 : Cross-species edge score conservation. Scatter plot of Spearman

\rho

for 25,876 matched TF–target edges between human and mouse lung.

Top $k$	Observed	Expected	Fold
100	26	0.1	484 $\times$
500	153	1.3	114 $\times$
1,000	289	5.4	54 $\times$
5,000	1,094	134.2	8.2 $\times$

Appendix

Table S4 : Top-

k

overlap between human and mouse edge rankings.

Figure S15 : Per-TF conservation of regulatory targets across species. (A) Horizontal bar chart of cross-species Spearman

\rho

for each TF’s target set (green:

\rho\geq 0.5

, orange:

0\leq\rho<0.5

, red:

\rho<0

). Lineage-specifying TFs (XBP1, EPAS1, RUNX3) show near-perfect conservation (

\rho>0.88

), while signaling-responsive TFs (MAX, BACH1, CTNNB1) show poor or negative conservation. (B) Conservation versus number of shared orthologous edges. Point size reflects mean absolute attention weight (

|\rho|

) in human. High-conservation TFs tend to have more shared edges, but low-conservation outliers (MAX, KLF6) persist despite moderate edge counts.

Figure S16 : Pseudotime directionality failures. Only 12/56 TF–target pairs (21.4%) show directionally consistent lag-based ordering.

Figure S17 : Pseudotime null comparisons. Mean directionality score marginally exceeds shuffled-pseudotime null (

p=0.068

) but not random gene-pair null (

p=0.37

).

Figure S18 : Cross-dataset leakage summary. Balanced accuracy and AUC for donor, batch, and method classification from edge-product features.

Figure S19 : Artifact Sensitivity Index distribution. Immune tissue has 54.6% of edges exceeding ASI

>0.5

.

Figure S20 : Edge score calibration. Reliability diagrams showing fraction of true positives vs. mean predicted probability. Platt scaling and isotonic regression reduce ECE by 4–7

\times

.

Figure S21 : Conformal prediction sets. Empirical coverage and average set size across methods at

\alpha=0.05

,

0.10

,

0.20

.

Config	$N$	States	Pooled F1	CSSI-max F1	Ratio
Small	200	2	$0.850\pm 0.053$	$0.957\pm 0.050$	1.13 $\times$
Medium	400	4	$0.657\pm 0.100$	$0.921\pm 0.071$	1.40 $\times$
Large	600	6	$0.486\pm 0.100$	$0.900\pm 0.069$	1.85 $\times$
XLarge	1000	8	$0.550\pm 0.089$	$0.967\pm 0.029$	1.76 $\times$
XXLarge	1000	10	$0.514\pm 0.083$	$0.932\pm 0.049$	1.81 $\times$
Massive	1500	12	$0.527\pm 0.041$	$0.942\pm 0.027$	1.79 $\times$

Appendix

Table S5 : CSSI mitigates top-

K

scaling degradation in synthetic experiments.

Figure S22 : CSSI-max null stress test. Oracle strata labels reproduce the CSSI-max gains, but shuffling, randomizing, or gene-permuting labels removes the advantage.

Figure S23 : Extended CSSI null and baseline tests. (A) CSSI-max with shuffled/random labels shows no inflation as

K

increases. (B) CSSI-max, CSSI-mean, and SCENIC-style aggregation perform equivalently. (C) Per-edge FDR control.

$N$	Pooled F1	CSSI-max F1	AUROC ${}_{\text{pool}}$	AUROC ${}_{\text{CSSI}}$
200	$0.405\pm 0.076$	$0.655\pm 0.042$	0.860	0.935
500	$0.560\pm 0.030$	$0.745\pm 0.015$	0.932	0.998
1,000	$0.640\pm 0.020$	$0.750\pm 0.000$	0.972	1.000
2,000	$0.695\pm 0.027$	$0.750\pm 0.000$	0.989	1.000
5,000	$0.750\pm 0.000$	$0.750\pm 0.000$	1.000	1.000

Appendix

Table S6 : CSSI on biologically structured data. 15 cell types, 15 cell-type-specific edges.

Layer	Pooled	Best CSSI	AUROC ${}_{\text{CSSI}}$	$\Delta$
0	0.552	cssi_mean	0.566	+0.014
1	0.600	cssi_mean	0.608	+0.007
2	0.564	cssi_mean	0.587	+0.022
3	0.573	cssi_mean	0.580	+0.007
4	0.610	cssi_range	0.609	-0.001
5	0.532	cssi_mean	0.551	+0.019

Appendix

Table S7: Per-layer GRN recovery from Geneformer attention on 497 human brain cells.

Figure S24 : Synthetic validation of mechanistic interpretability methods. (A) GRN recovery degrades with cell count. (B) Shapley values outperform single-component estimates by 91%. (C–D) Empirical detectability matches theoretical predictions (

r=0.887

).

Cells	Edges	TRRUST AUROC	DoRothEA AUROC
200	1.56M	0.444	0.473
500	3.27M	0.549	0.486
1000	5.38M	0.522	0.486

Appendix

Table S8 : Geneformer V1-10M attention-based GRN inference on DLPFC brain data.

	TRRUST AUROC		DoRothEA AUROC
Cells	scGPT	Geneformer	scGPT	Geneformer
200	0.51	0.444	0.50	0.473
500	0.49	0.549	0.48	0.486
1000	0.46	0.522	0.47	0.486

Appendix

Table S9 : Cross-model AUROC comparison for attention-based GRN inference.

Figure S25 : Attention–correlation

R^{2}

mapping. All cross-tissue

R^{2}

values are

<0.02

.

Figure S26 : Robust edge score residualization. Substantial TRRUST-predictive signal remains after residualization.

Figure S27 : Degree-preserving null models and per-TF evaluation. (A) Observed AUROC exceeds both null distributions. (B) Per-TF AUROC vs. TF out-degree. (C) AUROC decomposition. (D) Per-TF AUROC distribution.

Figure S28 : Bootstrap per-TF AUROC with uncertainty quantification. Forest plot of 18 evaluable TFs with 95% bootstrap CIs.

Figure S29 : Multi-model GRN validation summary. Both scGPT and Geneformer achieve near-random AUROC.

Figure S30 : Full 18-layer perturbation-first AUROC profile. Late layers cluster near or above the correlation reference.

Figure S31 : Nested layer selection protocol. L15 is independently chosen in all 5 folds.

Figure S32 : Attention-specific confound decomposition on K562. Attention edges are more expression-confounded.

Figure S33 : Head-level causal ablation (original 6 conditions). Zeroing top-5 regulatory heads has no effect on perturbation-first AUROC.

Figure S34 : Orthogonal causal interventions. Uniform attention replacement on TRRUST-ranked heads and MLP pathway ablation at regulatory layers produce exactly baseline AUROC, while random-layer MLP ablation causes significant degradation.

Figure S35 : Cross-context replication: Adamson CRISPRa. Attention significantly underperforms correlation.

Figure S36 : Cross-context replication: Shifrut T-cell CRISPRi. Attention and correlation are indistinguishable (

n=7

).

Figure S37 : Intervention-fidelity diagnostics. All conditions produce material representation perturbation despite null AUROC effects, confirming genuine functional redundancy.

Figure S38 : Propensity-matched perturbation benchmark. After matching, edge AUROCs drop to near chance and incremental value is zero.

Figure S39 : Hard-generalization incremental-value test. Gene-level features alone match or exceed models augmented with pairwise edge scores under all three split designs.

Section	Hypothesis	Test	Raw p	BH p	Effect	N	Sig
1. Scaling Behavior Analysis
Scaling TRRUST	Cell count improves GRN	Sign test	0.002	0.011	100% degrad.	9 runs	Yes
Scaling TRRUST	Cell count improves GRN	Wilcoxon	0.002	0.011	–	9 runs	Yes
Scaling DoRothEA	Cell count improves GRN	Sign test	0.002	0.011	100% degrad.	9 runs	Yes
Bootstrap CI	F1 confidence intervals	Bootstrap	–	–	–	10k res.	–
Robustness	Seed stability w/ scaling	Paired test	$<$ 0.001	0.003	46–48% drop	3 seeds	Yes

Appendix

Table S10 : Comprehensive Statistical Test Registry. All p-values reflect framework-level BH FDR correction (

\alpha=0.05

, 153 total tests, 95 confirmatory).

Headline Claim	Supporting Analysis	Key Test(s)	BH Sig
Top- $K$ scaling degradation	Scaling	Sign test (TRRUST)	Yes
Continuous AUROC improves	K-Sensitivity	Comparative	Descr.
Mediation non-additivity	Mediation Bias	Lower bound	Yes
Detectability theory validated	Detectability	Correlation $r{=}0.887$	Yes
Perturbation-first AUROC $>$ 0.5	Pert.-first	$t$ -test, all 27 cond.	Yes
Attn $\approx$ Corr on CRISPRi	Attn pert.-first	Wilcoxon $p{=}0.73$	No (null)

Appendix

Table S11 : Headline claim-to-evidence mapping.

Layer	STRING ≥700	STRING ≥900	TRRUST	Reactome	KEGG	GO BP
L0	0.640	0.644	0.558	0.505	0.530	0.526
L1	0.517	0.501	0.691	0.501	0.484	0.501
L4	0.546	0.530	0.686	0.516	0.525	0.515
L8	0.530	0.519	0.631	0.509	0.531	0.534
L13	0.574	0.559	0.708	0.523	0.548	0.530
L15	0.525	0.500	0.750	0.521	0.555	0.541

Appendix

Table S12 : AUROC of Geneformer attention edges against six biological reference databases across all 18 layers. Correlation baseline shown in last row.

Metric	Best VW Cosine	Best Raw Attn	Correlation	Variance
Pert-first AUROC	0.587 (L12)	0.787 (L14)	0.706	0.887
TRRUST AUROC	0.606 (L12)	0.718 (L15)	0.638	—

Appendix

Table S13 : Edge score comparison: value-weighted vs. raw attention.

TF	Category	Targets in HVG	AUROC	Successful
LEF1	master regulator	1	0.997	Yes
KLF1	master regulator	1	0.995	Yes
PPARG	master regulator	10	0.900	Yes
FOXA1	master regulator	2	0.874	Yes
MAL	signal-dependent	1	0.860	Yes
EGR1	signal-dependent	9	0.852	Yes

Appendix

Table S14 : Per-TF AUROC and characterization. 18 evaluable TFs.

Systematic Evaluation of Single-Cell Foundation Model Interpretability Reveals Attention Captures Co-Expression Rather Than Unique Regulatory Signal

Abstract

Figures & tables

Appendix

Mechanistic Interpretability

Sparse autoencoders reveal organized biological knowledge but minimal regulatory logic in single-cell foundation models: a comparative atlas of Geneformer and scGPT

Parameter-free representations outperform single-cell foundation models on downstream benchmarks

Exhaustive Circuit Mapping of a Single-Cell Foundation Model Reveals Massive Redundancy, Heavy-Tailed Hub Architecture, and Layer-Dependent Differentiation Control

Abstract

Mechanistic Interpretability

Explore similar work

Sparse autoencoders reveal organized biological knowledge but minimal regulatory logic in single-cell foundation models: a comparative atlas of Geneformer and scGPT

Parameter-free representations outperform single-cell foundation models on downstream benchmarks

Exhaustive Circuit Mapping of a Single-Cell Foundation Model Reveals Massive Redundancy, Heavy-Tailed Hub Architecture, and Layer-Dependent Differentiation Control