cs.LGNov 4, 2026

A low-rank decoder bottleneck bounds reliability in foundation-model perturbation prediction

Authors: Zheyu Zhang, Yuanhao Huang, Fan Feng, Jie Liu

Organizations: Computer Science and Engineering Division, University of Michigan, Ann Arbor, MI 48109 · Gilbert S. Omenn Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI 48109

Abstract

Foundation-model predictors of perturbation response are evaluated primarily by aggregate accuracy. Using their predictions to prioritize experiments requires assessing each prediction's reliability. We audit the state-transition (ST) decoder used in STATE across three encoders (SE-600M, scGPT, and Tahoe-x1) and four cell lines under CRISPRi. The decoder's first attention layer has a low-rank output. Projecting this output onto 4-32 fixed directions estimated from training activations preserves mean per-perturbation Pearson delta correlation (PDC). This low-rank attention bottleneck bounds the decoder's reliability by distance to training responses: PDC declines as the cosine distance from a perturbation's observed response to its nearest training response increases. We propose predicted-NN-distance, the cosine distance from a predicted response to its nearest training response, as a per-prediction confidence score. The score has a mean absolute Spearman correlation of 0.81 with PDC, close to the 0.88 obtained with the observed response. Thresholds calibrated on this score transfer across encoders and cell lines, and the score remains predictive of PDC for drug perturbations.

Figures & tables

Appendix figures & tables31 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Attribution-based diagnostics for single-cell foundation models

    Nov 2, 2026Farnoush Rezaei Jafari, Tristan Lazard, Kenza Bouzid +6Feature AttributionMolecular Foundation Models

  2. PerturbJEPA: Joint-Embedding Predictive Learning of Cellular Perturbation Responses

    Nov 16, 2026Jiangshuan Pang, Youzhe He, Yiting Deng +5Single-Cell Perturbation ModelingJoint-Embedding Predictive Architecture