Cellular world models aim to predict how cells respond to genetic perturbations, analogous to world models in reinforcement learning. Perturbation response models are evaluated almost exclusively by global Pearson correlation between predicted and observed transcriptomes. We show this metric is dominated by thousands of non-responsive genes and severely inflates apparent performance. On the Replogle K562 screen (509 unseen perturbations, 5000 genes), GEARS achieves global Pearson 0.977, but its delta Pearson is only 0.076. A trivial baseline predicting mean training delta achieves 0.267, which is 3.5x better than GEARS. Our primary contribution is methodological: we identify and quantify this delta gap as a systematic evaluation artifact, and propose delta-space evaluation as the primary benchmark. We provide two post-hoc corrections (RWR on co-expression graphs and kNN matching) and show that 2.8x of the remaining gap lies in perturbation representation geometry, not decoder capacity. We further show the gap is partially addressable end-to-end: delta-aligned training (lambda = 3.0) raises delta Pearson to 0.179 on Replogle (2.35x over GEARS) and surpasses the trivial baseline on direction accuracy (0.680 vs 0.567) and gene recovery (1.52 vs 1.19); combined with kNN it reaches 0.217. An amplitude-calibration variant (eta = 0.3) further raises gene recovery to 2.16 while reaching 0.218. Critically, these representations transfer zero-shot across cell lines: from Replogle K562 to unseen Norman A549, the delta-aligned model reaches 0.328 on 741 shared genes, 4.7x a trivial baseline and 2.1x standard GEARS. This is the first systematic quantification of the delta gap, and the first diagnostic framework localizing the residual gap to perturbation representation geometry via an oracle upper bound.
Figures & tables
Figure 1: Cellular world model framework. An agent (planner) issues a perturbation (action) to the world model f(s,a), which produces a next cell state s’; the observed post-perturbation transcriptome x’ feeds back to update the model. In the cellular setting, action = gene knockout perturbation, state = gene expression vector, s’ = post-perturbation transcriptome.
Figure 2: The delta gap as an evaluation problem. Step 1: the model’s global Pearson correlation is 0.977, so performance looks excellent. Step 2: measured on the perturbation-specific delta, the same model scores only 0.076. Step 3: this raises an open question - what does the gap reveal about model quality? We formalize and quantify it with the delta-gap metric in Sec. 4.3. The central question is answered in Sec. 4: the residual gap localizes to perturbation representation geometry (oracle kNN, Sec. 4.5), and delta-aligned training partially closes it (Sec. 4.8), transferring across cell lines (Sec. 4.9).
Figure 3: Left: Global vs delta Pearson on Replogle and Norman on a log scale, showing the 13x (Replogle) and 1.6x (Norman) gaps between global and delta performance. Right: RWR alpha sweep on Replogle; the best restart probability is alpha = 0.7.
Dataset
Cell line
Cells
Genes
Unseen perturbations
Replogle K562 essential [1]
K562
60,400
5,000
509
Norman [2]
A549
91,205
5,045
107
Table 1: Summary of datasets used in this study.
Dataset
Global Pearson
Delta Pearson
Gap
Replogle (509 holdout)
0.977
0.076 ± 0.006
-0.901
Norman (107 conditions)
0.877
0.548
-0.329
Table 2: Global vs delta Pearson comparison. The delta gap (Eq. 11) is 13x on Replogle and 1.6x on Norman.
Figure 4: Gene-level stratification by response strength on Replogle. Genes are grouped into deciles by their mean |true delta|; bars show pooled Pearson correlation between prediction and observation, in either absolute-expression space or delta space. The absolute-expression correlation rises with response strength and reaches 0.98 for the most responsive decile - this is what drives the global score of 0.977. Delta-space correlation, in contrast, collapses at every decile (GEARS 0.03-0.25; LinearPerturb 0.10-0.19). Metrics are computed in the GEARS pipeline space as in Figure S5:
Figure 5: Empirical CDF of absolute delta amplitude (pooled across gene-condition pairs) on Replogle, shown as step curves with shaded areas; vertical dotted lines mark the medians. Observed deltas (black, median 0.057) have the widest distribution; GEARS predicted deltas (red, median 0.052) collapse toward zero, and LinearPerturb predicted deltas (blue, median 0.009) are even more concentrated near zero. Metrics are computed in the GEARS pipeline space as in Figures S5 and 4.
Model
Metric
Delta definition
Pooling
Delta-align training?
Reported value
GEARS (this work)
rdelta
per-condition mean response, control-subtracted
pooled gene-condition pairs
No (global reconstruction)
0.076
SCALE
PDCorr
endpoint transport delta = z1 - z0
pooled
Yes (delta endpoint loss)
0.751
P2P
Effect Pearson
group effect delta = m up - m u0
pooled
Yes (effect-centric, largest weight)
0.70
HarmonyCell
DeltaPCC
observed - control mean
pooled
Partial (MCTS candidate search)
0.61-0.73
Lingshu-Cell
Pearson delta
delta across perturbation conditions
pooled
Yes (masked discrete diffusion)
reported
VCLMU
delta-PCC / delta-PCC@20
latent residual delta = xp - xc
pooled / top-20 genes
Yes (inverse dynamics)
joint plane
Table 3: Metric definition comparison across recent perturbation models. Delta-space metrics differ in how the delta is defined, how gene-condition pairs are pooled, and whether the training objective explicitly aligns with delta-space correlation. Values are as reported by the respective papers on Replogle K562 where available; the GEARS row reports our measurement.
Method
All delta Pearson
Top-20 DE Pearson
Direction
GEARS one-step (mean)
0.076
0.148
0.520
MLP (no GNN)
0.117
0.204
0.525
Flow matching (simplified)
0.207
0.266
0.534
GEARS + RWR (mean)
0.103
0.177
0.523
GEARS + kNN (k=50)
0.211
0.280
0.560
LinearPerturb (train mean)
0.267
0.309
0.567
Table 4: Method comparison on Replogle. The trivial LinearPerturb baseline outperforms all deep model variants.
Method
Global P.
Delta P.
Top-20
Direction
Recovery
GEARS
0.986
0.565
0.733
0.901
7.28
DA ( λ =3.0)
0.959
0.674
0.761
0.890
8.85
DA ( λ =3.0)+kNN
0.989
0.694
0.759
0.894
9.01
DA+amp ( η =0.3)
0.933
0.639
0.780
0.909
8.26
DA+amp+kNN
0.987
0.676
0.773
0.903
8.76
Table 5: Cross-dataset validation on Norman A549 (107 held-out conditions). GEARS delta Pearson is far higher here than on Replogle (0.565 vs 0.076) because overexpression perturbations produce stronger effects; the delta gap is consequently smaller (1.6x vs 13x). Delta-aligned training and kNN correction still transfer: DA+kNN achieves the best delta Pearson (0.694, +23% over GEARS) and recovery (9.01 vs 7.28). Amplitude calibration, in contrast, does not help on this low-gap dataset (delta 0.639 vs 0.674), consistent with amplitude collapse being most damaging where the gap is large.
Method
Global P.
Delta P.
Top-20
Direction
Recovery
GEARS
0.977
0.076
0.148
0.520
1.19
LinearPerturb
0.986
0.267
0.309
0.567
1.19
DA ( λ =1.0)
0.975
0.132
0.206
0.642
1.30
DA ( λ =3.0)
0.966
0.179
0.269
0.680
1.52
DA ( λ =3.0)+kNN
0.982
0.217
0.307
0.664
1.82
DA+amp ( λ =3.0, η =0.3), 3 seeds
0.934 ± 0.002
0.184 ± 0.007
0.327 ± 0.013
0.722 ± 0.004
1.75 ± 0.11
Table 6: Delta-aligned training on the Replogle holdout (509 conditions). All metrics are computed per condition and averaged, in the GEARS pipeline space (per-condition means), as in Table 4 and Figure S6. DA denotes delta-aligned training with the loss of Eq. 7; +kNN denotes blending with the kNN correction of Eq. 8 (k = 50, alpha = 0.7). DA+amp denotes the amplitude-calibration variant of Eq. 9 (eta = 0.3, k = 20); its kNN blending uses k = 25, alpha = 0.7. Alignment raises delta Pearson 2.35x over GEARS and surpasses LinearPerturb on direction and recovery; amplitude calibration lifts recovery to 2.73 +/- 0.52 across three seeds (129% over the trivial baseline). DA-rep denotes the representation-metric (cosine triplet) variant of Eq. 10; it does not improve delta Pearson (negative result, Section 4.8). Alignment raises delta Pearson 2.35x over GEARS and surpasses LinearPerturb on direction and recovery; kNN blending brings top-20 performance to parity with the trivial baseline. Two MLP rows (no graph prior) demonstrate that the alignment effect is architecture-agnostic.
Figure 6: The delta repair ladder on the Replogle holdout (509 conditions). Delta Pearson rises from 0.076 (GEARS) through the LinearPerturb baseline (0.267) and delta-aligned training (0.184 +/- 0.007), then kNN blending (0.261 +/- 0.038), stopping far short of the oracle kNN upper bound (0.592). Error bars are standard deviations over three seeds. The remaining gap to the oracle localizes to perturbation representation geometry (Sec. 4.5).
Method
Zero-shot Δ Pearson
Direction@20
vs LinearPerturb
LinearPerturb
0.070
0.55
1.0x
GEARS
0.158
0.50
2.3x
DA ( λ =3.0)
0.328
0.65
4.7x
DA+amp ( η =0.3)
0.329
0.75
4.7x
Table 7: Zero-shot cross-environment dynamics transfer from Replogle K562 to Norman A549. Models trained only on Replogle are applied to Norman without adaptation; metrics are pooled across 8 transferable conditions (5 perturbation genes) on 741 shared genes.
Figure 7: Zero-shot cross-environment dynamics transfer from Replogle K562 to Norman A549 (8 conditions, 5 genes, 741 shared genes; knockout to overexpression). Bars show pooled zero-shot delta Pearson; points show per-condition delta Pearson of the delta-aligned model (0.18-0.51), showing that the 0.328 pooled value is not driven by a single condition.
Metric
GEARS
LinearPerturb
LP / GEARS
Global Pearson
0.977
0.986
1.01
Delta Pearson (all genes)
0.082
0.201
2.45
Top-20 delta Pearson
0.163
0.311
1.91
Direction accuracy (top-20)
0.522
0.546
1.05
Top-20 gene recovery
1.19
1.19
1.00
Table 8: Metric sensitivity of model ranking on the Replogle holdout (509 conditions). All metrics are computed per condition and averaged, in the GEARS pipeline space as in Figures S5, 4 and 5. Global Pearson is computed on absolute expression (predicted + control vs observed + control); delta metrics on control-subtracted deltas; recovery counts the overlap between the predicted and the true top-20 responsive genes. GEARS does not win under any metric; the ratio column shows that the apparent gap depends on the metric choice, from 1.01x (global) to 2.45x (delta all genes).
Metric
Downsampled (30 cells/cond)
Full (310K cells)
All delta Pearson
0.267
0.251
Top-20 DE Pearson
0.309
0.299
Direction accuracy
0.567
0.564
Table 9: Downsampling vs full-data comparison for LinearPerturb baseline. The difference is within 0.02, confirming that downsampling does not affect conclusions.
Corr threshold
Edges
RWR all delta
RWR top20
0.1
45K
0.095
0.161
0.3
16K
0.103
0.177
0.5
6K
0.098
0.169
0.7
2K
0.087
0.152
Table 10: Ablation study: effect of co-expression graph threshold on RWR performance.
Table 12: Case study: top-5 up/down genes for the three strongest test perturbations.
Figure 8: t-SNE of GEARS perturbation embeddings (after GNN propagation) on Replogle K562. Training perturbations are shown in gray; the 509 held-out test perturbations are colored by their observed response strength (mean |delta| over the top-20 responsive genes). The three strongest test perturbations (SNRPD2, MED19, MED27) are annotated. No clear relationship between embedding position and response strength is visible: strong and weak responders are intermixed.
Figure 9: Per-condition direction accuracy versus observed response strength on the Replogle holdout (509 conditions). Left: GEARS one-step. Right: LinearPerturb (train-mean delta). The dashed line marks chance level (0.5). GEARS direction accuracy is near chance for weak responses and rises only modestly for strong responses; LinearPerturb shows the same pattern at slightly higher levels. Metrics are computed in the GEARS pipeline space as in Figures S5, 4 and 5.
Figure 10: Top-20 gene recovery on the Replogle holdout. For each of the 509 held-out conditions, the overlap between the predicted and the observed top-20 responsive genes is counted; histograms are shown on a log scale. Both GEARS one-step (red) and LinearPerturb (blue) recover on average 1.19 of the 20 true top genes per condition, barely above the chance level of 0.08 (dashed line). Metrics are computed in the GEARS pipeline space as in Figures S5, 4 and 5.
Figure 11: Case study for the three strongest held-out perturbations (SNRPD2, MED19, MED27): the top-10 responsive genes by observed |delta|, with observed (black), GEARS-predicted (red) and LinearPerturb-predicted (blue) delta values. GEARS predictions are pinned near zero for nearly every gene, while observed deltas reach -0.2 to -0.5; LinearPerturb predictions are intermediate. Metrics are computed in the GEARS pipeline space as in Figures S5, 4 and 5.
Figure 12: Per-condition prediction quality versus observed response strength on the Replogle holdout (509 conditions). Left: GEARS one-step. Right: LinearPerturb (train-mean delta). Each point is a held-out perturbation; the y-axis is the per-condition delta Pearson and the x-axis is the mean |delta| of its top-20 responsive genes. Black line: binned mean. Note: delta metrics here are computed inside the GEARS prediction pipeline (GEARS delta Pearson 0.082, LinearPerturb 0.202); these values differ from the downsampled-table numbers in Table 2 (0.076 / 0.267) because of the data-preparation pipeline, but the relative ordering - the trivial baseline outperforming GEARS - is identical in both settings.
Figure 13: Delta Pearson comparison across models on Replogle. The trivial LinearPerturb baseline outperforms GEARS by 3.5x.
Figure 14: Amplitude-calibration weight ablation on the Replogle holdout (509 conditions). Sweeping the top-20 amplitude-calibration weight eta (Eq. 9) from 0 to 1.0 shows eta = 0.3 maximizes delta Pearson (0.192) and direction accuracy (0.722), while larger weights buy modest additional recovery (1.90 at eta = 0.5) at the cost of global fidelity (0.816 at eta = 1.0).
Figure 15: Delta-aligned and amplitude-calibrated training results on the Replogle holdout (509 conditions). Bars show GEARS, LinearPerturb, the delta-aligned model at lambda = 3.0, its amplitude-calibrated variant (eta = 0.3), and the kNN-blended versions of both, on four metrics: delta Pearson, top-20 delta Pearson, direction accuracy, and gene recovery. Alignment repairs the direction and recovery deficits while improving delta Pearson 2.35x over GEARS; amplitude calibration further lifts recovery to 2.16 and direction accuracy above the oracle retrieval bound (0.728 vs 0.697).
Figure 16: Cross-dataset transfer of the repair mechanisms. Left: Replogle (high gap, 13x), where alignment and kNN raise delta Pearson from 0.076 to 0.261 (3.4x). Right: Norman (low gap, 1.6x), where the same mechanisms raise it from 0.565 to 0.694 (1.2x) and amplitude calibration is neutral-to-harmful. The repair benefit of each mechanism tracks the size of the delta gap.
Seed
One-step all
RWR all
One-step top20
RWR top20
One-step dir
RWR dir
1
0.069
0.101
0.140
0.191
0.519
0.523
2
0.080
0.107
0.152
0.166
0.522
0.524
3
0.078
0.101
0.152
0.178
0.519
0.522
mean
0.076
0.103
0.148
0.177
0.520
0.523
std
0.006
0.004
0.007
0.013
0.002
0.001
Table 13: Multi-seed stability of RWR improvement. RWR consistently improves delta Pearson across three random seeds with low variance.
k
alpha
All delta Pearson
Top-20 DE Pearson
Direction
1
0.0
0.167
0.205
0.551
5
0.0
0.196
0.262
0.557
10
0.0
0.204
0.266
0.559
30
0.0
0.206
0.268
0.560
50
0.0
0.211
0.280
0.560
50
0.3
0.204
0.282
0.558
Table 14: kNN matching results on Replogle. Increasing k improves perturbation matching with GEARS prediction (alpha=0) does not help.
Model
Global Pearson
Delta Pearson
Source
GEARS (ours)
0.977
0.076
this work
scGPT
~0.95*
0.367 (DEG only)
[14]
scLAMBDA
0.564 (RPE1)
not reported
[28]
GPO-VAE
~0.90*
not reported
[29]
Simple controls
~0.95*
outperform DL
[30]
LinearPerturb (ours)
N/A
0.267
this work
Table 15: Reported global Pearson correlations from published models on Replogle. All numbers are global (not delta), demonstrating the field’s reliance on inflated metrics.
Method
Global P.
Delta P.
Top-20
Direction
Recovery
DA ( λ =3.0), η =0
0.966
0.179
0.269
0.680
1.52
DA+amp, η =0.3
0.932
0.192
0.318
0.722
1.78
DA+amp, η =0.5
0.907
0.187
0.327
0.723
1.90
DA+amp, η =1.0
0.816
0.175
0.320
0.723
1.54
Table 16: Amplitude-weight ablation on the Replogle holdout (509 conditions). Sweeping the top-20 amplitude-calibration weight eta shows eta = 0.3 is optimal: larger weights degrade global reconstruction and reduce delta Pearson, confirming that heavy amplitude pressure buys behavioral gains at the expense of expression fidelity.
k
alpha
All delta Pearson
Top-20 DE Pearson
Direction
50
0.5
0.189
0.271
0.555
75
0.0
0.208
0.275
0.559
100
0.0
0.203
0.269
0.558
Table 17: Extended kNN matching sweep. Table S6 shows k = 1, 5, 10, 30, 50. This table extends the sweep to larger k values and additional mixing weights.
Predicting transcriptional responses to genetic perturbations is fundamental to functional genomics and therapeutic discovery. Recent deep learning models have shown promise in single-cell perturbation response prediction, but they typically generate each response in isolation, without explicitly leveraging related perturbations. We introduce PT-RAG (Perturbation-aware Two-stage Retrieval-Augmented Generation), a plug-in retrieval-and-conditioning module for generative cellular perturbation response. PT-RAG augments an existing perturbation-response backbone with learned access to related perturbation contexts. The key challenge is that relevance is not fixed in this setting: functionally related genes may elicit different effects across cell types. PT-RAG addresses this with a two-stage retrieval mechanism: GenePT-based semantic retrieval first identifies K candidate perturbations, after which a differentiable Gumbel-Softmax selector adaptively selects retrieved contexts conditioned on the control cell state, the query perturbation, and each candidate perturbation. We evaluate PT-RAG on two backbones, a STATE-style generator used as a frozen random reservoir and a fully trained scGPT, across cross-cell-type and cross-perturbation generalization tasks. PT-RAG consistently improves distributional similarity and often overall predictive quality; for example, on scGPT cross-cell-type results, the 2-Wasserstein distance drops by 5.9% relative to scGPT alone. The code to reproduce our experiments is available at https://github.com/difra100/PT-RAG_NIPS.
Andrea Giuseppe Di Francesco, Andrea Rubbi, Rishabh Jain +1
Predicting cellular responses to genetic perturbations helps prioritize experiments in single-cell genomics, where exhaustive measurement is infeasible. While computational models increasingly predict these responses, three evaluation deficiencies obscure what their scores demonstrate. First, absolute metrics cannot separate target-specific predictions from a shared background response. Second, common metrics remain high under gene shuffling, so gene-level accuracy is never verified. Third, a score at one training size says nothing about coverage, which depends on representation-space proximity and response-constraining power. We propose AmbiModBench, a specificity-aware, gene-resolved and coverage-aware benchmark. It pairs every score with a training-mean reference fitted on the same split, screens each readout by gene-coordinate permutation, and links embedding distance to response variation. Across K562, RPE1 and Norman, strong absolute scores largely reflect shared background rather than target-specific learning. Widely used readouts track response magnitude distributions rather than the affected genes. Detectable gain follows representation-space coverage rather than training-set size. Nonetheless, on RPE1 the protocol yields a reproducible target-specific gain across five additional splits and three gene selections, which absolute scores alone cannot distinguish from shared background.
Sikai Huang, Zhiwen Yang, Kai Yu +2
Fujian Agriculture and Forestry University · The Hong Kong Polytechnic University · Westlake University
Single-cell perturbation models can reduce costly wet-lab screening by predicting how cells respond transcriptionally to interventions. While recent generative models improve population-level prediction, individual generated cells are not explicitly checked for biological consistency. We introduce PerturbCellRL, a reinforcement learning (RL) framework that post-trains a pretrained single-cell transcriptomic generator using a suite of cell-level verifiers as rewards. These verifiers define four rewards: Pearson top-k similarity, RMSE top-k proximity, DE Spearman, and Pathway activity. The Pathway activity verifier rewards cells whose pathway responses match known perturbation biology. We evaluate PerturbCellRL on multiple genetic and chemical perturbation benchmarks. Across these benchmarks, PerturbCellRL improves over the pretrained flow-matching generator on reward-aligned evaluation metrics and a held-out evaluation metric. Moreover, PerturbCellRL remains competitive with state-of-the-art methods on population-level metrics. Together, these results frame trustworthy single-cell prediction as verifier-guided generative alignment, moving beyond matching expression distributions toward predictions whose single-cell perturbation effects are explicitly checked for biological consistency.
Dongxia Wu, Mingyu Li, Yuhui Zhang +4
Peking University Beijing, China · Stanford University Stanford, CA