Organizations: School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · Key Laboratory of Brain Cell Mapping of Zhejiang Province, BGI Research, Hangzhou 310030, China. · Center for Excellence in Brain Science and Intelligence Technology (Institute of Neuroscience), Chinese Academy of Sciences
Identifying the drug and dose that produced a cellular response requires preserving treatment effects within each cellular background. A joint-embedding predictor can match its targets while ignoring the intervention if its encoder preserves differences across backgrounds but erases differences between treated and control populations within each one. We call this failure interventional contrast collapse. PerturbJEPA encodes the difference in mean gene expression between treated and control populations as an effect vector. It aligns the displacement between their latent centroids with this vector and regularizes the vectors to preserve variation across population pairs. We show that the measured displacements cannot all coincide within a batch when effect-vector variation exceeds the root-mean-square alignment error. Trained with separate drug and dose ranking objectives, the model predicts candidate responses from a common control population and matches them to the observed response without cell pairing or expression reconstruction. We evaluate 3,592 Tahoe-100M queries on held-out combinations of known interventions and cellular backgrounds, each with over 700 drug-dose candidates in a fixed, background-specific catalog. PerturbJEPA achieves 91.46% macro joint drug-dose Recall@1, exceeding STATE by 3.40 percentage points (95% drug-cluster bootstrap interval of 1.72 to 5.07 points). Controlled diagnostics show that treatment-control differences can shrink even when representations remain globally dispersed. With the drug ranking objective fixed, adding effect-vector regularization to alignment raises joint Recall@1 from 18.57% to 87.47%.
Figures & tables
Figure 1: Global collapse and interventional contrast collapse are different failure modes. Colors denote cellular backgrounds; circles and triangles denote control and treated cells. (A) Global collapse erases both background variation and intervention contrasts. (B) ICC removes treatment–control separation within each background while retaining global dispersion. These are conceptual trajectories; Fig. 3 reports measured training dynamics.
Figure 2: PerturbJEPA learns cell population responses for source-intervention retrieval. (a) Population prediction. A shared set encoder embeds unpaired control and treated cell populations. An intervention-conditioned predictor minimizes population energy discrepancy (PED); state-marginal isotropy (SMI) regularizes each slot across population pairs, separately for each state. (b) Effect preservation. Pseudobulk-to-latent effect alignment (PEA) aligns the scaled, fixed centroid displacement Δzb/α with the expression-contrast embedding vb=g(Δxb) . Effect-space isotropy (ESI) regularizes these unscaled effect targets. (c) Intervention-discriminative ranking (IDR). Candidate predictions share the same control and are compared with the same treated population, using separate drug and dose softmax losses. Point clouds and matrices are schematic. The effect branch and loss terms are training-only; retrieval scores candidates by PED.
Joint
Drug
Dose
Model
R@1
MRR
R@1
Acc.
Mean
4.47
9.24
5.90
61.52
Linear
75.57
82.94
78.16
95.99
CPA
0.10
0.88
0.30
36.58
PerturbDiff (scratch)
1.92
4.17
2.90
49.38
PerturbDiff (fine-tuned)
2.93
5.96
3.42
48.70
Table 1: Overall intervention retrieval. All eight systems use the same 3,592 queries and hierarchical macro aggregation. R@1 and dose accuracy are percentages; MRR is ×100 . Dose accuracy supplies the true drug.
Row
Configuration
Enabled training objectives
Joint
Drug
Dose
PED
SMI
PEA
ESI
Drug IDR
Dose IDR
R@1
MRR
R@1
Acc.
A. Preserve intervention effects Shared encoder; drug IDR fixed
(a)
No effect branch
✓
✓
×
×
✓
×
1.02
2.36
1.92
37.89
(b)
+ PEA alignment
✓
✓
δ
×
✓
×
18.57
29.79
20.72
84.18
(d)
+ ESI (reference)
✓
✓
δ
✓
✓
×
87.47
92.07
89.83
96.47
B. Discriminate drugs and doses Shared encoder; complete PEA fixed
Table 2: Effect preservation and intervention discrimination. Panels A and B share reference (d). Panel C tests PEA with momentum targets. Panel D replaces only the PEA alignment in the full model (e). All rows report test results at validation-selected checkpoints.
Figure 3: Global diversity and intervention contrast require distinct constraints. Six controlled runs share common initial weights and actual sampled training batches. (a) Global centered latent scale; (b) energy discrepancy between encoded treated and control populations; (c) their dimension-normalized centroid distance. State regularization sustains global scale but permits contrast contraction; complete PEA sustains larger contrasts than alignment alone. Both IDR terms are disabled for all 10,000 warmup updates. Curves are centered 101-step moving means from one training seed, with no uncertainty bands. Diagnostics use observed training populations, include sampling variation, and are not prediction errors.
Denominator d(x)
Joint
Drug
Dose
R@1
MRR
R@1
Acc.
∥x∥2
87.47
92.07
89.83
96.47
1
87.55
91.99
89.99
96.43
∥x∥1
38.72
52.29
42.47
86.96
Table 3: Expression-weight normalization. For x=0 , cell tokens are t(x)=∑gxgEg/d(x) . Other settings follow the drug-IDR reference in Table 2 (d).
Conditioning
Joint
Drug
Dose
R@1
MRR
R@1
Acc.
AdaLN
87.47
92.07
89.83
96.47
Prefix tokens
73.64
81.80
78.17
92.50
Table 4: Intervention conditioning. Depth, width, and intervention encodings are matched. Both variants retain the reference input normalization and training objective from Table 2 (d).
Energy scoring
Sinkhorn scoring
Training
R@1
MRR
R@1
MRR
Energy
87.47
92.07
86.74
91.51
Sinkhorn
62.89
73.39
63.07
73.50
Table 5: Retrieval-distance sensitivity. The two independently validation-selected checkpoints are fixed while changing the test scorer. Both use complete PEA and drug IDR, with dose IDR disabled, and retain their original energy-based validation selection.
Joint
Drug
Dose
Input
Genes
R@1
MRR
R@1
Acc.
HVG2000
2,000
91.46
94.43
91.74
99.31
All genes
62,710
94.50
96.81
96.30
97.72
Table 6: Input-gene coverage. Both models use all six objectives and their own validation-selected checkpoints. Hidden width, depth, and optimization settings are matched; expanded input matrices increase parameter count. Units follow Table 1 .
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Model
R@1
R@5
R@10
MRR
Mean
4.47
11.93
16.67
9.24
Linear
75.57
92.27
94.98
82.94
CPA
0.10
0.61
0.97
0.88
PerturbDiff (scratch)
1.92
5.08
7.64
4.17
PerturbDiff (fine-tuned)
2.93
7.86
11.20
5.96
STATE
88.06
95.28
96.76
91.39
Appendix
Table 7: Additional overall joint-retrieval metrics. All rows use the same 3,592 queries, fixed candidate catalogs, and hierarchical macro aggregation as the main comparison. Recall is a percentage and MRR is ×100 .
Figure 4: Additional 30k validation of global dispersion and intervention contrast. All variants retain PED and population-axis SMI; alignment adds the PEA loss, and complete PEA also adds ESI. Both IDR losses are disabled. (a) Pooled, centered global state scale sz . (b) Matched observed treated–control energy contrast CE=B−1∑bDE(Zb0,Zb1) , using the implemented numerical distance floor. (c) Mean dimension-normalized centroid distance Cmean=B−1∑b∥Δzb∥2/D . These are encoder diagnostics, not predictor errors; Cmean differs from RΔ and SΔ in Equation ( 14 ). Lines and bands show the mean and sample SD across three seeds after within-seed smoothing. The gray interval marks the 10k warmup.
Figure 5: Effect-target variation and finite-batch transfer in the additional 30k runs. PED and SMI are common to both variants; complete PEA adds ESI to alignment. (a) Diagnostic I(V)/I0 using 256 fixed unit projections and the 17-point frequency grid of Equation ( 18 ); training ESI refreshes projections across calls. The reference line is the constant-batch lower limit; a raw value below 1 certifies nonconstant targets by Proposition 3 . (b) Target variation Sv (solid) and unweighted alignment error ϵ (dashed), with color identifying the variant. (c) Signed transfer margin mS=Sv−ϵ . (d) FULL’s measured SΔ and lower bound LS=0.2[mS]+ from Proposition 4 . SB has no effect branch and is omitted. Quantities are computed per batch before the smoothing and three-seed aggregation used in Figure 4 .
Figure 6: Additional checks for scale, sampling variation and shared context shifts. All three training variants retain PED and population-axis SMI with both IDR terms disabled. (a) Training-batch variation SΔ/sz , with the same three-seed mean, SD and smoothing as Figure 4 . (b) Cross-draw response Uz/sz,panel2 from Equation ( 29 ) and its matched control–control null at 30k. (c) Within-context cross-drug variation Udrug,z/sz,panel2 from Equation ( 30 ), with its corresponding null. Filled and hollow markers show response and null; each connecting line pairs one training seed, with circles, squares and triangles identifying seeds 42, 43 and 44. Panels (b,c) use identical symmetric-log limits, linear for absolute values at most 10−6 ; negative estimates are retained. The frozen panel comprises 60 conditions, five cell lines, 17 eligible contexts and 20 draws per condition.
Configuration
Selected (k updates)
Val. joint R@1
Val. joint MRR
Complete PEA, no IDR
180
77.89
85.35
Drug IDR, no effect
10
0.00
0.48
Drug IDR, alignment only
20
12.46
22.41
Complete PEA + drug IDR
120
86.67
91.45
Full HVG2000 model
160
91.58
94.92
Learned endpoints
70
79.65
87.07
Appendix
Table 8: Selected checkpoints and validation performance. Each configuration maximizes hierarchical macro joint Recall@1 on the common validation panel, with joint MRR and then the earliest update breaking ties. Recall is a percentage; MRR is multiplied by 100 .
Configuration
Joint R@1
Joint MRR
Dose Acc.
Complete PEA, no IDR
[80.65, 84.09]
[87.15, 89.69]
[95.10, 96.74]
Drug IDR, no effect
[0.45, 1.69]
[1.61, 3.20]
[36.81, 38.96]
Drug IDR, alignment only
[16.86, 20.30]
[28.06, 31.51]
[82.49, 85.81]
Complete PEA + drug IDR
[85.86, 88.92]
[90.97, 93.09]
[95.69, 97.18]
Full HVG2000 model
[90.01, 92.82]
[93.33, 95.43]
[98.98, 99.60]
Learned endpoints
[79.64, 83.77]
[86.21, 89.40]
[97.44, 98.92]
Appendix
Table 9: 95% target-drug cluster bootstrap intervals for the selected checkpoints. Joint Recall@1 and dose accuracy use percentage units; MRR is ×100 . These intervals condition on the trained models and frozen query cells.
Metric
HVG2000
All genes
Difference
Paired 95% interval
Joint Recall@1
91.46
94.50
+3.04
[1.57,4.56]
Joint MRR
94.43
96.81
+2.38
[1.38,3.42]
Drug Recall@1
91.74
96.30
+4.56
[3.16,6.09]
Conditional dose accuracy
99.31
97.72
−1.60
[−2.31,−0.92]
Log-dose MAE ↓
0.0093
0.0300
+0.0207
[0.0109,0.0311]
Appendix
Table 10: Paired full-gene versus HVG2000 comparison. Both models use the complete six-term objective and their independently validation-selected checkpoints. Differences are full-gene minus HVG2000, computed before rounding. Intervals are paired 95% target-drug cluster bootstrap intervals. Recall and accuracy use percentage units; MRR is multiplied by 100 . Both dose metrics supply the true drug.
Configuration
R@1
R@5
R@10
R@20
R@50
Joint MRR
nDCG@10
NR ↓
Complete PEA, no IDR
82.45
95.80
97.78
98.63
99.15
88.49
0.9073
0.002452
Drug IDR, no effect
1.02
2.55
3.56
5.53
10.74
2.36
0.0213
0.463000
Drug IDR, alignment only
18.57
41.53
51.54
62.62
77.99
29.79
0.3383
0.055814
Complete PEA + drug IDR
87.47
97.32
98.43
98.84
99.44
92.07
0.9362
0.001660
Full HVG2000 model
91.46
98.04
98.77
99.13
99.71
94.43
0.9547
0.000965
Learned endpoints
81.73
95.76
97.37
98.47
99.10
87.84
0.9012
0.003120
Appendix
Table 11: Complete joint retrieval metrics. Recall values are percentages, MRR is ×100 , and nDCG@10 is on [0,1] . Normalized rank (NR) is lower-is-better; all other columns are higher-is-better. Definitions follow Appendix D.5 .
Configuration
Drug R@1
Drug R@5
Drug R@10
Drug MRR
Dose Acc.
Log-dose error ↓
Complete PEA, no IDR
85.12
96.72
98.42
90.28
95.97
0.0540
Drug IDR, no effect
1.92
4.41
6.61
4.22
37.89
0.8613
Drug IDR, alignment only
20.72
44.90
56.26
32.75
84.18
0.2038
Complete PEA + drug IDR
89.83
97.98
98.94
93.57
96.47
0.0495
Full HVG2000 model
91.74
98.24
98.92
94.65
99.31
0.0093
Learned endpoints
82.23
96.21
97.79
88.35
98.24
0.0228
Appendix
Table 12: Complete drug and conditional-dose metrics. Recall and accuracy are percentages; MRR is ×100 . Dose error is mean absolute error in log10 dose and is lower-is-better. Both dose metrics supply the true drug.
Comparison
Δ Joint R@1
Δ Joint MRR
Change test scorer: Sinkhorn minus energy, checkpoint fixed
Energy-trained
-0.72 [-1.16, -0.31]
-0.56 [-0.83, -0.31]
Sinkhorn-trained
+0.18 [-0.17, 0.54]
+0.11 [-0.09, 0.31]
Compare checkpoints: Sinkhorn-trained minus energy-trained
Energy scoring
-24.57 [-26.89, -22.31]
-18.68 [-20.61, -16.87]
Sinkhorn scoring
-23.67 [-26.04, -21.30]
-18.01 [-19.93, -16.21]
Appendix
Table 13: Paired contrasts of test scorers and fixed checkpoints. Differences and 95% intervals use the same 4,000 target-drug bootstrap replicates across all four training/scoring combinations. Recall differences are percentage points. Each training recipe uses its independently validation-selected checkpoint, held fixed when changing the test scorer.
Metric
E/E
E/S
S/E
S/S
Joint R@1
87.47
86.74
62.89
63.07
Joint R@5
97.32
97.06
86.79
87.10
Joint R@10
98.43
98.19
92.31
92.43
Joint R@20
98.84
98.79
95.58
95.43
Joint R@50
99.44
99.35
97.70
97.77
Joint MRR
92.07
91.51
73.39
73.50
Appendix
Table 14: Complete metrics for retrieval-distance sensitivity. E and S denote energy and Sinkhorn, respectively; columns specify training/test discrepancy. Recall and dose accuracy are percentages, MRR is ×100 , and nDCG remains on [0,1] . Dose metrics supply the true drug. Each training recipe uses one fixed checkpoint for both test scorers.
Department of Biomedical Data Science Stanford University Stanford, CA 94305, USA · Biology Research & AI Development, Genentech DNA Way, South San Francisco, CA 94080, USA