Cell Painting microscopy captures how cells change after a chemical or genetic perturbation. Connecting these images to the perturbations that produced them could make large imaging screens easier to search and interpret, but the task remains difficult because biological effects are subtle and technical variation is substantial. We introduce MorphoCLIP, a contrastive model that links Cell Painting profiles with text descriptions of compounds, CRISPR knockouts, and ORF overexpressions. The model keeps its vision and language backbones frozen and trains only a compact cross-channel module and projection layers, so it can be trained on a single consumer GPU. On held-out CPJUMP1 data, MorphoCLIP searches in both directions: from a cell image to its perturbation description and from a description to matching cell images. In both cases, a correct match appears among the top ten results much more often than expected by chance. Adding a replicate-alignment loss makes profiles from repeated experiments more consistent, although this improvement does not yet translate into reliable gene-compound matching. Gene-aware labels and plate correction also show no consistent retrieval benefit. These findings suggest that text supervision can help organize chemical and genetic Cell Painting data. Matching compounds with genetic perturbations, however, remains an open problem.
Figures & tables
Fig. 1: Overview of MorphoCLIP. A frozen DINOv3 encoder processes each fluorescence channel separately, and the CrossChannelFormer combines the resulting tokens. Site representations are averaged into a well representation and projected into the shared embedding space. A frozen BioClinical ModernBERT encodes the perturbation prompt, followed by a trainable text projection. Optional plate correction is applied to the image embedding before the training losses.
Ch
Stain
Target
1
MitoTracker / Alexa 647
Mitochondria
2
Phalloidin / Alexa 568
Actin
3
WGA / Alexa 488
Golgi / plasma membrane
4
Concanavalin A / Alexa 488
Endoplasmic reticulum
5
Hoechst 33342
DNA / nucleus
TABLE I: Cell Painting fluorescence channels used in MorphoCLIP.
Split
Direction
R@1
R@5
R@10
Med. rank
Val
well image → text
5.9
23.3
39.4
14
well text → image
4.1
15.3
23.5
30
pert. image → text
4.1
20.4
38.8
13
pert. text → image
5.1
18.4
39.8
13
random, single positive
1.0
5.1
10.2
49.5
random, well text → image
1.0
5.0
9.7
99
TABLE II: Base-model retrieval on validation and test data. Image-to-text retrieval ranks one description per perturbation. Text-to-image retrieval ranks either individual wells or perturbation-level mean profiles. Random rows show analytic chance performance.
Validation
Test
Run
Change from control
i → t
t → i
Med.
i → t
t → i
Med.
Base model
100-epoch schedule, random batches
38.8
39.8
13
37.2
44.2
14
Control
none
37.8
37.8
13
50.0
41.9
10
Soft labels
α=0.6
37.8
42.9
14
44.2
47.7
12
Replicate loss
λrep=0.3
44.9
41.8
13
47.7
46.5
12
Plate offsets
condition-relative correction
41.8
38.8
13
41.9
43.0
12
TABLE III: Single-factor ablations from a shared control. Values are perturbation-level Recall@10 in percent and median rank. Random Recall@10 is 10.2 on validation and 11.6 on test.
Track
Base
Replicate loss
Plate offsets
All three
Compound A549
short
0.353 (0.392)
0.435 (0.458)
0.327 (0.384)
0.497 (0.493)
long
0.513 (0.526)
0.650 (0.583)
0.503 (0.513)
0.703 (0.626)
Compound U2OS
short
0.297 (0.344)
0.379 (0.404)
0.310 (0.351)
0.422 (0.439)
long
0.369 (0.413)
0.471 (0.474)
0.395 (0.421)
0.503 (0.496)
CRISPR A549
short
0.173 (0.286)
0.271 (0.343)
0.092 (0.239)
0.340 (0.375)
long
0.183 (0.296)
0.376 (0.356)
0.134 (0.282)
0.366 (0.385)
TABLE IV: Standard CPJUMP1 benchmark. Each entry gives fraction retrieved, with mAP in parentheses. Summary fractions are unweighted across tracks; summary mAP is pooled across scored profiles.
Model
Compound A549
Compound U2OS
CRISPR A549
CRISPR U2OS
ORF U2OS
Mean
CellCLIP [ 8 ]
0.366
0.353
0.020
0.118
0.119
0.195
MorphoCLIP base
0.353
0.297
0.173
0.026
0.043
0.178
+ replicate loss
0.435
0.379
0.271
0.049
0.099
0.247
MorphoCLIP, all three
0.497
0.422
0.340
0.078
0.099
0.287
TABLE V: Short-timeline CPJUMP1 replicability on tracks shared by CellCLIP and MorphoCLIP. Values are fraction retrieved; the final column is the unweighted mean of the five tracks. CellCLIP uses the released checkpoint and its own preprocessing path.
Method
Dataset
Task
Metric
Result
CellProfiler [ 4 ]
CPJUMP1
Compound target matching
FR range
4.3–25.1%
MorphoCLIP base
CPJUMP1
Compound target matching
FR range
7.5–31.5%
CellCLIP [ 8 ]
RxRx3-core
CORUM gene–gene
Recall
0.354
CWA-MSN [ 6 ]
RxRx3-core
CORUM gene–gene
Recall
0.386
TABLE VI: Published benchmark context. Results are comparable within each dataset–task group, not across groups. FR denotes fraction retrieved.
Fig. 2: Variance decomposition of the cached DINOv3 tokens on plate BR00116991: between-channel, between-site and residual components.
Fig. 3: Mean cosine similarity between fluorescence channels in the cached token space. The DNA channel is the most distinct.
Fig. 4: First two principal components of the cached 1024-d channel tokens, colored by fluorescence channel.
Department of Electrical and Computer Engineering, The University of Hong Kong · School of Computing and Data Science, The University of Hong Kong · Nvidia AI Technology Center +2
Department of Computer Science and Engineering, The Ohio State University · Department of Biomedical Informatics, The Ohio State University · Department of Biostatistics, Yale University +1