MolEmb: Multimodal Large Language Models Can Be Strong Molecular Embedding Models

Xinjian Zhao; Xiangru Jian; Yaoyao Xu; Xiaozhuang Song; Wei Pang; Lei Bai; Tianshu Yu

cs.AIAug 24, 2026

MolEmb: Multimodal Large Language Models Can Be Strong Molecular Embedding Models

Authors: Xinjian Zhao, Xiangru Jian, Yaoyao Xu, Xiaozhuang Song, Wei Pang, Lei Bai, Tianshu Yu

Organizations: The Chinese University of Hong Kong, Shenzhen · Shanghai Artificial Intelligence Laboratory · Cheriton School of Computer Science, University of Waterloo

Abstract

Molecular embedding models can serve as foundational infrastructure for computational chemistry and drug discovery, where reusable vector representations support property prediction, virtual screening, and retrieval. Most molecular encoders are specialist models built around a single molecular view, producing unconditional vectors with no language interface for varying the representation. We ask whether multimodal large language models (MLLMs), which natively process images, text, and symbolic inputs, can instead serve as general molecular embedding models that produce embeddings conditioned on both a molecular profile and a natural-language semantic context. We introduce MolEmb, a lightweight framework that adapts MLLMs by aligning molecular profiles with textual descriptions in a shared embedding space using a bidirectional contrastive objective. The resulting embedding model is competitive on molecular property prediction and supports cross-modal molecule--text retrieval in the same space. We further introduce MolCAR, a diagnostic benchmark for context-aware retrieval, and find that context-aware molecular embedding is primarily a data property of the supervision. These results suggest that MLLMs are not merely chemistry assistants or generators, but a viable and extensible route to general molecular embedding models.

Figures & tables

Figure 1: From specialist encoders to general molecular embedding models. (A) The MLLM-to-embedding route shifts molecular representation from fixed-view specialist encoders to an MLLM-based embedding interface that accepts multi-view molecular profiles and natural-language task instructions. (B) MolEmb instantiates this route by adapting a pretrained MLLM with lightweight LoRA modules through molecule–text contrastive alignment; the EOS hidden state is used as the molecular embedding. (C) The resulting reusable embedding interface supports property prediction, cross-modal molecule–text retrieval, and context-aware embedding by varying the task instruction.

Supervised GNN
	Regression (RMSE $\downarrow$ )			Classification (ROC-AUC% $\uparrow$ )
Model	ESOL	Lipo	FreeSolv	BACE	BBBP	ClinTox	Tox21	SIDER
S-GIN	1.173 $\pm$ 0.057	0.757 $\pm$ 0.018	2.755 $\pm$ 0.349	72.97 $\pm$ 4.00	68.17 $\pm$ 1.48	88.14 $\pm$ 2.51	74.91 $\pm$ 0.51	57.60 $\pm$ 1.40
R-GIN	1.706 $\pm$ 0.180	1.075 $\pm$ 0.022	7.526 $\pm$ 2.119	75.07 $\pm$ 2.23	64.48 $\pm$ 2.46	72.29 $\pm$ 4.15	71.53 $\pm$ 0.74	62.29 $\pm$ 1.12
Pre-training Methods
InfoGraph	1.344 $\pm$ 0.178	1.005 $\pm$ 0.023	10.005 $\pm$ 4.819	74.74 $\pm$ 3.64	66.33 $\pm$ 2.79	64.50 $\pm$ 5.32	69.74 $\pm$ 0.57	60.54 $\pm$ 0.90

Table 1: Main downstream comparison with representative molecular representation methods. Regression columns report RMSE, where lower is better; classification columns report ROC-AUC in percent, where higher is better. For each column, the best and second-best results are highlighted.

Model	m2t R@1	m2t R@5	m2t R@10	t2m R@1	t2m R@5	t2m R@10
Qwen3-VL-Embedding-8B	0.0347	0.0838	0.1185	0.0306	0.0828	0.1229
Qwen3.5-0.8B + MolEmb (MolTextNet)	0.8397	0.9818	0.9963	0.8458	0.9805	0.9926
Intern-S1-mini + MolEmb (MolTextNet)	0.8539	0.9811	0.9926	0.8488	0.9785	0.9923
Qwen3-VL-8B + MolEmb (MolTextNet)	0.7343	0.9549	0.9842	0.7502	0.9572	0.9828

Table 2: Molecule-text retrieval on MolTextNet test split. m2t denotes molecule-to-text retrieval, t2m text-to-molecule retrieval, and R@

K

denotes recall at

K

. All values are reported as fractions.

Model	Metric	Base			+ Continued Alignment
		Inst (%)	NoInst (%)	$\Delta$ (pp)	Inst (%)	NoInst (%)	$\Delta$ (pp)
Intern-S1-mini	Mol. R@1	10.5	10.6	-0.1	24.9	16.0	+8.9
	Doc. R@1	4.5	4.3	+0.2	24.8	6.3	+18.5
	Doc. MRR	11.3	10.7	+0.6	39.8	14.6	+25.2
	Doc. R@5	16.6	15.3	+1.3	57.4	21.1	+36.3
	Context R@1	44.4	40.4	+4.0	99.8	40.4	+59.4

Table 3: MolCAR-Structured results. Base : alignment-pretrained on the Mixed corpus. + Continued Alignment : after continued alignment on MolCAR-Train. Inst : query with task-specific instruction; NoInst : generic query without task specification.

\Delta

= Inst

-

NoInst (percentage points). Mol. R@1 : correct molecule in top-1. Doc. R@1 : correct document (molecule + context) in top-1. Doc. MRR :

100\times

mean reciprocal rank of the correct document. Doc. R@5 : correct document in top-5. Context R@1 : correct context ranked first within the same molecule.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Task	Train	Valid	Test	Metric
ESOL	Regression	902	113	113	RMSE
Lipophilicity	Regression	3360	420	420	RMSE
FreeSolv	Regression	513	64	64	RMSE
BACE	Binary cls.	1210	152	152	ROC-AUC
BBBP	Binary cls.	1631	204	204	ROC-AUC
ClinTox	Binary cls.	1181	148	148	ROC-AUC

Appendix

Table 4: Downstream prediction datasets. Train/valid/test sizes follow the OGB scaffold split.

Corpus	Train	Valid	Test
MolTextNet	290,962	2,969	2,970
KnowMol-100k	94,968	969	970
ChEBI-20-MM	30,630	312	314
Mixed	411,709	4,149	4,165

Appendix

Table 5: Alignment-pretraining corpus sizes (molecule–text pairs). Pre-dedupe rows are each corpus’s own 98/1/1 split. Mixed reflects the post-dedupe global merge across the three sources; merging drops 5,041 duplicate pairs at the canonical-SMILES level.

Model	Metric	Base			+ Continued Alignment
		Inst (%)	NoInst (%)	$\Delta$ (pp)	Inst (%)	NoInst (%)	$\Delta$ (pp)
Intern-S1-mini	Mol. R@1	14.4	14.4	-0.1	22.9	18.1	+4.8
	Doc. R@1	6.2	6.0	+0.2	22.8	7.5	+15.3
	Doc. MRR	14.5	13.9	+0.6	38.9	17.1	+21.8
	Doc. R@5	21.4	20.1	+1.3	57.6	25.0	+32.6
	Context R@1	44.3	40.4	+3.9	99.2	40.4	+58.8

Appendix

Table 6: MolCAR-Natural results. We report Inst / NoInst /

\Delta

for Base and +Continued Alignment.

\Delta

is computed from unrounded scores before display; small discrepancies are due to rounding.

Figure 2: MolTextNet retrieval remains strong after MolCAR continued alignment, for both Intern-S1-mini (left) and Qwen3.5-0.8B (right). Bars compare m2t/t2m R@1 on the held-out MolTextNet test split for the alignment-pretrained (Mixed) checkpoint and the same checkpoint after MolCAR continued alignment; the MolTextNet-only checkpoint is shown as a reference point. Here, CL denotes continued alignment.

Figure 3: Controlled MolCAR instruction-probe t-SNE on a fixed molecule set for Intern-S1-mini. Each unique molecule is expanded into all eight MolCAR task instructions, so changes in geometry reflect instruction conditioning rather than sample-composition drift. The first two panels show the model before continued alignment, and the last two panels show the model after continued alignment; each setting is evaluated with generic queries (NoInst) and task-instructed queries (Inst). Clear task-wise separation appears only after continued alignment and only when task instructions are present. PreCL and PostCL denote before and after continued alignment, respectively.

Figure 4: Multimodal profile versus SMILES-only input for Intern-S1-mini under direct adaptation. Using both the 2D depiction and canonical SMILES yields modest gains on most regression and classification tasks, supporting the value of the multi-view input interface.

Molecular Representation Learning

Molecular representation learning encodes chemical structures and context for prediction, generation, and discovery across chemistry and biology.

See all 207 papers

MolE-RAG: Molecular Structure-Enhanced Retrieval-Augmented Generation for Chemistry
Jun 4, 2026Joey Chan, Wonbin Kweon, Ashley Shin +4Retrieval-Augmented Generation LLM Inference
University of Illinois Urbana-Champaign · University of California, San Diego
Bolek: A Multimodal Language Model for Molecular Reasoning
May 4, 2026Frederic Grabowski, Jacek Szczerbiński, Maciej Jaśkowski +4LLM Grounding Scientific Reasoning
1Ingenix.ai, Warsaw, Poland · Laboratory of Bioinformatics and Computational Genomics, Warsaw University of Technology, Warsaw, Poland
Mol-JEPA: A multimodal Joint Embedding Predictive Architecture for Molecules
Aug 23, 2026Florian Rottach, Sebastian Schieferdecker, William Rudman +2Joint-Embedding Predictive Architecture Multimodal Pretraining
University of Tübingen · 2Boehringer Ingelheim · 3The University of Texas at Austin +1

Jun 4, 2026J/K move · Enter open · S save

MolE-RAG: Molecular Structure-Enhanced Retrieval-Augmented Generation for Chemistry

Joey Chan, Wonbin Kweon, Ashley Shin, Niharika Bhattacharjee+3

University of Illinois Urbana-Champaign · University of California, San Diego

Retrieval-Augmented Generation LLM Inference Molecular Property Prediction

Open paper AskPDF arXiv

Large language models (LLMs) have shown promise for molecular property prediction, but their ability to reason over chemical structures remains limited, as molecular representations such as SMILES differ substantially from the natural language on which LLMs are primarily trained. To bridge this semantic and chemical knowledge gap, we propose MolE-RAG, a training-free, molecule-centric retrieval-augmented generation framework for LLM-based molecular property prediction. MolE-RAG augments each prediction with three complementary sources of inference-time context: retrieved chemistry literature, molecule-specific information including compound synonyms, identifiers, functional group annotations, and physicochemical descriptors, and structurally similar molecules retrieved from the training set. We evaluate MolE-RAG across nine molecular property prediction tasks using proprietary, chemistry-specialized, and open-source LLMs. Across general-purpose LLMs, MolE-RAG improves ROC-AUC by up to 28 percentage points on classification tasks and reduces regression RMSE by up to 67% relative to a SMILES-only baseline. We further find that the utility of each context source varies across models and tasks, with different models benefiting most from textual retrieval, molecular context, or structural retrieval. These results suggest that molecule-centric retrieval can improve LLM-based molecular property prediction without model fine-tuning while providing a flexible framework for integrating heterogeneous chemical knowledge at inference time.

Abstract

Molecular Representation Learning

Explore similar work

MolE-RAG: Molecular Structure-Enhanced Retrieval-Augmented Generation for Chemistry

Bolek: A Multimodal Language Model for Molecular Reasoning

Mol-JEPA: A multimodal Joint Embedding Predictive Architecture for Molecules