Monroe: A Molecular Foundation Model for In-Context Probabilistic Inference

Blazej Banaszewski; Andrew W. Fitzgibbon

cs.LGAug 19, 2026

Monroe: A Molecular Foundation Model for In-Context Probabilistic Inference

Authors: Blazej Banaszewski, Andrew W. Fitzgibbon

Organizations: Graphcore, Bristol, UK · Max Planck Institute of Biochemistry, Munich, DE · Graphcore, Cambridge, UK

Abstract

Bioassay activity prediction is often data-limited because drug-discovery datasets rely on time-consuming and expensive wet-lab experiments for data generation and evaluation. This challenge has inspired recent research into molecular foundation models (MFMs), which aim to encode general-purpose chemical knowledge into molecular representations that generalize well in data-constrained scenarios. This paper presents Monroe, a new MFM with several innovations over the existing state of the art: increased scale allowing pre-training on over 81 million molecules from the PM6 quantum chemistry dataset; improved graph representation of stereochemistry; improved training losses including conformer denoising and embedding decorrelation; improved multi-task learning; and the use of a prior-data-fitted model (TabPFN) for downstream in-context prediction. Our evaluations use a principled pairwise comparison framework that measures statistically significant performance differences. Across established Polaris benchmarks, Monroe matches or exceeds existing MFMs, while on activity cliff benchmarks, designed to assess utility for molecular discovery, it achieves significant improvements over prior methods. Finally, ablation and transfer experiments show that PFN-based downstream predictors also substantially improve two leading existing models, MiniMol and CheMeleon, yielding new state-of-the-art variants we call MiniMol_PFN and CheMeleon_PFN, suggesting that our downstream adaptation strategy generalizes beyond Monroe. Source code is at github.com/blazejba/monroe.

Figures & tables

Figure 1: A. Monroe multi-task pretraining with structural and decorrelation losses under uncertainty weighting. B. Graph construction including featurization and stereochemistry-aware rewiring. C. In-context probabilistic inference: embeddings generated by attention pooling of the pretrained model together with training labels are passed to TabPFN, which outputs an approximate posterior-predictive distribution for the test molecule in a single forward pass.

Figure 2: Stereochemical E/Z isomers and R/S configurations.

Figure 3: Comparison between {Monroe, $\text{MiniMol}_{\mathsf{PFN}}$ , $\text{CheMeleon}_{\mathsf{PFN}}$ } (methods new to this paper) and {MiniMol, CheMeleon, MolFormer}, the leading previous models, plus a classical QSAR, on 28 tasks from Polaris (left) and 30 tasks from MoleculeACE (right). The pairwise matrices in the middle show the number of tasks that the model on the y-axis won against the model on the x-axis. The barplots show the win rates (see Section 4 ) in % when all seven methods are taken into account.

	Polaris		MoleculeACE
Model	Win %	Rank $\downarrow$	Win %	RMSE $\downarrow$	Cliff RMSE $\downarrow$
MolFormer	10.7	17.57	0 0.0	0.748 $\pm$ 0.018	0.854 $\pm$ 0.027
CheMeleon	46.4	13.25	36.7	0.662 $\pm$ 0.016	0.760 $\pm$ 0.026
MiniMol	42.9	11.39	10.0	0.703 $\pm$ 0.018	0.794 $\pm$ 0.031
$\text{CheMeleon}_{\mathsf{PFN}}$	42.9	11.46	80.0	0.641 $\pm$ 0.013	0.741 $\pm$ 0.020
GBM	46.4	10.04	16.7	0.680 $\pm$ 0.020	0.769 $\pm$ 0.023

Table 1: Model performance on Polaris and MoleculeACE. Error bars report the mean standard deviation across seeds.

Figure 4: Per-task Cliff RMSE on MoleculeACE for each method.

Model	${}^{\text{PT}}_{\text{PFN}}$	${}^{\text{FT}}_{\text{PFN}}$	Native
MiniMol	92.9	85.7	57.1
CheMeleon	75.0	96.4	92.9
MolFormer	39.3	71.4	92.9

Table 2: Downstream adaptation comparison on Polaris. In this test we compare within variants of each method, not across methods.

Multi-task balancing
	Polaris		MoleculeACE
Variant	Win %	Rank $\downarrow$	RMSE $\downarrow$	Cliff RMSE $\downarrow$
UW	75.0	3.71	0.629 $\pm$ 0.003	0.737 $\pm$ 0.005
EW	67.9	3.71	0.633 $\pm$ 0.003	0.743 $\pm$ 0.006
RLW	42.9	3.96	0.634 $\pm$ 0.004	0.746 $\pm$ 0.006
STCH	39.3	4.32	0.627 $\pm$ 0.003	0.738 $\pm$ 0.006

Table 3: Ablation study on Polaris and MoleculeACE. Pairwise testing is performed within each group separately. Error bars report the mean standard deviation across 3 seeds.

	Polaris		MoleculeACE
Strategy	Win %	Rank $\downarrow$	RMSE $\downarrow$	Cliff RMSE $\downarrow$
TabPFN(v2)	100.0	3.93	0.637 $\pm$ 0.002	0.740 $\pm$ 0.003
MLP decoders	28.6	9.04	0.723 $\pm$ 0.001	0.806 $\pm$ 0.007
LoRA $\to$ TabPFN	89.3	4.93	0.653 $\pm$ 0.001	0.749 $\pm$ 0.003

Table 4: Downstream adaptation strategies on Monroe. Win % is computed pairwise within this group.

Metric	$n$ tasks	Monroe	$\text{MiniMol}_{\mathsf{PFN}}$	$\text{CheMeleon}_{\mathsf{PFN}}$
AUROC	8	0.853 $\pm$ 0.034	0.847 $\pm$ 0.032	0.811 $\pm$ 0.049
AUPRC	5	0.706 $\pm$ 0.035	0.701 $\pm$ 0.045	0.625 $\pm$ 0.048
$1-\text{MAE}/\sigma_{y}$	5	0.610 $\pm$ 0.029	0.583 $\pm$ 0.032	0.547 $\pm$ 0.038
Pearson $r$	4	0.762 $\pm$ 0.022	0.741 $\pm$ 0.019	0.642 $\pm$ 0.026
Spearman $r$	4	0.579 $\pm$ 0.064	0.558 $\pm$ 0.079	0.541 $\pm$ 0.078

Table 5: Polaris under repeated Bemis–Murcko scaffold splits grouped by metric.

1-\text{MAE}/\sigma_{y}

is used for meaningful aggregation across scales. Scores are not comparable to Table 1 , which uses the official benchmark split.

	Polaris Rank $\downarrow$			MoleculeACE RMSE $\downarrow$			MoleculeACE Cliff RMSE $\downarrow$
Support	Monroe	$\text{MiniMol}_{\mathsf{PFN}}$	$\text{CheMeleon}_{\mathsf{PFN}}$	Monroe	$\text{MiniMol}_{\mathsf{PFN}}$	$\text{CheMeleon}_{\mathsf{PFN}}$	Monroe	$\text{MiniMol}_{\mathsf{PFN}}$	$\text{CheMeleon}_{\mathsf{PFN}}$
1%	23.29	23.79	24.36	1.173 $\pm$ 0.019	1.162 $\pm$ 0.030	1.247 $\pm$ 0.021	1.195 $\pm$ 0.036	1.181 $\pm$ 0.043	1.244 $\pm$ 0.032
5%	20.64	21.86	23.68	1.014 $\pm$ 0.010	1.015 $\pm$ 0.010	1.116 $\pm$ 0.006	1.048 $\pm$ 0.016	1.050 $\pm$ 0.016	1.140 $\pm$ 0.011
10%	17.54	19.39	22.96	0.914 $\pm$ 0.008	0.920 $\pm$ 0.006	1.022 $\pm$ 0.009	0.968 $\pm$ 0.013	0.977 $\pm$ 0.007	1.052 $\pm$ 0.012
25%	11.93	16.29	21.57	0.786 $\pm$ 0.008	0.794 $\pm$ 0.006	0.854 $\pm$ 0.010	0.867 $\pm$ 0.014	0.880 $\pm$ 0.010	0.924 $\pm$ 0.009
50%	0 7.29	10.57	18.50	0.707 $\pm$ 0.005	0.717 $\pm$ 0.004	0.747 $\pm$ 0.004	0.809 $\pm$ 0.008	0.822 $\pm$ 0.006	0.851 $\pm$ 0.004

Table 6: Downstream performance as a function of labelled support-set size.

Method	Polaris Rank $\downarrow$	MACE RMSE $\downarrow$	MACE Cliff RMSE $\downarrow$
Monroe	3.71	0.629	0.737
$\text{Monroe}_{\mathsf{ENS}}(2)$	3.18	0.623	0.733
$\text{Monroe}_{\mathsf{ENS}}(4)$	2.86	0.620	0.730
$\text{Monroe}_{\mathsf{ENS}}(8)$	3.00	0.621	0.732
$\text{Monroe}_{\mathsf{ENS}}(9)$	2.71	0.619	0.732
$\text{Monroe}_{\mathsf{ENS}}(10)$	2.68	0.615	0.726

Table 7: Prediction-level ensembling of Monroe.

\text{Monroe}_{\mathsf{ENS}}(k)

pools TabPFN predictions from the

k

strongest models.

\text{MiniMol}_{\mathsf{PFN}}

added at

k{=}9

and

\text{CheMeleon}_{\mathsf{PFN}}

added at

k{=}10

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Size	Metric	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
CYP2D6 Veith	13,130	AUPRC ( $\uparrow$ )	0.801	0.790	0.749	0.739	0.739	0.737	0.726	0.723	0.721	0.718	0.716	0.705	0.698	0.685	0.673
CYP3A4 Veith	12,328	AUPRC ( $\uparrow$ )	0.919	0.916	0.904	0.902	0.901	0.895	0.895	0.884	0.884	0.881	0.876	0.875	0.874	0.873	0.867
CYP2C9 Veith	12,092	AUPRC ( $\uparrow$ )	0.895	0.859	0.839	0.835	0.833	0.829	0.826	0.825	0.816	0.796	0.789	0.786	0.783	0.777	0.776
Solubility AqSolDB	9,982	MAE ( $\downarrow$ )	0.657	0.681	0.719	0.726	0.743	0.744	0.761	0.775	0.776	0.789	0.792	0.796	0.796	0.827	0.828
LD50 Zhu	5,907	MAE ( $\downarrow$ )	0.533	0.547	0.552	0.555	0.573	0.576	0.588	0.593	0.606	0.612	0.614	0.614	0.621	0.622	0.625
Ames	5,821	AUROC ( $\uparrow$ )	0.881	0.875	0.871	0.870	0.869	0.868	0.868	0.866	0.866	0.865	0.862	0.852	0.850	0.847	0.847

Appendix

Table 8: Consolidated leaderboard from Polaris and TDC ADMET benchmarks.

\text{Monroe}_{\mathsf{ENS}}(10)

(black) is the ensemble of Table 7 . We replace as-reported results with our own evaluations for Monroe ,

\text{MiniMol}_{\mathsf{PFN}}

\text{CheMeleon}_{\mathsf{PFN}}

, MiniMol , CheMeleon and MolFormer .

Figure 5: Per-task pairwise comparisons on 28 ADMET tasks from Polaris. Error bars are 95% confidence intervals; interval overlap corresponds to an uncorrected pairwise comparison. The colour reflects the multiplicity-corrected verdict against the best method (Tukey HSD followed by Benjamini–Hochberg FDR): blue marks the best-mean method on the task (a winner); gray marks methods tied with the best (also winners, counted as ties for first) whose interval overlaps the best’s; amber marks methods that are also counted as winners under FDR correction but whose interval does not overlap the best’s, meaning the intervals separate but the correction still classifies them as tied; red marks methods significantly worse than the best (losers). Dashed vertical lines mark the best method’s 95% CI; all error bars in a panel are equal-length under the pooled-variance assumption. MAE-based tasks are reported as scaled MAE (

1-\mathrm{MAE}/\mathrm{scale}

, see [ 1 ] ) so that higher is better across all panels.

Figure 6: Per-task pairwise comparisons on 30 activity-cliff tasks from MoleculeACE. Same color and CI conventions as Figure 5 : blue = best on this task, gray = tied with the best and interval overlaps, amber = tied with the best under FDR correction but interval does not overlap, red = significantly worse than the best. Each panel reports the overall test RMSE on one dataset (lower is better, so the best-mean method has the smallest RMSE).

Figure 7: PCBA per-assay distributions of the three filter signals: number of non-NaN labels (left), positive labels (middle), and negative labels (right) per assay. Dashed crimson lines mark the training-time cutoffs (1000 / 100 / 100). An assay is dropped if it fails any single threshold; this leaves 1089 of the 1328 input assays. The positive-count threshold accounts for the bulk of the rejected assays (235 below the cutoff), reflecting strong class imbalance in high-throughput screens.

Feature	CheMeleon	MiniMol	MolFormer
Strategy	Full finetuning	Frozen features + Head	Full finetuning
Task head	chemprop default FFN	Residual MLP	Linear on CLS token
Epochs	20	Max 50 (patience=5)	5
Model selection	Best val loss	Best val loss	Final epoch
Batch size	64	64	8
Optimizer	Adam	Adam	AdamW

Appendix

Table 10: Model-specific finetuning details.

Molecular Foundation Models

See all 20 papers

Apr 17, 2026

Tabular foundation models for in-context prediction of molecular properties

Accurate molecular property prediction is central to drug discovery, catalysis, and process design, yet real-world applications are often limited by small datasets. Molecular foundation models provide a promising direction by learning transferable molecular representations; however, they typically involve task-specific fine-tuning, require machine learning expertise, and often fail to outperform classical baselines. Tabular foundation models (TFMs) offer a fundamentally different paradigm: they perform predictions through in-context learning, enabling inference without task-specific training. Here, we evaluate TFMs in the low- to medium-data regime across both standardized pharmaceutical benchmarks and chemical engineering datasets. We evaluate both frozen molecular foundation model representations, as well as classical descriptors and fingerprints. Across the benchmarks, the approach shows excellent predictive performance while reducing computational cost, compared to fine-tuning, with these advantages also transferring to practical engineering data settings. In particular, combining TFMs with CheMeleon embeddings yields up to 100% win rates on 30 MoleculeACE tasks, while compact RDKit2d and Mordred descriptors provide strong descriptor-based alternatives. Molecular representation emerges as a key determinant in TFM performance, with molecular foundation model embeddings and 2D descriptor sets both providing substantial gains over classic molecular fingerprints on many tasks. These results suggest that in-context learning with TFMs provides a highly accurate and cost-efficient alternative for property prediction in practical applications.

Karim K. Ben Hicham, Jan G. Rittig, Martin Grohe +1

RWTH Aachen University, Process Systems Engineering (AVT.SVT), Aachen, Germany. · Lehrstuhl Informatik 7, RWTH Aachen University, Aachen, Germany. · JARA-CSD, Aachen, Germany. +1

Tabular Foundation Models Molecular Property Prediction Tabular ICL Molecular Representation Learning Molecular Foundation Models

Ask PDF

Aug 23, 2026

Mol-JEPA: A multimodal Joint Embedding Predictive Architecture for Molecules

Despite recent advances in molecular foundation models, several limitations remain, such as chemically invalid augmentations, modality collapse, and incomplete representation of biochemical environments. To address these challenges, we present Mol-JEPA, a scalable framework for learning molecular world models. Rather than relying on suboptimal molecular perturbations, our model uses modality masking to exploit information from molecular structures, cellular phenotypes, binding affinities, ADMET profiles, quantum chemistry simulations and other drug discovery data. Across various benchmarks, we show that the representations learned by Mol-JEPA deliver strong performance, demonstrating the value of incorporating biochemical context through latent space prediction.

Florian Rottach, Sebastian Schieferdecker, William Rudman +2

University of Tübingen · 2Boehringer Ingelheim · 3The University of Texas at Austin +1

Joint-Embedding Predictive Architecture Multimodal Pretraining Molecular Foundation Models Molecular Representation Learning Drug Discovery

Ask PDF

Mar 3, 2026

MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery

General-purpose large language models (LLMs) that rely on in-context learning do not reliably deliver the scientific understanding and performance required for drug discovery tasks. Simply increasing model size or introducing reasoning tokens does not yield significant performance gains. To address this gap, we introduce the MMAI Gym for Science, a one-stop shop molecular data formats and modalities as well as task-specific reasoning, training, and benchmarking recipes designed to teach foundation models the 'language of molecules' in order to solve practical drug discovery problems. We use MMAI Gym to train an efficient Liquid Foundation Model (LFM) for these applications, demonstrating that smaller, purpose-trained foundation models can outperform substantially larger general-purpose or specialist models on molecular benchmarks. Across essential drug discovery tasks - including molecular optimization, ADMET property prediction, retrosynthesis, drug-target activity prediction, and functional group reasoning - the resulting model achieves near specialist-level performance and, in the majority of settings, surpasses larger models, while remaining more efficient and broadly applicable in the domain.

Maksim Kuznetsov, Zulfat Miftahutdinov, Rim Shayakhmetov +17

Insilico Medicine · Liquid AI

Molecular Optimization Molecular Property Prediction Molecular Foundation Models Retrosynthesis Molecular Representation Learning Drug Discovery

Ask PDF

Abstract

Molecular Foundation Models

Explore similar work

Tabular foundation models for in-context prediction of molecular properties

Mol-JEPA: A multimodal Joint Embedding Predictive Architecture for Molecules

MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery