Separability Is Not Alignability: Representations for Spectrum-to-Molecule Retrieval

Karol Musielinski; Fanurs C. E. Teh; Takahiko Koyama; Artur Dubrawski

cs.LGOct 27, 2026

Separability Is Not Alignability: Representations for Spectrum-to-Molecule Retrieval

Authors: Karol Musielinski, Fanurs C. E. Teh, Takahiko Koyama, Artur Dubrawski

Organizations: Auton Lab, Carnegie Mellon University · University of Warsaw · Keio University

Abstract

Identifying a small molecule from its tandem mass spectrum is limited by paired data. Foundation models offer a way around this scarcity: pretrain a spectrum encoder on unlabeled spectra and a molecule encoder on a large corpus of structures, then align the two frozen representations, as in vision-language and as the Platonic Representation Hypothesis suggests. We audit this route on MassSpecGym with spectrum encoders retrained free of test-fold contamination. Alignment needs a metric relating distinct molecules, but the objectives that produce spectrum encoders never see molecular structure: we prove that some predictive objectives fix molecular identity without fixing that metric. Moreover, we find that a candidate-based objective rewards a spectrum-independent prior over the benchmark's decoys, which we isolate. What ultimately addresses data scarcity is not architectural: simulated spectra surpass frozen geometry.

Figures & tables

Figure 1: Separability is not alignability. (a–d) UMAP of the same test spectra (12 molecules with

{\geq}12

spectra) using four different representations. Color is the molecule. (e) Separability against cross-modal agreement with MoLFormer, annotated with frozen MRR (Table 3 ).

Tower	Negatives	MRR	$\Delta_{0}$	$\Delta_{s}$
ChemBERTa	in-batch	0.147	$-0.004$	0.127
ChemBERTa	candidate	0.211	0.122	0.064
MoLFormer	in-batch	0.227	$-0.006$	0.209
MoLFormer	candidate	0.303	0.144	0.135

Table 1: Spectrum-blind ranking (left) and improved baselines (right). Left: frozen DreaMS contrastive;

\Delta_{0}=\mathrm{MRR}_{\text{blind}}-\mathrm{MRR}_{\text{rand}}

\Delta_{s}=\mathrm{MRR}_{\text{full}}-\mathrm{MRR}_{\text{blind}}

. Right: Ablation of simulated pretraining, fine-grained scoring, and candidate-aware training for fresh and MS-JEPA-pretrained encoders (App. J ).

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

	DreaMS (reproduced)	MS-JEPA	fresh encoder
Blocks / width / heads / MLP	7 / 1,024 / 8 / 4,096	8 / 768 / 12 / 3,072	8 / 768 / 12 / 3,072
Parameters	95.5M	64.1M encoder; 87.7M as (A)	87.7M, as MS-JEPA (A)
Peaks per spectrum	60 (SSL), 100 (contrastive)	100	100
$m/z$ encoding	fixed Fourier	fixed Fourier, $m/z$ and neutral loss	as MS-JEPA
Attention bias	no own parameters, shared	learned, per head and block	as MS-JEPA
Encoder conditioning	none	FiLM gains: adduct, charge, polarity	as MS-JEPA

Appendix

Table 2: The three spectrum encoders. The DreaMS and MS-JEPA columns describe pretraining; the fresh encoder is trained end to end with the recipe of App. D . The last two rows describe retrieval training (App. B.4 , B.5 ).

Within-modality (partition)
Metric	DreaMS SSL	DreaMS contr.	MS-JEPA	MS-JEPA [CLS]
Same-molecule silhouette ( $\uparrow$ )	+0.141	+0.282	+0.490	+0.381
$k$ NN purity@1 ( $\uparrow$ )	0.878	0.922	0.971	0.965
$k$ NN purity@10, normalized ( $\uparrow$ )	0.683	0.782	0.905	0.881
Fisher trace ratio ( $\uparrow$ )	1.85	2.12	6.45	3.56
Calinski–Harabasz ( $\uparrow$ )	10.3	11.9	36.1	20.0

Appendix

Table 3: Frozen-geometry panel, MassSpecGym test fold. MS-JEPA is the aggregated representation; MS-JEPA [CLS] is the encoder’s [CLS] token, which sees only the ion condition. Arrows give the preferred direction. Frozen MRR: in-batch MoLFormer adapter (App. D ); the adapter receives adduct and collision energy for DreaMS, no conditions for MS-JEPA (its encoder reads them), and collision energy and instrument for [CLS].

Figure 2: Cosine against Morgan Tanimoto for four representations of the same

3{,}170

test molecules (one random spectrum each), as a pair density with the mean cosine per Tanimoto bin (line). Top: DreaMS’s protocol,

40

Tanimoto bins with

2{,}500

pairs each drawn with replacement; Pearson

0.370

(mass null),

0.626

(DreaMS SSL),

0.661

(DreaMS contrastive),

0.548

(MS-JEPA). Bottom:

200{,}000

uniformly drawn molecule pairs; Pearson

0.189

0.228

0.185

0.200

in the same order.

Figure 3: Representational alignment across nine representations on the $2{,}994$ MassSpecGym test molecules covered by all of them. Lower triangle: mutual- $k$ NN at $k{=}10$ (chance $0.003$ ); upper triangle: linear CKA. Crimson lines separate molecule (top left) from spectrum (bottom right) representations; each triangle is colored on its own scale.

Tower	Negatives	MRR	blind	$\Delta_{0}$	$\Delta_{s}$	blind	$\Delta_{0}$	$\Delta_{s}$
			expected score			mean embedding
ChemBERTa	in-batch	0.147	0.020	$-0.004$	0.127	0.039	$+0.015$	0.108
ChemBERTa	candidate	0.211	0.147	$+0.122$	0.064	0.155	$+0.131$	0.056
MoLFormer	in-batch	0.227	0.018	$-0.006$	0.209	0.034	$+0.009$	0.193
MoLFormer	candidate	0.303	0.169	$+0.144$	0.135	0.160	$+0.136$	0.143

Appendix

Table 4: Spectrum-blind controls, frozen DreaMS contrastive, MassSpecGym test fold. Expected score: primary control (Table 1 , left); mean embedding: secondary control, conditions kept.

Recipe	Runs	MRR	Blind	MRR	Blind
		JESTR-NR		Change with penalty
Published	9	0.230	0.118	$-0.004$	$+0.005$
Simplified	5	0.163	0.029	$+0.013$	$+0.056$
Reweighted	3	0.235	0.118	$+0.020$	$+0.068$

Appendix

Table 5: Effect of JESTR’s candidate-list penalty, MassSpecGym test fold. Change: JESTR minus JESTR-NR, averaged over paired runs (the published runs share their first

970

epochs). Blind: expected-score control (random

0.024

). Reweighted: published recipe with

w=0.58

and the decoys in the positives’ BatchNorm batch.

(a)
Arm	MRR	R@1	$\Delta$ MRR [95% CI]
DreaMS contrastive, backbone LR; $\Delta$ vs. frozen
frozen	0.227	0.122
$5\times 10^{-6}$	0.255	0.144	$+0.028$ [0.016, 0.041]
$1.5\times 10^{-5}$	0.255	0.136	$+0.028$ [0.016, 0.041]
$5\times 10^{-5}$	0.244	0.130	$+0.018$ [0.005, 0.029]

Appendix

Table 6: Improved baselines , MassSpecGym test fold, one seed each.

\Delta

: paired MRR difference with a

95\%

interval from

1{,}000

bootstrap draws over test molecules (InChIKey-14). (a) Arms trained on real spectra;

\Delta

against the reference named in each block. (b) One-change steps of Table 1 (right) and pretraining on simulated spectra alone (sim only), for the fresh encoder and the pretrained MS-JEPA (A) encoder.

Molecular Representation Learning

Molecular representation learning encodes chemical structures and context for prediction, generation, and discovery across chemistry and biology.

See all 209 papers

MSAlign: Aligning Molecule and Mass Spectra Foundation Models for Metabolite Identification
May 19, 2026Paul Krzakala, Gabriel Melo, Camille Lançon +4Distribution Shift Cross-Modal Alignment
LTCI, Télécom Paris & CMAP, Ecole Polytechnique, Institut Polytechnique de Paris · LTCI, Télécom Paris, Institut Polytechnique de Paris · CEA, INRAE, MetaboHUB, Université Paris-Saclay +1
MS-GPT: Rethinking MS/MS De Novo Structure Elucidation as Spectrum-Induced Posterior Querying of a Molecule-Language Model
Jul 26, 2026Xin Zhao, Yumin Liu, Zhuo Li +5Language Modeling Molecular Generation
MoE Key Lab of Artificial Intelligence, AI Institute, School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · Instrumental Analysis Center, Shanghai Jiao Tong University, Shanghai, China · ByteDance Inc., Beijing, China +1
CoRe-Gen: Robust Spectrum-to-Structure Generation under Imperfect Fingerprint Conditions
May 13, 2026Tianbo Liu, Chixiang Lu, Jing Hao +4Autoregressive Generation Molecular Generation
The University of Hong Kong · The Chinese University of Hong Kong · Zhejiang Shuren University

May 19, 2026J/K move · Enter open · S save

MSAlign: Aligning Molecule and Mass Spectra Foundation Models for Metabolite Identification

Paul Krzakala, Gabriel Melo, Camille Lançon, Charlotte Laclau+3

LTCI, Télécom Paris & CMAP, Ecole Polytechnique, Institut Polytechnique de Paris · LTCI, Télécom Paris, Institut Polytechnique de Paris · CEA, INRAE, MetaboHUB, Université Paris-Saclay+1

Distribution Shift Cross-Modal Alignment Cross-Modal Retrieval Mass Spectrometry

Open paper AskPDF arXiv

Accurately identifying metabolites i.e. small molecules from mass spectrometry data remains a core challenge in metabolomics, with broad applications in drug discovery, environmental analysis, and clinical research. We address the Molecule Retrieval task, which consists in recovering the chemical structure of a metabolite from its MS/MS spectrum given a set of candidate molecules. While the recent release of benchmark datasets such as MassSpecGym and Spectraverse has considerably accelerated the development of novel machine learning approaches, the complexity of data preprocessing pipelines and the lack of unified implementations make methods and results difficult to reproduce and compare. We make three contributions. First, we propose a unified framework encompassing recent approaches based on representation alignment and contrastive learning. Second, we introduce MSAlign, inspired by multimodal alignment in vision-language models, which learns a shared representation space by aligning two frozen foundation models (DreaMS for mass spectra and ChemBERTa for molecules) through lightweight MLP projections trained with a candidate-based contrastive objective. MSAlign is simple to implement, fast to train and consistently outperforms existing approaches across all benchmarks. Third, we investigate a long-standing evaluation problem: data splitting strategies in molecule retrieval implicitly trade off data leakage against domain shift. We formalize this tension by introducing a quantitative measure of distribution shift, and use it to evaluate splitting strategies in existing benchmarks. All datasets, splits, candidate sets, and a unified implementation of MSAlign and baselines are publicly released to support reproducible research.

Abstract

Molecular Representation Learning

Explore similar work

MSAlign: Aligning Molecule and Mass Spectra Foundation Models for Metabolite Identification

MS-GPT: Rethinking MS/MS De Novo Structure Elucidation as Spectrum-Induced Posterior Querying of a Molecule-Language Model

CoRe-Gen: Robust Spectrum-to-Structure Generation under Imperfect Fingerprint Conditions