Critical Mass: Structure-Disjoint Splits Drift With the Libraries They Are Built On

Tomo Oga; Devesh Shah; Cailum M. K. Stienstra; Gabriel Asher; Antonio H. de O. Fonseca; Niall O'Connor; Michael Widrich

cs.LGOct 8, 2026

Critical Mass: Structure-Disjoint Splits Drift With the Libraries They Are Built On

Authors: Tomo Oga, Devesh Shah, Cailum M. K. Stienstra, Gabriel Asher, Antonio H. de O. Fonseca, Niall O'Connor, Michael Widrich

Organizations: Matterworks, Inc. Somerville, MA 02143, USA

Abstract

Molecular machine learning evaluates out-of-distribution performance with structure disjoint splits, a construction rule fixed once at the scale its library happened to have: threshold a pairwise distance, then assign whole connected components to folds. For predicting structures from mass spectra, the field has converged on single-linkage clustering under the myopic Maximum Common Edge Subgraph (MCES) distance at a threshold of 10. This convention was set on 29,000 molecules but public libraries now hold 227,690 molecules. MCES is NP-hard per pair over a quadratically growing set, and so the criterion has never been evaluated at that scale. We propose an optimized version of this criterion, allowing for computationally feasible application on the public libraries: Admissible bounds and a union-find skip reduce $2.6 \times 10^{10}$ molecular pairs to only $1.15 \times 10^{7}$ integer programs, a $2{,}259\times$ saving, while provably producing the lossless partition. We find that at this scale the graph percolates. A single component holds $95.5\%$ of the library, capping any test set at $4.5\%$ . What remains is not merely smaller, but systematically different: a held-out compound outweighs a training compound 88.1% of the time, because a fixed absolute budget is a tighter relative tolerance on a larger molecule. Large library benchmarks using this splitting rule naively therefore conflate structural novelty with extrapolation to larger, more complex chemistry.

Figures & tables

Figure 1: The union-find skip turns a quadratic problem into a near-linear one. (a) Integer programs solved against library size

N

: all pairs and myopic-MCES both scale as

N^{2}

, ours as

N^{0.93}

, a

70\times

gap at the full union. (b) The union-skip is the only stage that improves substantially with

N

. (c) One 24-structure cluster: of the 276 pairs among its members, 30 were solved near, 23 merging components, and 7 closing cycles, 11 were solved and returned far, and 235 never reached the solver. Myopic-MCES counts in (a) are calculated with

5\times 10^{6}

-pair samples per rung (Appendix C ).

Figure 2: The vanishing holdout of MCES with increasing dataset size skews high mass . Structure-disjoint splits at MCES-10. (a) The largest cluster is indivisible, so the fraction outside it, $1-g/N$ , upper-bounds any evaluation set; horizontal lines mark what each split ratio would require. (b) The mass holdout AUC, the probability that a compound outside the giant component is heavier than one inside it, reaches 0.881 for the union and 0.836 for Enveda-180. Bands are min-max across seeds.

Figure 3: MCES splitting results in high-mass skew for holdout set, irrespective of dataset mass distributions. (a) The two libraries the public union pools. Enveda-180 supplies 80.6% of the union and its shape dominates. (b) The union’s giant component against the available holdout. (c) The same on Enveda-180 alone.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

	endpoint elements	elements $+$ bond weight
class	C-C	C-C:1.5	C-C:1.0
benzene $a_{\ell}$	$9.0$	$9.0$	$0.0$
cyclohexane $b_{\ell}$	$6.0$	$0.0$	$6.0$
$\|a_{\ell}-b_{\ell}\|$	$3.0$	$9.0$	$6.0$
bound $\sum_{\ell}\|a_{\ell}-b_{\ell}\|$	$\mathbf{3.0}$	$\mathbf{15.0}$

Appendix

Table 1: Bond order must not enter the class label. Benzene and cyclohexane have the same six-carbon skeleton and differ only in bond order, so their true distance is

3.0

: match all six bonds, each costing

|1.5-1.0|=0.5

. Under the correct label both molecules occupy the single class C-C , their totals meet, and the bound is exactly

d

. Folding bond weight into the label splits them into disjoint classes in which each molecule holds zero weight in the other’s, so the

l_{1}

sum accumulates where it should cancel, overstating

d

five-fold. Any threshold in

(3,15]

would then discard a pair at distance

3

	Test	Pairs removed	Pairs left	Saving
	all pairs $\binom{N}{2}$	—	25,921,254,205	—
vectorized, over all pairs
1	total bond weight	7,395,768,861	18,525,485,344	$1.40\times$
2	per-class weight imbalance	14,891,197,847	3,634,287,497	$5.10\times$
3	per-element degree sequences	282,443,691	3,351,843,806	$1.08\times$
per pair

Appendix

Table 2: The pipeline on the public spectral union,

N=227{,}690

T=10

. What each test removes. Stage 1 and 2 are vectorized bounds that we introduce; Stage 3 is a vectorized version of filter1 ; Stage 4 is our union-find based skipping we introduce; Stage 5 is the per-pair neighborhood bound ( filter2 ), applied immediately before the solver. 99.956% of pairs are pruned before ever touching a solver.

Figure 4: Both per-pair reductions strengthen along dispatch order. Candidates are dispatched cheapest-first, in ascending order of their vectorized bound. Together they concentrate integer programs to the cheaper end of the dispatch order, which is also the end that builds components. The last 4 buckets hold 31.8% of dispatched pairs but only 3.0% of the programs. (a) The union-find skip cannot skip at all in the first few buckets as no components exist yet, but removes 96% of the last bucket, because by then the giant has absorbed most of the library and a candidate’s endpoints are usually already connected. (b) The per-pair bound’s rejection rate climbs because a pair whose weaker bound already sits near

T

is exactly the pair a stronger bound most easily pushes past. It rises from skipping 2.5% of pairs to 99.8% of them

How the integer program terminates	Programs	Share	Verdict
proves infeasibility	10,531,154	$91.76\%$	far
proves an optimum, $d\geq T$	200,403	$1.75\%$	far
proves an optimum, $d<T$	689,280	$6.01\%$	near
budget expires, unproven solution	2,415	$0.02\%$	near
budget expires, no solution found	53,063	$0.46\%$	near, conservatively
total	11,476,315	$100\%$

Appendix

Table 3: The exit outcomes of the 11,476,315 pairs that survived every pruning measure. Infeasibility proves farness, by construction of the program. The near verdicts form a graph of 744,758 edges, of which 220,598 merge components and 524,160 close cycles, for a total of 7,092 clusters with a giant of 217,440 (95.50%). The dispatch stage took 3,813 core hours, against 58,901 for myopic-MCES under MassSpecGym’s settings.

$N$	timeouts	giant, all far	giant, all near	interval
$10{,}000$	$993$	$63.02\%$	$63.45\%$	$0.58$ pp
$20{,}000$	$2{,}587$	$76.89\%$	$77.13\%$	$0.30$ pp
$40{,}000$	$5{,}652$	$86.08\%$	$86.22\%$	$0.17$ pp
$80{,}000$	$12{,}764$	$91.31\%$	$91.38\%$	$0.08$ pp
$160{,}000$	$32{,}198$	$94.41\%$	$94.51\%$	$0.17$ pp
$227{,}690$	$55{,}478$	$95.33\%$	$95.50\%$	$0.17$ pp

Appendix

Table 4: Every rung of the ladder is bounded. Timeout counts grow with

N

while the interval does not. The public union at

T=10

; rungs with several seeds are averaged and the interval is the widest of them.

Arm	Pool	Rungs $n$	Seeds $s$
Public union	227,690	10,000 / 20,000 / 40,000	$0,1,2$
		80,000 / 160,000	$0,1$
		227,690 (full)	—
Enveda-180 alone	183,476	10,000 / 20,000 / 40,000	$0,1,2$
		80,000 / 160,000	$0,1$
		183,476 (full)	—

Appendix

Table 5: Every run reported. Seeds are

1000+s

for

s

in the listed range; a full-library rung is unsampled and unseeded. All runs use

T=10

, a

5

s per-program budget, 240 workers and a dispatch block of

2{,}000{,}000

, except the threshold sweep as noted.

Figure 5: Probability that a uniformly drawn library member is near a molecule of the given total bond weight on MassSpecGym’s complete matrix, split by Equation 3 . Because of the fixed absolute width, the probability a library member has a near neighbor falls with total bond weight.

Figure 6: At a lower threshold a conventional split exists and is nearly mass unbiased, but its holdout is less novel. (a) Train against holdout for an achievable 80/10/10 split of the public union at $T=5$ . Median mass is 331.2 Da against 343.1 Da, compared to a $T=10$ , where the giant component and the available holdout differ by 122.1 Da. (b) Maximum ECFP4 Tanimoto to any training compound. Medians are 0.427 at $T=10$ , 0.566 at $T=5$ , and 0.648 at random, so the stricter threshold does buy a novel holdout.

Figure 7: Homologous pairs that are near-identical to ECFP4 yet provably distant under MCES. Green marks the maximum common edge subgraph; red marks the unmatched bonds that constitute the MCES distance. (a) Tetradecanoic vs. pentacosanoic acid and (b) tetra- vs. octaethylene glycol are fingerprint-identical (Tanimoto 1.000) despite MCES distances of 11 and 12; (c) quercetin di- vs. triglycoside (Tanimoto 0.983, MCES 12) and (d) diphenylamine vs. $N,N^{\prime}$ -diphenyl- $p$ -phenylenediamine (Tanimoto 0.923, MCES 11) show the same repeat-unit blindness at high but sub-unit Tanimoto.

Figure 8: A natural-product library behaves the same way. (a) Achievable holdout falls monotonically, $40.6\%$ to $14.6\%$ . (b) Mass holdout AUC rises monotonically, $0.647$ to $0.726$ , (c) Its two sources against their union. SpectraVerse and COCONUT overlap closely, medians $406.3$ and $412.3$ Da, which follows from both covering natural products and small metabolites. (d) The split at $N=160{,}000$ : the giant sits at median $393.1$ Da and everything outside it at $538.4$ Da.

OOD Generalization

Out-of-distribution generalization studies how machine-learning models perform when test data, domains, tasks, or environments differ from training.

See all 395 papers

When Molecular Similarity Works: Property Cliffs Reveal Hidden Errors
May 17, 2026Di Hu, Kun Li, Haojie Rao +6Molecular Property Prediction
School of Economics and Management · School of Computer Science, Wuhan University Wuhan, China · Department of Data Science and Artificial Intelligence, Monash University Victoria, Australia +2
FRIGID: Scaling Diffusion-Based Molecular Generation from Mass Spectra at Training and Inference Time
Apr 17, 2026Montgomery Bohde, Hongxuan Liu, Mrunali Manjrekar +4Inference-Time Scaling Molecular Generation
Massachusetts Institute of Technology, Cambridge, MA, United States · Texas A&M University, College Station, TX, United States
When should we trust the annotation? Selective prediction for molecular structure retrieval from mass spectra
Mar 11, 2026Mira Jürgens, Gaetan De Waele, Morteza Rakhshaninejad +1Selective Prediction Mass Spectrometry
Department of Data Analysis and Mathematical Modeling, Ghent University, Coupure Links 653, Ghent, 9000, Belgium. · Department of Computer Science, University of Antwerp, Middelheimlaan 1, Antwerp, 2020, Belgium.

May 17, 2026J/K move · Enter open · S save

When Molecular Similarity Works: Property Cliffs Reveal Hidden Errors

Di Hu, Kun Li, Haojie Rao, Longtao Hu+5

School of Economics and Management · School of Computer Science, Wuhan University Wuhan, China · Department of Data Science and Artificial Intelligence, Monash University Victoria, Australia+2

Molecular Property Prediction

Open paper AskPDF arXiv

Accurate prediction of molecular properties underpins drug discovery and material design, yet even state-of-the-art models remain vulnerable to localized failure modes that aggregate metrics cannot detect. The places where molecular similarity should be most helpful are also places where standard evaluation can be most misleading. Property cliffs expose this gap: structurally similar molecules can still differ sharply in target property, so models with competitive overall performance may fail in high-risk local neighborhoods. To expose and mitigate this failure mode, CliffSplit, a cliff-aware evaluation protocol that constructs locally supported, cliff-exposed test cases, and CliffLoss, a model-agnostic train-only mitigation mechanism for cliff-sensitive errors, are introduced. Experiments on three QM9 targets and three MoleculeNet tasks across five backbones show that CliffSplit reveals at least 15% higher error in cliff-heavy QM9 regions, while CliffLoss reduces the cliff-to-smooth error gap by up to 30% on Lipophilicity and improves overall MAE by 9.7%. Together, these results turn molecular similarity failure from a descriptive anomaly into a benchmarked evaluation problem for molecular machine learning. The code is available at https://anonymous.4open.science/r/Cliff_Loss.

Abstract

OOD Generalization

Explore similar work

When Molecular Similarity Works: Property Cliffs Reveal Hidden Errors

FRIGID: Scaling Diffusion-Based Molecular Generation from Mass Spectra at Training and Inference Time

When should we trust the annotation? Selective prediction for molecular structure retrieval from mass spectra