May 13, 2026 · cs.LGJ/K move · Enter open · S save
Gordan Prastalo, Kevin Maik Jablonka
Helmholtz-Zentrum Berlin für Materialien und Energie GmbH, Hahn-Meitner-Platz 1, 14109 Berlin, Germany · HIPOLE Jena (Helmholtz Institute for Polymers in Energy Applications Jena), Lessingstrasse 12–14, 077432026 Jena, Germany · Laboratory of Organic and Macromolecular Chemistry (IOMC), Friedrich Schiller University Jena, Humboldtstrasse 10, 07743 Jena, Germany · Center for Energy and Environmental Chemistry Jena (CEEC Jena), Friedrich Schiller University Jena,May Philosophenweg 7a, 07743 Jena, Germany · Jena Center for Soft Matter (JCSM), Friedrich Schiller University Jena, Philosophenweg 7, 07743 Jena, Germany
Scientific machine learning reports predictive performance. It does not report whether the same prediction would survive a different draw of training data. Across
9 chemistry benchmarks, two classifiers trained on independent bootstraps of the same training set agree on aggregate accuracy to within
1.3–4.2 percentage points but disagree on the class label of
8.0–21.8% of test molecules. We call this gap \emph{cross-sample prediction churn}. The standard parameter-side techniques (deep ensembles, MC dropout, stochastic weight averaging) do not reduce this gap; two data-side methods do. The first is
K-bootstrap bagging, which cuts the rate
40–54% on every dataset at no accuracy cost (
K×-ERM compute). The second is \emph{twin-bootstrap}, our proposal: two networks trained jointly on independent bootstraps with a sym-KL consistency loss between their predictions, which at matched
2×-ERM compute reduces churn a further median
45% beyond bagging-
K=2. Cross-sample prediction churn deserves a column alongside predictive performance in scientific-ML benchmark reports, because without it the parameter-side and data-side methods are indistinguishable on the metric they actually differ on.