Systematic Evaluation of Training Data Generation Strategies for Machine Learning-Based Molecular and Materials Property Prediction
Organizations: Institute of Chemistry and Biochemistry, Freie Universität Berlin, Arnimallee 22, 14195 Berlin, Germany · Department of Polymer and Functional Materials, CSIR-Indian Institute of Chemical Technology, 500007, Hyderabad, India
Abstract
Machine-learning (ML) models for molecular and materials property prediction are often trained on large datasets curated through arbitrary or generic sampling strategies. Such approaches can lead to the over- or under-representation of regions of chemical and materials space that are most relevant to the prediction task. An important challenge is therefore how to efficiently curate training data to maximize predictive performance while minimising the number of labelled examples required. Here, we systematically compare training data acquisition strategies for molecular and materials property prediction, ranging from random sampling to various active learning approaches. We evaluate their performance across different model classes, representations, datasets, and target properties. Our results show that the effectiveness of data acquisition strategies depend strongly on the interplay between the model, dataset, and prediction task, although uncertainty-based sampling generally outperforms alternative strategies. Based on these findings, we derive guidelines for selecting data acquisition strategies under a given labelling budget.
Figures & tables
| Dataset | Train | Val. | Test | |||
|---|---|---|---|---|---|---|
| QM9 | 107,106 | 13,386 | 13,389 | 7,106 | 5 000 | 9 |
| QM9small | 5,356 | 670 | 669 | 356 | 500 | 10 |
| Matbench MP | 106,201 | 13,276 | 13,275 | 5,000 | 5,000 | 9 |
| Pourbaix Stability | 5,496 | 687 | 687 | 496 | 500 | 10 |
| Dataset | Model | RMSE | MAE | |
|---|---|---|---|---|
| QM9 | Ensemble | 0.339 | 0.257 | 0.931 |
| SchNet | 0.090 | 0.062 | 0.995 | |
| QM9small | Ensemble | 0.387 | 0.303 | 0.907 |
| SchNet | — | |||
| Matbench MP | Ensemble | 0.268 | 0.140 | 0.947 |
| SchNet | 0.070 | 0.036 | 0.996 | |
| Dataset | Model | RS | US | DS |
|---|---|---|---|---|
| QM9 | Ensemble | 26 000 (25%) | 19 000 (17%) | 24 000 (23%) |
| QM9small | Ensemble | 2 500 (46%) | 2 100 (40%) | 2 800 (52%) |
| Matbench MP | Ensemble | 16 000 (15%) | 14 000 (13%) | 17 000 (16%) |
| Pourbaix stability | Ensemble | 4 200 (77%) | 3 000 (54%) | 3 600 (66%) |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | Parameter | QM9 | QM9small | MP | HSE06 |
|---|---|---|---|---|---|
| RF | n_estimators | 100 | 100 | 500 | 500 |
| max_features | sqrt | sqrt | 0.8 | 0.8 | |
| min_samples_leaf | 2 | 2 | 1 | 1 | |
| KRR | kernel | Laplacian | Laplacian | Laplacian | Laplacian |
| 0.01 | 0.01 | 0.01 | 0.01 | ||
| 0.01 | 0.01 | 0.01 | 0.01 |
| Parameter | QM9 | Matbench MP |
|---|---|---|
| hidden channels | 64 | 64 |
| filters | 256 | 128 |
| interaction blocks | 6 | 6 |
| Gaussians | 50 | 50 |
| cutoff (Å) | 6.0 | 5.0 |
| readout | mean | mean |
Training Data Selection
Training data selection studies how to choose, filter, weight, or schedule examples to improve learning while reducing data and compute costs.
See all 196 papers