Chem World: A Large-Scale Benchmark and Physics-Aware Framework for Trustworthy Molecular Property Prediction
Organizations: Institute of Automation, Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · Cleer Science · Imperial College London ·
Abstract
Molecular property prediction is fundamental to computational chemistry and drug formulation research, yet existing benchmarks remain fragmented across datasets, properties, and evaluation protocols, limiting systematic evaluation of model generalization. We introduce Chem World, a large-scale benchmark integrating 17 public datasets with over 800,000 molecular and chemical-system instances across 10 property tracks, covering single-molecule and multi-component systems, including drug solubility. Chem World provides a unified testbed for evaluating molecular representations and compositional modeling across diverse chemical properties. Building on the benchmark, we propose PACM (Physics-Aware Compositional Modeling), which combines molecular representations with composition-aware modeling and domain-specific physical constraints. Extensive experiments show that molecular language models provide strong and transferable representations, while PACM consistently improves prediction across diverse chemical systems. Chem World and our PACM-powered drug-property prediction Agent support generalizable chemical modeling and offer a foundation for AI-assisted biomedical research and drug formulation development.
Figures & tables
| Benchmark | Scale | Property Coverage | Single-Molecule Tasks | Multi-Component Systems | Unified Protocol | OOD Evaluation |
| QM9 | 130K | Quantum chemical | Yes | No | Task-specific | Limited |
| MoleculeNet | Multiple datasets | Multiple | Yes | Limited | Suite-level | Task-dependent |
| CheMixHub | 500K | Multiple | Limited | Yes | Benchmark-specific | Yes |
| MixtureSolDB | 175K+ | Solubility | Limited | Yes | Dataset-specific | No |
| MolSets | 1K | Multiple | Set-based | Yes | Dataset-specific | No |
| Chem World | 800K+ | 10 tracks | Yes | Yes | Unified schema | Multiple settings |
| Encoder/Features | Model | Macro NRMSE | Macro | 95% CI (NRMSE) |
| RDKit descriptors | XGBoost | 0.478 | 0.768 | [0.454, 0.494] |
| Molecular graph | GCN/GIN | 0.532 | 0.714 | [0.511, 0.552] |
| MolFormer | Mean Pooling | 0.445 | 0.795 | [0.421, 0.467] |
| MolT5 | Mean Pooling | 0.458 | 0.786 | [0.439, 0.486] |
| MolFormer | DeepSets | 0.433 | 0.806 | [0.412, 0.447] |
| MolT5 | Set Transformer | 0.416 | 0.821 | [0.393, 0.431] |
| Property Track | MAE | RMSE | 95% CI (RMSE) | |
| Conductivity | 0.689 | 0.906 | 0.914 | [0.881, 0.924] |
| Density | 0.681 | 0.917 | 0.921 | [0.892, 0.937] |
| Enthalpy of Mixing | 0.793 | 1.008 | 0.883 | [0.970, 1.049] |
| Fusion Enthalpy | 1.474 | 1.904 | 0.769 | [1.822, 1.989] |
| Heat of Vaporization | 0.892 | 1.186 | 0.852 | [1.135, 1.236] |
| Melting Temperature | 1.249 | 1.642 | 0.787 | [1.574, 1.718] |
| Evaluation Protocol | NRMSE | |
| Group-aware | 0.367 | 0.858 |
| Molecule-disjoint | 0.422 | 0.824 |
| Composition-disjoint | 0.476 | 0.785 |
| Source-held-out | 0.509 | 0.754 |
| Condition extrapolation | 0.463 | 0.791 |
| Configuration | Macro NRMSE | |
| Full PACM | 0.367 | 0.858 |
| pairwise interactions | 0.389 | 0.843 |
| composition regularization | 0.398 | 0.835 |
| applicable temperature constraints | 0.382 | 0.848 |
| physical-boundary constraints | 0.412 | 0.821 |
| Constraint | Baseline (%) | PACM (%) |
| Composition-related criterion | 12.8 | 3.6 |
| Interaction-related criterion | 15.4 | 5.2 |
| Temperature-dependent criterion | 11.6 | 4.1 |
| Physical-boundary criterion | 9.7 | 1.8 |
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| Item | Policy |
| Unified schema | All source datasets are converted into a common schema with standardized property labels, target values, component identifiers (e.g., canonicalized SMILES where applicable), and optional context fields such as mole fractions, temperature, and pressure. Dataset-specific metadata and source identifiers are retained when available. |
| Primary benchmark split | The default protocol uses group-aware 5-fold cross-validation. Before splitting, each record is assigned a group identifier based on its chemical system and the applicable experimental conditions. All records belonging to the same group are assigned to a single partition and cannot be distributed across training, validation, and test sets. Each fold reserves approximately 20% of groups for testing; the remaining groups are divided into training and validation sets, yielding an approximate 70/10/20 train/validation/test ratio. |
| Group definition | Records with the same chemical composition measured under the same applicable conditions are grouped together. Records for the same composition under different temperatures or pressures are not automatically merged into one record or treated as identical measurements. The grouping key includes the relevant conditions when they are available and applicable to the target property. |
| Alternative OOD splits | The codebase additionally supports molecule-disjoint, composition-disjoint, source-held-out, and condition-extrapolation evaluations. These protocols test generalization beyond the default group-aware split and should be reported separately from the primary benchmark results. |
| Canonicalization | Explicit RDKit canonical SMILES normalization is applied in the MON and NIST-logV processing pipelines. Other datasets retain source SMILES or use curated name-to-SMILES mappings where applicable. The canonicalization procedure used for each dataset is documented in its processing script. |
| Unit and label normalization | Units and label transformations are handled at the dataset-processing level. Log-transformed targets and other property-specific transformations are documented separately; the stored unit metadata are not assumed to describe a transformed target without qualification. |
| Method | Cond. | Dens. | Enth. | Fusion | Vap. | Melt. | MON | Odor | Sol. | Visc. | Avg. |
| RDKit + XGBoost | 0.427 | 0.411 | 0.506 | 0.499 | 0.446 | 0.508 | 0.435 | 0.624 | 0.482 | 0.442 | 0.478 |
| GCN/GIN + Mean Pooling | 0.477 | 0.468 | 0.537 | 0.597 | 0.519 | 0.588 | 0.468 | 0.634 | 0.535 | 0.497 | 0.532 |
| MolFormer + Mean Pooling | 0.392 | 0.387 | 0.435 | 0.519 | 0.419 | 0.483 | 0.388 | 0.554 | 0.396 | 0.477 | 0.445 |
| MolT5 + Mean Pooling | 0.402 | 0.383 | 0.431 | 0.537 | 0.435 | 0.509 | 0.395 | 0.558 | 0.416 | 0.514 | 0.458 |
| MolFormer + DeepSets | 0.373 | 0.356 | 0.427 | 0.525 | 0.373 | 0.523 | 0.358 | 0.580 | 0.423 | 0.392 | 0.433 |
| MolT5 + DeepSets | 0.407 | 0.392 | 0.468 | 0.549 | 0.461 | 0.547 | 0.391 | 0.602 | 0.457 | 0.425 | 0.469 |
| Baseline Model | Baseline Macro NRMSE | (vs. Proposed) | Raw -value | Holm |
| RDKit + XGBoost (Descriptor) | 0.478 | +0.111 | ||
| GCN/GIN + Mean Pooling (Graph) | 0.532 | +0.165 | ||
| MolFormer + Mean Pooling | 0.445 | +0.078 | ||
| MolT5 + Mean Pooling | 0.458 | +0.091 | ||
| MolFormer + DeepSets | 0.433 | +0.066 | ||
| MolT5 + Set Transformer | 0.416 | +0.049 |
Molecular Property Prediction
Molecular property prediction estimates chemical, physical, and biological properties from molecular structure and context.
See all 226 papers