WEECFP-SuRGE: A Position-Aware Substructure Encoding Method for Molecular Property Prediction
Organizations: Eli Lilly and Company Lilly Corporate Center, Indianapolis, IN, United States of America
Abstract
Computational molecular property prediction requires representations that capture local chemistry, long-range interactions, and molecular topology. Conventional fingerprints provide efficient local substructure features, whereas learned graph and sequence models can represent broader context but often rely on pretraining or three-dimensional conformers. We introduce Wide Encoded Extended Connectivity Fingerprints (WEECFP) with Substructure Rotary Graph-distance Encoding (SuRGE), a tokenized hierarchical Morgan representation in which graph-distance-dependent rotations are applied at the input and within transformer self-attention. Across MoleculeNet and the Therapeutic Data Commons ADMET benchmarks, WEECFP-SuRGE is competitive with recent pretrained and geometry-aware methods without external pretraining or conformer generation. We also show that the rotated token representation remains structurally informative. A guided confirmed-handshake overlap procedure reconstructs the correct constitutional isomer for 92.6% of a 4,200-molecule self-library evaluation. Together, the predictive and reconstruction results indicate that WEECFP-SuRGE preserves local substructure identity while making relative topology available to the model.
Figures & tables
| Method | Regression (RMSE ↓) | Classification (ROC-AUC ↑) | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| FreeSolv N=642 | ESOL 1,128 | Lipo 4,200 | QM9 133,885 | QM9 gap 133,885 | QM9 133,885 | QM9 133,885 | ClinTox 1,478 | BACE 1,513 | BBBP 2,039 | Tox21 7,831 | HIV 41,127 | |
| WEECFP-SuRGE Ensemble | 2.060 2 ± 0.171 | 1.038 1 ± 0.132 | 0.666 1 ± 0.009 | 0.765 1 ± 0.089 | 0.0107 1 ± 0.0008 | 1.521 ± 0.541 | 0.661 ± 0.199 | 0.828 3 ± 0.041 | 0.810 2 ± 0.052 | 0.919 2 ± 0.012 | 0.681 3 ± 0.063 | 0.820 2 ± 0.032 |
| GotenNet | 2.259 3 ± 0.092 | 1.309 3 ± 0.322 | 0.714 2 ± 0.031 | 0.771 2 ± 0.100 | 0.0131 ± 0.0018 | 0.930 2 ± 0.197 | 0.492 2 ± 0.078 | 0.688 ± 0.052 | 0.757 3 ± 0.022 | 0.859 ± 0.041 | 0.655 ± 0.047 | 0.785 3 ± 0.036 |
| MolGraph- xLSTM | 2.862 ± 0.613 | 1.320 ± 0.184 | 0.737 ± 0.028 | 0.774 3 ± 0.115 | 0.0118 2 ± 0.0010 | 0.904 1 ± 0.161 | 0.430 1 ± 0.048 | 0.880 1 ± 0.032 | 0.743 ± 0.050 | 0.924 1 ± 0.042 | 0.707 2 ± 0.074 | 0.654 ± 0.047 |
| ViSNet | 2.620 ± 0.313 | 1.433 ± 0.292 | 0.734 3 ± 0.033 | 0.786 ± 0.095 | 0.0129 3 ± 0.0019 | 0.989 3 ± 0.293 | 0.516 3 ± 0.096 | 0.715 ± 0.056 | 0.742 ± 0.033 | 0.851 ± 0.045 | 0.626 ± 0.050 | 0.731 ± 0.050 |
| Best Base Model | 1.690 1 ± 0.167 | 1.231 2 ± 0.136 | 0.802 ± 0.028 | 0.850 ± 0.098 | 0.0136 ± 0.0013 | 1.653 ± 0.286 | 0.824 ± 0.209 | 0.844 2 ± 0.046 | 0.824 1 ± 0.033 | 0.898 3 ± 0.019 | 0.732 1 ± 0.058 | 0.832 1 ± 0.020 |
| Method | Regression rank (n) | Classification rank (n) | Overall rank (n) | Uses pretraining? |
|---|---|---|---|---|
| MapLight + GNN | 5.3 (9) | 3.5 (13) | 4.3 (22) | Yes (GIN) |
| WEECFP-SuRGE Ensemble* (ours) | 3.9 (9) | 5.9 (13) | 5.1 (22) | No |
| MiniMol | 6.4 (9) | 4.2 (12) | 5.2 (21) | Yes |
| WEECFP-NoSuRGE Ensemble* (ours) | 4.1 (9) | 6.2 (13) | 5.3 (22) | No |
| MapLight | 5.4 (9) | 6.0 (13) | 5.8 (22) | No |
| CFA | 9.2 (9) | 7.2 (12) | 8.0 (21) | No |
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| Ensemble member | Regression (RMSE ↓) | Classification (ROC-AUC ↑) | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| FreeSolv N=642 | ESOL 1,128 | Lipo 4,200 | QM9 133,885 | QM9 gap 133,885 | QM9 133,885 | QM9 133,885 | ClinTox 1,478 | BACE 1,513 | BBBP 2,039 | Tox21 7,831 | HIV 41,127 | |
| WEECFP- SuRGE (Input, md) | 2.229 3 ± 0.248 | 1.011 1 ± 0.177 | 0.725 3 ± 0.010 | 0.843 ± 0.105 | 0.0125 3 ± 0.0007 | 1.045 ± 0.229 | 0.468 2 ± 0.052 | 0.754 ± 0.122 | 0.778 3 ± 0.080 | 0.855 ± 0.033 | 0.676 ± 0.069 | 0.741 ± 0.036 |
| WEECFP- SuRGE+AE (Input, md) | 2.308 ± 0.218 | 1.237 3 ± 0.252 | 0.752 ± 0.012 | 0.919 ± 0.106 | 0.0127 3 ± 0.0008 | 1.569 ± 0.744 | 0.641 ± 0.300 | 0.684 ± 0.059 | 0.789 3 ± 0.061 | 0.886 ± 0.002 | 0.662 ± 0.067 | 0.767 ± 0.054 |
| Morgan- SuRGE (Input, md) | 2.972 ± 0.243 | 1.225 2 ± 0.249 | 0.765 ± 0.020 | 0.859 ± 0.111 | 0.0131 ± 0.0007 | 1.228 ± 0.195 | 0.511 ± 0.035 | 0.695 ± 0.089 | 0.805 3 ± 0.037 | 0.862 ± 0.038 | 0.648 ± 0.046 | 0.761 ± 0.047 |
| Morgan- SuRGE+AE (Input, md) | 2.428 ± 0.122 | 1.242 3 ± 0.231 | 0.740 ± 0.013 | 0.897 ± 0.102 | 0.0125 3 ± 0.0011 | 1.470 ± 0.621 | 0.654 ± 0.291 | 0.696 ± 0.060 | 0.785 3 ± 0.061 | 0.874 ± 0.016 | 0.656 ± 0.052 | 0.780 ± 0.050 |
| MLP + WEECFP+AE | 1.881 2 ± 0.257 | 1.002 1 ± 0.067 | 0.723 3 ± 0.019 | 0.861 ± 0.087 | 0.0117 2 ± 0.0010 | 1.405 ± 0.263 | 0.670 ± 0.161 | 0.682 ± 0.089 | 0.772 3 ± 0.048 | 0.891 ± 0.028 | 0.666 ± 0.049 | 0.810 3 ± 0.027 |
| Fingerprint | Regression (RMSE ↓) | Classification (ROC-AUC ↑) | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| FreeSolv N=642 | ESOL 1,128 | Lipo 4,200 | QM9 133,885 | QM9 gap 133,885 | QM9 133,885 | QM9 133,885 | ClinTox 1,478 | BACE 1,513 | BBBP 2,039 | Tox21 7,831 | HIV 41,127 | |
| AtomPair | 2.445 ± 0.164 | 1.272 3 ± 0.130 | 0.851 ± 0.012 | 0.890 ± 0.103 | 0.0169 ± 0.0024 | 2.168 ± 0.357 | 0.986 ± 0.113 | 0.573 ± 0.046 | 0.824 1 ± 0.033 | 0.831 ± 0.033 | 0.709 2 ± 0.053 | 0.790 3 ± 0.034 |
| Avalon | 2.052 2 ± 0.149 | 1.631 ± 0.212 | 0.829 ± 0.008 | 0.850 ± 0.098 | 0.0154 ± 0.0024 | 5.121 ± 1.052 | 2.243 ± 0.545 | 0.576 ± 0.047 | 0.796 3 ± 0.056 | 0.838 ± 0.036 | 0.691 3 ± 0.064 | 0.832 1 ± 0.020 |
| ErG | 2.667 ± 0.097 | 1.857 ± 0.303 | 0.857 ± 0.020 | 1.017 ± 0.118 | 0.0330 ± 0.0011 | 5.825 ± 0.798 | 2.888 ± 0.341 | 0.769 ± 0.029 | 0.754 ± 0.062 | 0.898 3 ± 0.019 | 0.711 2 ± 0.056 | 0.747 ± 0.042 |
| MACCS | 1.690 1 ± 0.167 | 1.593 ± 0.367 | 0.920 ± 0.013 | 0.886 ± 0.102 | 0.0165 ± 0.0020 | 6.211 ± 3.231 | 2.249 ± 1.038 | 0.598 ± 0.067 | 0.781 3 ± 0.049 | 0.839 ± 0.029 | 0.711 2 ± 0.053 | 0.801 3 ± 0.036 |
| MHFP6 | 2.762 ± 0.267 | 1.701 ± 0.079 | 0.885 ± 0.012 | 0.897 ± 0.098 | 0.0178 ± 0.0028 | 4.445 ± 0.745 | 1.983 ± 0.266 | 0.601 ± 0.047 | 0.802 3 ± 0.038 | 0.835 ± 0.031 | 0.693 3 ± 0.059 | 0.793 3 ± 0.029 |
| Fingerprint | Regression (RMSE ↓) | Classification (ROC-AUC ↑) | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| FreeSolv N=642 | ESOL 1,128 | Lipo 4,200 | QM9 133,885 | QM9 gap 133,885 | QM9 133,885 | QM9 133,885 | ClinTox 1,478 | BACE 1,513 | BBBP 2,039 | Tox21 7,831 | HIV 41,127 | |
| AtomPair | 2.612 ± 0.158 | 1.231 2 ± 0.136 | 0.910 ± 0.036 | 0.911 ± 0.111 | 0.0144 ± 0.0012 | 1.653 ± 0.286 | 0.824 ± 0.209 | 0.655 ± 0.067 | 0.821 2 ± 0.040 | 0.891 ± 0.022 | 0.614 ± 0.073 | 0.752 ± 0.039 |
| Avalon | 2.064 3 ± 0.178 | 1.681 ± 0.074 | 0.870 ± 0.016 | 0.900 ± 0.086 | 0.0153 ± 0.0056 | 4.404 ± 1.134 | 1.870 ± 0.593 | 0.697 ± 0.049 | 0.736 ± 0.050 | 0.876 ± 0.022 | 0.652 ± 0.059 | 0.803 3 ± 0.025 |
| ErG | 3.109 ± 0.427 | 2.011 ± 0.266 | 0.891 ± 0.033 | 1.029 ± 0.121 | 0.0318 ± 0.0019 | 5.881 ± 0.876 | 2.922 ± 0.395 | 0.713 ± 0.042 | 0.724 ± 0.081 | 0.850 ± 0.033 | 0.668 ± 0.035 | 0.740 ± 0.024 |
| MACCS | 1.888 2 ± 0.241 | 1.523 ± 0.308 | 0.938 ± 0.051 | 0.901 ± 0.092 | 0.0147 ± 0.0007 | 4.280 ± 1.336 | 1.636 ± 0.492 | 0.844 2 ± 0.046 | 0.773 3 ± 0.062 | 0.867 ± 0.034 | 0.699 3 ± 0.074 | 0.783 ± 0.041 |
| MHFP6 | 2.886 ± 0.256 | 1.882 ± 0.138 | 0.946 ± 0.024 | 0.915 ± 0.092 | 0.0136 ± 0.0013 | 4.083 ± 0.449 | 1.919 ± 0.220 | 0.776 ± 0.063 | 0.769 3 ± 0.043 | 0.853 ± 0.032 | 0.646 ± 0.058 | 0.751 ± 0.021 |
| Method | Regression (RMSE ↓) | Classification (ROC-AUC ↑) | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| FreeSolv N=642 | ESOL 1,128 | Lipo 4,200 | QM9 133,885 | QM9 gap 133,885 | QM9 133,885 | QM9 133,885 | ClinTox 1,478 | BACE 1,513 | BBBP 2,039 | Tox21 7,831 | HIV 41,127 | |
| Morgan Transformer | 2.829 ± 0.311 | 1.223 2 ± 0.098 | 0.758 ± 0.025 | 0.835 ± 0.098 | 0.0136 ± 0.0007 | 2.679 ± 0.462 | 1.218 ± 0.351 | 0.805 ± 0.078 | 0.769 3 ± 0.059 | 0.881 ± 0.032 | 0.647 ± 0.065 | 0.774 ± 0.039 |
| WEECFP- NoSuRGE | 2.159 3 ± 0.459 | 0.932 1 ± 0.061 | 0.719 3 ± 0.027 | 0.872 ± 0.108 | 0.0138 ± 0.0006 | 2.462 ± 0.296 | 1.238 ± 0.281 | 0.829 3 ± 0.060 | 0.774 3 ± 0.081 | 0.873 ± 0.022 | 0.690 3 ± 0.045 | 0.759 ± 0.058 |
| WEECFP- InputSuRGE (md) | 1.852 2 ± 0.203 | 0.861 1 ± 0.085 | 0.711 2 ± 0.027 | 0.852 ± 0.095 | 0.0126 3 ± 0.0007 | 1.062 ± 0.138 | 0.449 2 ± 0.020 | 0.809 ± 0.068 | 0.783 3 ± 0.054 | 0.853 ± 0.029 | 0.689 3 ± 0.057 | 0.756 ± 0.031 |
| WEECFP- InputSuRGE (ref3) | 2.074 3 ± 0.330 | 0.864 1 ± 0.056 | 0.723 3 ± 0.029 | 0.872 ± 0.102 | 0.0129 3 ± 0.0007 | 1.257 ± 0.180 | 0.596 ± 0.146 | 0.836 3 ± 0.065 | 0.787 3 ± 0.062 | 0.862 ± 0.031 | 0.689 3 ± 0.058 | 0.756 ± 0.046 |
| WEECFP- LayerSuRGE (md) | 2.241 3 ± 0.422 | 0.866 1 ± 0.050 | 0.706 2 ± 0.028 | 0.845 ± 0.109 | 0.0129 3 ± 0.0008 | 1.977 ± 0.124 | 0.891 ± 0.131 | 0.827 ± 0.044 | 0.785 3 ± 0.055 | 0.889 ± 0.013 | 0.689 3 ± 0.054 | 0.766 ± 0.030 |
| Seq. length S | Pairwise (MiB, fp16) | Ref3 (KiB, fp32) | Ratio |
|---|---|---|---|
| 50 | 19.5 | 818.8 | 48.9× |
| 100 | 78.1 | 1,637.5 | 97.7× |
| 200 | 312.5 | 3,275.0 | 195.4× |
| 500 | 1,953.1 | 8,187.5 | 488.5× |
| 1,000 | 7,812.5 | 16,375.0 | 977.1× |
| Benchmark | Metric | LB #1 (method) | LB #1 NoPT (method) | SuRGE Ensemble | NoSuRGE Ensemble | Rank [#all, #NoPT] |
|---|---|---|---|---|---|---|
| Caco-2 | MAE ↓ | 0.256 (CaliciBoost) | 0.256 (CaliciBoost) | 0.276 | 0.284 | [4, 4] |
| Lipophil. | MAE ↓ | 0.456 (MiniMol) | 0.467 (Chemprop-RDKit) | 0.447* | 0.460 | [1, 1] |
| Solubility | MAE ↓ | 0.741 (MiniMol) | 0.761 (Chemprop-RDKit) | 0.745 | 0.762 | [2, 1] |
| PPBR | MAE ↓ | 7.440 (Gradient Boost) | 7.440 (Gradient Boost) | 7.569 | 7.499 | [3, 2] |
| VDss | Spearman ↑ | 0.713 (MapLight+GNN) | 0.707 (MapLight) | 0.617 | 0.619 | [5, 4] |
| Half-life | Spearman ↑ | 0.576 (CFA) | 0.576 (CFA) | 0.431 | 0.433 | [9, 7] |
| Benchmark | Metric | LB #1 (method) | LB #1 NoPT (method) | SuRGE Ensemble | NoSuRGE Ensemble | Rank [#all, #NoPT] |
|---|---|---|---|---|---|---|
| HIA (Hou) | AUROC ↑ | 0.993 (MiniMol) | 0.990 (DeepMol (AutoML)) | 0.9914 | 0.9932* | [2, 1] |
| Pgp (Broccatelli) | AUROC ↑ | 0.938 (MapLight+GNN) | 0.935 (ZairaChem) | 0.9452* | 0.9383* | [1, 1] |
| Bioavailability (Ma) | AUROC ↑ | 0.942 (MiniMol) | 0.935 (ZairaChem) | 0.7056 | 0.6869 | [21, 17] |
| BBB (Martins) | AUROC ↑ | 0.924 (MiniMol) | 0.920 (CFA) | 0.9161 | 0.9159 | [3, 2] |
| CYP2C9 Inhib. (Veith) | AUPRC ↑ | 0.859 (MapLight+GNN) | 0.789 (ADMETrix) | 0.7684 | 0.7684 | [9, 5] |
| CYP2D6 Inhib. (Veith) | AUPRC ↑ | 0.790 (MapLight+GNN) | 0.723 (MapLight) | 0.6969 | 0.6853 | [7, 3] |
| Hyperparameter | Value |
|---|---|
| Token dimension | 1024 (WEECFP); 3387 (+AE variants) |
| d_model | (transformer and CNN) |
| Feed-forward width | 256 (transformer) |
| Layers L | 3 (transformer) / 3 residual Conv1D blocks (CNN) |
| Heads | (transformer); |
| Max sequence length | 300 tokens (+1 [CLS]) |
Molecular Representation Learning
Molecular representation learning encodes chemical structures and context for prediction, generation, and discovery across chemistry and biology.
See all 205 papers