Classical structural priors improve neural molecular representations in molecule language models
Authors: Donghyeok Lee, Sangkwon Park, Munyoung Chang, Sungroh Yoon
Organizations: Interdisciplinary Program in Artificial Intelligence, Seoul National University, Seoul 08826, Republic of Korea · Department of Electrical and Computer Engineering, Seoul National University, Seoul 08826, Republic of Korea · Education and Research Program for Future ICT Pioneers, Seoul National University, Seoul 08826, Republic of Korea · Department of Otorhinolaryngology-Head & Neck Surgery, Seoul National University Bundang Hospital, Seongnam 13620, Republic of Korea
Classical molecular similarity is central to chemical information systems, yet molecule-language models trained on paired structure-text data do not necessarily organize molecular embeddings according to graded structural relationships. Here we introduce Structure-Aware Guided Alignment (SAGA), a training-only regularizer that uses Centered Kernel Alignment to align neural molecular representations with fingerprint-based Tanimoto similarity, without adding parameters or requiring fingerprints at inference. In MolCA and Atomas, SAGA improves molecule-text retrieval, shifts non-exact retrieval toward closer structural analogues, and increases average scaffold-split property-prediction performance, while molecular description and text-to-molecule generation are improved or preserved in single-run ablations. These results show that established chemoinformatics priors can make molecule-language representations more chemically faithful while retaining their multimodal utility.
Figures & tables
Figure 1: Overview of SAGA. A CKA-based auxiliary loss aligns pairwise similarities among neural molecular embeddings with fingerprint-based structural similarities, transferring a classical chemoinformatics prior into molecule-language models without adding trainable parameters or requiring fingerprints at inference.
Figure 2: Structural similarity and relational alignment. (a) Structure-property relationship showing median property difference (regression) and label agreement rate (classification) across Tanimoto similarity bins. (b) Statistical analysis on 20 TDC ADMET datasets. ρ : Spearman correlation; Effect: Cliff’s δ (regression) / Cohen’s h (classification). (c) CKA comparison of matched and shuffled structure-text pairs.
Text → Molecule
Molecule → Text
Model
R@1
R@5
R@10
MRR
R@1
R@5
R@10
MRR
MolCA
31.48
48.30
54.09
39.31
29.15
44.33
48.73
36.32
+ SAGA
35.82
57.55
65.42
45.88
31.55
49.73
56.73
40.29
Δ
+4.34
+9.25
+11.33
+6.57
+2.40
+5.40
+8.00
+3.97
(%)
(+13.8%)
(+19.2%)
(+21.0%)
(+16.7%)
(+8.2%)
(+12.2%)
(+16.4%)
(+10.9%)
Atomas-B
33.56
58.62
68.85
45.33
31.05
55.75
65.19
42.90
Table 1: Molecule–text retrieval performance on the ChEBI-20 validation split. R@K denotes Recall at K and MRR denotes Mean Reciprocal Rank. Δ : absolute difference. Atomas-B and Atomas-L denote Base and Large variants, respectively. Best results for each base model are shown in bold .
Figure 3: SAGA improves the structural quality of retrieved neighbours. (a) Exclude-self structural neighbourhoods for MolCA Text → Molecule retrieval on the ChEBI-20 validation split ( n=3,301 ). For each query, the exact target was removed from the candidate pool and the Tanimoto similarity between the target and the next retrieved molecules was measured using Morgan fingerprints (radius 2, 2,048 bits). Bars summarize mean Tanimoto values for the top-1, top-2, and top-3 non-exact neighbours, and box plots show the per-query distributions. Differences are significant by one-sided Wilcoxon signed-rank test ( p<10−100 ). (b) Per-query structural shift versus exact-match rank gain in Text → Molecule retrieval. Points above the diagonal indicate queries for which MolCA + SAGA retrieves structurally closer non-exact neighbours, and colour denotes the signed log-rank improvement of the exact target.
Model
BBBP
Tox21
ToxCast
SIDER
ClinTox
MUV
HIV
BACE
Avg
MolCA
0.681 ± 0.008
0.704 ± 0.005
0.687 ± 0.003
0.628 ± 0.007
0.672 ± 0.030
0.536 ± 0.030
0.749 ± 0.013
0.756 ± 0.008
0.677
+ SAGA
0.688 ± 0.013
0.701 ± 0.007
0.715 ± 0.007
0.652 ± 0.006
0.669 ± 0.042
0.651 ± 0.009
0.754 ± 0.007
0.809 ± 0.010
0.705
Atomas-B
0.703 ± 0.003
0.659 ± 0.004
0.670 ± 0.005
0.584 ± 0.015
0.955 ± 0.003
0.579 ± 0.011
0.739 ± 0.009
0.742 ± 0.016
0.704
+ SAGA
0.704 ± 0.007
0.676 ± 0.002
0.677 ± 0.007
0.621 ± 0.009
0.994 ± 0.001
0.605 ± 0.007
0.714 ± 0.009
0.753 ± 0.009
0.718
Atomas-L
0.726 ± 0.007
0.683 ± 0.006
0.661 ± 0.008
0.604 ± 0.003
0.959 ± 0.002
0.632 ± 0.015
0.716 ± 0.022
0.741 ± 0.015
0.715
+ SAGA
0.735 ± 0.012
0.690 ± 0.006
0.677 ± 0.004
0.614 ± 0.005
0.978 ± 0.014
0.591 ± 0.025
0.741 ± 0.006
0.771 ± 0.012
0.725
Table 2: Molecular property prediction performance (ROC-AUC) on scaffold-split MoleculeNet classification tasks. Results are reported as mean ± standard deviation over multiple runs. Atomas-B and Atomas-L denote Base and Large variants, respectively. Best results for each base model are shown in bold .
Model
BLEU ↑
Exact ↑
Lev. ↓
MACCS ↑
RDK ↑
Morgan ↑
Valid. ↑
Atomas-B
79.20
17.20
20.86
85.70
74.60
68.30
92.50
+ SAGA
79.40
17.10
20.83
86.10
75.10
68.70
92.30
Atomas-L
82.80
24.10
17.24
89.00
80.10
74.50
94.50
+ SAGA
82.60
24.10
17.25
89.10
80.30
74.40
94.50
Table 3: Generation and description performance on ChEBI-20. Rows compare each baseline with its SAGA counterpart. Values are single-run results. Bounded metrics are multiplied by 100, whereas Levenshtein is reported as raw edit distance. Text → Molecule generation is evaluated for Atomas on the validation split. Molecule → Text description is evaluated on the MolCA test split and the Atomas validation split. Best results within each model pair are shown in bold .
Figure 4: Representation space analysis. t-SNE of MolCA graph embeddings (a–d) and quantitative metrics for all models (e). (a) Base vs SAGA ( λ=100 , Morgan), colored by Tanimoto-based clusters. (b) Effect of regularization weight λ . (c) Fingerprint type comparison. (d) Base vs SAGA colored by Murcko scaffold identity. Parenthetical values in (a–d) denote molecule–fingerprint CKA. (e) CKA alignment and clustering metrics (Silhouette, Calinski-Harabasz) across all models; bold marks the column maximum within each model section. Atomas and text-side embeddings are shown in Supplementary Figs. 5–7.
Figure 1: Supplementary Fig. 1 ∣ Structure-property alignment analysis for all 20 TDC ADMET datasets. Relationship between Tanimoto similarity (x-axis) and property alignment for each task. Regression tasks (blue) show median absolute property difference with interquartile range shading (lower is better); classification tasks (green) show label agreement rate with 95% Wilson confidence interval shading (higher is better). Spearman ρ is annotated in each panel.
Text → Molecule
Molecule → Text
Setting
R@1
R@5
R@10
MRR
R@1
R@5
R@10
MRR
(a) Regularization weight λ
Base (no SAGA)
31.88
47.17
52.98
39.18
28.87
42.59
47.56
35.67
λ=0.1
32.75
49.05
54.62
40.53
29.75
43.56
48.65
36.53
Δ
+0.87
+1.88
+1.64
+1.35
+0.88
+0.97
+1.09
+0.86
(%)
(+2.7%)
(+4.0%)
(+3.1%)
(+3.4%)
(+3.0%)
(+2.3%)
(+2.3%)
(+2.4%)
Table 1: Supplementary Table 1 ∣ Ablation study results on MolCA. Validation set performance on ChEBI-20. For (a) and (b), the batch size is fixed at 64. For (c), since the CKA sample size n equals the batch size in MolCA, a separate baseline is trained for each n to isolate the effect of SAGA. Δ : absolute difference from Base. %: relative improvement.
Text → Molecule
Molecule → Text
Setting
R@1
R@5
R@10
MRR
R@1
R@5
R@10
MRR
(a) Regularization weight λ ( n=128 , Morgan)
Base (no SAGA)
43.15
69.21
77.96
55.23
41.56
65.01
73.70
52.49
λ=0.1
44.66
71.83
80.37
57.06
40.89
66.34
75.09
52.66
Δ
+1.51
+2.62
+2.41
+1.83
− 0.67
+1.33
+1.39
+0.17
(%)
(+3.5%)
(+3.8%)
(+3.1%)
(+3.3%)
( − 1.6%)
(+2.0%)
(+1.9%)
(+0.3%)
Table 2: Supplementary Table 2 ∣ Ablation study results on Atomas-L. Validation set performance on ChEBI-20. The training batch size is fixed at 16; the CKA sample size n is decoupled via the fingerprint queue, with the fixed settings for each ablated factor given in the sub-section headers. Δ : absolute difference from Base. %: relative improvement.
Text → Molecule
Molecule → Text
Setting
R@1
R@5
R@10
MRR
R@1
R@5
R@10
MRR
(a) Regularization weight λ ( n=128 , Morgan)
Base (no SAGA)
33.56
58.62
68.85
45.33
31.05
55.75
65.19
42.90
λ=0.1
31.18
57.39
68.40
43.47
29.57
54.15
63.68
41.02
Δ
− 2.38
− 1.23
− 0.45
− 1.86
− 1.48
− 1.60
− 1.51
− 1.88
(%)
( − 7.1%)
( − 2.1%)
( − 0.7%)
( − 4.1%)
( − 4.8%)
( − 2.9%)
( − 2.3%)
( − 4.4%)
Table 3: Supplementary Table 3 ∣ Ablation study results on Atomas-B. Validation set performance on ChEBI-20. The training batch size is fixed at 16; the CKA sample size n is decoupled via the fingerprint queue, with the fixed settings for each ablated factor given in the sub-section headers. Δ : absolute difference from Base. %: relative improvement.
Figure 2: Supplementary Fig. 2 ∣ Validation MRR curves for SAGA configurations on MolCA. Mol-to-Text (left) and Text-to-Mol (right) retrieval on ChEBI-20 across training epochs. (a) Effect of SAGA regularization weight λ with batch size 64. Darker green indicates higher λ ; λ=100 achieves optimal performance. (b) Comparison of fingerprint types (Morgan, MACCS, RDKit) with λ=100 . (c) Effect of batch size with λ=100 ; solid lines denote SAGA, dashed lines denote baseline.
Figure 3: Supplementary Fig. 3 ∣ Validation MRR curves for SAGA configurations on Atomas-B (MolT5-base). Mol-to-Text (left) and Text-to-Mol (right) retrieval on ChEBI-20 across training steps. (a) Effect of SAGA regularization weight λ at CKA sample size n=128 . (b) Comparison of fingerprint types (Morgan, MACCS, RDKit) at λ=100 , n=128 . (c) Effect of CKA sample size n at λ=100 ; solid lines denote SAGA, dashed lines denote baseline.
Figure 4: Supplementary Fig. 4 ∣ Validation MRR curves for SAGA configurations on Atomas-L (MolT5-large). Mol-to-Text (left) and Text-to-Mol (right) retrieval on ChEBI-20 across training steps. (a) Effect of SAGA regularization weight λ at CKA sample size n=128 ; λ=1000 achieves the best performance. (b) Comparison of fingerprint types (Morgan, MACCS, RDKit) at λ=100 , n=128 . (c) Effect of CKA sample size n at λ=100 ; solid lines denote SAGA, dashed lines denote baseline.
Top 10 scaffolds
Scaffolds with ≥20 samples
Mol Clustering
Text Clustering
Mol Clustering
Text Clustering
Setting
Sil
CH
Sil
CH
Sil
CH
Sil
CH
MolCA (Graph Embedding)
Base
− 0.058
4.73
− 0.010
2.90
− 0.008
7.47
− 0.006
3.25
λ=1
− 0.043
6.59
− 0.010
2.98
− 0.012
11.97
+0.006
3.29
λ=100 (Ours)
− 0.067
13.71
+0.008
4.49
+0.018
33.09
+0.023
5.90
Table 4: Supplementary Table 4 ∣ Scaffold-based clustering metrics on molecular and text embeddings. Silhouette and Calinski–Harabasz computed with Murcko scaffold identity as cluster labels. Because scaffold frequencies are long-tailed, we report both Top 10 scaffolds (matching the scaffold-coloured t-SNE panel in the main text) and ≥20 samples (which drops small-cluster noise). Scaffold identity is not used during SAGA training. Bold marks the column maximum within each model section.
Figure 5: Supplementary Fig. 5 ∣ t-SNE of MolCA text embeddings on the ChEBI-20 validation set. (a) Base vs SAGA ( λ=100 , Morgan), colored by Tanimoto-based clusters. (b) Effect of regularization weight ( λ=1 vs λ=1000 ). (c) Fingerprint type comparison (MACCS vs RDKit). (d) Base vs SAGA colored by Murcko scaffold identity. Parenthetical values denote text–fingerprint CKA.
Figure 6: Supplementary Fig. 6 ∣ t-SNE of Atomas-B embeddings on the ChEBI-20 validation set. Top two rows: SMILES embeddings; bottom two rows: text embeddings. Within each modality, (a) Base vs SAGA ( λ=100 , Morgan), colored by Tanimoto-based clusters. (b) Effect of regularization weight ( λ=1 vs λ=1000 ). (c) Fingerprint type comparison (MACCS vs RDKit). (d) Base vs SAGA colored by Murcko scaffold identity. Parenthetical values denote embedding–fingerprint CKA.
Figure 7: Supplementary Fig. 7 ∣ t-SNE of Atomas-L embeddings on the ChEBI-20 validation set. Top two rows: SMILES embeddings; bottom two rows: text embeddings. Within each modality, (a) Base vs SAGA ( λ=100 , Morgan), colored by Tanimoto-based clusters. (b) Effect of regularization weight ( λ=1 vs λ=1000 ). (c) Fingerprint type comparison (MACCS vs RDKit). (d) Base vs SAGA colored by Murcko scaffold identity. Parenthetical values denote embedding–fingerprint CKA.
Figure 8: Supplementary Fig. 8 ∣ Structural quality of MolCA Molecule → Text retrieval neighbours. Exclude-self structural neighbourhoods on the ChEBI-20 validation split ( n=3,301 ). For each molecule query, the exact paired text was excluded before ranking, and the Tanimoto similarity between the query molecule and the molecules paired with the next retrieved texts was measured using Morgan fingerprints (radius 2, 2,048 bits). The mean top-3 Tanimoto similarity increased from 0.211 to 0.272 with SAGA, and SAGA retrieved structurally closer neighbours for 73.8% of queries (one-sided Wilcoxon signed-rank test, p=1.63×10−229 ).
Figure 9: Supplementary Fig. 9 ∣ Structural quality of Atomas-B Text → Molecule retrieval neighbours. Exclude-self structural neighbourhoods on the ChEBI-20 validation split ( n=3,299 ). The mean top-3 Tanimoto similarity increased from 0.273 to 0.309 with SAGA, and SAGA retrieved structurally closer neighbours for 57.8% of queries (one-sided Wilcoxon signed-rank test, p=5.77×10−37 ).
Figure 10: Supplementary Fig. 10 ∣ Structural quality of Atomas-B Molecule → Text retrieval neighbours. Exclude-self structural neighbourhoods on the ChEBI-20 validation split ( n=3,299 ). The mean top-3 Tanimoto similarity increased from 0.250 to 0.282 with SAGA, and SAGA retrieved structurally closer neighbours for 60.0% of queries (one-sided Wilcoxon signed-rank test, p=1.91×10−46 ).
Figure 11: Supplementary Fig. 11 ∣ Structural quality of Atomas-L Text → Molecule retrieval neighbours. Exclude-self structural neighbourhoods on the ChEBI-20 validation split ( n=3,299 ). The mean top-3 Tanimoto similarity increased from 0.286 to 0.338 with SAGA, and SAGA retrieved structurally closer neighbours for 63.9% of queries (one-sided Wilcoxon signed-rank test, p=9.99×10−98 ).
Figure 12: Supplementary Fig. 12 ∣ Structural quality of Atomas-L Molecule → Text retrieval neighbours. Exclude-self structural neighbourhoods on the ChEBI-20 validation split ( n=3,299 ). The mean top-3 Tanimoto similarity increased from 0.261 to 0.302 with SAGA, and SAGA retrieved structurally closer neighbours for 62.5% of queries (one-sided Wilcoxon signed-rank test, p=2.79×10−69 ).
Figure 13: Supplementary Fig. 13 ∣ Description-level structural-class overlap for MolCA retrieval examples. For representative Text → Molecule queries discussed in the main text, each panel shows the target molecule, query description, and top-3 non-exact molecules retrieved by the MolCA baseline and MolCA + SAGA. Retrieved candidates are annotated with the native text–graph match score and their Morgan-fingerprint Tanimoto similarity to the target molecule. Red text denotes structural-class terms shared with the query description. The examples show that structurally closer SAGA neighbours also recover description terms corresponding to shared chemotypes or substructures.
Figure 13: Supplementary Fig. 13 ∣ Description-level structural-class overlap for MolCA retrieval examples, continued. Same layout and annotation scheme as the first panel; red text marks structural-class terms shared with the query description.
Figure 13: Supplementary Fig. 13 ∣ Description-level structural-class overlap for MolCA retrieval examples, continued. Same layout and annotation scheme as the first panel; red text marks structural-class terms shared with the query description.
Figure 14: Supplementary Fig. 14 ∣ Description-level structural-class overlap for Atomas-B retrieval examples. Text → Molecule retrieval examples for Atomas-B, shown with the same annotation scheme as Supplementary Fig. 13 . Scores are native Atomas text–SMILES cosine similarities, and red text marks structural-class terms shared with the query description. The SAGA neighbours have higher structural similarity to the target and frequently share chemotype terms with the query descriptions.
Figure 14: Supplementary Fig. 14 ∣ Description-level structural-class overlap for Atomas-B retrieval examples, continued. Same layout and annotation scheme as the first panel; red text marks structural-class terms shared with the query description.
Figure 14: Supplementary Fig. 14 ∣ Description-level structural-class overlap for Atomas-B retrieval examples, continued. Same layout and annotation scheme as the first panel; red text marks structural-class terms shared with the query description.
Figure 15: Supplementary Fig. 15 ∣ Description-level structural-class overlap for Atomas-L retrieval examples. Text → Molecule retrieval examples for Atomas-L, shown with the same annotation scheme as Supplementary Fig. 13 . Scores are native Atomas text–SMILES cosine similarities, and red text marks structural-class terms shared with the query description. These examples provide a text-side view of the same structural-neighbour effect measured by Tanimoto similarity.
Figure 15: Supplementary Fig. 15 ∣ Description-level structural-class overlap for Atomas-L retrieval examples, continued. Same layout and annotation scheme as the first panel; red text marks structural-class terms shared with the query description.
Figure 15: Supplementary Fig. 15 ∣ Description-level structural-class overlap for Atomas-L retrieval examples, continued. Same layout and annotation scheme as the first panel; red text marks structural-class terms shared with the query description.