Towards Multiscale Molecular Representation Learning through Geometry-Based Fragmentation for Polymer Property Prediction
Authors: Makoto Takamoto, Chaitanya Sampara, Viktor Zaverkin, Daniel Oñoro-Rubio, Mathias Niepert
Organizations: NEC Laboratories Europe, Heidelberg, Germany · Heidelberg University, Heidelberg, Germany · Present address: INM – Leibniz Institute for New Materials, Saarbrücken, Germany; Saarland University, Department of Computer Science, Saarbrücken, Germany; German Research Center for Artificial Intelligence (DFKI), Saarbrücken, Germany · University of Stuttgart, Institute for Artificial Intelligence, Stuttgart, Germany
It is well established that molecular substructures can contribute unequally to chemical and physical properties, with specific motifs or side chains often playing dominant roles. Fragment-based representations have therefore emerged as an effective strategy for incorporating chemically meaningful inductive biases, as exemplified by molecular fingerprints and fragment-based attention mechanisms. However, many existing fragmentation-based machine learning methods rely on predefined chemical rules, which may yield suboptimal fragmentation for previously unseen or chemically unfamiliar structures and often overlook the importance of multiscale molecular representations. To address these limitations, we formulate molecular fragmentation as a radius-constrained optimization problem inspired by the k-center formulation, where the number of clusters is not fixed a priori but emerges from the specified radius, and propose a geometry-based fragmentation method that partitions arbitrary molecular or material structures using largely geometric information, without relying on predefined chemical fragmentation rules. The method provides flexible control over fragment size, enabling the construction of multiscale fragment representations for scale-adaptive modeling. Building on this framework, we develop a geometric fragment-based attention mechanism that integrates multiscale structural information and incorporate it into a multimodal machine learning model combining p-SMILES representations with three-dimensional (3D) conformational information. Experiments on a benchmark polymer dataset show that incorporating 3D conformational information improves prediction accuracy for several properties over models based exclusively on p-SMILES representations. Explainable artificial intelligence analyses further provide insights into the physical relevance of the learned representations.
Figures & tables
Figure 1: Schematic illustration of the geometric fragment of a monomer, []CC(O)[]. (A) Example of a fragmentation result with rpool=1A˚ and rcutoff=0.5A˚ with an explicit description of the representative atoms, rpool and rcutoff . Here, rpool sets the spacing of the representative atoms (centers), whereas rcutoff is an atom-centered radius that determines fragment membership. Following the ASE neighbor-list convention, two atoms are neighbors when their radius- rcutoff spheres overlap, so the effective pairwise cutoff is 2×cutoff . (B) Illustration of fragmentation at multiple scales, with rpool=1,2,3A˚ and corresponding rcutoff=0.5,1.0,1.5A˚ , respectively.
Figure 2: A schematic illustration of the fragment-based attention architecture. The top pathway represents the global information flow, while the lower three pathways correspond to large-, medium-, and small-scale flows, respectively.
Figure 3: Overview of the multi-modal machine learning model architecture. (A) Schematic illustration of the overall model structure. The snow mark above 3D conf. Encoder denotes the model weight of the 3D conformation embedding (MACE-MP) is frozen. (B) Schematic illustration of the modality fusion module, where “Norm” denotes layer normalization and “Add” indicates a residual connection. (C) Schematic illustration of the decoder module, where “SA Pooling” refers to self-attention pooling and “MLP” denotes a multilayer perceptron.
Figure 4: Fragmentation results of a monomer: []CCOC(=O)NCCOCCOCCNC(=O)O[].
Model Name
Modality
Test RMSE
Egc (eV)
Egb (eV)
Eea (eV)
Ei (eV)
Xc (%)
EPS
nc
TP (scratch) a
SMILES
0.58 ±0.01
0.60 ±0.03
0.36 ±0.03
0.46 ±0.04
18.9 ±0.51
0.57 ±0.11
0.099 ±0.015
TP (pretrained) a
SMILES
0.44 ±0.01
0.53 ±0.04
0.31 ±0.03
0.39 ±0.07
17.5 ±0.69
0.54 ±0.05
0.098 ±0.017
PB+MACE
SMILES
0.44 ±0.02
0.48 ±0.05
0.26 ±0.01
0.39 ±0.05
17.1 ±0.59
0.49 ±0.04
0.087 ±0.013
PB+MACE
3D
0.48 ±0.03
0.50 ±0.05
0.29 ±0.03
0.40 ±0.04
17.7 ±0.94
0.49 ±0.06
0.085 ±0.016
PB+MACE
SMILES+3D
0.42 ±0.01
0.46 ±0.04
0.27 ±0.02
0.39 ±0.05
17.1 ±0.48
0.47 ±0.05
0.085 ±0.012
Table 1: Benchmark results on polymer datasets. The performance is measured using root mean-square error (RMSE). TP denotes TransPolymer, and PB denotes PolyBERT. The number written in bold font indicates the best performance for each dataset.
Model Name
Branch
Test RMSE
Egc (eV)
Egb (eV)
Eea (eV)
Ei (eV)
Xc (%)
EPS
nc
PB+MACE
pSMILES+3D+Fragment
0.42 ±0.01
0.46 ±0.04
0.27 ±0.02
0.39 ±0.05
17.1 ±0.48
0.47 ±0.05
0.085 ±0.012
PB+MACE
3D+Fragment
0.42 ±0.01
0.48 ±0.05
0.28 ±0.02
0.39 ±0.05
17.1 ±0.47
0.50 ±0.05
0.084 ±0.012
PB+MACE
pSMILES+Fragment
0.42 ±0.02
0.46 ±0.05
0.27 ±0.03
0.38 ±0.05
17.1 ±0.48
0.51 ±0.05
0.085 ±0.012
PB+MACE
pSMILES+3D
0.41 ±0.02
0.45 ±0.04
0.27 ±0.02
0.38 ±0.05
17.5 ±0.52
0.48 ±0.04
0.081 ±0.013
TP+MACE
pSMILES+3D+Fragment
0.42 ±0.01
0.45 ±0.05
0.28 ±0.02
0.38 ±0.05
16.6 ±0.43
0.50 ±0.05
0.083 ±0.013
Table 2: Ablation Study of the branch dependence on polymer datasets. The performance is measured using root mean-square error (RMSE). TP denotes TransPolymer, and PB denotes PolyBERT. The number written in bold font indicates a result showing a relatively notable difference from the other results, although the difference remains within one standard deviation.
Figure 5: Scatter plot of latent vector embedded by MAP4 Capecchi et al. (2020) and fragment-attention branch. The embedded vectors are mapped onto two-dimensional space with tSNE. The color map shows the heavy atom number in the monomer.
Figure 6: A plot of importance-score of the input structure in the case of []CCC(=O)O[]. Only oxygen atoms are labeled, whereas carbon and hydrogen atoms are left unlabeled for clarity. The plotted importance-score is normalized by the maximum value of the original importance-score.
Figure 7: A plot of importance-score of the input structure in the case of []CCCCCCC(=O)N[]. Only oxygen and nitrogen atoms are labeled, whereas carbon and hydrogen atoms are left unlabeled for clarity. The plotted importance-score is normalized by the maximum value of the original importance-score.
Functional group
Structure
Mechanism
Carbonyl
C=O
Strong electron withdrawing Peet et al. (2007) ; Ansari et al. (2018) , planar conjugation enhances π -overlap Vu et al. (2018)
Fluoro
C–F, CF 3
High electronegativity Peet et al. (2007) ; Ansari et al. (2018) ; bulky groups disrupt packing Wu et al. (2025)
Ether
C–O–C
Breaks conjugation Vu et al. (2018) , adds dipole Mogbojuri et al. (2025)
Alkyl
–C n H 2n+1
increases torsion/free volume Wedler et al. (2020)
Hydroxyl
–OH
H-bonding increases packing Aziz et al. (2025) but breaks conjugation Vu et al. (2018)
Amine
–NH 2
Electron-donating Peet et al. (2007) ; Ansari et al. (2018)
Table 3: Summary of functional group that can effect on polymer electronic, structural, and optical properties.
Figure 8: A statistics of importance-score of each dataset. ”Av” denotes the average of the all the connectivity and atoms in the input data. The importance-score of each sample is normalized by the each sample’s maximum value of the importance-score.
Fragment-level representations provide a natural way to capture recurring molecular substructures and reuse their learned representations across molecules. However, a shared fragment identity alone may not fully describe how a fragment is instantiated in a particular molecule, since the same fragment can exhibit different chemical behavior depending on its surrounding atomic environment. Effective fragment-based molecular learning therefore requires representations that are both reusable across molecules and sensitive to local atomic context. We introduce BiScale-GTR, a self-supervised molecular representation framework built around context-grounded shared fragment tokens. BiScale-GTR constructs a reusable graph Byte Pair Encoding (graph-BPE) vocabulary using Weisfeiler-Lehman (WL)-based fragment identity, chemical validity filtering, and recursive out-of-vocabulary (OOV) decomposition. Each shared fragment token is then grounded with atom-level GNN representations through atom-to-fragment pooling and gated fusion, allowing the same fragment identity to acquire context-dependent representations in different molecular environments. A structure-aware fragment Transformer performs global reasoning over these atom-grounded tokens, capturing reusable substructure identity, local chemical context, and long-range molecular dependencies. Experiments on MoleculeNet, PharmaBench, and the Long Range Graph Benchmark demonstrate strong performance across classification and regression tasks. Attribution analysis further shows that BiScale-GTR highlights chemically meaningful recurring motifs, providing interpretable links between molecular structure and predicted properties.
Yi Yang, Ovidiu Daescu
Department of Computer Science, The University of Texas at Dallas
Most machine learning models for molecular property prediction rely on a single molecular representation (either a sequence, a graph, or a 3D structure) and treat molecular geometry as static. We present MolFM-Lite, a multi-modal model that jointly encodes SELFIES sequences (1D), molecular graphs (2D), and conformer ensembles (3D) through cross-attention fusion, while conditioning predictions on experimental context via Feature-wise Linear Modulation (FiLM). Our main methodological contributions are: (1) a conformer ensemble attention mechanism that combines learnable attention with Boltzmann-weighted priors over multiple RDKit-generated conformers, capturing the thermodynamic distribution of molecular shapes; and (2) a cross-modal fusion layer where each modality can attend to others, enabling complementary information sharing. We evaluate on four MoleculeNet scaffold-split benchmarks using our model's own splits, and report all baselines re-evaluated under the same protocol. Comprehensive ablation studies across all four datasets confirm that each architectural component contributes independently, with tri-modal fusion providing 7-11% AUC improvement over single-modality baselines and conformer ensembles adding approximately 2% over single-conformer variants. Pre-training on ZINC250K (~250K molecules) using cross-modal contrastive and masked-atom objectives enables effective weight initialization at modest compute cost. We release all code, trained models, and data splits to support reproducibility.
Syed Omer Shah, Mohammed Maqsood Ahmed, Danish Mohiuddin Mohammed +2
Department of Computer Science and Engineering, University at Buffalo, Buffalo, NY, USA · Khoury College of Computer Sciences, Northeastern University, Boston, MA, USA · Department of Computer Science, Muffakham Jah College of Engineering and Technology, Hyderabad, India +1
Molecular property prediction requires representations that generalize from limited labeled data to structurally novel compounds. Existing molecular pretraining methods often rely on a single view: graph-based approaches model atom-bond topology but provide limited fragment-level supervision, whereas fingerprint descriptors encode chemical patterns but are typically used as fixed auxiliary features. We propose HiFi-Mol, a multi-view framework that separately pretrains a hierarchical graph encoder and a contextualized fingerprint encoder before downstream integration. The graph branch uses fragment-aware masking with multi-resolution supervision to capture substructure-aware representations, while the fingerprint branch tokenizes active entries from seven fingerprint families and applies masked language modeling to learn contextualized embeddings. During fine-tuning, HiFi-Mol combines projected multi-resolution graph features with fingerprint embeddings for downstream prediction. Evaluated on MoleculeNet benchmarks under the scaffold split, HiFi-Mol achieves a 2.77% improvement in average ROC-AUC over the best baseline across eight classification tasks while maintaining competitive performance on three regression tasks. Further analyses reveal that fragment-aware masking improves graph representation quality, and classification results demonstrate dataset-dependent strengths of the individual graph and fingerprint variants, confirming that the two views provide complementary predictive signals.
Gwang-Hyeon Yun, Jong-Hoon Park, Bing Hu +3
Yonsei University · Wonju-si, Republic of Korea · University of Waterloo +1