Hierarchical Molecular Representation Learning via Fragment-Based Self-Supervised Embedding Prediction

Jiele Wu; Haozhe Ma; Zhihan Guo; Thanh Vinh Vo; Tze Yun Leong

cs.LGFeb 23, 2026

Hierarchical Molecular Representation Learning via Fragment-Based Self-Supervised Embedding Prediction

Authors: Jiele Wu, Haozhe Ma, Zhihan Guo, Thanh Vinh Vo, Tze Yun Leong

Organizations: School of Computing, National University of Singapore, Singapore · Computer Science and Engineering, The Chinese University of Hong Kong, China

Abstract

Graph self-supervised learning (GSSL) has demonstrated strong potential for generating expressive graph embeddings without the need for human annotations, making it particularly valuable in domains with high labeling costs such as molecular graph analysis. However, existing GSSL methods mostly focus on node- or edge-level information, often ignoring chemically relevant substructures which strongly influence molecular properties. In this work, we propose Graph Semantic Predictive Network (GraSPNet), a hierarchical self-supervised framework that explicitly models both atomic-level and fragment-level semantics. GraSPNet decomposes molecular graphs into chemically meaningful fragments without predefined vocabularies and learns node- and fragment-level representations through multi-level message passing with masked semantic prediction at both levels. This hierarchical semantic supervision enables GraSPNet to learn multi-resolution structural information that is both expressive and transferable. Extensive experiments on multiple molecular property prediction benchmarks demonstrate that GraSPNet learns chemically meaningful representations and consistently outperforms state-of-the-art GSSL methods in transfer learning settings.

Figures & tables

Figure 1: An example of hierarchical representation learning on molecular graph. The molecule is represented as a string-based notations (SMILES) and encoded at three semantic levels—node (atoms), fragment (e.g., functional groups), and graph to support various downstream tasks.

Panel of Figure 2 — Figure 2: Illustration of our graph fragmentation process. The left figure shows graph $G_{1}$ and $G_{2}$ which can not be distinguished by WL test while higher-level fragment graph $F_{1}$ and $F_{2}$ exhibit different connections that can be distinguished by WL. The right figure shows an example of our graph fragmentation. The ring is first selected, followed by the extraction of multiple paths. The articulation points are designated as unique fragment types to prevent cycles in the fragment graph.

Figure 3: Overview of the Graph Semantic Predictive Network (GraSPNet) framework. The original molecule is fragmented to form a higher-level fragment graph. Masked node and fragment graphs are input into the context encoder, while the target encoder processes the original unmasked graphs. The predictor uses context representations to predict node and fragment embeddings, and the loss minimizes the distance between the prediction and the target encoder’s representations.

	BBBP	Tox21	ToxCast	Sider	MUV	HIV	Bace	Clintox
No Pre-train	67.8 $\pm$ 1.4	73.9 $\pm$ 0.8	62.4 $\pm$ 0.4	58.3 $\pm$ 1.8	73.4 $\pm$ 2.5	76.3 $\pm$ 1.2	76.8 $\pm$ 2.6	62.6 $\pm$ 4.4
MAE	68.7 $\pm$ 1.3	75.5 $\pm$ 0.5	62.0 $\pm$ 0.8	58.0 $\pm$ 1.0	69.7 $\pm$ 1.5	74.2 $\pm$ 2.2	77.2 $\pm$ 1.6	70.1 $\pm$ 3.2
Infomax	69.2 $\pm$ 0.7	74.6 $\pm$ 0.5	61.8 $\pm$ 0.8	60.1 $\pm$ 0.7	74.8 $\pm$ 1.5	75.0 $\pm$ 1.3	76.3 $\pm$ 1.8	71.2 $\pm$ 2.5
Attr mask	65.6 $\pm$ 0.9	77.2 $\pm$ 0.4	63.3 $\pm$ 0.8	59.6 $\pm$ 0.7	74.7 $\pm$ 1.9	77.9 $\pm$ 1.2	78.3 $\pm$ 1.1	77.5 $\pm$ 3.1
CP	72.1 $\pm$ 1.5	74.3 $\pm$ 0.5	63.5 $\pm$ 0.3	60.2 $\pm$ 1.2	70.2 $\pm$ 2.8	74.4 $\pm$ 0.8	79.2 $\pm$ 0.9	70.2 $\pm$ 2.6
ADGCL	70.5 $\pm$ 1.8	74.5 $\pm$ 0.4	63.0 $\pm$ 0.5	59.1 $\pm$ 0.9	71.5 $\pm$ 2.2	75.9 $\pm$ 1.8	74.2 $\pm$ 2.4	78.5 $\pm$ 3.7

Table 1: Evaluation on molecular property prediction tasks. For each downstream dataset, we report the mean and standard deviation of the ROC-AUC (%) scores over three random scaffold splits. The best and second best scores are marked bold and underline, respectively.

Table 5

Methods	FreeSolv	ESOL	Lipophilicity
ContextPred	$3.195\pm 0.058$	$2.190\pm 0.026$	$1.053\pm 0.048$
AttrMasking	$4.023\pm 0.039$	$2.954\pm 0.087$	$0.982\pm 0.052$
EdgePred	$3.192\pm 0.023$	$2.368\pm 0.070$	$1.085\pm 0.061$
Infomax	$3.033\pm 0.026$	$2.953\pm 0.049$	$0.970\pm 0.023$
JOAO	$3.282\pm 0.002$	$1.978\pm 0.029$	$1.093\pm 0.097$
GraphCL	$3.166\pm 0.027$	$1.390\pm 0.363$	$1.014\pm 0.018$

Table 4: Performance comparison on regression tasks in terms of RMSE(

\downarrow

Figure 4: Performance of incorporating fragment information after different GINE layers in the context encoder.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Description	Number of Graphs	Number of Tasks
BBBP	Blood-brain barrier permeability	2,039	1
Tox21	Toxicology on 12 biological targets	7,831	12
ToxCast	Toxicology via high-throughput screening	8,575	617
SIDER	Adverse drug reactions of marketed medicines	1,427	27
ClinTox	Clinical trial failures due to toxicity	1,478	2
MUV	Validation of virtual screening techniques	93,087	17

Appendix

Table 5: Summary of molecular property prediction benchmarks.

Figure 5: Distribution of structural components in molecular graphs. Each subplot shows the distribution of (a) fragment sizes, (b) number of rings, (c) number of paths, and (d) number of articulation points per graph. The x-axis represents the count of each structure, and the y-axis shows the number of graphs with that count.

Figure 6: Distribution of (a) Rings and (b) Paths size in ZINC dataset.

Fragment	epoxide	lactam	morpholine	oxazole	tetrazole	NO	ether	furan	guanido	halogen	piperdine
Acc	99.5	99.95	99.45	99.10	98.85	99.60	91.75	99.60	99.55	94.35	95.70

Appendix

Table 6: RDKiT fragments prediction accuracy of GraSPNet.

Fragment	thiazole	thiophene	urea	allylic oxid	amide	amidine	azo	benzene	imidazole	imide	piperzine	pyridine
Acc	99.05	99.70	99.75	95.45	94.65	99.65	99.80	92.25	98.75	99.4	97.2	97.15

Appendix

Table 12

Type	Clintox	BBBP	Sider	Bace	Mean
w/o Arti	70.16	66.84	58.14	77.81	68.24
Full	70.35	67.30	59.61	76.86	68.53

Appendix

Table 7: Ablation study on articulation point.

Model	GraphMAE	Mole-BERT	S2GAE	GraphMAE2	SimSGT	GraSPNet
Pretrain Time	527 min	2199 min	1763 min	1195 min	645 min	769 min

Appendix

Table 8: Training time.

Panel of Figure 7 — Figure 7: $G_{1}$ (top left) and graph $G_{2}$ (top right) with their corresponding higher-level graph on bottom left and bottom right. Graph $G_{1}$ and graph $G_{2}$ that are indistinguishable by 1-WL but distinguishable by Fragment-WL with higher-level graph.

Setting	Self-supervised Learning	Fine-tuning
Backbone GNN Type	GIN	GIN
Context Layer	5	5
Target Layer	1	1
PE type	None	None
Backbone Neurons	[300]	[300]
Fragment layer	[2,3]	[2,3]

Appendix

Table 9: Hyperparameters for Experiments in Self-supervised learning and fine-tuning

Explore similar work

Multi-level Self-supervised Pretraining on Compositional Hierarchical Graph for Molecular Property Prediction
May 15, 2026Xiayu Liu, Zhengyi Lu, Hou-biao LiSelf-Supervised Pre-Training Hierarchical Representation Learning
School of Mathematical Sciences, University of Electronic Science and Technology of China, No.2006, Xiyuan Avenue, West Hi-tech Zone,2026 611731, Chengdu, China · Department of Computer Science and Engineering, Oakland University, 201 Meadow Brook RD, Rochester Hills, 48309, MI, USA
BiScale-GTR: Fragment-Aware Graph Transformers for Multi-Scale Molecular Representation Learning
Apr 7, 2026Yi Yang, Ovidiu DaescuMolecular Representation Learning Graph Transformer
Department of Computer Science, The University of Texas at Dallas
Self-Supervised Pretraining of Molecular Graph Encoders with LeJEPA
Sep 2, 2026Michał Kulczykowski, Rafał ŁabędzkiSelf-Supervised Pre-Training Molecular Representation Learning
SGH Warsaw School of Economics

May 15, 2026J/K move · Enter open · S save

Multi-level Self-supervised Pretraining on Compositional Hierarchical Graph for Molecular Property Prediction

Xiayu Liu, Zhengyi Lu, Hou-biao Li

School of Mathematical Sciences, University of Electronic Science and Technology of China, No.2006, Xiyuan Avenue, West Hi-tech Zone,2026 611731, Chengdu, China · Department of Computer Science and Engineering, Oakland University, 201 Meadow Brook RD, Rochester Hills, 48309, MI, USA

Self-Supervised Pre-Training Hierarchical Representation Learning Molecular Property Prediction Molecular Representation Learning

Open paper AskPDF arXiv

Self-supervised pretraining on molecular graphs has emerged as a promising approach for molecular property prediction, yet most existing methods operate at a single structural granularity and treat bond information as auxiliary edge attributes rather than as an independent semantic layer. In this work, we propose MolCHG, a multi-level self-supervised pretraining framework built upon a novel Compositional Hierarchical Graph that organizes molecular structure into four types of nodes across three semantic levels. By introducing a bond graph that operates in parallel with the atom graph, our architecture elevates bond-level information to independently evolving node representations, enabling fragment nodes to aggregate atom-level and bond-level semantics on an equal footing. We design three level-specific pretraining objectives: an atom-bond cross-view contrastive task that aligns the atom-view and bond-view representations within each fragment, a fragment-level functional group prediction task to inject domain-relevant chemical knowledge, and graph-level structure prediction tasks to encode global molecular topology. Experiments on nine MoleculeNet benchmarks demonstrate that MolCHG achieves the best performance on seven datasets across both classification and regression tasks, remaining competitive with the strongest baselines on the rest. Ablation studies further confirm that the multi-level supervision signals are complementary and that each component contributes to the overall performance.

Abstract

Explore similar work

Multi-level Self-supervised Pretraining on Compositional Hierarchical Graph for Molecular Property Prediction

BiScale-GTR: Fragment-Aware Graph Transformers for Multi-Scale Molecular Representation Learning

Self-Supervised Pretraining of Molecular Graph Encoders with LeJEPA