MoTIF-X: A Multimodal Tokenized Framework for Interpretable and Extensible Molecular Representation Learning
Authors: Linqing Mo, Jiayu Zhou, Bin Chen
Organizations: Department of Computer Science and Engineering, Michigan State University, East Lansing, 48824, MI, USA. · School of Information, University of Michigan, Ann Arbor, 48109, MI, USA. · Department of Pediatrics and Human Development, Michigan State University, Grand Rapids, MI, 49503, USA. · Department of Pharmacology and Toxicology, Michigan State University, Grand Rapids, MI, 49503, USA. · Center for AI-enabled Drug Discovery, College of Human Medicine, Michigan State University, Grand Rapids, MI, 49503, USA.
Molecular representation learning is central to computer-aided drug discovery. Molecular graphs, SMILES strings, and 3D conformations provide complementary structural information, yet many multimodal approaches encode these views independently and align them only at a later stage, limiting fine-grained cross-modal interaction and substructure-level interpretability. To address these limitations, we introduce MoTIF-X, a motif-centered framework that uses graph-grounded chemical motifs as shared anchors for multimodal integration and interpretation. Its first pretraining stage learns motif representations through hierarchical contrastive learning across atomic, motif, and molecular scales. The second stage contextualizes these representations with SMILES and torsion-angle tokens through multimodal masked token modeling. After pretraining on drug-like molecules with multiple conformers, MoTIF-X achieved the lowest mean absolute error on all nine OpenADMET ExpansionRx endpoints and the best overall performance among the evaluated methods. Significance analyses supported its advantage in the vast majority of endpoint-baseline comparisons after multiple-testing correction. Ablation studies supported the complementary contributions of motif-token contextualization, multimodal integration, and two-stage pretraining. Beyond molecular properties, the framework extended to drug-target interaction prediction, achieving the best average classification performance across the evaluated benchmarks and generalizing to an external drug-cold-start dataset without additional fine-tuning. Its motif-centered design also enabled substructure-level interpretation: higher motif attribution scores were associated with larger experimentally measured activity shifts. Together, these findings support MoTIF-X as a transferable and interpretable framework for molecular modeling.
Figures & tables
Figure 1: Overview of the MoTIF-X framework. a Pretraining Stage I: hierarchical contrastive learning grounds motif representations by aligning atom-, motif-, and molecule-level representations using molecular and fragment graphs, enabling motif representations to bridge atomic detail and molecular context. b Pretraining Stage II: multimodal tokenization converts motif, SMILES, and 3D torsion information into tokens, which are jointly modeled using a Transformer with a masked token modeling objective to integrate topological, symbolic, and conformational molecular information. c Downstream molecular property prediction by fine-tuning the pretrained Transformer using motif and SMILES tokens. d Downstream drug-target interaction prediction by jointly modeling molecular motif tokens and pretrained protein representations, with attention over motif tokens indicating the substructures emphasized by the model.
Figure 2: MoTIF-X achieves strong molecular property prediction performance and benefits from two-stage multimodal pretraining on OpenADMET ExpansionRx. a-b , Comparison with nine baseline models. MoTIF-X achieves the lowest mean MAE on all nine endpoints and the lowest MA-RAE among the compared methods. a , Mean absolute error (MAE) across nine ADME endpoints. b , Overall performance measured by macro-averaged relative absolute error (MA-RAE). c-d , Comparison of modality combinations and stage-wise pretraining ablations. c , MA-RAE across pretraining configurations. d , Endpoint-specific MAE across the same configurations.
Figure 3: Multimodal pretraining redistributes attention across token types and shapes representation geometry in MoTIF-X. a-b , Attention redistribution across token types. Pretraining with 3D tokens shifts downstream attention toward motif tokens and promotes interactions among motif, SMILES, and 3D tokens during pretraining. a , Incoming attention received by the classification token ( [CLS] ) from motif and SMILES tokens during downstream prediction across MoleculeNet benchmarks. b , Final-layer incoming attention composition for motif, SMILES, and 3D target tokens during pretraining. Top: models pretrained without 3D information. Bottom: models pretrained with 3D information. c-d , Representation geometry induced by multimodal pretraining. Multimodal pretraining yields more structured and separable representations. c , Cluster-level positive rate on BBBP, showing stronger alignment between embedding clusters and downstream labels after pretraining with 3D tokens. d , Uniform manifold approximation and projection (UMAP) of learned [CLS] representations on BBBP. From left to right: motif+SMILES+3D, motif+SMILES, and extended-connectivity fingerprint (ECFP) representations.
Figure 4: MoTIF-X transfers to drug-target interaction prediction. a , In-domain half-maximal inhibitory concentration (IC 50 ) regression on the BindingDB test set, shown as predicted versus measured pIC 50 values, where pIC 50 denotes the negative base-10 logarithm of the molar IC 50 value. Pearson correlation coefficient (PCC) indicates agreement with measured binding affinities. b , External evaluation under a drug cold-start setting on a ChEMBL IC 50 dataset without additional fine-tuning, evaluated by PCC, indicating generalization to previously unseen compounds across datasets. c , DTI classification performance across multiple benchmarks, reported as area under the precision-recall curve (AUPR; %) with comparison to baseline methods. Mol pretrain only uses pretrained molecular embeddings, while the Transformer is initialized randomly before downstream fine-tuning and no DTI-specific pretraining is applied. DTI pretrain only initializes the molecular encoders randomly before DTI-specific pretraining, rather than using encoders initialized from molecular pretraining. Single-motif token and Multi-motif tokens both use pretrained molecular embeddings and DTI-specific pretraining, with Multi-motif tokens denoting the standard MoTIF-X formulation that represents each molecule using multiple motif tokens. MoTIF-X achieves the highest average AUPR and performs best on generalization-oriented settings.
Figure 5: Motif-level attribution in MoTIF-X aligns with activity variation and highlights chemically informative substructures. a-b , Motif-level attribution aligns with experimental activity variation, with activity-relevant motifs receiving higher attribution and being preferentially ranked by the model. a , Top- k hit rate versus motif selection ratio. Higher values at lower ratios indicate that the most activity-relevant motifs are preferentially ranked. b , Examples of top-ranked motifs plotted by activity change ( ΔpIC50 ) and motif attribution. MoTIF-X assigns high attribution to motifs associated with both increased and decreased activity and frequently prioritizes them within molecules. c-e , Motif-level attribution identifies localized and consistent substructure-level signals across structurally diverse hepatocellular carcinoma (HCC)-related compounds, with the Salicylamide motif highlighted in orange-red and consistently receiving high attribution. c , Niclosamide against STAT3. d-e , Additional HCC-related compounds showing a similar attribution pattern despite structural variation. f , Structural plausibility of motif-level attribution examined by molecular docking. The top-ranked predicted Niclosamide-STAT3 pose yields a favorable docking score, with predicted protein-ligand contacts involving the Salicylamide motif.
Abbreviation
Definition
2D
two-dimensional
3D
three-dimensional
ADMET
absorption, distribution, metabolism, excretion, and toxicity
Phenotypic drug discovery enables the discovery of functional relationships between molecular structures and cellular responses. However, existing multimodal representation learning methods often optimize cross-modal alignment without considering the intrinsic organization of chemical space, resulting in distorted molecular representations and loss of structural information. We propose PhenMol, a structure-preserving framework for phenotype-aware molecular representation learning. PhenMol disentangles molecular and cellular representations into shared and private components, enabling phenotype-guided alignment while preserving chemical structures through a dedicated molecular branch. This design integrates cellular phenotype information without disrupting molecular neighborhood organization. Experiments on approximately 3.04×104 molecule--cell morphology pairs demonstrate that PhenMol improves molecular property prediction across 270 bioactivity tasks, molecule--phenotype retrieval, and clinical trial outcome prediction. Moreover, ECFP4-based structural analysis shows that PhenMol better preserves molecular neighborhoods and reduces embedding distortion compared with existing multimodal alignment methods. These results highlight the importance of structure-aware constraints in multimodal molecular representation learning and provide an effective approach for integrating cellular phenotypes with chemical knowledge for drug discovery.
Xuan Lin, Jingyu Sheng, Tengfei Ma +2
School of Computer Science, Xiangtan University, Xiangtan, Hunan, China · College of Computer Science and Electronic Engineering, Hunan University, Changsha, Hunan, China · School of Computer Science, Beijing University of Posts and Telecommunications, Beijing, China +1
Molecular property prediction requires representations that generalize from limited labeled data to structurally novel compounds. Existing molecular pretraining methods often rely on a single view: graph-based approaches model atom-bond topology but provide limited fragment-level supervision, whereas fingerprint descriptors encode chemical patterns but are typically used as fixed auxiliary features. We propose HiFi-Mol, a multi-view framework that separately pretrains a hierarchical graph encoder and a contextualized fingerprint encoder before downstream integration. The graph branch uses fragment-aware masking with multi-resolution supervision to capture substructure-aware representations, while the fingerprint branch tokenizes active entries from seven fingerprint families and applies masked language modeling to learn contextualized embeddings. During fine-tuning, HiFi-Mol combines projected multi-resolution graph features with fingerprint embeddings for downstream prediction. Evaluated on MoleculeNet benchmarks under the scaffold split, HiFi-Mol achieves a 2.77% improvement in average ROC-AUC over the best baseline across eight classification tasks while maintaining competitive performance on three regression tasks. Further analyses reveal that fragment-aware masking improves graph representation quality, and classification results demonstrate dataset-dependent strengths of the individual graph and fingerprint variants, confirming that the two views provide complementary predictive signals.
Gwang-Hyeon Yun, Jong-Hoon Park, Bing Hu +3
Yonsei University · Wonju-si, Republic of Korea · University of Waterloo +1
Despite recent advances in molecular foundation models, several limitations remain, such as chemically invalid augmentations, modality collapse, and incomplete representation of biochemical environments. To address these challenges, we present Mol-JEPA, a scalable framework for learning molecular world models. Rather than relying on suboptimal molecular perturbations, our model uses modality masking to exploit information from molecular structures, cellular phenotypes, binding affinities, ADMET profiles, quantum chemistry simulations and other drug discovery data. Across various benchmarks, we show that the representations learned by Mol-JEPA deliver strong performance, demonstrating the value of incorporating biochemical context through latent space prediction.
Florian Rottach, Sebastian Schieferdecker, William Rudman +2
University of Tübingen · 2Boehringer Ingelheim · 3The University of Texas at Austin +1