MoTIF-X: A Multimodal Tokenized Framework for Interpretable and Extensible Molecular Representation Learning
Authors: Linqing Mo, Jiayu Zhou, Bin Chen
Organizations: Department of Computer Science and Engineering, Michigan State University, East Lansing, 48824, MI, USA. · School of Information, University of Michigan, Ann Arbor, 48109, MI, USA. · Department of Pediatrics and Human Development, Michigan State University, Grand Rapids, MI, 49503, USA. · Department of Pharmacology and Toxicology, Michigan State University, Grand Rapids, MI, 49503, USA. · Center for AI-enabled Drug Discovery, College of Human Medicine, Michigan State University, Grand Rapids, MI, 49503, USA.
Molecular representation learning is central to computer-aided drug discovery. Molecular graphs, SMILES strings, and 3D conformations provide complementary structural information, yet many multimodal approaches encode these views independently and align them only at a later stage, limiting fine-grained cross-modal interaction and substructure-level interpretability. To address these limitations, we introduce MoTIF-X, a motif-centered framework that uses graph-grounded chemical motifs as shared anchors for multimodal integration and interpretation. Its first pretraining stage learns motif representations through hierarchical contrastive learning across atomic, motif, and molecular scales. The second stage contextualizes these representations with SMILES and torsion-angle tokens through multimodal masked token modeling. After pretraining on drug-like molecules with multiple conformers, MoTIF-X achieved the lowest mean absolute error on all nine OpenADMET ExpansionRx endpoints and the best overall performance among the evaluated methods. Significance analyses supported its advantage in the vast majority of endpoint-baseline comparisons after multiple-testing correction. Ablation studies supported the complementary contributions of motif-token contextualization, multimodal integration, and two-stage pretraining. Beyond molecular properties, the framework extended to drug-target interaction prediction, achieving the best average classification performance across the evaluated benchmarks and generalizing to an external drug-cold-start dataset without additional fine-tuning. Its motif-centered design also enabled substructure-level interpretation: higher motif attribution scores were associated with larger experimentally measured activity shifts. Together, these findings support MoTIF-X as a transferable and interpretable framework for molecular modeling.
Figures & tables
Figure 1: Overview of the MoTIF-X framework. a Pretraining Stage I: hierarchical contrastive learning grounds motif representations by aligning atom-, motif-, and molecule-level representations using molecular and fragment graphs, enabling motif representations to bridge atomic detail and molecular context. b Pretraining Stage II: multimodal tokenization converts motif, SMILES, and 3D torsion information into tokens, which are jointly modeled using a Transformer with a masked token modeling objective to integrate topological, symbolic, and conformational molecular information. c Downstream molecular property prediction by fine-tuning the pretrained Transformer using motif and SMILES tokens. d Downstream drug-target interaction prediction by jointly modeling molecular motif tokens and pretrained protein representations, with attention over motif tokens indicating the substructures emphasized by the model.
Figure 2: MoTIF-X achieves strong molecular property prediction performance and benefits from two-stage multimodal pretraining on OpenADMET ExpansionRx. a-b , Comparison with nine baseline models. MoTIF-X achieves the lowest mean MAE on all nine endpoints and the lowest MA-RAE among the compared methods. a , Mean absolute error (MAE) across nine ADME endpoints. b , Overall performance measured by macro-averaged relative absolute error (MA-RAE). c-d , Comparison of modality combinations and stage-wise pretraining ablations. c , MA-RAE across pretraining configurations. d , Endpoint-specific MAE across the same configurations.
Figure 3: Multimodal pretraining redistributes attention across token types and shapes representation geometry in MoTIF-X. a-b , Attention redistribution across token types. Pretraining with 3D tokens shifts downstream attention toward motif tokens and promotes interactions among motif, SMILES, and 3D tokens during pretraining. a , Incoming attention received by the classification token ( [CLS] ) from motif and SMILES tokens during downstream prediction across MoleculeNet benchmarks. b , Final-layer incoming attention composition for motif, SMILES, and 3D target tokens during pretraining. Top: models pretrained without 3D information. Bottom: models pretrained with 3D information. c-d , Representation geometry induced by multimodal pretraining. Multimodal pretraining yields more structured and separable representations. c , Cluster-level positive rate on BBBP, showing stronger alignment between embedding clusters and downstream labels after pretraining with 3D tokens. d , Uniform manifold approximation and projection (UMAP) of learned [CLS] representations on BBBP. From left to right: motif+SMILES+3D, motif+SMILES, and extended-connectivity fingerprint (ECFP) representations.
Figure 4: MoTIF-X transfers to drug-target interaction prediction. a , In-domain half-maximal inhibitory concentration (IC 50 ) regression on the BindingDB test set, shown as predicted versus measured pIC 50 values, where pIC 50 denotes the negative base-10 logarithm of the molar IC 50 value. Pearson correlation coefficient (PCC) indicates agreement with measured binding affinities. b , External evaluation under a drug cold-start setting on a ChEMBL IC 50 dataset without additional fine-tuning, evaluated by PCC, indicating generalization to previously unseen compounds across datasets. c , DTI classification performance across multiple benchmarks, reported as area under the precision-recall curve (AUPR; %) with comparison to baseline methods. Mol pretrain only uses pretrained molecular embeddings, while the Transformer is initialized randomly before downstream fine-tuning and no DTI-specific pretraining is applied. DTI pretrain only initializes the molecular encoders randomly before DTI-specific pretraining, rather than using encoders initialized from molecular pretraining. Single-motif token and Multi-motif tokens both use pretrained molecular embeddings and DTI-specific pretraining, with Multi-motif tokens denoting the standard MoTIF-X formulation that represents each molecule using multiple motif tokens. MoTIF-X achieves the highest average AUPR and performs best on generalization-oriented settings.
Figure 5: Motif-level attribution in MoTIF-X aligns with activity variation and highlights chemically informative substructures. a-b , Motif-level attribution aligns with experimental activity variation, with activity-relevant motifs receiving higher attribution and being preferentially ranked by the model. a , Top- k hit rate versus motif selection ratio. Higher values at lower ratios indicate that the most activity-relevant motifs are preferentially ranked. b , Examples of top-ranked motifs plotted by activity change ( ΔpIC50 ) and motif attribution. MoTIF-X assigns high attribution to motifs associated with both increased and decreased activity and frequently prioritizes them within molecules. c-e , Motif-level attribution identifies localized and consistent substructure-level signals across structurally diverse hepatocellular carcinoma (HCC)-related compounds, with the Salicylamide motif highlighted in orange-red and consistently receiving high attribution. c , Niclosamide against STAT3. d-e , Additional HCC-related compounds showing a similar attribution pattern despite structural variation. f , Structural plausibility of motif-level attribution examined by molecular docking. The top-ranked predicted Niclosamide-STAT3 pose yields a favorable docking score, with predicted protein-ligand contacts involving the Salicylamide motif.
Abbreviation
Definition
2D
two-dimensional
3D
three-dimensional
ADMET
absorption, distribution, metabolism, excretion, and toxicity
School of Computer Science, Xiangtan University, Xiangtan, Hunan, China · College of Computer Science and Electronic Engineering, Hunan University, Changsha, Hunan, China · School of Computer Science, Beijing University of Posts and Telecommunications, Beijing, China +1