Drug-Target Interaction Prediction via Hierarchical Sequential Cross-Attention over Chemical and Protein Language Models
Authors: Khadidja Henni, Hamza Abdelali, Abdelkrim Aries, Neila Mezghani, Brigitte Vannier, Sara Magdouli, Lina Abou-Abbas
Organizations: I2A institute, T´ELUQ University, Montreal, QC, Canada · LIO, CRCHUM, Montreal, QC, Canada · LCSI, ESI, Algiers, Algeria · CoMeT UR 24344, Universite de Poitiers, Poitiers, France · Dept. of Civil Engineering, University of Ottawa, Ottawa, ON, Canada · Dept. of Electrical and Computer Engineering, Lebanese American University, Byblos, Lebanon
Predicting Drug-Target Interactions~(DTIs) is a central task in computational drug discovery, with direct applications in virtual screening, drug repurposing, and therapeutic candidate prioritization. Although recent deep learning methods have improved DTI prediction, many sequence-based models still process drugs and proteins independently and only combine their representations at a late prediction stage. This limits their ability to explicitly model cross-molecular dependencies between chemical substructures and protein sequence regions. In this paper, we propose a sequence-only DTI prediction architecture that combines two pre-trained language models, ChemBERTa for drug SMILES strings and ESM-2 for protein amino acid sequences, with a hierarchical interaction module. The proposed model first extracts contextual representations using pre-trained encoders, then applies 1D convolutional layers to condense local sequence patterns, followed by a sequential bidirectional cross-attention mechanism inspired by the induced-fit view of molecular recognition. Finally, attention-based pooling constructs fixed-size interaction-aware vectors for binary prediction. Experiments on BIOSNAP, Davis, and BindingDB show that the proposed model achieves the best performance on BIOSNAP, matches the best AUROC on Davis, and remains competitive on BindingDB while using only 25.2 million trainable parameters. Ablation results confirm the contribution of both the CNN and cross-attention modules, and cold-start experiments indicate promising generalization to unseen proteins and drugs.
Figures & tables
Fig. 1: Overview of the proposed architecture. Drug SMILES and protein sequences are encoded using ChemBERTa and ESM-2, condensed through 1D CNN layers, fused using sequential bidirectional cross-attention, and aggregated through attention-based pooling before final prediction.
Dataset
Drugs
Proteins
Positive Pairs
Negative Pairs
Davis
68
379
1,043 / 160 / 303
1,043 / 2,846 / 5,708
BIOSNAP
4,510
2,181
9,619 / 1,374 / 2,748
9,619 / 1,374 / 2,748
BindingDB
10,665
1,413
6,334 / 927 / 1,905
6,334 / 5,717 / 11,384
TABLE I: Dataset statistics. Interaction pairs are reported as train / validation / test.
Hyperparameter
Value
Optimizer
AdamW
Base learning rate
5×10−5
Learning-rate schedule
Linear warmup + cosine decay
Effective batch size
32
Maximum epochs
30
Unfrozen encoder layers
Top 2 layers
TABLE II: Main hyperparameters of the proposed model.
BIOSNAP
Davis
BindingDB
Model
AUROC
AUPRC
Sens.
Spec.
AUROC
AUPRC
Sens.
Spec.
AUROC
AUPRC
Sens.
Spec.
MolTrans [ 5 ]
0.895
0.901
0.775
0.851
0.907
0.404
0.800
0.876
0.914
0.622
0.797
0.896
DeepDTA [ 2 ]
0.834
0.849
0.726
0.813
0.792
0.272
0.688
0.778
0.902
0.604
0.757
0.904
HyperAttentionDTI [ 18 ]
0.825
0.830
0.800
0.812
0.841
0.483
0.708
0.722
0.900
0.876
0.801
0.791
Fine-tuning BERT [ 8 ]
0.914
0.900
0.862
0.847
0.920
0.395
0.824
0.802
0.922
0.623
0.814
0.793
DLM-DTI [ 13 ]
0.914
0.914
0.848
0.844
0.895
0.373
0.833
0.766
0.912
0.643
0.846
0.916
TABLE III: Performance comparison on BIOSNAP, Davis, and BindingDB. Best result per metric per dataset is in bold .
Test set
Training
AUROC
AUPRC
Sens.
Spec.
Davis
Individual
0.920
0.356
0.901
0.801
Integrated
0.939
0.448
0.911
0.848
BIOSNAP
Individual
0.920
0.924
0.854
0.830
Integrated
0.920
0.840
0.886
0.835
BindingDB
Individual
0.920
0.642
0.816
0.874
Integrated
0.918
0.764
0.852
0.850
TABLE IV: Individual training vs. integrated training.
Accurate prediction of drug-target interactions (DTI) is critical for drug discovery. Existing methods often rely on single-modal representations (e.g., sequences or graphs) or combine only two modalities, overlooking 3D structural features. To address this challenge, we propose TriMod-DTI, a triple-modal contrastive learning framework that incorporates 1D sequences, 2D graphs, and 3D structures of drugs and proteins, obtaining the universal and complementary feature representations for DTI prediction. We design a Feature Extractor to capture drug and target features across the three modalities, thereby enriching their representations. We further propose a triple-modal contrastive learning strategy to align different modal representations of the same drug or protein in the latent space. By constructing cross-modal positive and negative sample pairs, this approach enhances the model's discriminative ability. Experiments on three benchmark datasets demonstrate that TriMod-DTI outperforms state-of-the-art methods. The ablation studies validate the contributions of each modality. Moreover, case studies highlight its practical potential for DTI prediction and drug discovery.
Le Xu, Xi Zhang, Dan Luo +2
School of Computer Science, Xiangtan University, Xiangtan 411105, China
Drug-target interaction (DTI) and affinity (DTA) predictors increasingly achieve strong benchmark scores, yet their internal use of sequence, fingerprint, and graph features often remains opaque. We present an interpretability audit of BridgeDPI architecture on three different datasets including Gao, Human, and C.elegans. This study combines gradient-based attributions -- integrated gradients, saliency, layer-wise relevance propagation, SmoothGrad, and SmoothGrad-IG -- with feature-wise occlusion ablation and strict intersection consensus across methods to reduce single-explainer bias. We summarize sensitivity and signed effects at raw inputs, at the bridge similarity scaffold, and through the graph convolution, including edge-level sensitivities and targeted edge removals. The results show that explainability is most informative when treated as model criticism: it reveals modality dominance, padding and special-token artifacts, dataset-dependent cooperative versus suppressive effects across layers, and chemistry-consistent fragment and composition motifs where methods agree. These analyses do not substitute for structural or experimental ground truth, yet they can provide testable hypotheses for downstream validation in computational drug discovery pipelines. More broadly, applying modern XAI to contemporary DTI/DTA models is still an early pass over the rich structure implicit in trained weights and data -- yet even this first layer of scrutiny already helps researchers relate predictions to drug- and target-side representations and to prioritize external validation.
Ali Vefghi, Zahed Rahmati, Mohammad Akbari
Department of Mathematics and Computer Science, Amirkabir University of Technology, Tehran, Iran
Deep learning models for drug--target interaction (DTI) prediction often achieve strong benchmark performance without necessarily relying on mechanistically meaningful molecular features, a limitation that standard accuracy-based evaluation cannot detect. We introduce ISAAC (Intervention-based Structural Auditing Approach for Causal Reasoning), a post-hoc framework that evaluates prior-relative structural sensitivity by probing frozen models through matched mechanistic and spurious input-level interventions, independently of predictive accuracy. Applied to three sequence-based DTI architectures on the Davis benchmark, ISAAC reveals approximately 25% relative differences in reasoning scores across models with comparable AUROC (within around 3%), stable across training and intervention seeds and two distinct perturbation operators. These discrepancies, undetectable under conventional accuracy metrics, motivate the use of post-hoc structural auditing as a complement to standard performance evaluation in scientific machine learning for molecular modeling.
Barbara Tarantino, Sun Kim, Yijingxiu Lu +1
Department of Economics and Management, University of Pavia, Pavia, Italy · Department of Computer Science and Engineering, Seoul National University, Gwanak-gu 08826, Korea.