DPTM-DT: Dual-Pretrained Transformer Multitask Representation Learning for Drug-Target Prediction

Ge Kong

cs.LGAug 28, 2026

DPTM-DT: Dual-Pretrained Transformer Multitask Representation Learning for Drug-Target Prediction

Authors: Ge Kong

Organizations: School of Biological Science and Medical Engineering, Beihang University, Beijing 100191, China

Abstract

Drug-target relation prediction supports candidate screening, drug repositioning, and mechanism analysis. Existing models often use incomplete drug or protein representations, model cross-modal interactions shallowly, or train affinity regression and interaction classification separately, although these tasks describe closely related views of the same drug-target pair. This paper presents DPTM-DT, a dual-pretrained Transformer framework for multitask drug-target prediction. DPTM-DT combines GROVER molecular graph embeddings, ESM protein language-model embeddings, and CTD physicochemical descriptors, then exchanges drug-target information through bidirectional cross-modal attention. A shared pair representation is used for continuous affinity regression, high-affinity binary classification, and six-level affinity classification. Experiments on Davis and KIBA cover random 80/20 and DeepDTA-style standard splits. On the random 80/20 split, DPTM-DT achieves MSE/CI values of 0.193/0.917 on Davis and 0.120/0.918 on KIBA. It also reports binary AUPR/MCC values of 0.727/0.654 and 0.798/0.689, and six-class Macro-F1/Top-2 values of 0.800/0.932 and 0.815/0.962 on Davis and KIBA, respectively. Across the reported regression, binary classification, and multiclass classification settings, DPTM-DT achieves the best overall performance among the compared methods. Results under the standard split show the same relative trend. Ablations indicate that dual target representation, gated fusion, and cross-modal attention each contribute to the final performance. Code and supplementary materials are available at: anonymous.4open.science/r/DPCM-DT-74E0.

Figures & tables

Figure 1: Overall architecture of DPTM-DT. The model encodes drug SMILES with GROVER, encodes target FASTA sequences with ESM-2 and CTD descriptors, fuses target-side features, performs bidirectional cross-modal Transformer interaction, and predicts affinity regression, binary interaction, and six-level affinity classes from the shared pair representation.

Figure 2: Structure of the Cross-modal Joint Interaction Module.

Figure 3: Construction of the pair representation after cross-modal interaction. The updated drug and target vectors are concatenated with their element-wise product and absolute difference to form the final drug-target representation used by the prediction heads.

Figure 4: Multitask prediction heads and training objectives.

Item	Davis	KIBA
Drugs	68	2,111
Targets	442	229
Valid pairs	30,056	118,254
Continuous label	pKd	KIBA score
Label range	5.00–9.94	0.00–17.20
Mean / std.	5.45 / 0.89	11.72 / 0.84

Table 1: Dataset, Label, and Task Summary

Output	Label construction	Main screening role
Regression	Original continuous pKd or KIBA score	Rank candidate pairs and refine affinity estimates
Binary	pKd $\geq 7.0$ for Davis; KIBA score $\geq 12.1$ for KIBA	Filter likely high-affinity interactions
Six-class	Elbow-guided K-means on continuous affinities, with $C=6$ and centers sorted from low to high affinity	Provide coarse affinity-level triage and communicate binding strength

Table 2: Task Outputs, Label Construction, and Screening Role

Split	Model	Davis		KIBA
		CI $\uparrow$	MSE $\downarrow$	CI $\uparrow$	MSE $\downarrow$
80/20	DPTM-DT	0.917	0.193	0.918	0.120
80/20	TEFDTA	0.895	0.194	0.865	0.179
80/20	MFR-DTA	0.910	0.216	0.903	0.131
80/20	GLCN-DTA	0.908	0.210	0.878	0.122
80/20	DeepDTA	0.883	0.256	0.868	0.189

Table 3: Affinity Regression Results on Davis and KIBA

Split	Dataset	Model	AUROC $\uparrow$	AUPR $\uparrow$	F1 $\uparrow$	MCC $\uparrow$
80/20	Davis	DPTM-DT	0.948 $\pm$ 0.006	0.727 $\pm$ 0.024	0.681 $\pm$ 0.028	0.654 $\pm$ 0.022
80/20	Davis	TEFDTA	0.931 $\pm$ 0.004	0.698 $\pm$ 0.011	0.655 $\pm$ 0.011	0.622 $\pm$ 0.027
80/20	Davis	MFR-DTA	0.941 $\pm$ 0.008	0.718 $\pm$ 0.020	0.670 $\pm$ 0.011	0.640 $\pm$ 0.029
80/20	Davis	GLCN-DTA	0.928 $\pm$ 0.010	0.692 $\pm$ 0.012	0.648 $\pm$ 0.015	0.615 $\pm$ 0.014
80/20	Davis	DeepDTA	0.915 $\pm$ 0.006	0.680 $\pm$ 0.017	0.632 $\pm$ 0.021	0.598 $\pm$ 0.016
80/20	Davis	WideDTA	0.921 $\pm$ 0.009	0.688 $\pm$ 0.010	0.640 $\pm$ 0.017	0.608 $\pm$ 0.017

Table 4: Binary Classification Results

Split	Dataset	Model	Acc. $\uparrow$	Macro-F1 $\uparrow$	W-F1 $\uparrow$	Top-2 $\uparrow$
80/20	Davis	DPTM-DT	0.826 $\pm$ 0.006	0.800 $\pm$ 0.020	0.819 $\pm$ 0.008	0.932 $\pm$ 0.006
80/20	Davis	TEFDTA	0.812 $\pm$ 0.006	0.759 $\pm$ 0.017	0.799 $\pm$ 0.009	0.911 $\pm$ 0.003
80/20	Davis	MFR-DTA	0.814 $\pm$ 0.008	0.760 $\pm$ 0.015	0.804 $\pm$ 0.007	0.917 $\pm$ 0.004
80/20	Davis	GLCN-DTA	0.797 $\pm$ 0.007	0.752 $\pm$ 0.022	0.783 $\pm$ 0.007	0.894 $\pm$ 0.007
80/20	Davis	DeepDTA	0.772 $\pm$ 0.004	0.733 $\pm$ 0.025	0.765 $\pm$ 0.010	0.892 $\pm$ 0.007
80/20	Davis	WideDTA	0.781 $\pm$ 0.006	0.722 $\pm$ 0.019	0.768 $\pm$ 0.008	0.878 $\pm$ 0.007

Table 5: Six-Class Affinity-Level Classification Results

Variant	MSE $\downarrow$	RMSE $\downarrow$	$R^{2}\uparrow$	Pearson $\uparrow$	CI $\uparrow$	AUPR $\uparrow$	MCC $\uparrow$	Macro-F1 $\uparrow$	Top-2 $\uparrow$
DPTM-DT	0.193	0.440	0.754	0.869	0.917	0.727	0.654	0.800	0.932
ESM-only	0.199	0.446	0.753	0.866	0.906	0.680	0.610	0.584	0.818
CTD-only	0.216	0.465	0.726	0.853	0.886	0.675	0.605	0.538	0.772
Concat	0.198	0.445	0.752	0.867	0.897	0.722	0.645	0.777	0.928
No cross-attn	0.200	0.447	0.750	0.865	0.892	0.715	0.638	0.773	0.925

Table 6: Representative Davis 80/20 Ablation Results

Explore similar work

Sep 2, 2026

Chemical and geometric representation fidelity improves drug--target affinity prediction

Predicting drug--target binding affinity (DTA) requires models to distinguish subtle chemical and structural determinants underlying molecular recognition. Although recent approaches increasingly incorporate richer drug and protein information, such information may be compressed, homogenized or discretized during representation construction, causing affinity-relevant distinctions to be lost before interaction modelling. We hypothesized that this representation-stage information loss constitutes an upstream bottleneck that cannot be reliably overcome by increasingly complex interaction predictors. To test this hypothesis, we developed ReGeoDTA, a representation-preserving framework that maintains affinity-relevant chemical heterogeneity in molecular representations and continuous geometric relationships in protein structures. Across three benchmark datasets, ReGeoDTA consistently improved affinity prediction, and the proposed representation-preserving strategies retained their benefits across diverse DTA architectures. Controlled representation degradation progressively reduced predictive performance, whereas increasing downstream predictor complexity failed to recover information lost during representation construction. These findings identify representation fidelity as an upstream design principle for accurate and generalizable drug--target affinity prediction, with potential implications for computational compound prioritization.

Yixiao Li, Yining Qian, Yefan Chen +5

College of Information Science and Engineering, Northeastern University, Shenyang 110819, China · School of Computer and Communication Engineering, Northeastern University at Qinhuangdao, Qinhuangdao 066004, China · School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China +1

Protein-Ligand Binding Affinity Prediction Representation Learning Molecular Representation Learning Geometric Representation Learning Protein Representation Learning

Ask PDF

Sep 28, 2026

Drug-Target Interaction Prediction via Hierarchical Sequential Cross-Attention over Chemical and Protein Language Models

Predicting Drug-Target Interactions~(DTIs) is a central task in computational drug discovery, with direct applications in virtual screening, drug repurposing, and therapeutic candidate prioritization. Although recent deep learning methods have improved DTI prediction, many sequence-based models still process drugs and proteins independently and only combine their representations at a late prediction stage. This limits their ability to explicitly model cross-molecular dependencies between chemical substructures and protein sequence regions. In this paper, we propose a sequence-only DTI prediction architecture that combines two pre-trained language models, ChemBERTa for drug SMILES strings and ESM-2 for protein amino acid sequences, with a hierarchical interaction module. The proposed model first extracts contextual representations using pre-trained encoders, then applies 1D convolutional layers to condense local sequence patterns, followed by a sequential bidirectional cross-attention mechanism inspired by the induced-fit view of molecular recognition. Finally, attention-based pooling constructs fixed-size interaction-aware vectors for binary prediction. Experiments on BIOSNAP, Davis, and BindingDB show that the proposed model achieves the best performance on BIOSNAP, matches the best AUROC on Davis, and remains competitive on BindingDB while using only 25.2 million trainable parameters. Ablation results confirm the contribution of both the CNN and cross-attention modules, and cold-start experiments indicate promising generalization to unseen proteins and drugs.

Khadidja Henni, Hamza Abdelali, Abdelkrim Aries +4

I2A institute, T´ELUQ University, Montreal, QC, Canada · LIO, CRCHUM, Montreal, QC, Canada · LCSI, ESI, Algiers, Algeria +3

Topic pending

Ask PDF

Mar 16, 2026

LaPro-DTA: Latent Dual-View Drug Representations and Salient Protein Feature Extraction for Generalizable Drug--Target Affinity Prediction

Drug--target affinity prediction is pivotal for accelerating drug discovery, yet existing methods suffer from significant performance degradation in realistic cold-start scenarios (unseen drugs/targets/pairs), primarily driven by overfitting to training instances and information loss from irrelevant target sequences. In this paper, we propose LaPro-DTA, a framework designed to achieve robust and generalizable DTA prediction. To tackle overfitting, we devise a latent dual-view drug representation mechanism. It synergizes an instance-level view to capture fine-grained substructures with stochastic perturbation and a distribution-level view to distill generalized chemical scaffolds via semantic remapping, thereby enforcing the model to learn transferable structural rules rather than memorizing specific samples. To mitigate information loss, we introduce a salient protein feature extraction strategy using pattern-aware top-

k

pooling, which effectively filters background noise and isolates high-response bioactive regions. Furthermore, a cross-view multi-head attention mechanism fuses these purified features to model comprehensive interactions. Extensive experiments on benchmark datasets demonstrate that LaPro-DTA significantly outperforms state-of-the-art methods, achieving an 8% MSE reduction on the Davis dataset in the challenging unseen-drug setting, while offering interpretable insights into binding mechanisms.

Zihan Dun, Liuyi Xu, An-Yang Lu +2

College of Information Science and Engineering, Northeastern University, Shenyang 110819, China · School of Artificial Intelligence, Beihang University, Beijing 100000, China · School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China

Protein-Ligand Binding Affinity Prediction Molecular Representation Learning Protein Representation Learning

Ask PDF