DPTM-DT: Dual-Pretrained Transformer Multitask Representation Learning for Drug-Target Prediction

Ge Kong

cs.LGAug 28, 2026

DPTM-DT: Dual-Pretrained Transformer Multitask Representation Learning for Drug-Target Prediction

Authors: Ge Kong

Organizations: School of Biological Science and Medical Engineering, Beihang University, Beijing 100191, China

Abstract

Drug-target relation prediction supports candidate screening, drug repositioning, and mechanism analysis. Existing models often use incomplete drug or protein representations, model cross-modal interactions shallowly, or train affinity regression and interaction classification separately, although these tasks describe closely related views of the same drug-target pair. This paper presents DPTM-DT, a dual-pretrained Transformer framework for multitask drug-target prediction. DPTM-DT combines GROVER molecular graph embeddings, ESM protein language-model embeddings, and CTD physicochemical descriptors, then exchanges drug-target information through bidirectional cross-modal attention. A shared pair representation is used for continuous affinity regression, high-affinity binary classification, and six-level affinity classification. Experiments on Davis and KIBA cover random 80/20 and DeepDTA-style standard splits. On the random 80/20 split, DPTM-DT achieves MSE/CI values of 0.193/0.917 on Davis and 0.120/0.918 on KIBA. It also reports binary AUPR/MCC values of 0.727/0.654 and 0.798/0.689, and six-class Macro-F1/Top-2 values of 0.800/0.932 and 0.815/0.962 on Davis and KIBA, respectively. Across the reported regression, binary classification, and multiclass classification settings, DPTM-DT achieves the best overall performance among the compared methods. Results under the standard split show the same relative trend. Ablations indicate that dual target representation, gated fusion, and cross-modal attention each contribute to the final performance. Code and supplementary materials are available at: anonymous.4open.science/r/DPCM-DT-74E0.

Figures & tables

Figure 1: Overall architecture of DPTM-DT. The model encodes drug SMILES with GROVER, encodes target FASTA sequences with ESM-2 and CTD descriptors, fuses target-side features, performs bidirectional cross-modal Transformer interaction, and predicts affinity regression, binary interaction, and six-level affinity classes from the shared pair representation.

Figure 2: Structure of the Cross-modal Joint Interaction Module.

Figure 3: Construction of the pair representation after cross-modal interaction. The updated drug and target vectors are concatenated with their element-wise product and absolute difference to form the final drug-target representation used by the prediction heads.

Figure 4: Multitask prediction heads and training objectives.

Item	Davis	KIBA
Drugs	68	2,111
Targets	442	229
Valid pairs	30,056	118,254
Continuous label	pKd	KIBA score
Label range	5.00–9.94	0.00–17.20
Mean / std.	5.45 / 0.89	11.72 / 0.84

Table 1: Dataset, Label, and Task Summary

Output	Label construction	Main screening role
Regression	Original continuous pKd or KIBA score	Rank candidate pairs and refine affinity estimates
Binary	pKd $\geq 7.0$ for Davis; KIBA score $\geq 12.1$ for KIBA	Filter likely high-affinity interactions
Six-class	Elbow-guided K-means on continuous affinities, with $C=6$ and centers sorted from low to high affinity	Provide coarse affinity-level triage and communicate binding strength

Table 2: Task Outputs, Label Construction, and Screening Role

Split	Model	Davis		KIBA
		CI $\uparrow$	MSE $\downarrow$	CI $\uparrow$	MSE $\downarrow$
80/20	DPTM-DT	0.917	0.193	0.918	0.120
80/20	TEFDTA	0.895	0.194	0.865	0.179
80/20	MFR-DTA	0.910	0.216	0.903	0.131
80/20	GLCN-DTA	0.908	0.210	0.878	0.122
80/20	DeepDTA	0.883	0.256	0.868	0.189

Table 3: Affinity Regression Results on Davis and KIBA

Split	Dataset	Model	AUROC $\uparrow$	AUPR $\uparrow$	F1 $\uparrow$	MCC $\uparrow$
80/20	Davis	DPTM-DT	0.948 $\pm$ 0.006	0.727 $\pm$ 0.024	0.681 $\pm$ 0.028	0.654 $\pm$ 0.022
80/20	Davis	TEFDTA	0.931 $\pm$ 0.004	0.698 $\pm$ 0.011	0.655 $\pm$ 0.011	0.622 $\pm$ 0.027
80/20	Davis	MFR-DTA	0.941 $\pm$ 0.008	0.718 $\pm$ 0.020	0.670 $\pm$ 0.011	0.640 $\pm$ 0.029
80/20	Davis	GLCN-DTA	0.928 $\pm$ 0.010	0.692 $\pm$ 0.012	0.648 $\pm$ 0.015	0.615 $\pm$ 0.014
80/20	Davis	DeepDTA	0.915 $\pm$ 0.006	0.680 $\pm$ 0.017	0.632 $\pm$ 0.021	0.598 $\pm$ 0.016
80/20	Davis	WideDTA	0.921 $\pm$ 0.009	0.688 $\pm$ 0.010	0.640 $\pm$ 0.017	0.608 $\pm$ 0.017

Table 4: Binary Classification Results

Split	Dataset	Model	Acc. $\uparrow$	Macro-F1 $\uparrow$	W-F1 $\uparrow$	Top-2 $\uparrow$
80/20	Davis	DPTM-DT	0.826 $\pm$ 0.006	0.800 $\pm$ 0.020	0.819 $\pm$ 0.008	0.932 $\pm$ 0.006
80/20	Davis	TEFDTA	0.812 $\pm$ 0.006	0.759 $\pm$ 0.017	0.799 $\pm$ 0.009	0.911 $\pm$ 0.003
80/20	Davis	MFR-DTA	0.814 $\pm$ 0.008	0.760 $\pm$ 0.015	0.804 $\pm$ 0.007	0.917 $\pm$ 0.004
80/20	Davis	GLCN-DTA	0.797 $\pm$ 0.007	0.752 $\pm$ 0.022	0.783 $\pm$ 0.007	0.894 $\pm$ 0.007
80/20	Davis	DeepDTA	0.772 $\pm$ 0.004	0.733 $\pm$ 0.025	0.765 $\pm$ 0.010	0.892 $\pm$ 0.007
80/20	Davis	WideDTA	0.781 $\pm$ 0.006	0.722 $\pm$ 0.019	0.768 $\pm$ 0.008	0.878 $\pm$ 0.007

Table 5: Six-Class Affinity-Level Classification Results

Variant	MSE $\downarrow$	RMSE $\downarrow$	$R^{2}\uparrow$	Pearson $\uparrow$	CI $\uparrow$	AUPR $\uparrow$	MCC $\uparrow$	Macro-F1 $\uparrow$	Top-2 $\uparrow$
DPTM-DT	0.193	0.440	0.754	0.869	0.917	0.727	0.654	0.800	0.932
ESM-only	0.199	0.446	0.753	0.866	0.906	0.680	0.610	0.584	0.818
CTD-only	0.216	0.465	0.726	0.853	0.886	0.675	0.605	0.538	0.772
Concat	0.198	0.445	0.752	0.867	0.897	0.722	0.645	0.777	0.928
No cross-attn	0.200	0.447	0.750	0.865	0.892	0.715	0.638	0.773	0.925

Table 6: Representative Davis 80/20 Ablation Results

Explore similar work

Chemical and geometric representation fidelity improves drug--target affinity prediction
Sep 2, 2026Yixiao Li, Yining Qian, Yefan Chen +5Protein-Ligand Binding Affinity Prediction Representation Learning
College of Information Science and Engineering, Northeastern University, Shenyang 110819, China · School of Computer and Communication Engineering, Northeastern University at Qinhuangdao, Qinhuangdao 066004, China · School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China +1
Drug-Target Interaction Prediction via Hierarchical Sequential Cross-Attention over Chemical and Protein Language Models
Sep 28, 2026Khadidja Henni, Hamza Abdelali, Abdelkrim Aries +4
I2A institute, T´ELUQ University, Montreal, QC, Canada · LIO, CRCHUM, Montreal, QC, Canada · LCSI, ESI, Algiers, Algeria +3
LaPro-DTA: Latent Dual-View Drug Representations and Salient Protein Feature Extraction for Generalizable Drug--Target Affinity Prediction
Mar 16, 2026Zihan Dun, Liuyi Xu, An-Yang Lu +2Protein-Ligand Binding Affinity Prediction Molecular Representation Learning
College of Information Science and Engineering, Northeastern University, Shenyang 110819, China · School of Artificial Intelligence, Beihang University, Beijing 100000, China · School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China

Sep 2, 2026J/K move · Enter open · S save

Chemical and geometric representation fidelity improves drug--target affinity prediction

Yixiao Li, Yining Qian, Yefan Chen, Zenghui Chen+4

College of Information Science and Engineering, Northeastern University, Shenyang 110819, China · School of Computer and Communication Engineering, Northeastern University at Qinhuangdao, Qinhuangdao 066004, China · School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China+1

Protein-Ligand Binding Affinity Prediction Representation Learning Molecular Representation Learning Geometric Representation Learning Protein Representation Learning

Open paper AskPDF arXiv

Predicting drug--target binding affinity (DTA) requires models to distinguish subtle chemical and structural determinants underlying molecular recognition. Although recent approaches increasingly incorporate richer drug and protein information, such information may be compressed, homogenized or discretized during representation construction, causing affinity-relevant distinctions to be lost before interaction modelling. We hypothesized that this representation-stage information loss constitutes an upstream bottleneck that cannot be reliably overcome by increasingly complex interaction predictors. To test this hypothesis, we developed ReGeoDTA, a representation-preserving framework that maintains affinity-relevant chemical heterogeneity in molecular representations and continuous geometric relationships in protein structures. Across three benchmark datasets, ReGeoDTA consistently improved affinity prediction, and the proposed representation-preserving strategies retained their benefits across diverse DTA architectures. Controlled representation degradation progressively reduced predictive performance, whereas increasing downstream predictor complexity failed to recover information lost during representation construction. These findings identify representation fidelity as an upstream design principle for accurate and generalizable drug--target affinity prediction, with potential implications for computational compound prioritization.

Abstract

Explore similar work

Chemical and geometric representation fidelity improves drug--target affinity prediction

Drug-Target Interaction Prediction via Hierarchical Sequential Cross-Attention over Chemical and Protein Language Models

LaPro-DTA: Latent Dual-View Drug Representations and Salient Protein Feature Extraction for Generalizable Drug--Target Affinity Prediction