cs.LGOct 29, 2026
SaveTACFormer: Transformer-Based Generative Model for Target-Conditioned De Novo PROTAC Design
Organizations: AI Laboratory for Molecular Engineering (AIME), Department of Computer Science and Engineering, Chalmers University of Technology & University of Gothenburg, Gothenburg, Sweden · Science for Life Laboratory (SciLifeLab), Gothenburg, Sweden
Abstract
Targeted protein degradation (TPD) represents a fundamental shift in drug discovery, moving beyond traditional target inhibition toward the event-driven removal of disease-causing proteins. A prominent strategy for TPD is the use of proteolysis-targeting chimeras (PROTACs), heterobifunctional molecules that selectively route a protein of interest (POI) to the ubiquitin-proteasome system for degradation. While PROTAC design still relies heavily on trial-and-error synthesis and experimental testing, deep generative models offer a promising means to explore PROTAC chemical space in silico and generate novel molecules. However, many existing generative methods formulate PROTAC design primarily as a linker-generation problem, with POI-binding (warhead) and E3-ligase-binding components fixed in advance. This restricts exploration of the full PROTAC chemical space and often leaves the POI implicit through the predefined warhead, rather than using protein-level information as an explicit conditioning signal during generation. To address these limitations, we propose TACFormer: a transformer-based generative model for target-aware de novo PROTAC design. The model conditions a GPT-style molecular decoder on pretrained ESM-2 representations of POI sequences, allowing POI-level information to steer full-molecule generation. Our results show that TACFormer generates highly novel and chemically diverse PROTACs while showing consistent POI-dependent shifts in generated chemical space. In particular, molecules generated under specific target conditions preferentially populate chemical neighborhoods associated with POIs in the held-out dataset, while extrapolating to novel chemotypes. By natively integrating biological context into the generative process, TACFormer moves PROTAC design toward a more biology-informed exploration of degrader chemical space.
Figures & tables
Figure 1: Overview of the TACFormer architecture. The protein of interest (POI) amino-acid sequence is encoded using pretrained ESM-2. The resulting residue representations are mean-pooled and projected into a 512-dimensional POI embedding. A GPT-style decoder generates PROTAC SMILES autoregressively. Protein conditioning is introduced in each decoder block by adding the projected POI embedding to the hidden representations after self-attention, scaled by a learnable layer-specific gate. The model generates a complete PROTAC SMILES conditioned on the specified POI.
Figure 2: POI-specific PROTAC chemistry is globally heterogeneous but locally organized. All analyses use 2048-bit Morgan fingerprints with radius 2. (a,b) The same two-dimensional embedding of PROTAC chemical space, colored by (a) E3-ligase identity and (b) POI identity. Both panels contain 3,000 molecules, with 300 sampled from each of the 10 most represented POIs. E3 identity broadly structures chemical space, while individual POIs show local enrichment across distinct regions. (c) Morgan–Tanimoto similarity between same-POI and different-POI molecular pairs across the seen POIs. We sample 10,000 pairs of each type. Same-POI similarity is higher overall (0.385 versus 0.190), within CRBN (0.421 versus 0.223), and within VHL (0.535 versus 0.329), indicating that the POI-specific similarity signal persists after controlling for E3 identity. (d) Local POI purity within CRBN and VHL chemical space. Purity measures the fraction of the nearest neighbors sharing the query’s POI, with –30. Solid lines show observed purity, while dashed lines represent the null obtained from 300 within-E3 label permutations. At , observed purity remains high for CRBN (0.977) and VHL (0.933), well above the shuffled-label null. Strong local enrichment despite moderate global similarity indicates that individual POIs can span multiple locally coherent chemotypes.
| POI | Validity (%) | Uniqueness (%) | Novelty (%) | Diversity | 5-NN (%) | 10-NN (%) | 20-NN (%) | |
|---|---|---|---|---|---|---|---|---|
| AR | 99.65 (0.07) | 48.64 (0.53) | 79.38 (0.09) | 0.689 (0.004) | 0.278 | 97.89 (0.15) | 97.32 (0.16) | 96.33 (0.15) |
| ESR1 | 99.53 (0.11) | 41.12 (0.38) | 80.54 (0.13) | 0.656 (0.001) | 0.366 | 97.94 (0.20) | 97.12 (0.25) | 94.78 (0.23) |
| CDK4 | 99.58 (0.05) | 30.60 (0.14) | 81.86 (0.20) | 0.566 (0.006) | 0.618 | 95.17 (0.10) | 93.62 (0.23) | 92.66 (0.36) |
| KRAS | 99.46 (0.06) | 49.80 (0.26) | 85.59 (0.18) | 0.641 (0.004) | 0.498 | 97.91 (0.08) | 97.18 (0.06) | 95.56 (0.09) |
| EZH2 | 98.40 (0.01) | 31.50 (0.55) | 94.57 (0.14) | 0.629 (0.003) | 0.358 | 82.38 (0.25) | 74.63 (0.13) | 56.15 (0.30) |
| BTK | 97.85 (0.15) | 43.06 (0.19) | 87.63 (0.31) | 0.716 (0.003) | 0.428 | 88.58 (0.21) | 85.50 (0.20) | 81.07 (0.21) |
Table 1: Target-wise generation quality and conditional fidelity across the seen POIs. Generation metrics and nearest-neighbor purity are reported as mean (standard deviation) across three generation seeds. Best per-target values in each column are shown in bold.
Figure 3: Generated PROTACs span distinct warhead scaffolds under the same POI condition. Rows show BRD4, MAP4K1, and BTK, each with one reference PROTAC and three generated examples. These targets have the highest warhead internal diversity ( , , and , respectively). Gold highlighting identifies the warhead predicted by PROTAC-Splitter in each generated molecule. Bemis–Murcko scaffolds are extracted from the parent warheads to group molecules by their core structures. For each POI, the generated examples represent three distinct warhead-scaffold groups.
| POI | (pp) | (pp) | (pp) |
|---|---|---|---|
| AR | 75.14 (0.02) | 74.14 (0.10) | 72.32 (0.10) |
| ESR1 | 88.03 (0.18) | 86.86 (0.24) | 84.10 (0.21) |
| CDK4 | 89.38 (0.06) | 87.94 (0.15) | 87.01 (0.27) |
| KRAS | 87.95 (0.06) | 87.13 (0.05) | 85.41 (0.10) |
| EZH2 | 80.82 (0.27) | 73.01 (0.12) | 54.64 (0.28) |
| BTK | 82.32 (0.23) | 79.23 (0.20) | 74.75 (0.20) |
Table 2: Improvement in POI-neighborhood purity over unconditional generation. measures the difference between conditional and unconditional -NN purity for each POI. Values are reported in percentage points as mean (standard deviation) across three generation seeds.
| POI | Validity (%) | Uniq. (%) | Novelty (%) | Diversity | 5-NN (%) | 10-NN (%) | 20-NN (%) | |
|---|---|---|---|---|---|---|---|---|
| WDR5 | 75.44 (0.53) | 99.21 (0.08) | 99.96 (0.01) | 0.737 (0.002) | 56.23 (0.10) | 55.35 (0.35) | 50.58 (0.44) | |
| DHFR | 95.33 (0.03) | 88.59 (0.20) | 99.57 (0.01) | 0.669 (0.003) | 31.05 (0.32) | 37.06 (0.28) | 45.18 (0.14) | |
| FGFR3 | 96.96 (0.04) | 45.95 (0.57) | 97.31 (0.08) | 0.716 (0.003) | 0.080 | 20.26 (0.15) | 20.51 (0.07) | 20.70 (0.05) |
| MAP3K13 | 99.29 (0.12) | 36.23 (0.22) | 95.90 (0.25) | 0.727 (0.004) | 0.096 | 14.57 (0.46) | 13.27 (0.31) | 12.71 (0.12) |
| Macro | 91.76 (0.11) | 67.50 (0.15) | 98.19 (0.04) | 0.712 (0.001) | 30.52 (0.06) | 31.55 (0.11) | 32.29 (0.14) |
Table 3: Generation quality and conditional fidelity for the four zero-shot POIs. WDR5, DHFR, FGFR3, and MAP3K13 are excluded entirely from conditional training and provided to the model only at inference. Generation metrics and nearest-neighbor purity are reported as mean (standard deviation) across three generation seeds.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Dataset composition and component-level scaffold similarity. (a) Number of molecules of seen POIs. (b) Cumulative fraction of the dataset covered by POIs ranked by molecule count. (c) E3-ligase distribution across the complete dataset. (d) E3-ligase composition across the seen POIs. (e) Pairwise scaffold similarity across the seen POIs, comparing predicted POI-warhead scaffolds (lower triangle) and E3-ligand scaffolds (upper triangle). Similarity is measured using symmetric nearest-scaffold Tanimoto similarity.
| Role | POI | Records | Closest top-20 POI | Cosine similarity |
|---|---|---|---|---|
| Most dissimilar | WDR5 | 66 | PTPN1 | 0.760 |
| Dissimilar | DHFR | 43 | EGFR | 0.894 |
| Medium | FGFR3 | 58 | MAP4K1 | 0.969 |
| Very similar | MAP3K13 | 50 | AR | 0.987 |
Table 4: Protein-representation similarity of the four zero-shot POIs. For each zero-shot POI, the closest seen POI is identified by cosine similarity between mean-pooled ESM-2 embeddings.
| POI | Total | Train | Val | Test |
|---|---|---|---|---|
| AR | 4933 | 3947 | 493 | 493 |
| KRAS | 2776 | 2220 | 278 | 278 |
| ESR1 | 2212 | 1770 | 221 | 221 |
| EGFR | 1617 | 1293 | 162 | 162 |
| CDK4 | 1297 | 1037 | 130 | 130 |
| BTK | 1293 | 1035 | 129 | 129 |
Table 5: Target-wise counts for the seen POIs. Data for each POI are split into 80% training, 10% validation, and 10% test subsets while keeping molecular structures disjoint across partitions.
Figure 5: Physicochemical distributions of PROTAC-like, reference PROTAC, and generated molecules. Normalized density distributions for PROTAC-like structures, reference PROTACs, and molecules generated by the conditional model. Dashed vertical lines indicate distribution means. (a) Molecular weight (MW). (b) LogP. (c) Topological polar surface area (TPSA). (d) Number of rotatable bonds. (e) Number of aromatic rings. (f) Number of non-aromatic rings. (g) Number of hydrogen-bond acceptors (HBA). (h) Number of hydrogen-bond donors (HBD).
| POI | Unique PROTACs | Unique warheads | Warhead uniqueness (%) | Warhead novelty (%) | Warhead diversity |
|---|---|---|---|---|---|
| AR | 4,847 (50) | 1,005 (30) | 20.75 (0.40) | 61.20 (0.39) | 0.811 (0.001) |
| ESR1 | 4,093 (37) | 893 (33) | 21.85 (0.64) | 69.09 (0.50) | 0.776 (0.005) |
| CDK4 | 3,047 (13) | 619 (15) | 20.31 (0.52) | 72.99 (1.88) | 0.780 (0.004) |
| KRAS | 4,954 (25) | 898 (29) | 18.12 (0.62) | 80.01 (1.10) | 0.749 (0.005) |
| EZH2 | 3,100 (54) | 788 (41) | 25.42 (1.13) | 96.14 (0.23) | 0.758 (0.005) |
| BTK | 4,214 (24) | 1,350 (50) | 32.04 (1.05) | 84.86 (0.53) | 0.817 (0.001) |
Table 6: Warhead-level generation metrics across the seen POIs. Warheads are identified using PROTAC-Splitter and converted to parent structures. Uniqueness is measured over successfully processed parent warheads, and novelty is evaluated against parent warheads extracted from training PROTACs for the same POI. Values are reported as mean (sample standard deviation) across three seeds. Best per-target values in each column are shown in bold.
| POI | Splitter coverage (%) | Warhead processing (%) |
|---|---|---|
| AR | 100.00 (0.00) | 99.91 (0.03) |
| ESR1 | 99.99 (0.01) | 99.88 (0.07) |
| CDK4 | 100.00 (0.00) | 100.00 (0.00) |
| KRAS | 100.00 (0.00) | 100.00 (0.00) |
| EZH2 | 100.00 (0.00) | 99.96 (0.02) |
| BTK | 100.00 (0.00) | 100.00 (0.00) |
Table 7: PROTAC-Splitter and warhead processing coverage for seen POIs. Splitter coverage measures the proportion of unique valid generated PROTACs with a usable three-component decomposition. Warhead processing coverage measures the proportion of extracted POI warheads successfully converted to valid parent structures. Values are reported as mean (sample standard deviation) across three generation seeds. The macro row reports the unweighted mean across POIs.
| POI | Evaluation | 5-NN | 10-NN | 20-NN |
|---|---|---|---|---|
| AR | Reference | 98.50 | 98.05 | 96.68 |
| TACFormer | 97.89 (0.15) | 97.32 (0.16) | 96.33 (0.15) | |
| Unconditional | 22.75 (0.14) | 23.18 (0.13) | 24.01 (0.15) | |
| ESR1 | Reference | 97.74 | 97.42 | 94.89 |
| TACFormer | 97.94 (0.20) | 97.12 (0.25) | 94.78 (0.23) | |
| Unconditional | 9.91 (0.02) | 10.27 (0.02) | 10.68 (0.03) |
Table 8: Reference, conditional, and unconditional -NN purity for seen POIs. Reference purity is fixed across seeds. TACFormer and unconditional results are mean (sample standard deviation) across three seeds. All values are percentages; macro values are unweighted means across seen POIs. Bold values indicate the highest target-wise results within each evaluation type and neighborhood size.
| POI | Ref. 5-NN | Gen. 5-NN | Ref. 10-NN | Gen. 10-NN | Ref. 20-NN | Gen. 20-NN |
|---|---|---|---|---|---|---|
| WDR5 | 100.00 | 56.23 (0.10) | 97.88 | 55.35 (0.35) | 94.09 | 50.58 (0.44) |
| DHFR | 98.60 | 31.05 (0.32) | 94.42 | 37.06 (0.28) | 90.70 | 45.18 (0.14) |
| FGFR3 | 100.00 | 20.26 (0.15) | 100.00 | 20.51 (0.07) | 100.00 | 20.70 (0.05) |
| MAP3K13 | 98.40 | 14.57 (0.46) | 96.80 | 13.27 (0.31) | 95.30 | 12.71 (0.12) |
| Macro | 99.25 | 30.52 (0.06) | 97.27 | 31.55 (0.11) | 95.02 | 32.29 (0.14) |
Table 9: Reference versus conditional -NN purity for four zero-shot POIs. Reference values are fixed across seeds. Generated values are reported as mean (sample standard deviation) across three generation seeds. All values are percentages. Macro values are unweighted means across the four POIs.
Figure 6: Generated PROTACs retain the heterogeneous chemical-space organization observed in reference data. Two-dimensional t-SNE projection of novel molecules generated under the seen POI conditions, colored by conditioning POI. Molecules are represented using 2048-bit Morgan fingerprints with radius 2 and projected using a single common t-SNE fit. Example structures are novel generated PROTACs, with arrows indicating their conditioning POIs.
Figure 7: Generation quality during molecular pretraining, unconditional fine-tuning, and conditional fine-tuning. Panels (a,b) show the molecular prior, (c,d) the unconditional model, and (e,f) the conditional model. (a,c,e) Validity, uniqueness among valid molecules, and novelty among unique valid molecules across training epochs. (b,d,f) Internal diversity among unique valid molecules. For the conditional model, solid lines represent the mean across the seen POIs, and shaded regions indicate standard deviation across targets.
Figure 8: Emergence of POI-specific conditioning during fine-tuning. (a) and (b) 5-NN purity across training epochs. Solid lines show the mean across the seen POIs, and shaded regions indicate standard deviation. Likelihood evaluation uses a fixed set of validation molecules, while nearest-neighbor purity uses a fixed validation reference pool. At each monitored epoch, 500 molecules are generated per POI, with up to 25 unique valid molecules per target selected for 5-NN evaluation.
| Epoch | Validity (%) | Uniq. (%) | Novelty (%) | Diversity | 5-NN (%) | Score (%) | |||
|---|---|---|---|---|---|---|---|---|---|
| 46 | 98.40 | 72.90 | 74.13 | 0.795 | 0.4058 | 87.56 | 0.8063 | 0.9247 | 86.34 |
| 60 | 98.84 | 71.30 | 72.33 | 0.796 | 0.4154 | 87.12 | 0.7981 | 0.9332 | 86.30 |
| 52 | 98.63 | 72.03 | 72.85 | 0.795 | 0.4106 | 87.44 | 0.8009 | 0.9295 | 86.28 |
| 56 | 98.66 | 71.34 | 71.38 | 0.796 | 0.4144 | 86.76 | 0.7952 | 0.9302 | 86.00 |
| 58 | 98.78 | 71.04 | 71.77 | 0.796 | 0.4155 | 89.72 | 0.7957 | 0.9472 | 86.82 |
Table 10: Conditional checkpoint selection during fine-tuning. The five highest-scoring checkpoints are reported. Generation quality combines validity, uniqueness, novelty, and internal diversity. Conditional fidelity combines generated 5-NN purity and normalized . The final score balances both components. Bold values indicate the highest values among the reported checkpoints.
| Epoch | Validity (%) | Uniq. (%) | Novelty (%) | Diversity |
|---|---|---|---|---|
| 51 | 99.80 | 95.99 | 67.43 | 0.785 |
| 58 | 99.80 | 96.79 | 66.46 | 0.790 |
Table 11: Unconditional checkpoint selection during fine-tuning. The two checkpoints with the highest observed validity are compared. Metrics are computed from 500 generated molecules per epoch. Bold values indicate the highest values in each column.