CODesign: Consistency from Data to Trajectory in All-Atom Protein Binder Co-Design

Yuanle Mo; Bo Qiang; Haitao Lin; Qinghan Wang; Gang Du; Odin Zhang; Pheng Ann Heng

cs.CEOct 1, 2026

CODesign: Consistency from Data to Trajectory in All-Atom Protein Binder Co-Design

Authors: Yuanle Mo, Bo Qiang, Haitao Lin, Qinghan Wang, Gang Du, Odin Zhang, Pheng Ann Heng

Organizations: Valhalla Technology · The Chinese University of Hong Kong · University of Washington · Westlake University · Zhejiang University

Abstract

The central challenge in de novo protein design is generating plausible, mutually compatible structures and sequences, such that each designed sequence folds into its intended structure and the structure accommodates that sequence. Compared to typical two-stage design methods, which decouple the modeling of the interdependent modalities, co-design models improve the cross-modal consistency by jointly generating sequences and structures. However, naively generating sequences and structures simultaneously does not ensure their consistency. To address this challenge, we propose CODesign framework. We improve data consistency by generating approximately 105,000 consistency-distilled dimers. We further promote consistency through a multimodal joint flow model that captures the joint distribution of sequences, backbone structures, and local atomic configurations, together with a consistency-aware joint resampling strategy that iteratively refines sequences and side chains. Experiments show that CODesign achieves state-of-the-art performance with the highest in silico success rates on both protein- and ligand-target binder design. Ablation studies also demonstrate our distilled dataset increases performance by 70.9%, which can be further improved by our proposed resampling mechanism with negligible additional computational cost. Code, model weights and the new dataset will be completely open-source.

Figures & tables

Figure 1 : CODesign jointly generates sequence, backbone, and local atomic states conditioned on either a protein or small-molecule target. A shared co-design denoiser is trained with independent modality-time sampling and predicts sequence logits, backbone updates, and local-atom updates. During generation, we apply consistency-aware joint resampling for residue and side-chain refining. This overall framework generates protein binders with consistent all-atom structures.

Figure 2 : Overview of CODesign and CAR. (a) CODesign jointly generates a binder sequence and all-atom structure conditioned on a protein or small-molecule target using coupled residue- and atom-level denoising with self-conditioning. (b) CAR evaluates decoded residues against the evolving backbone, selects incompatible identities using the model’s conditional sequence predictions, and jointly resamples their sequence and local atomic states.

Model	# Unique Successes $\uparrow$			scRMSD $\downarrow$ / $\leq 2$ Å (%) $\uparrow$			Diversity $\uparrow$	Novelty $\downarrow$
	Self	MPNN@1	MPNN@8	Self	MPNN@1	MPNN@8
two-stage
RFDiffusion3	–	6.46	10.96	–	7.24 / 36.12	4.17 / 53.02	25.66	0.91
PXDesign	–	4.59	7.01	–	5.21 / 51.15	2.29 / 74.54	15.39	0.92
Protpardelle-1c	–	0.28	0.92	–	16.44 / 10.26	12.61 / 20.40	75.15	0.88
co-design

Table 1 : Results of protein-conditioned protein binder design benchmark. scRMSD is reported as the mean value / percentage of samples with scRMSD

\leq 2

Å. Best and second-best results are highlighted in bold and underlined , respectively. Complete results with detailed standard deviations are provided in Appendix F.2 .

Model	# Unique Successes $\uparrow$					Diversity $\uparrow$	Novelty $\downarrow$	Time (s) $\downarrow$
	SAM	OQO	FAD	IAI	Mean
two-stage
RFdiffusionAA	2.0	3.4	1.6	5.0	3.0	135.65	0.74	73.78
RFDiffusion3	9.4	14.8	20.8	11.0	14.0	52.50	0.83	14.96
co-design
Complexa	3.2	3.0	5.8	8.4	5.1	147.70	0.76	10.27

Table 2 : Results of ligand-conditioned protein binder design benchmark. Best and second-best results are highlighted in bold and underlined , respectively. Complete results with detailed standard deviations are provided in Appendix F.3 .

Figure 3 : Impact of Codimer on the model’s performance. (a) Average unique successes across 5 random seeds on each target. Complete results with detailed standard deviations are provided in Appendix F.4 . (b) Distributions of different confidence and consistency metrics.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Source	Reference dimers	Codimer pairs
AFDB high-confidence heterodimers	25,837	23,823
HumanPPI	9,757	5,177
PINDER	156,242	75,528
Total	191,836	104,528

Appendix

Table 3 : Composition of Codimer . Each reference dimer produces four co-folded candidates and contributes at most one sequence–structure pair to the final dataset.

Dataset	Protein I	Protein II	Ligand adaptation
Teddymer	0.80	0.70	–
PDB binder–target examples	0.19	0.10	–
PDB monomers	0.01	0.01	–
Codimer	–	0.19	–
CDDB monomers	–	–	0.60
PLINDER	–	–	0.40

Appendix

Table 4 : Source-sampling probabilities at each training stage. Codimer is treated as a single distilled dataset. PDB monomers combine repeated-entity and single-entity examples.

Hyperparameter	Protein-conditioned	Ligand-conditioned
Residue trunk
Representation dimension $d_{\mathrm{res}}$	768	768
Conditioning dimension	256	256
Pair dimension $d_{\mathrm{pair}}$	256	256
Transformer layers	14	14
Attention heads	12	12

Appendix

Table 5 : CODesign architecture. Protein- and ligand-conditioned variants use the same dimensions and layer counts, with different target feature encoders.

Setting	Protein I	Protein II	Ligand adaptation
Number of GPUs	24	64	32
Batch size per GPU	5	5	3
Global batch size	120	320	96
Updates at selected checkpoint	60,000	10,000	10,000
Trainable parameters	Full model	Full model	LoRA adapters
Optimizer	Adam	Adam	Adam

Appendix

Table 6 : Training settings. Checkpoint steps are optimizer updates within the corresponding stage. Global batch sizes equal the per-GPU batch sizes multiplied by the number of GPUs.

Setting	Backbone $\mathbf{x}$	Sequence $\mathbf{r}$	Local atoms $\mathbf{z}$
Active updates	500	500	1,000
Time schedule	Exponential	Quadratic	Exponential
Noise-variance factor	0.3	–	0.6
Categorical temperature	–	0.3	–
Discrete stochasticity $\eta$	–	5	–
ODE switching time	0.98	–	0.98

Appendix

Table 7 : Default sampling settings of CODesign. A dash indicates a parameter that does not apply to that modality.

Model	Sequence	Unique successes $\uparrow$	scRMSD (Å) $\downarrow$	$\leq 2$ Å (%) $\uparrow$
RFDiffusion3	MPNN@1	$6.46\,\pm\,0.43$	$7.24\,\pm\,0.11$	$36.12\,\pm\,0.95$
RFDiffusion3	MPNN@8	$10.96\,\pm\,0.31$	$4.17\,\pm\,0.04$	$53.02\,\pm\,0.93$
PXDesign	MPNN@1	$4.59\,\pm\,0.25$	$5.21\,\pm\,0.10$	$51.15\,\pm\,0.72$
PXDesign	MPNN@8	$7.01\,\pm\,0.51$	$2.29\,\pm\,0.04$	$74.54\,\pm\,0.47$
Protpardelle-1c	MPNN@1	$0.28\,\pm\,0.10$	$16.44\,\pm\,0.29$	$10.26\,\pm\,0.50$
Protpardelle-1c	MPNN@8	$0.92\,\pm\,0.24$	$12.61\,\pm\,0.21$	$20.40\,\pm\,0.53$

Appendix

Table 8 : Protein-target benchmark: aggregate unique successes and complex

C_{\alpha}

scRMSD. Entries are means

\pm

standard deviations across five seeds; the threshold column is in percent.

Target	Complexa	CODesign
IFNAR2	$35.00\,\pm\,3.61$	$30.00\,\pm\,4.30$
BHRF1	$22.00\,\pm\,2.65$	$43.60\,\pm\,6.15$
BBF14	$12.80\,\pm\,1.64$	$8.00\,\pm\,1.58$
DerF21	$20.60\,\pm\,2.88$	$36.40\,\pm\,4.51$
TrkA	$10.00\,\pm\,1.73$	$2.60\,\pm\,1.34$
PD1	$13.20\,\pm\,3.96$	$2.00\,\pm\,0.00$

Appendix

Table 9 : Per-target unique successes with Self sequences. Entries report means

\pm

standard deviations across five seeds.

Target	RFDiffusion3	PXDesign	Protpardelle-1c	Complexa	CODesign
IFNAR2	$24.20\,\pm\,3.96$	$11.80\,\pm\,2.95$	$0.80\,\pm\,0.84$	$39.20\,\pm\,3.49$	$41.20\,\pm\,2.39$
BHRF1	$9.20\,\pm\,1.10$	$5.00\,\pm\,1.00$	$1.20\,\pm\,0.84$	$19.20\,\pm\,1.64$	$40.60\,\pm\,9.99$
BBF14	$1.80\,\pm\,1.79$	$3.60\,\pm\,1.82$	$0.00\,\pm\,0.00$	$10.40\,\pm\,2.70$	$10.40\,\pm\,1.52$
DerF21	$1.80\,\pm\,0.45$	$8.00\,\pm\,1.58$	$2.60\,\pm\,1.14$	$23.60\,\pm\,1.82$	$42.80\,\pm\,5.81$
TrkA	$10.20\,\pm\,1.79$	$7.80\,\pm\,0.84$	$0.20\,\pm\,0.45$	$12.80\,\pm\,2.39$	$2.40\,\pm\,2.07$
PD1	$10.60\,\pm\,2.41$	$6.00\,\pm\,1.87$	$0.00\,\pm\,0.00$	$13.80\,\pm\,1.79$	$2.20\,\pm\,1.64$

Appendix

Table 10 : Per-target unique successes with MPNN@1 sequences. Entries report means

\pm

standard deviations across five seeds.

Target	RFDiffusion3	PXDesign	Protpardelle-1c	Complexa	CODesign
IFNAR2	$35.60\,\pm\,4.16$	$19.00\,\pm\,2.92$	$2.60\,\pm\,1.14$	$58.80\,\pm\,2.77$	$62.60\,\pm\,4.39$
BHRF1	$11.60\,\pm\,0.55$	$7.20\,\pm\,1.92$	$4.80\,\pm\,1.10$	$23.00\,\pm\,2.55$	$67.60\,\pm\,6.50$
BBF14	$4.80\,\pm\,1.30$	$7.40\,\pm\,2.07$	$0.20\,\pm\,0.45$	$19.20\,\pm\,4.09$	$19.80\,\pm\,3.42$
DerF21	$5.20\,\pm\,0.84$	$11.40\,\pm\,4.67$	$3.40\,\pm\,1.14$	$34.80\,\pm\,5.45$	$63.00\,\pm\,5.00$
TrkA	$14.40\,\pm\,2.88$	$11.20\,\pm\,2.49$	$0.80\,\pm\,1.30$	$20.20\,\pm\,1.64$	$6.00\,\pm\,2.45$
PD1	$17.40\,\pm\,2.30$	$8.20\,\pm\,1.10$	$0.40\,\pm\,0.89$	$26.20\,\pm\,2.95$	$4.00\,\pm\,1.41$

Appendix

Table 11 : Per-target unique successes with MPNN@8 sequences. Entries report means

\pm

standard deviations across five seeds.

Target	Complexa	CODesign
IFNAR2	$4.26\,\pm\,0.25$	$2.17\,\pm\,0.24$
BHRF1	$1.44\,\pm\,0.16$	$0.90\,\pm\,0.08$
BBF14	$9.36\,\pm\,0.42$	$8.71\,\pm\,0.48$
DerF21	$3.53\,\pm\,0.14$	$4.26\,\pm\,0.24$
TrkA	$4.42\,\pm\,0.20$	$4.73\,\pm\,0.35$
PD1	$5.08\,\pm\,0.37$	$4.66\,\pm\,0.50$

Appendix

Table 12 : Per-target mean complex

C_{\alpha}

scRMSD with Self sequences. Values are in Å; lower is better. Entries report means

\pm

standard deviations across five seeds.

Target	RFDiffusion3	PXDesign	Protpardelle-1c	Complexa	CODesign
IFNAR2	$2.56\,\pm\,0.21$	$4.72\,\pm\,0.36$	$5.39\,\pm\,0.34$	$4.50\,\pm\,0.49$	$2.17\,\pm\,0.21$
BHRF1	$2.63\,\pm\,0.29$	$2.18\,\pm\,0.19$	$7.82\,\pm\,0.45$	$1.22\,\pm\,0.15$	$0.87\,\pm\,0.10$
BBF14	$10.16\,\pm\,0.28$	$7.59\,\pm\,0.44$	$45.26\,\pm\,2.20$	$7.37\,\pm\,0.20$	$6.33\,\pm\,0.37$
DerF21	$10.32\,\pm\,0.24$	$3.48\,\pm\,0.11$	$19.21\,\pm\,2.51$	$2.85\,\pm\,0.31$	$4.92\,\pm\,0.40$
TrkA	$2.81\,\pm\,0.23$	$2.20\,\pm\,0.10$	$7.80\,\pm\,0.33$	$3.77\,\pm\,0.27$	$4.38\,\pm\,0.32$
PD1	$3.57\,\pm\,0.35$	$4.46\,\pm\,0.65$	$7.81\,\pm\,0.15$	$3.79\,\pm\,0.15$	$4.66\,\pm\,0.55$

Appendix

Table 13 : Per-target mean complex

C_{\alpha}

scRMSD with MPNN@1 sequences. Values are in Å; lower is better. Entries report means

\pm

standard deviations across five seeds.

Target	RFDiffusion3	PXDesign	Protpardelle-1c	Complexa	CODesign
IFNAR2	$1.09\,\pm\,0.05$	$1.62\,\pm\,0.24$	$2.55\,\pm\,0.18$	$1.91\,\pm\,0.20$	$1.06\,\pm\,0.05$
BHRF1	$1.31\,\pm\,0.13$	$1.15\,\pm\,0.13$	$4.83\,\pm\,0.30$	$0.82\,\pm\,0.03$	$0.57\,\pm\,0.02$
BBF14	$5.97\,\pm\,0.30$	$4.07\,\pm\,0.39$	$42.49\,\pm\,2.47$	$3.56\,\pm\,0.16$	$3.14\,\pm\,0.27$
DerF21	$7.76\,\pm\,0.22$	$1.32\,\pm\,0.10$	$13.80\,\pm\,2.35$	$1.53\,\pm\,0.08$	$2.40\,\pm\,0.25$
TrkA	$1.23\,\pm\,0.09$	$0.86\,\pm\,0.08$	$4.47\,\pm\,0.13$	$1.83\,\pm\,0.08$	$2.25\,\pm\,0.27$
PD1	$1.47\,\pm\,0.13$	$1.62\,\pm\,0.11$	$4.42\,\pm\,0.31$	$1.58\,\pm\,0.17$	$1.91\,\pm\,0.17$

Appendix

Table 14 : Per-target mean complex

C_{\alpha}

scRMSD with MPNN@8 sequences. Values are in Å; lower is better. Entries report means

\pm

standard deviations across five seeds.

Target	Complexa	CODesign
IFNAR2	$58.10\,\pm\,1.08$	$79.20\,\pm\,2.95$
BHRF1	$94.00\,\pm\,1.27$	$96.90\,\pm\,1.14$
BBF14	$26.50\,\pm\,2.78$	$31.10\,\pm\,3.96$
DerF21	$42.10\,\pm\,2.58$	$69.20\,\pm\,3.05$
TrkA	$48.70\,\pm\,2.75$	$34.50\,\pm\,3.48$
PD1	$45.60\,\pm\,4.36$	$46.10\,\pm\,6.05$

Appendix

Table 15 : Per-target percentage of designs with complex

C_{\alpha}

scRMSD

\leq 2

Å with Self sequences. Values are percentages; higher is better. Entries report means

\pm

standard deviations across five seeds.

Target	RFDiffusion3	PXDesign	Protpardelle-1c	Complexa	CODesign
IFNAR2	$80.90\,\pm\,3.19$	$57.60\,\pm\,2.90$	$40.30\,\pm\,2.08$	$57.90\,\pm\,2.41$	$80.20\,\pm\,2.51$
BHRF1	$78.80\,\pm\,3.63$	$83.90\,\pm\,2.22$	$25.80\,\pm\,3.33$	$94.60\,\pm\,1.08$	$97.10\,\pm\,1.47$
BBF14	$4.80\,\pm\,1.68$	$20.30\,\pm\,3.07$	$0.10\,\pm\,0.22$	$27.30\,\pm\,1.44$	$37.60\,\pm\,3.17$
DerF21	$0.80\,\pm\,0.57$	$68.60\,\pm\,1.56$	$8.40\,\pm\,2.16$	$57.50\,\pm\,3.66$	$66.00\,\pm\,2.32$
TrkA	$65.70\,\pm\,2.51$	$75.10\,\pm\,2.10$	$14.90\,\pm\,3.29$	$54.50\,\pm\,3.71$	$44.70\,\pm\,2.68$
PD1	$58.10\,\pm\,2.92$	$54.10\,\pm\,1.92$	$15.50\,\pm\,1.62$	$53.20\,\pm\,3.09$	$32.20\,\pm\,3.65$

Appendix

Table 16 : Per-target percentage of designs with complex

C_{\alpha}

scRMSD

\leq 2

Å with MPNN@1 sequences. Values are percentages; higher is better. Entries report means

\pm

standard deviations across five seeds.

Target	RFDiffusion3	PXDesign	Protpardelle-1c	Complexa	CODesign
IFNAR2	$92.30\,\pm\,0.76$	$83.80\,\pm\,3.37$	$65.00\,\pm\,3.59$	$80.60\,\pm\,2.38$	$90.00\,\pm\,1.90$
BHRF1	$88.30\,\pm\,2.33$	$89.50\,\pm\,2.21$	$41.20\,\pm\,4.91$	$96.50\,\pm\,0.50$	$99.70\,\pm\,0.45$
BBF14	$13.30\,\pm\,2.84$	$39.40\,\pm\,6.66$	$0.30\,\pm\,0.27$	$46.90\,\pm\,2.51$	$60.30\,\pm\,3.55$
DerF21	$2.80\,\pm\,2.08$	$90.50\,\pm\,1.00$	$21.70\,\pm\,3.58$	$81.50\,\pm\,2.47$	$80.90\,\pm\,2.41$
TrkA	$86.80\,\pm\,2.71$	$94.90\,\pm\,1.14$	$30.00\,\pm\,3.43$	$75.20\,\pm\,2.33$	$66.50\,\pm\,5.26$
PD1	$83.20\,\pm\,2.20$	$84.00\,\pm\,1.22$	$30.80\,\pm\,3.27$	$77.50\,\pm\,4.56$	$63.90\,\pm\,4.17$

Appendix

Table 17 : Per-target percentage of designs with complex

C_{\alpha}

scRMSD

\leq 2

Å with MPNN@8 sequences. Values are percentages; higher is better. Entries report means

\pm

standard deviations across five seeds.

Model	SAM	OQO	FAD	IAI	Mean
RFdiffusionAA	$2.0\,\pm\,1.4$	$3.4\,\pm\,1.3$	$1.6\,\pm\,1.1$	$5.0\,\pm\,2.5$	$3.0\,\pm\,0.4$
RFDiffusion3	$9.4\,\pm\,3.2$	$14.8\,\pm\,2.4$	$20.8\,\pm\,2.6$	$11.0\,\pm\,1.2$	$14.0\,\pm\,1.1$
Complexa	$3.2\,\pm\,1.5$	$3.0\,\pm\,1.2$	$5.8\,\pm\,2.9$	$8.4\,\pm\,1.7$	$5.1\,\pm\,0.7$
Pallatom-Ligand	$0.2\,\pm\,0.4$	$4.4\,\pm\,1.1$	$2.0\,\pm\,1.0$	$3.2\,\pm\,1.3$	$2.5\,\pm\,0.5$
DISCO	$10.6\,\pm\,3.1$	$14.0\,\pm\,2.5$	$25.4\,\pm\,3.8$	$12.8\,\pm\,0.8$	$15.7\,\pm\,1.0$
CODesign	$5.8\,\pm\,1.6$	$37.2\,\pm\,3.8$	$21.4\,\pm\,2.3$	$18.6\,\pm\,2.7$	$20.8\,\pm\,1.3$

Appendix

Table 18 : Ligand-target unique successes. Entries report means

\pm

standard deviations across five seeds. The final column averages the four targets within each seed.

Variant	Codimer	CAR	Unique successes $\uparrow$	Time (s)
Without Codimer , without CAR	No	No	$5.54\,\pm\,0.48$	–
Without CAR	Yes	No	$9.46\,\pm\,0.65$	27.96
CODesign	Yes	Yes	$9.84\,\pm\,0.25$	28.41

Appendix

Table 19 : Controlled ablations on protein targets using Self sequences. Quality entries report means

\pm

standard deviations across five seeds; times correspond to Figure 4 . A dash indicates that generation time was not evaluated.

Target	Without Codimer without CAR	With Codimer without CAR	CODesign
IFNAR2	$13.60\,\pm\,1.14$	$30.20\,\pm\,7.12$	$30.00\,\pm\,4.30$
BHRF1	$20.60\,\pm\,6.27$	$43.00\,\pm\,4.85$	$43.60\,\pm\,6.15$
BBF14	$1.00\,\pm\,1.00$	$7.40\,\pm\,1.52$	$8.00\,\pm\,1.58$
DerF21	$23.40\,\pm\,3.91$	$34.20\,\pm\,3.27$	$36.40\,\pm\,4.51$
TrkA	$3.80\,\pm\,0.84$	$2.40\,\pm\,0.55$	$2.60\,\pm\,1.34$
PD1	$4.40\,\pm\,1.34$	$1.40\,\pm\,0.55$	$2.00\,\pm\,0.00$

Appendix

Table 20 : Per-target unique successes in the Codimer and CAR ablations. All variants use Self sequences. Entries report means

\pm

standard deviations across five seeds.

Explore similar work

May 5, 2026q-bio.QM

A-CODE: Fully Atomic Protein Co-Design with Unified Multimodal Diffusion

We present A-CODE, a fully atomic unified one-stage protein co-design model that simultaneously refines discrete atom types and continuous atom coordinates. Unlike predominant two-stage methods that cascade structure design with amino acid-level sequence design, our approach is fully atomic within a unified multimodal diffusion framework, in which residue identities are inferred solely from atom-level predictions. Built upon the powerful all-atom architecture, A-CODE achieves superior designability for unconditional protein generation, outperforming all existing one-stage and two-stage design models. For binder design, A-CODE rivals and even outperforms existing state-of-the-art two-stage design models and, compared with the existing one-stage co-design model, achieves a drastic tenfold improvement in success rate on hard tasks. The inherent flexibility of our atomic formulation enables, for the first time, seamless adaptation to non-canonical amino acid (ncAA) modeling. Our fully atomic framework establishes a new, versatile foundation for all-atom generative modeling that can be naturally extended to complex biomolecular systems.

Chaoran Cheng, Jiaqi Guan, Milong Ren +5

University of Illinois Urbana-Champaign · 2ByteDance Seed

Protein Design Molecular Generation

Ask PDF

Jul 26, 2026cs.LG

Chamaileon: Cross-Context Binder Design with Contextualized Modeling and Mixed Sampling

The rapid evolution of generative models has unlocked new potentials in protein binder design, a pivotal task in structural biology, by facilitating end-to-end generation via joint sequence-structure modeling or hallucination. However, existing approaches are predominantly implemented under a single-target, single-state assumption, limiting their ability to model multi-target or multi-state interactions required for advanced function-oriented protein design. Here, we introduce Chamaileon, which unifies multi-target and multi-state binder design by formulating the problem as cross-context binding landscape modeling. The framework is underpinned by a training paradigm termed In-Context Complex Co-Design (I3CD) for context-aware sequence-structure co-modeling. During inference, we employ Mixture-of-Paths Sampling (MoPS), a scalable strategy that optimizes a single sequence across contexts while alleviating the scarcity of high-quality multi-conformational paired data. Extensive evaluation on our newly constructed benchmark, CROSS, demonstrates that Chamaileon effectively generates sequences adaptable to diverse conformational landscapes and multi-target requirements. The code is available on https://github.com/caohengyuan/Chamaileon.

Hengyuan Cao, Shizhuo Cheng, Mingxuan Liu +5

Zhejiang University, Hangzhou, China · Fudan University, Shanghai, China · Shanghai Institute for Advanced Study Zhejiang University, Shanghai, China +1

Protein Design Conformational Ensembles Sequence Modeling

Ask PDF

Sep 3, 2026cs.LG

SimpleDesign: A Joint Model for Protein Sequence and Structure Codesign

Proteins are fundamental to biological processes, with their function determined by the complex interplay between the amino acid sequence and the three-dimensional structure. Developing generative models capable of understanding this intrinsically multi-modal relationship is crucial for fields like drug discovery and protein engineering. Existing models often rely on a multi-stage training process where autoencoders that tokenize data into latent representations are trained in a first stage. Secondly, a generative model is trained on the latent representation of the autoencoder(s), i.e., generative modeling in a latent space. We hypothesize that this multi-stage training is not necessary to obtain performant co-design models and thus present SimpleDesign, an effective multi-modal protein design model trained directly in the data space. SimpleDesign leverages a single-stage end-to-end objective that combines discrete cross-entropy for sequences and a regression objective for structures. In order to effectively model the difference in sequence and structure modalities, we develop a Mixture-of-Transformer architecture that allows modality-specific processing while keeping global self-attention over both modalities. We train SimpleDesign on over 2M sequence-structure pairs achieving strong performance across co-design and unconditional sequence/structure generation benchmarks.

Jiarui Lu, Yuyang Wang, Yizhe Zhang +4

Mila, Université de Montréal, Apple · Apple

Protein Design Generative Models Modalities Multi-Stage Training

Ask PDF