Trion-1: Unifying Language, Protein Sequence, and Structure with Diffusion Language Models

Tianyu Lu; Kaicheng Yang; Jiangtao Feng; Keyue Qiu; Yawen Ouyang; Yixin Wu; Xinbo Zhang; Lihao Wang

cs.AIOct 22, 2026

Trion-1: Unifying Language, Protein Sequence, and Structure with Diffusion Language Models

Authors: Tianyu Lu, Kaicheng Yang, Jiangtao Feng, Keyue Qiu, Yawen Ouyang, Yixin Wu, Xinbo Zhang, Lihao Wang,

Abstract

We envision a general-purpose protein foundation model as a unified, interactive model of sequence, structure, and function, capable of understanding, generation, and cross-modal translation while retaining the instruction-following and reasoning capabilities of modern LLMs. Toward this vision, we introduce Trion-1, the first native, unified, multimodal diffusion language model (dLLM) for proteins. Trion-1 is built on two key design choices: (1) a unified discrete token space that jointly represents sequence, structure, and natural language; (2) a dLLM backbone that replaces strict left-to-right factorization with bidirectional or semi-bidirectional token interactions, enabling global residue communication and long-range contact modeling that are naturally suited to structure modeling. This formulation unifies six core capabilities: folding, inverse folding, sequence understanding, structure understanding, sequence design, and structure design, while retaining the native language capabilities of general-purpose LLMs. On established benchmarks, Trion-1 achieves broadly competitive performance. In folding, Trion-1 achieves a TM-score above 0.8 with a single sample under strict homology control, matching or exceeding the existing tri-modal model using 200 samples. For sequence understanding, Trion-1 performs on par with top-tier LLMs such as Claude Opus 4.7 (62.37%). These results support well-aligned sequence-structure-language representations and strong cross-modal generative capabilities.

Figures & tables

	Stage 1	Stage 2	Stage 3
Training Paradigm	AR-CPT	A2D-CPT	SFT
Context Length	4096	4096	4096
Block Size	1	$8\rightarrow 16\rightarrow 32$	32 (bdlm) / 4096 (mdlm)
Assistant Mask	$\times$	$\times$	$\checkmark$
Document Mask	$\times$	$\times$	$\checkmark$
Task Type	T + S + ST	T + S + ST	X2X

Table 1: Training configuration. T/S/B: text/sequence/backbone; ST: sequence–text pairs; X2X: all six cross-modal directions. Lengths are in tokens.

Frontier LLMs
Method	Params	GeneralQA					StructureMCQA
		Overall	F&F	I&B	L&M	Other	Choice	Semantic
		(481)	(150)	(152)	(153)	(26)	(3,918)	(3,918)
Claude Opus 4.7	–	62.37	61.33	62.50	66.67	42.31	–	–
Claude Opus 4.6	–	48.02	32.67	48.68	58.17	73.08	–	–
Gemini 3 Flash Thinking	–	49.48	42.00	46.05	59.48	53.85	–	–

Table 3: Sequence and structure understanding. Accuracy (%,

\uparrow

). F&F: Function & Family; I&B: Interaction & Binding; L&M: Location & Modification; Other: physicochemical/structural properties. Choice: option accuracy; Semantic: final-answer semantic correctness. Yellow: our models. Gray: per-metric group best (ties included); –: unavailable.

\dagger

: BioMatrix uses sequence+structure input and nearest-option mapping for Choice; its Semantic score uses a different judge prompt.

Frontier LLMs
Method	Params	SR $\uparrow$	Repetition		Foldability	Function	Novelty
Method	Params	SR $\uparrow$	Rep2 $\downarrow$	Rep5 $\downarrow$	pLDDT $\uparrow$	IPRRecovery $\uparrow$	Hard (Seq) $\uparrow$
Claude Opus 4.7	-	84.14	44.65	0.65	71.33	65.95	36.54
Claude Opus 4.6	-	90.57	48.44	1.86	42.17	25.32	45.91
Gemini 3.1 Pro Thinking	-	100.00	44.21	7.17	48.67	25.92	45.59
GPT-5.5	-	99.89	53.19	3.86	43.95	25.21	44.21

Table 4: Text-conditioned protein design. All metrics use a 0–100 scale. SR covers all 870 prompts; other metrics average over evaluated sequences. Yellow: our models. Gray: per-metric group best (ties included); –: unavailable; Params: total model size. ‡ : likelihood-ranked best-of-10; † : pLDDT on 819 sequences.

Figure 3: Text-conditioned PrcB backbone designs. The reference at left is enlarged; the four designs share the same scale. Pink: helices; blue: sheets; lavender: loops; gray: aligned AlphaFold reference. scTMbb measures ProteinMPNN–ESMFold self-consistency against each design backbone.

Panel of Figure 4 — Figure 4: Geometric organization of the shared token space. (a) Embedding similarity decreases monotonically with Hamming distance in the quantized n-bit structure-code space, as described in 3.1 . (b) UMAP visualization shows that structure tokens form an organized manifold while sequence tokens occupy the same global embedding space; color intensity indicates the Hamming weight of each structure token, i.e., the number of 1s in its n-bit code.

Figure 5: Sequence–structure attention on 6WEB. Head-averaged maps for DPLM2, Trion-1 (MDLM), and Trion-1 (BDLM). Red boxes mark structure queries attending to sequence keys. Each layer is selected by maximal conditional attention mass within $\pm 2$ residues of the corresponding sequence position.

Symbol	Description
$\mathcal{V},\ V$	Token vocabulary and its size (excluding the mask)
$x_{0},\ x_{t}$	Clean and corrupted sequences of length $L$
$x_{0}^{i},\ x_{t}^{i}$	Corresponding one-hot tokens at position $i$
$\mathbf{M}$	Absorbing mask token, $e_{V+1}$
$s,\ t$	Diffusion times, $0\leq s<t\leq 1$
$\alpha_{t}$	Token-retention probability, $\alpha_{t}=1-t$

Table 5: Summary of notation.

Task	Source	Prompt	CoT	Answer
S2B	DPLM2; UC40	Generate a plausible backbone structure for the sequence below: { sequence }	$\times$	{ backbone }
B2S	DPLM2; UC40	{ backbone } Use the protein structure above to construct a plausible sequence.	$\times$	{ sequence }
S2T	Swiss-Prot	Examine the protein sequence and indicate which targeting signals (e.g., signal peptide, mitochondrial targeting peptide, nuclear localization signal) are present. The protein is { sequence }	$\checkmark$	The protein contains an N-terminal signal peptide; no mitochondrial targeting sequence or nuclear localization signal is predicted.
T2S	Swiss-Prot	Formulate an original protein sequence featuring the Photosystem II PsbT protein family. Design a complete, biologically relevant protein that exhibits the core features and actions of this protein family. Embrace the normal structural conformations, molecular dimensions, and biological activities of the Photosystem II PsbT (protein family).	$\checkmark$	The { motif } is a single N-terminal hydrophobic helix followed by a short basic C-terminal tail; it could be constructed into the { sequence }.
B2T	OPI-Struc	{ backbone } Could you share a detailed analysis of the function attributed to the protein?	$\times$	Involved in oxygen transport from the lung to the various peripheral tissues.
T2B	Prot2Text	Generate a protein backbone of approximately 354 residues for a Tryptophanyl-tRNA synthetase with the following function: Catalyzes the attachment of tryptophan to tRNA(Trp).	$\times$	{ backbone }

Table 6: SFT examples and data sources. Italicized placeholders replace biological token spans; { motif } denotes a segment within the generated sequence. Modality and motif special tokens are omitted for clarity.

Figure 6: SFT corpus by task and source ( database.collection ). Labels show actual token counts and shares (B: billion; M: million). Small slices are enlarged to at least

5^{\circ}

for readability, so areas are not proportional to token counts.

Protein Language Models

Protein language models learn patterns in amino-acid sequences to predict protein properties, interpret mutations, and design new proteins.

See all 69 papers

AMix-2: Establishing Protein as a Native Modality in Large Language Models
May 29, 2026Keyue Qiu, Yixin Wu, Lihao Wang +19Diffusion Language Models Protein Language Models
1Shanghai Artificial Intelligence Laboratory · Institute for AI Industry Research (AIR), Tsinghua University
Protein Language Models Diverge from Natural Language: Comparative Analysis and Improved Inference
Feb 24, 2026Anna Hart, Chi Han, Jeonghwan Kim +2Protein Language Models Protein Representation Learning
Siebel School of Data Science and Computing, University of Illinois Urbana-Champaign, 201 N Goodwin Ave, Urbana, IL 61801, USA · DOE Center for Advanced Bioenergy and Bioproducts Innovation, University of Illinois Urbana-Champaign, 1206 W Gregory Drive, Urbana, IL 61801, USA · Chemical and Biomolecular Engineering, University of Illinois Urbana-Champaign, 600 S Matthews Ave, Urbana, IL 61801, USA
Learning Latent Protein Languages for Autoregressive Generation
Oct 2, 2026Mahdi Pourmirzaei, Farzaneh Esmaili, Amir Ziashahabi +2Protein Structure Generation Protein Language Models
University of Missouri · University of Southern California · Independent Researcher

May 29, 2026J/K move · Enter open · S save

AMix-2: Establishing Protein as a Native Modality in Large Language Models

Keyue Qiu, Yixin Wu, Lihao Wang, Yawen Ouyang+18

1Shanghai Artificial Intelligence Laboratory · Institute for AI Industry Research (AIR), Tsinghua University

Diffusion Language Models Protein Language Models Protein Design Protein Representation Learning Multimodal Large Language Models Molecular Foundation Models+1

Open paper AskPDF arXiv

We present AMix-2, a protein-text foundation model that establishes protein as a native modality in large language models (LLMs), unifying protein understanding and sequence design within a single foundation model. AMix-2 is built upon two key ideas: (1) a unified protein-text formulation that embeds natural language and protein sequence in a shared token space, enabling one model to perform biological reasoning and conditional design instead of separate downstream task-specialized models; and (2) a block-wise diffusion language modeling backbone that combines causal generation across blocks with bidirectional context and iterative refinement within blocks. This scheme better matches the intrinsic nature of proteins than a strict left-to-right factorization. To evaluate protein foundation models under realistic generalization settings, we further introduce ProteinArena, a comprehensive benchmark with time-aware and homology-aware protocols across various understanding and design tasks, and with baselines covering classical bioinformatics tools, protein-specialized models and LLMs. On ProteinArena, AMix-2 outperforms frontier LLMs and demonstrates competitive performance to task-specific protein models. Controlled experiments further show that the diffusion-based paradigm generally surpasses its autoregressive counterpart, highlighting the advantage of flexible generation order for protein sequences. We release both AMix-2 and ProteinArena to facilitate open research in protein foundation models.

Abstract

Protein Language Models

Explore similar work

AMix-2: Establishing Protein as a Native Modality in Large Language Models

Protein Language Models Diverge from Natural Language: Comparative Analysis and Improved Inference

Learning Latent Protein Languages for Autoregressive Generation