Generative modeling of intrinsically disordered protein regions by reinforcing sparse autoencoder features

Jason X. Liu; Sebastian Ibarraran; Frank Hu; Soojung Yang; Xinyu A. Feng; Abigail Park; Anagha Aneesh; Lacramioara Bintu

cs.LGOct 1, 2026

Generative modeling of intrinsically disordered protein regions by reinforcing sparse autoencoder features

Authors: Jason X. Liu, Sebastian Ibarraran, Frank Hu, Soojung Yang, Xinyu A. Feng, Abigail Park, Anagha Aneesh, Lacramioara Bintu, +2 more

Organizations: Stanford University

Abstract

Intrinsically disordered protein regions (IDRs) play central roles in cellular processes such as transcriptional regulation, signal transduction, and subcellular localization, yet their functional design remains challenging. Structure-based design methods do not readily apply to IDRs, and existing protein language models are trained on full-length protein sequences, thus learning a prior that is biased towards folded domains. Here, we present IDiom, an autoregressive protein language model trained on IDiom-DB, a dataset of 54 million predicted IDRs curated from the AlphaFold Database. IDiom generates diverse sequences that recapitulate the composition, patterning, motifs, and predicted disorder of natural IDRs. To control function-associated sequence patterns, we also introduce reinforcement learning with sparse autoencoder features (RL-SAE), a post-training method that rewards the generation of sequences that activate specified feature sets. Across eight IDR design tasks, RL-SAE sequences activate, on average, 90% of 30 targeted features, compared to 24% for activation steering. We demonstrate that RL-SAE improves the predicted subcellular localization and transcriptional activity of generated IDRs compared to steering and supervised fine-tuning, and enables features associated with distinct biological functions to be combined within individual sequences. Thus, IDiom and RL-SAE enable interpretable and composable IDR design through explicit control of function-associated sequence features. More broadly, RL-SAE could extend to other protein design settings where interpretable features provide useful design targets. Code is available at https://github.com/rotskoff-group/idiom.

Figures & tables

Figure 1: Training on 54M curated IDRs enables the generation of diverse and biologically plausible IDRs. (a) IDiom-DB curation workflow. (b) AlphaFold2 pLDDT distribution of curated IDRs and non-IDRs. (c) Fraction of curated training IDRs and experimental DisProt IDRs which are located at the N-terminus, C-terminus, and internally. (d) IDiom model perplexity on DisProt, validation, and generated IDRs. Solid and hatched bars indicate unprompted and prompted IDRs, respectively. (e) Amino acid compositional enrichment for training, generated, and DisProt IDRs, relative to the folded CATH sequences’ compositions. This panel’s legend also applies to panels (f) and (g). (f) Sequence identity of generated IDRs relative to training set IDRs, calculated with MMseqs2. (g) Predicted disorder scores of training, generated, and DisProt IDRs, alongside CATH sequences (higher is more disordered).

Figure 2: Enriched SAE features recover canonical IDR sequence grammars. Top row: Statistical significance

|Z|

versus log 2 odds ratio (LOR) for all 16,384 IDiomSAE features in nuclear pore complex, nuclear speckle, transcriptional activation, and repression sequence sets. Dashed lines mark LOR

=1

(vertical) and the false discovery rate threshold of

10^{-3}

(horizontal). Features in the upper right quadrant pass both thresholds. Bottom row: Information content logos for each set’s top-four enriched features, showing residue likelihoods within a

\pm 7

-residue window around peak firing positions (vertical dashed line position). Orange shading indicates the feature activation strength at each residue position. Logo construction is described in Appendix R .

					RL-SAE
Compartment	Base	Nat.	SFT	Steering ( $\alpha$ )	top-30	private-30
Nucleolus	$0.14_{.01}$	$\underline{0.48}_{.02}$	$0.30_{.01}$	$0.41_{.01}$ (0.35)	$\mathbf{0.99}_{.00}$	$0.94_{.07}$
Chromosome	$0.26_{.01}$	$\underline{0.52}_{.01}$	$0.37_{.00}$	$0.51_{.02}$ (0.21)	$\mathbf{0.71}_{.03}$	$0.67_{.08}$
Speckle	$0.00_{.00}$	$\underline{0.15}_{.01}$	$0.07_{.00}$	$0.11_{.01}$ (0.20)	$\mathbf{0.36}_{.15}$	$0.30_{.24}$
Stress gran.	$0.05_{.00}$	$0.15_{.01}$	$0.06_{.01}$	$\underline{0.16}_{.01}$ (0.15)	$0.32_{.09}$	$\mathbf{0.51}_{.36}$
P-body	$0.17_{.01}$	$\underline{0.27}_{.02}$	$0.20_{.01}$	$0.19_{.01}$ (0.00)	$0.05_{.02}$	$\mathbf{0.32}_{.10}$

Table 1: RL-SAE improves predicted association with target biomolecular compartments. ProtGPS-predicted biomolecular compartment association scores (0 to 1). Bold indicates the best score per row, underline indicates the best non-RL-SAE method. The private-30 column targets the top 30 private features of each compartment. Subscripted values are standard deviations over three seeds, except for natural IDRs (Nat.), which report the standard error of the mean (Appendix I ).

Figure 3: Targeting private SAE features reduces predicted cross-talk between compartments. (a) Number of private (greyed diagonal) and shared (off-diagonal) features among the top-30 between the 6 considered compartments. (b) Heatmap of ProtGPS scores for sequences generated after RL-SAE on the top-30 features of each compartment. (c) Heatmap of ProtGPS scores of sequences generated after RL-SAE on the private-30 features of each compartment.

	Readout	Base	Nat.	SFT	Steering ( $\alpha$ )	RL-SAE
Activation	PADDLE	$1.03_{.03}$	$5.34_{.18}$	$4.75_{.08}$	$\underline{6.81}_{.01}$ (0.43)	$\mathbf{11.16}_{.11}$
Repression	$\psi$ KxE	$0.07_{.02}$	$\underline{0.60}_{.05}$	$0.37_{.04}$	$0.56_{.02}$ (0.35)	$\mathbf{5.63}_{.60}$
	NDSM	$0.01_{.01}$	$0.10_{.02}$	$0.06_{.00}$	$\underline{0.21}_{.02}$ (0.35)	$\mathbf{1.80}_{.44}$

Table 2: RL-SAE generates sequence features associated with transcriptional regulation. Values are PADDLE scores and densities of SUMOylation motifs per sequence. Bold indicates the best score per row, underline the second best. Motif regexes are in Appendix Q . Subscripted values are standard deviations over three seeds, except for natural IDRs (Nat.), which report the standard error of the mean (Appendix I ).

	Activation	Repression		Chromosome / nuclear
RL-SAE target	PADDLE	$\psi$ KxE	NDSM	ProtGPS	DeepLoc	cNLS
chromosome $+$ AD	$11.00_{.10}$	$0.00_{.00}$	$0.00_{.00}$	$0.54_{.44}$	$0.76_{.14}$	$0.61_{.53}$
chromosome $+$ RD	$-0.09_{.05}$	$2.89_{.10}$	$1.73_{.77}$	$0.73_{.14}$	$0.92_{.02}$	$0.99_{.02}$

Table 3: RL-SAE combines localization and transcriptional effector features. Evaluation of the two composed designs (chromosome

+

AD/RD), each targeting nuclear localization together with one gene-regulatory function. Values are PADDLE scores, SUMOylation motif density, predicted localization, and nuclear localization signal presence (cNLS). Grey cells mark the values which are biologically expected to score highly. Subscripted values are standard deviations over three seeds (Appendix I ).

Figure 4: Feature compatibility determines the organization of sequences with composed features. (a) Fraction of features activated in each target feature set by sequences generated under each RL-SAE condition. The chromosome feature set is private-30. (b) Distribution of the per-sequence Jaccard overlap between residues which fire on chromosome association features versus on activation domain or repression domain features. Curves show each design and a position-shuffled null. (c) Distribution of charge patterning $\kappa$ parameter for chromosome $+$ AD/RD, and scrambled variants. (d) Example generated chromosome $+$ AD/RD sequences showing feature firing positions for chromosome association (light blue) and the activation domain (AD, upper, red) or repression domain (RD, lower, dark blue).

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Characterization of IDiom-DB. (a) Length distribution of full-length protein sequences within which the IDRs are located, of the final curated dataset. (b) Length distribution of IDRs within the final curated dataset. (c) Distribution of the fraction of residues of the full-length sequence which are IDR residues. (d) Distribution of sequence entropy for IDR versus non-IDR regions in the final curated dataset. The vertical red dashed line indicates the sequence entropy threshold at the lower 5th percentile.

Figure 6: Characterization of training IDRs, IDiom-generated IDRs, DisProt IDRs, and CATH sequences. (a) Distribution of training, generated, and DisProt IDR lengths. The legend in this panel applies to all other panels in this figure. (b) Distribution of the fraction of charged residues (FCR) for training, generated, and DisProt IDRs, and CATH sequences. (c) Distribution of charge patterning

\kappa

parameter. (d) Distribution of sequence hydropathy decoration (SHD). (e) Distribution of sequence entropy. (f) Distribution of AlphaFold2 scores for CATH sequences, IDiom-generated IDRs, unprompted (upper) and prompted (lower), as well as DisProt IDRs without surrounding context (upper) and with context (lower).

Figure 7: Sparse autoencoder layer sweep. Fraction of variance unexplained (FVU) per layer, evaluated on 10K held-out validation records (

k=32

, expansion

\times 16

). The layer-18 is marked by the vertical dashed line.

Figure 8: ProtGPS dose-response against top-

N

. Left: Average ProtGPS score for sequences generated after RL-SAE on the top-

N

features per compartment. Right: Average ProtGPS score for sequences generated after RL-SAE on the top-

N

features which are private to the given compartment. Stress granule and P-body only have 36 and 55 private features, respectively.

Figure 9: Transcriptional effector domain dose-response against top-

N

. Left: PADDLE score for activation domains generated after RL-SAE on the top-

N

activation domain features. Right:

\psi

KxE and NDSM motif density in sequences generated after RL-SAE on the top-

N

repression domain features.

Figure 10: Feature activation fraction versus top-

N

. For all 8 sequence sets (2 transcriptional effector sequences and 6 compartment association sequences) the average fraction of target features which fire within generated sequences,

R_{S}

, as a function of top-

N

, for RL-SAE (solid colored lines) and activation steering (grey dashed lines).

R_{S}

for steering is presented at the best

\alpha

value across the sweep, regardless of whether that

\alpha

passes the sequence entropy threshold (i.e. steering

R_{S}

values presented here are a generous upper bound).

Figure 11: Steering dose-response at

N\!=\!30

. Here, we plot the various scoring metrics (PADDLE,

\psi

KxE, NDSM, and ProtGPS) versus activation steering strength

\alpha

at top-

N=30

. The metrics are plotted as colored curves (left axis), and the generated sequence entropies are plotted as grey curves (right axis).

	Target feature coverage $R_{S}$			On-target ProtGPS
Target	Base	random-30	top-30	Base	random-30	top-30
Nucleolus	$0.03_{.00}$	$0.59_{.04}$	$0.99_{.00}$	$0.14_{.01}$	$0.17_{.28}$	$0.99_{.00}$
Chromosome	$0.04_{.00}$	$0.55_{.06}$	$0.99_{.00}$	$0.26_{.01}$	$0.22_{.26}$	$0.71_{.03}$
Speckle	$0.01_{.00}$	$0.58_{.07}$	$0.93_{.00}$	$0.00_{.00}$	$0.00_{.00}$	$0.36_{.15}$
Stress gran.	$0.01_{.00}$	$0.59_{.05}$	$0.95_{.00}$	$0.05_{.00}$	$0.05_{.06}$	$0.32_{.09}$
P-body	$0.02_{.00}$	$0.61_{.09}$	$0.69_{.04}$	$0.17_{.01}$	$0.27_{.40}$	$0.05_{.02}$

Appendix

Table 4: RL-SAE on randomly chosen target features (random-30) compared to the base model and to RL-SAE on the enriched top-30. Subscripted values are standard deviations over three seeds (Appendix I ).

Compartment	RL-GPS ProtGPS
Nucleolus	$1.00_{.00}$
Chromosome	$0.99_{.00}$
Speckle	$0.98_{.00}$
Stress gran.	$0.99_{.00}$
P-body	$1.00_{.00}$
NPC	$0.99_{.00}$

Appendix

Table 5: ProtGPS-predicted compartment association scores (0 to 1) for the oracle RL baseline (RL-GPS). Subscripted values are standard deviations over three training seeds (Appendix I ).

	RL-SAE top-30		RL-SAE private
Compartment	on-target	scramble	on-target	scramble
Nucleolus	$0.99_{.00}$	$0.50_{.03}$	$0.94_{.07}$	$0.39_{.02}$
Chromosome	$0.71_{.03}$	$0.38_{.04}$	$0.67_{.08}$	$0.44_{.02}$
Speckle	$0.36_{.15}$	$0.11_{.02}$	$0.30_{.24}$	$0.14_{.05}$
Stress gran.	$0.32_{.09}$	$0.68_{.04}$	$0.51_{.36}$	$0.56_{.07}$
P-body	$0.05_{.02}$	$0.33_{.03}$	$0.32_{.10}$	$0.17_{.01}$

Appendix

Table 6: ProtGPS scores of RL-SAE localization sequences and their composition-matched scrambles. Subscripted values are standard deviations over three training seeds (Appendix I ).

Readout	RL-SAE	scramble
PADDLE	$11.16_{.11}$	$8.73_{.10}$
$\psi$ KxE	$5.63_{.60}$	$0.49_{.08}$
NDSM	$1.80_{.44}$	$0.12_{.02}$

Appendix

Table 7: Transcriptional effector readouts of RL-SAE sequences and their composition-matched scrambles. Subscripted values are standard deviations over three training seeds (Appendix I ).

Target	Nat.	SFT	Steering ( $\alpha$ )	RL-SAE	RL-GPS
	Metapredict V3
Nucleolus	$0.85_{.00}$	$0.81_{.01}$	$0.87_{.00}$ (0.35)	$0.93_{.00}$	$0.95_{.00}$
Chromosome	$0.86_{.00}$	$0.81_{.01}$	$0.88_{.00}$ (0.21)	$0.92_{.01}$	$0.91_{.00}$
Speckle	$0.83_{.01}$	$0.82_{.00}$	$0.89_{.00}$ (0.20)	$0.84_{.07}$	$0.94_{.00}$
Stress gran.	$0.86_{.00}$	$0.80_{.01}$	$0.86_{.01}$ (0.15)	$0.91_{.01}$	$0.93_{.01}$
P-body	$0.82_{.01}$	$0.80_{.00}$	$0.64_{.01}$ (0.00)	$0.92_{.01}$	$0.91_{.01}$

Appendix

Table 8: Additional disorder predictions for generated and natural sequences, using Metapredict V3 and IUPred3. Values range from 0 to 1, and higher values indicate a prediction of greater disorder. Subscripted values are standard deviations over three seeds, except for natural IDRs (Nat.), which report the standard error of the mean (Appendix I ).

					RL-SAE
Compartment	Base	Nat.	SFT	Steering ( $\alpha$ )	top-30	private-30	RL-GPS
	$P(\text{Nucleus})$
Nucleolus	$0.43_{.01}$	$0.66_{.01}$	$0.59_{.01}$	$0.47_{.01}$ (0.35)	$\mathbf{0.90}_{.01}$	$\underline{0.87}_{.02}$	$0.70_{.03}$
Chromosome	$0.45_{.01}$	$0.67_{.01}$	$0.58_{.01}$	$0.57_{.01}$ (0.21)	$\mathbf{0.91}_{.00}$	$\underline{0.85}_{.01}$	$0.75_{.02}$
Speckle	$0.44_{.02}$	$0.67_{.02}$	$0.56_{.01}$	$0.63_{.01}$ (0.20)	$0.80_{.04}$	$\underline{0.82}_{.05}$	$\mathbf{0.84}_{.03}$
Stress gran.	$0.43_{.02}$	$0.61_{.01}$	$0.53_{.00}$	$0.54_{.01}$ (0.15)	$0.78_{.02}$	$0.56_{.03}$	$0.61_{.16}$

Appendix

Table 9: Orthogonal DeepLoc localization predictions. Bold indicates the best score and underline the second best within each grey shaded row. The private-30 column targets the top 30 private features of a compartment. RL-GPS is included for comparison here because DeepLoc is an orthogonal predictor, unlike in the main text Table 1 . The grey shaded rows indicate the biologically expected localization. Subscripted values are standard deviations over three seeds, except for natural IDRs (Nat.), which report the standard error of the mean (Appendix I ).

Compartment	Motif	Base	Nat.	SFT	Steering ( $\alpha$ )	top-30	private-30	RL-GPS
						RL-SAE
Nucleolus	cNLS	$0.13_{.02}$	$0.54_{.02}$	$0.37_{.00}$	$0.38_{.01}$ (0.35)	$\mathbf{1.00}_{.00}$	$\underline{1.00}_{.00}$	$1.00_{.00}$
Chromosome	cNLS	$0.13_{.02}$	$0.43_{.02}$	$0.27_{.00}$	$0.82_{.02}$ (0.21)	$\mathbf{1.00}_{.00}$	$\underline{0.96}_{.03}$	$0.64_{.14}$
Speckle	RS repeat	$0.00_{.00}$	$0.08_{.01}$	$0.02_{.00}$	$0.21_{.02}$ (0.20)	$\underline{0.97}_{.04}$	$\mathbf{1.00}_{.01}$	$0.27_{.26}$
Stress gran.	RG/RGG	$0.04_{.01}$	$0.15_{.01}$	$0.09_{.00}$	$0.45_{.01}$ (0.15)	$\mathbf{1.00}_{.00}$	$\underline{0.93}_{.04}$	$0.66_{.52}$
P-body	RG/RGG	$0.04_{.01}$	$0.13_{.02}$	$0.10_{.01}$	$0.04_{.00}$ (0.00)	$\underline{0.34}_{.56}$	$0.00_{.00}$	$\mathbf{0.51}_{.02}$

Appendix

Table 10: Fraction of sequences containing the specified biological motif (density per 100 aa for GLFG). Bold indicates the best score per row, underline the second best. The private-30 column targets the compartment’s private top-30 features. RL-GPS is included for comparison here because the motifs are orthogonal to ProtGPS. Motif regexes are in Appendix Q . Subscripted values are standard deviations over three seeds, except for natural IDRs (Nat.), which report the standard error of the mean (Appendix I ).

Motif	Regex / rule	Compartment
RS/SR repeat	(?:RS\|SR){3,}	Nuclear speckle
RG/RGG cluster	$\geq\!2$ of { RGG , RG not in an RGG } within 30 residues	Stress granule, P-body
GLFG repeat	(?=(GLFG)) (overlapping matches)	Nuclear pore complex
$\psi$ KxE	[VILMF]K.E	Repression domain
NDSM	[VILMF]K.E[DE]+	Repression domain

Appendix

Table 11: Regular expressions used in Tables 10 and 2 .

ELM class	Regex
TRG_NLS_MonoCore_2	ˆ DE[KRP][KR][ ˆ DE]
TRG_NLS_MonoExtN_4	(([PKR].{0,1}[ ˆ DE])\|([PKR]))((K[RK])\|(RK))(([ ˆ DE][KR])\|([KR][ ˆ DE]))[ ˆ DE]
TRG_NLS_MonoExtC_3	ˆ DE(([ ˆ DE][KR])\|([KR][ ˆ DE]))(([PKR])\|([ ˆ DE][DE]))
TRG_NLS_Bipartite_1	[KR][KR].{7,15} ˆ DE(([ ˆ DE][KR])\|([KR][ ˆ DE]))[ ˆ DE]

Appendix

Table 12: Eukaryotic Linear Motif classical-NLS (cNLS) regular expressions used in Table 10 .

Figure 12: Feature enrichment plots for all eight sequence sets. Feature enrichment plots (hypergeometric $|Z|$ versus log 2 odds ratio (LOR)) for all 8 sequence sets. The top-30 features as ranked by LOR are the colored dots. The horizontal dashed line marks the FDR threshold, and the vertical dashed line marks the LOR $\geq$ 1 threshold.

Figure 13: Nucleolus. Logo plots of the top-30 enriched features.

Figure 14: Chromosome. Logo plots of the top-30 enriched features.

Figure 15: Nuclear speckle. Logo plots of the top-30 enriched features.

Figure 16: Stress granule. Logo plots of the top-30 enriched features.

Figure 17: P-body. Logo plots of the top-30 enriched features.

Figure 18: Nuclear pore complex. Logo plots of the top-30 enriched features.

Figure 19: Activation domains. Logo plots of the top-30 enriched features.

Figure 20: Repression domains. Logo plots of the top-30 enriched features.

Explore similar work

Proteo-R1: Reasoning Foundation Models for De Novo Protein Design
May 1, 2026Fang Wu, Weihao Xuan, Heli Qi +26Protein Design Moleculenet
Stanford University
ProteinZero: Self-Improving Protein Generation via Online Reinforcement Learning
Jun 9, 2025Ziwen Wang, Jiajun Fan, Ruihan Guo +3Protein Design Protein
Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign, Urbana, IL, USA · DOE Center for Advanced Bioenergy and Bioproducts Innovation, University of Illinois Urbana-Champaign, Urbana, IL, USA
VFUSE: Virulent Feature Understanding with Sparse autoEncoders
Jun 8, 2026Michael Yu, Matthew L. OlsonProtein Design Sparse Autoencoder Features
Raft Bioworks · Independent

May 1, 2026, cs.LGJ/K move · Enter open · S save

Proteo-R1: Reasoning Foundation Models for De Novo Protein Design

Fang Wu, Weihao Xuan, Heli Qi, Hanqun Cao+25

Stanford University

Protein Design Moleculenet Generative Models Wireless Foundation Models

Open paper AskPDF arXiv

Deep learning in \emph{de novo} protein design has achieved atomic-level fidelity. However, existing models remain largely non-deliberative: they directly synthesize molecular geometries without explicitly reasoning about which residues or interactions are functionally essential. As a result, design decisions are entangled with continuous sampling dynamics, limiting interpretability, controllability, and systematic reuse of biochemical knowledge. We introduce \textbf{Proteo-R1}, a reasoning-guided protein design framework that explicitly decouples \emph{molecular understanding} from \emph{geometric generation}. Proteo-R1 adopts a dual-expert architecture in which a multimodal large language model (MLLM) serves as an \emph{understanding expert}, analyzing protein sequences, structures, and textual context to identify key functional residues that govern binding and specificity. These residue-level decisions are then passed as hard constraints to a separate diffusion-based \emph{generation expert}, which performs conditional co-design while respecting the fixed interaction anchors. This factorization mirrors how human experts approach molecular engineering: first, reasoning about critical interactions, then optimizing geometry subject to those constraints. By operationalizing reasoning as explicit residue-level commitments rather than latent textual guidance, Proteo-R1 achieves stable, interpretable, and modular integration of LLM reasoning with state-of-the-art geometric generative models. Code, data, and demos are available at https://smiles724.github.io/r1/.

Abstract

Explore similar work

Proteo-R1: Reasoning Foundation Models for De Novo Protein Design

ProteinZero: Self-Improving Protein Generation via Online Reinforcement Learning

VFUSE: Virulent Feature Understanding with Sparse autoEncoders