RIDER: 3D RNA Inverse Design with Reinforcement Learning-Guided Diffusion

Tianmeng Hu; Yongzheng Cui; Biao Luo; Ke Li

cs.LGFeb 18, 2026

RIDER: 3D RNA Inverse Design with Reinforcement Learning-Guided Diffusion

Authors: Tianmeng Hu, Yongzheng Cui, Biao Luo, Ke Li

Organizations: University of Exeter · Central South University

Abstract

The inverse design of RNA three-dimensional (3D) structures is crucial for engineering functional RNAs in synthetic biology and therapeutics. While recent deep learning approaches have advanced this field, they are typically optimized and evaluated using native sequence recovery, which is a limited surrogate for structural fidelity, since different sequences can fold into similar 3D structures and high recovery does not necessarily indicate correct folding. To address this limitation, we propose RIDER, an RNA Inverse DEsign framework with Reinforcement learning that directly optimizes for 3D structural similarity. First, we develop and pre-train a GNN-based generative diffusion model conditioned on the target 3D structure, achieving a 9% improvement in native sequence recovery over state-of-the-art methods. Then, we fine-tune the model with an improved policy gradient algorithm using four task-specific reward functions based on 3D self-consistency metrics. Experimental results show that RIDER improves structural similarity by over 100% across all metrics and discovers designs that are distinct from native sequences.

Figures & tables

Figure 1: Visualization of sequences (a), (b), and (c), and their corresponding 3D structures ( $\alpha$ ), ( $\beta$ ), and ( $\gamma$ ) predicted by RhoFold [ 49 ] . Although sequences (a) and (b) differ by only 3 nucleotides, and (b) and (c) by 5 nucleotides, their folded structures exhibit clear differences.

Figure 2: Overview of the RIDER framework. RNA tertiary structures are processed by a GVP-GNN encoder to produce structural embeddings. These embeddings condition the diffusion model for sequence generation, which is further optimized by RL to maximize structural similarity.

Figure 3: Results of supervised learning pre-training. A . Comparison of native sequence recovery on the test set. RIDE is compared against RiboDiffusion and gRNAde. The best NSR among $16$ designs per target (sampled at temperature $0.1$ ) is reported. B . Relationship between NSR and structural similarity (GDT_TS and RMSD) for RIDE designs. Color denotes RMSD. C . Correlation among GDT_TS, TM-score, and RMSD for the designed structures.

Figure 4: Results of reinforcement learning fine-tuning. A . GDT_TS comparison on $14$ RNA structures of interest [ 9 ] for gRNAde, RIDE (pre-trained), and RIDER (fine-tuned with $R^{\texttt{gdt\_rmsd}}$ ). B . Comparison of native sequence recovery before ( RIDE ) and after ( RIDER ) RL fine-tuning. Color indicates GDT_TS after RL fine-tuning. The results for the other two metrics are provided in Appendix D.4 .

Panel of Figure 5 — Figure 5: Visualization of designed examples. Structures folded from sequences generated by RIDER are shown in color, while the target structures are shown in semi-transparent yellow.

Method	Reward function	3D self-consistency metrics			NSR
Method	Reward function	GDT_TS $\uparrow$	RMSD $\downarrow$	TM-score $\uparrow$	NSR
RIDER	$R^{\texttt{tm}}$	$\mathbf{0.62\pm 0.04(72\%)}$	$4.31\pm 0.70(31\%)$	$\mathbf{0.61\pm 0.03(72\%)}$	$0.45\pm 0.04$
	$R^{\texttt{gdt}}$	$0.58\pm 0.05(63\%)$	$4.60\pm 0.55(30\%)$	$0.56\pm 0.05(64\%)$	$0.47\pm 0.06$
	$R^{\texttt{rmsd}}$	$0.56\pm 0.03(60\%)$	$4.09\pm 0.40(31\%)$	$0.55\pm 0.06(64\%)$	$0.49\pm 0.05$
	$R^{\texttt{gdt\_rmsd}}$	$\mathbf{0.62\pm 0.02(72\%)}$	$\mathbf{3.35\pm 0.44(33\%)}$	$0.56\pm 0.03(68\%)$	$0.47\pm 0.05$
RIDE	–	$0.33\pm 0.03(31\%)$	$10.36\pm 0.19(8\%)$	$0.33\pm 0.03(36\%)$	$\mathbf{0.61\pm 0.02}$

Table 1: Performance comparison of different methods and reward functions on the test set. Results are reported as the mean and standard deviation over 5 independent runs. Percentages in parentheses indicate the proportion of designs that meet the structural designability thresholds (GDT_TS

\geq 0.5

, TM-score

\geq 0.45

, RMSD

\leq 2

Å).

Model	Reward function	GDT_TS $\uparrow$	RMSD $\downarrow$	TM-score $\uparrow$
RIDER (AlphaFold3)	$R^{\texttt{tm}}$	$0.56\pm 0.07(64\%)$	$4.69\pm 0.79(30\%)$	$0.56\pm 0.06(65\%)$
RIDER (AlphaFold3)	$R^{\texttt{gdt\_rmsd}}$	$\mathbf{0.57\pm 0.06(65\%)}$	$\mathbf{3.80\pm 0.61(32\%)}$	$\mathbf{0.56\pm 0.05(67\%)}$
gRNAde (AlphaFold3)	–	$0.26\pm 0.04(26\%)$	$9.96\pm 0.77(3\%)$	$0.28\pm 0.04(26\%)$

Table 2: Cross-predictor validation using the independent AlphaFold3 oracle.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: The proposed framework consists of two stages: RIDE (pre-training via supervised learning) and RIDER (fine-tuning via reinforcement learning). A multi-layer GVP-GNN encoder-decoder backbone is used to iteratively denoise RNA sequences conditioned on the target 3D structure. Structural fidelity is evaluated using a separate RNA structure predictor and fed back through a reward function.

Hyperparameter	Value
Number of GVP-GNN layers ( $L_{E}$ , $L_{D}$ )	5
Node input dimensions (scalar, vector)	(15, 4)
Node hidden dimensions (scalar, vector)	(256, 24)
Edge input dimensions (scalar, vector)	(131, 3)
Edge hidden dimensions (scalar, vector)	(128, 4)
Dropout rate	0.5

Appendix

Table 3: RIDE architecture and associated hyperparameters.

Hyperparameter	Value
SDE schedule	Linear
Initial noise scale $\beta_{0}$	0.1
Final noise scale $\beta_{1}$	20.0
Total diffusion time $T$	1.0
Minimum time step $\epsilon_{\texttt{time}}$	0.001
Number of epochs	150

Appendix

Table 4: Training hyperparameters for RIDE pre-training.

Hyperparameter	Value
Learning rate	$5\times 10^{-5}$
Optimizer	Adam
Number of RL training epochs	80
Policy updates per epoch	2
Gradient accumulation steps	60
Clip range $\epsilon_{\texttt{clip}}$	0.5

Appendix

Table 5: Training hyperparameters for RIDER fine-tuning.

Component	Scaling Factor
GDT_TS weight ( $w_{\texttt{gdt\_scale}}$ )	5
TM-score weight ( $w_{\texttt{tm\_scale}}$ )	5
RMSD weight ( $w_{\texttt{rmsd\_scale}}$ )	0.5
GDT_TS weight for bonus reward ( $w_{\texttt{bonus\_gdt}}$ )	100
RMSD weight for bonus reward ( $w_{\texttt{bonus\_rmsd}}$ )	20

Appendix

Table 6: Scaling factors for reward function components.

Figure 7: Comparison of RIDER , RIDER-RWD , and RIDER-ADV on the GDT_TS metric across the test set. Bar heights represent the mean, and red error bars indicate the Standard Error of the Mean (SEM).

Figure 8: Comparison of RIDER , RIDER-RWD , and RIDER-ADV on the RMSD metric across the test set. Bar heights represent the mean, and red error bars indicate the SEM.

Figure 9: Comparison of RIDER , RIDER-RWD , and RIDER-ADV on the TM-score metric across the test set. Bar heights represent the mean, and red error bars indicate the SEM.

Figure 10: GDT_TS scores of RIDER , RIDER-RWD , and RIDER-ADV on 14 target RNA structures.

Figure 11: RMSD scores of RIDER , RIDER-RWD , and RIDER-ADV on 14 target RNA structures. Lower values indicate better structural alignment.

Figure 12: TM-score values of RIDER , RIDER-RWD , and RIDER-ADV on 14 target RNA structures. Higher values indicate better structural similarity.

Figure 13: Comparison of four reward functions (

R^{\texttt{tm}}

R^{\texttt{rmsd}}

R^{\texttt{gdt}}

R^{\texttt{gdt\_rmsd}}

) on the GDT_TS metric across the test set. Bar heights represent the mean, and red error bars indicate the SEM.

Figure 14: Comparison of four reward functions on the RMSD metric across the test set. Bar heights represent the mean, and red error bars indicate the SEM.

Figure 15: Comparison of four reward functions on the TM-score metric across the test set. Bar heights represent the mean, and red error bars indicate the SEM.

Optimizer	Activation function	Initial LR	GVP-GNN layers ( $L_{E},L_{D}$ )	Native sequence recovery
Adam	SiLU	$3\times 10^{-4}$	5	$\mathbf{0.61\pm 0.02}$
		$3\times 10^{-4}$	8	$0.58\pm 0.04$
		$5\times 10^{-4}$	5	$0.60\pm 0.02$
		$5\times 10^{-4}$	8	$0.52\pm 0.08$
	ReLU	$3\times 10^{-4}$	5	$0.58\pm 0.04$
		$3\times 10^{-4}$	8	$0.59\pm 0.03$

Appendix

Table 7: Ablation study of RIDE hyperparameters on native sequence recovery. Results are reported as the mean and standard deviation over five independent runs.

Figure 16: RMSD comparison of gRNAde, RIDE , and RIDER on 14 RNA structures of interest. Lower RMSD indicates better structural alignment.

Figure 17: TM-score comparison of gRNAde, RIDE , and RIDER on 14 RNA structures of interest. Higher TM-score reflects better global fold similarity.

Explore similar work

MORAD: Reinforcement Learning with Multi-Objective Rewards for RNA Inverse Design
Oct 9, 2026Jixin Tang, Ji Guo, Jun ZhaoInverse Design Reward Design for RL
Fudan University
La-Ribo: RNA Co-Design via Geometry-Latent Flow Matching
Oct 8, 2026Runze Ma, Will Hua, Shuangjia ZhengInverse Design Flow-Based Generative Modeling
Shanghai Jiao Tong University · Independent Researcher
RNA Design via Conditioned Flow Matching and Finite-Policy Reinforcement Learning
Sep 29, 2026Zefeng Lin, Xianyong Fang, Tianfan Fu +1Inverse Design Molecular Generation
School of Computer Science and Technology, University of Science and Technology of China, Hefei, Anhui, China · School of Computer Science and Technology, Anhui University, Hefei, Anhui, China · State Key Laboratory for Novel Software Technology at Nanjing University, School of Computer Science, Nanjing University, Nanjing, Jiangsu, China

Oct 9, 2026J/K move · Enter open · S save

MORAD: Reinforcement Learning with Multi-Objective Rewards for RNA Inverse Design

Jixin Tang, Ji Guo, Jun Zhao

Fudan University

Inverse Design Reward Design for RL RL for Diffusion Models Multi-Objective Reinforcement Learning Conditional Diffusion Models

Open paper AskPDF arXiv

We introduce MORAD, an online reinforcement learning framework for backbone-conditioned RNA diffusion models that trains a single shared policy across diverse target backbones. Structure-conditioned RNA inverse design seeks nucleotide sequences compatible with a target three-dimensional backbone, yet existing reinforcement learning approaches either optimize a separate policy for each target backbone or rely on offline preference optimization, leaving online post-training of a reusable backbone-conditioned model largely unexplored. MORAD combines backbone-conditioned group sampling, group-relative reward normalization, and a forward-process diffusion objective. Beyond existing RL rewards that primarily target tertiary structural quality, MORAD jointly optimizes six measurements spanning tertiary geometry, base pairing, ensemble behavior, and sequence composition. To aggregate these heterogeneous objectives despite differences in scale and optimization direction, MORAD maps each measurement to a bounded desirability score and combines them with a weighted geometric mean. Compared with a tertiary-structure-only reward, this multi-objective formulation improves 10 of the 11 evaluation metrics that have a preferred direction. Applied to pretrained RIDE, MORAD improves GDT-TS by 16.0% (from 0.3300 to 0.3827) and pairing MCC by 19.4% (from 0.6110 to 0.7295), while reducing normalized ensemble defect by 27.5% (from 0.3906 to 0.2830). On the test set, MORAD outperforms RiboDiffusion, gRNAde, and RDesign across tertiary structural accuracy, secondary-structure pairing, and thermodynamic quality, establishing a new state of the art in structure-conditioned RNA inverse design.

Abstract

Explore similar work

MORAD: Reinforcement Learning with Multi-Objective Rewards for RNA Inverse Design

La-Ribo: RNA Co-Design via Geometry-Latent Flow Matching

RNA Design via Conditioned Flow Matching and Finite-Policy Reinforcement Learning