CocoRNA: Collective RNA Design with Cooperative Multi-agent Reinforcement Learning

Tianmeng Hu; Biao Luo; Ke Li

cs.LGOct 16, 2026

CocoRNA: Collective RNA Design with Cooperative Multi-agent Reinforcement Learning

Authors: Tianmeng Hu, Biao Luo, Ke Li

Organizations: Living Systems Institute, University of Exeter, United Kingdom · Department of Computer Science, University of Exeter, United Kingdom · School of Automation, Central South University, China

Abstract

Designing RNA sequences that reliably fold into specific secondary structures is essential for understanding their biological functions but remains a challenging computational problem. We propose CocoRNA, a cooperative multi-agent reinforcement learning framework for RNA inverse design. CocoRNA simplifies the design task by decomposing it into smaller sub-problems, each solved collaboratively by multiple agents. This approach reduces the complexity of the problem and improves the exploration of design policies. During training, a centralized critic uses global structural information to guide the agents, enabling them to jointly optimize their design strategies. As a result, CocoRNA learns high-quality RNA design policies that generalize effectively to unseen structures without additional training. Experiments on the Rfam dataset demonstrate that CocoRNA substantially outperforms state-of-the-art methods in both success rate and design speed. Further experiments on other biological sequence design tasks highlight the effectiveness and broad potential of CocoRNA for complex design tasks.

Figures & tables

Figure 1: A . Examples of RNA sequences and structures visualized using Forna [ 39 ] . $(a)$ is an unfolded RNA sequence. Structures $(b)$ and $(c)$ show two different folded forms of this sequence. Structure $(c)$ is specifically generated by RNAfold [ 45 ] , which uses a minimum free energy algorithm. Structure $(d)$ is folded from a different sequence, yet it shares the same secondary structure as $(c)$ . B . Structure density surface for RNA sequences of length $150$ . The plot illustrates the distribution of Hamming distances between RNA sequences and their corresponding secondary structure distances.

Figure 2: Overview of the CocoRNA workflow.

Method	Number of solved samples	Mean success rate	Average diversity
CocoRNA	$\mathbf{635.40\pm 2.58}$	$\mathbf{97.75\%}$	$\mathbf{0.405}$
LEARNA- $30$ s	$63.80\pm 3.03$	$9.82\%$	0.311
LEARNA- $300$ s	$368.20\pm 6.18$	$56.65\%$	0.339
LEARNA- $600$ s	$382.20\pm 4.82$	$58.80\%$	0.336
Meta-LEARNA-Adapt- $30$ s	$381.20\pm 5.63$	$58.65\%$	0.377
Meta-LEARNA-Adapt- $300$ s	$564.20\pm 4.09$	$86.80\%$	0.392

Table 1: Evaluation of CocoRNA and baseline methods on the RNA inverse design task. The time limit for CocoRNA is set to 30 seconds. The suffixes in baseline names indicate different time constraints; for example, “

-600

s" denotes a time limit of

600

seconds per structure.

Objectives	Success rate	GC	MFE
Struct.	$97.75\%$	$0.62$	$-125.1$
Struct. + GC	$97.20\%$	$0.51$	–
Struct. + MFE	$97.07\%$	–	$-158.5$
Struct. + GC + MFE	$97.26\%$	$0.53$	$-151.2$

Table 2: Multi-objective design with CocoRNA .

Figure 3: CPU time distribution of CocoRNA and four baselines over successfully solved cases.

Figure 4: ( Left ) Cross-oracle validation experiment using RibonanzaNet. ( Middle ) Matrix representation and pseudoknot design results. ( Right ) Success rate breakdown across RNA families.

Figure 5: ( Left ) Comparison of sequence recovery rates during training between the Single-agent variant and CocoRNA . ( Right ) Effect of removing the SAE heuristic from CocoRNA .

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Number of solved samples	Mean success rate
CocoRNA	$633.33\pm 5.47$	$97.44\%$
Meta-LEARNA-Adapt-600s	$578.17\pm 9.60$	$88.95\%$
SAMFEO-600s	$568.16\pm 6.09$	$87.41\%$

Appendix

Table 3: Performance of CocoRNA , Meta-LEARNA-Adapt-600s, and SAMFEO-600s across six different train-test splits. The mean and standard deviation are reported.

Method	Number of solved samples	Mean success rate
CocoRNA -PBD	$633.33\pm 5.47$	$97.44\%$
CocoRNA -SBD	$628.50\pm 4.99$	$96.69\%$
CocoRNA -random	$596.33\pm 6.70$	$91.74\%$

Appendix

Table 4: Performance of CocoRNA with different problem decomposition schemes on Rfam dataset across six different train-test splits. The mean and standard deviation over are reported.

Method	Number of solved samples	Mean success rate
CocoRNA	$635.40\pm 2.58$	$97.75\%$
CocoRNA -ablated	$599.60\pm 7.64$	$92.25\%$
Single-agent version	$340.80\pm 5.63$	$52.43\%$

Appendix

Table 5: Performance of CocoRNA and ablated versions on Rfam dataset. The mean and standard deviation over five independent runs are reported.

Figure 6: Learning curves under different reward settings on the Rfam dataset. Each experiment is performed over

6

independent runs with different random seeds. The shaded areas represent the

95\%

confidence intervals. Left : Learning curves using delayed reward ( 22 ). Right : Learning curves using terminal reward ( 23 ).

Reward Setting / Method	Number of solved samples	Success rate
Delayed Reward / CocoRNA	$634/650$	$97.54\%$
Delayed Reward / CocoRNA -ablated	$621/650$	$95.54\%$
Terminal Reward / CocoRNA	$604/650$	$92.92\%$
Terminal Reward / CocoRNA -ablated	$570/650$	$87.69\%$

Appendix

Table 6: Performance of CocoRNA and CocoRNA -ablated under different reward settings on the Rfam dataset.

Figure 7: Learning curves of CocoRNA under different agent size settings on the Rfam dataset. Each experiment is performed over

6

independent runs with different random seeds. The shaded areas represent the

95\%

confidence intervals.

Number of Agents ( $n$ )	Number of solved samples	Success rate
$n=2$	$601/650$	$92.46\%$
$n=4$	$636/650$	$97.85\%$
$n=6$	$629/650$	$96.77\%$
$n=8$	$615/650$	$94.62\%$

Appendix

Table 7: Performance of CocoRNA under different agent size settings on the Rfam dataset.

Figure 8: Learning curves of CocoRNA -PBD and CocoRNA -PBD-shared on the Rfam dataset. Each experiment is performed over

6

independent runs with different random seeds. The shaded areas represent the

95\%

confidence intervals.

Method	Number of solved samples	Success rate
CocoRNA -PBD	$636/650$	$97.85\%$
CocoRNA -PBD-shared	$622/650$	$95.69\%$
CocoRNA -SBD	$629/650$	$96.77\%$
CocoRNA -SBD-shared	$621/650$	$95.54\%$

Appendix

Table 8: Performance of CocoRNA with independent and shared policies on the Rfam dataset.

Figure 9: Learning curves of CocoRNA using different learning rates. Each experiment is performed over

6

independent runs with different random seeds. The shaded areas represent the

95\%

confidence intervals.

Figure 10: Number of solved samples as a function of the allotted runtime.

	TF Bind 8	TF Bind 10	ChEMBL	UTR
CocoRNA	$1.000\pm 0.000$	$1.000\pm 0.000$	$0.811\pm 0.001$	$0.810\pm 0.001$
Single-agent version	$0.901\pm 0.093$	$0.725\pm 0.140$	$0.595\pm 0.166$	$0.690\pm 0.010$

Appendix

Table 9: Evaluation of CocoRNA and the single-agent ablated version on biological sequence design tasks. Results represent the average of

8

independent runs, with standard deviations also reported.

Method	$n_{\text{i}}=1$	$1<n_{\text{i}}\leq 15$	$n_{\text{i}}>15$
CocoRNA	$621$	$29$	$0$
LEARNA	$1$	$16$	$633$
Meta-LEARNA	$22$	$137$	$491$

Appendix

Table 10: Distribution of RNA sequences by redesign attempts

n_{\text{i}}

intervals for different methods on the Rfam dataset.

Figure 11: Length distributions of RNA secondary structures in different datasets.

Dataset	Sample Size	Average Length	Max Length	Min Length
Rfam (Training Set)	$60000$	$243.73$	$450$	$50$
Rfam (Test Set)	$650$	$238.78$	$450$	$50$

Appendix

Table 11: Statistics of the datasets.

Hyperparameter	Value
Learning Rate	$0.00001$
Discount Factor ( $\gamma$ )	$0.99$
GAE Parameter ( $\lambda$ )	$0.95$
PPO Clip Range ( $\epsilon$ )	$0.2$
Entropy Coefficient	$0.02$
Horizon Length	$256$

Appendix

Table 12: Training hyperparameters.

Hyperparameter	Value
Actor Network Layers	$[128,64]$
Critic Network Layers	$[256,64]$
CNN Filter Size	$[8,4,3]$
CNN Number of Filters	$[32,64,64]$
Activation Function	ReLU
Optimizer	Adam

Appendix

Table 13: Network hyperparameters.

Parameter	Value
Maximum Steps per Episode (Training)	$350$
Maximum Steps per Episode (Testing)	$400$
Observation Radius	$30$
Local Observation Window Size	$61$
Global State Dimension	$900$
Total Training Steps	$2.5\times 10^{7}$

Appendix

Table 14: Additional hyperparameters.

Inverse Design

Inverse design maps desired properties or behaviors to materials, geometries, molecules, and systems that can realize them.

See all 111 papers

RNA Design via Conditioned Flow Matching and Finite-Policy Reinforcement Learning
Sep 29, 2026Zefeng Lin, Xianyong Fang, Tianfan Fu +1Inverse Design Molecular Generation
School of Computer Science and Technology, University of Science and Technology of China, Hefei, Anhui, China · School of Computer Science and Technology, Anhui University, Hefei, Anhui, China · State Key Laboratory for Novel Software Technology at Nanjing University, School of Computer Science, Nanjing University, Nanjing, Jiangsu, China
TANGO: RNA Topology and Geometry Co-Design
Oct 14, 2026Tianmeng Hu, Biao Luo, Ke LiInverse Design
University of Exeter · Earlham Institute · Central South University
La-Ribo: RNA Co-Design via Geometry-Latent Flow Matching
Oct 8, 2026Runze Ma, Will Hua, Shuangjia ZhengInverse Design Flow-Based Generative Modeling
Shanghai Jiao Tong University · Independent Researcher

Sep 29, 2026J/K move · Enter open · S save

RNA Design via Conditioned Flow Matching and Finite-Policy Reinforcement Learning

Zefeng Lin, Xianyong Fang, Tianfan Fu, Xiaohua Xu

School of Computer Science and Technology, University of Science and Technology of China, Hefei, Anhui, China · School of Computer Science and Technology, Anhui University, Hefei, Anhui, China · State Key Laboratory for Novel Software Technology at Nanjing University, School of Computer Science, Nanjing University, Nanjing, Jiangsu, China

Inverse Design Molecular Generation Flow Matching RL for Generative Models

Open paper AskPDF arXiv

RNA design aims to identify sequences that fold into specified secondary structures. Existing methods formulate the task as target-specific search or conditional generation. However, natural RNA evolution proceeds through sequence variation and selection, with compensatory substitutions, whereas these methods do not explicitly model this process. To address this limitation, we propose a two-stage framework comprising RNA Inverse-Folding Flow (RNA-IFlow) and RNA-IFlow-RL. RNA-IFlow uses structure-conditioned Dirichlet Flow Matching to model coordinated variation across the sequence, while RNA-IFlow-RL maps the learned flow to a pairing-preserving finite policy and refines it with thermodynamic feedback. Our framework achieves leading performance on multiple benchmarks, reaching 85.19% Pass@1 on Rfam-27. Further analyses reveal thermodynamic gains, policy dynamics, and robustness across settings. Our work couples coordinated variation with thermodynamic selection, offering a novel paradigm for RNA design.

Abstract

Inverse Design

Explore similar work

RNA Design via Conditioned Flow Matching and Finite-Policy Reinforcement Learning

TANGO: RNA Topology and Geometry Co-Design

La-Ribo: RNA Co-Design via Geometry-Latent Flow Matching