CrossLLM-Mamba: Multimodal State Space Fusion of LLMs for RNA Interaction Prediction

Rabeya Tus Sadia; Qiang Ye; Qiang Cheng

q-bio.GNFeb 23, 2026

CrossLLM-Mamba: Multimodal State Space Fusion of LLMs for RNA Interaction Prediction

Authors: Rabeya Tus Sadia, Qiang Ye, Qiang Cheng

Organizations: Department of Computer Science, University of Kentucky, Lexington, KY, USA · Department of Mathematics, University of Kentucky, Lexington, KY, USA · Institute for Biomedical Informatics, University of Kentucky, Lexington, KY, USA

Abstract

Accurate prediction of RNA-associated interactions is essential for understanding cellular regulation and advancing drug discovery. While Biological Large Language Models (BioLLMs) such as ESM-2 and RiNALMo provide powerful sequence representations, existing methods rely on static fusion strategies that fail to capture the dynamic, context-dependent nature of molecular binding. We introduce CrossLLM-Mamba, a novel framework that reformulates interaction prediction as a state-space alignment problem. By leveraging bidirectional Mamba encoders, our approach enables deep ``crosstalk'' between modality-specific embeddings through hidden state propagation, modeling interactions as dynamic sequence transitions rather than static feature overlaps. The framework maintains linear computational complexity, making it scalable to high-dimensional BioLLM embeddings. We further incorporate Gaussian noise injection and Focal Loss to enhance robustness against hard-negative samples. Comprehensive experiments across three interaction categories, RNA-protein, RNA-small molecule, and RNA-RNA demonstrate that CrossLLM-Mamba achieves state-of-the-art performance. On the RPI1460 benchmark, our model attains an MCC of 0.892, surpassing the previous best by 5.2%. For binding affinity prediction, we achieve Pearson correlations exceeding 0.95 on riboswitch and repeat RNA subtypes. These results establish state-space modeling as a powerful paradigm for multi-modal biological interaction prediction.

Figures & tables

Figure 1: Sequence Embedding and Feature Extraction Pipeline. The proposed framework utilizes specialized pre-trained large language models to encode biological entities into high-dimensional feature vectors. (Top) Protein amino acid sequences are encoded using ESM2 . (Middle) RNA nucleotide sequences are processed via RiNALMo . (Bottom) Small molecule graphs are constructed and encoded using MoleBERT . These distinct embedding streams serve as the initial input features for the downstream dual-path Mamba architecture.

Figure 2: The CrossLLM-Mamba Model Architecture. The framework processes multi-modal inputs (Protein, RNA, or Molecule feature vectors) through a dual-path pipeline. First, feature vectors are projected and aligned using a linear transformation with Gaussian noise injection ( $\mathcal{N}(0,\sigma^{2})$ ) to enhance robustness. These aligned features are encoded by parallel BiMamba Encoders (detailed in the right panel), which capture bidirectional sequential dependencies. The encoded representations are fused in the Cross-Mamba Interaction Module via sequence stacking and a BiMamba Mixer to explicitly model interaction flows. Finally, global average pooling aggregates the features for the MLP prediction head to output the interaction probability.

Method	MCC	ACC	F1	Precision	Recall	AUC–ROC
RPISeq-RF [ 14 ]	0.570	0.780	0.780	0.790	0.780	0.790
IPMIner [ 15 , 16 ]	0.520	0.760	0.770	0.720	0.830	0.801
CFRP [ 3 ]	0.630	0.810	0.820	0.830	0.780	0.834
RPITER [ 17 ]	0.412	0.690	0.510	0.610	0.480	0.720
LPI-CSFFR [ 6 ]	0.600	0.830	0.840	0.780	0.910	0.820
RNAincoder [ 22 ]	0.760	0.880	0.840	0.810	0.940	0.915

Table 1: Performance comparison on the RPI1460 dataset. Best values are in bold.

Metric	Method	Aptamer	Repeats	Ribosomal	Riboswitch	Viral	miRNA
Pearson	BioLLMNet	0.7713	0.9340	0.8933	0.9462	0.8004	0.8712
	RLaffinity	0.7400	0.9100	0.8500	0.9250	0.7890	0.8750
	RSAPred	0.7180	0.8940	0.8360	0.9090	0.7840	0.8810
	CrossLLM-Mamba (Ours)	0.8254	0.9521	0.9012	0.9562	0.829	0.869
MAE	BioLLMNet	0.5726	0.3572	0.5743	0.5236	0.5582	0.3827
	RLaffinity	0.5900	0.3630	0.6400	0.5300	0.6010	0.4000

Table 2: Comparison of BioLLMNet [ 1 ] , RLaffinity [ 20 ] , RSAPred [ 8 ] , and CrossLLM-Mamba. Best values are in bold.

Model	ATH-GMA	ATH-MTR	GMA-ATH	GMA-MTR	MTR-ATH	MTR-GMA
PMLIPred [ 7 ]	65	70	55	87	53	71
CORAIN [ 23 ]	69	74	67	93	58	84
BioLLMNet [ 1 ]	69	72	69	84	68	85
CrossLLM-Mamba (Ours)	72	74	73	87	75	85

Table 3: Comparison of CrossLLM-Mamba on miRNA–lncRNA interaction datasets based on accuracy (%), where the best values are shown in bold.

Figure 3: Performance Comparison on the RPI1460 Dataset. The boxplots illustrate the distribution of performance metrics (MCC, ACC, F1, Precision, Recall, and Specificity) across 5-fold cross-validation for various state-of-the-art methods. Our proposed CrossLLM-Mamba (shown in red) consistently outperforms existing baselines, achieving the highest median scores and lowest variance across all major metrics, particularly in MCC and F1-score, demonstrating its robustness and superior predictive capability.

Figure 4: Performance impact of removing specific architectural components. The full CrossMamba-Bio model (blue) significantly outperforms variants lacking the cross-modal state mixing or bidirectional context.

Figure 5: Ablation study on the number of BiMamba blocks in the modality-specific encoders and fusion module. Performance peaks at moderate depth ( enc_blocks =3, Fusion blocks =2–3), while deeper stacks show diminishing returns.

Cross-Modal Representation Learning

Cross-modal representation learning connects information across data types such as images, text, audio, biosignals, and sensor measurements.

See all 210 papers

ZeroFold: Protein-RNA Binding Affinity Predictions from Pre-Structural Embeddings
Mar 24, 2026Josef Hanke, Sebastian Pujalte Ojeda, Shengyu Zhang +3Molecular Property Prediction Molecular Representation Learning
Yusuf Hamied Department of Chemistry, University of Cambridge, UK · Centre for Misfolding Diseases, Yusuf Hamied Department of Chemistry, University of Cambridge, Cambridge, UK · Medicinal Chemistry, Research and Early Development, Respiratory and Immunology, BioPharmaceuticals R and D, AstraZeneca, Sweden +1
Enhancing Protein-Protein Interaction Prediction with Hierarchical Motif-based Multimodal Protein Embedding
May 30, 2026Zaifei Yang, Samuel Ping-Man Choi, James KwokHierarchical Representation Learning Multimodal Embedding
Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Clear Water Bay, Hong Kong. · Lee Shau Kee School of Business and Administration, The Hong Kong Metropolitan University, Ho Man Tin, Kowloon, Hong Kong.
Multimodal Alignment and Preference Optimization for Zero-Shot Conditional RNA Generation
May 12, 2026Roman Klypa, Alberto Bietti, Sergei GrudininMolecular Generation Conditional Generation
Univ. Grenoble Alpes, CNRS, Grenoble INP, LJK · Grenoble, France · Center for Computational Mathematics, Flatiron Institute +1

Mar 24, 2026J/K move · Enter open · S save

ZeroFold: Protein-RNA Binding Affinity Predictions from Pre-Structural Embeddings

Josef Hanke, Sebastian Pujalte Ojeda, Shengyu Zhang, Werngard Czechtizky+2

Yusuf Hamied Department of Chemistry, University of Cambridge, UK · Centre for Misfolding Diseases, Yusuf Hamied Department of Chemistry, University of Cambridge, Cambridge, UK · Medicinal Chemistry, Research and Early Development, Respiratory and Immunology, BioPharmaceuticals R and D, AstraZeneca, Sweden+1

Molecular Property Prediction Molecular Representation Learning

Open paper AskPDF arXiv

The accurate prediction of protein-RNA binding affinity remains an unsolved problem in structural biology, limiting opportunities in understanding gene regulation and designing RNA-targeting therapeutics. A central obstacle is the structural flexibility of RNA, as, unlike proteins, RNA molecules exist as dynamic conformational ensembles. Thus, committing to a single predicted structure discards information relevant to binding. Here, we show that this obstacle can be addressed by extracting pre-structural embeddings, which are intermediate representations from a biomolecular foundation model captured before the structure decoding step. Pre-structural embeddings implicitly encode conformational ensemble information without requiring predicted structures. We build ZeroFold, a transformer-based model that combines pre-structural embeddings from Boltz-2 for both protein and RNA molecules through a cross-modal attention mechanism to predict binding affinity directly from sequence. To support training and evaluation, we construct PRADB, a curated dataset of 2,621 unique protein-RNA pairs with experimentally measured affinities drawn from four complementary databases. On a held-out test set constructed with 40% sequence identity thresholds, ZeroFold achieves a Spearman correlation of 0.65, a value approaching the ceiling imposed by experimental measurement noise. Under progressively fairer evaluation conditions that control for training-set overlap, ZeroFold compares favourably with respect to leading structure-based and leading sequence-based predictors, with the performance gap widening as sequence similarity to competitor training data is reduced. These results illustrate how pre-structural embeddings offer a representation strategy for flexible biomolecules, opening a route to affinity prediction for protein-RNA pairs for which no structural data exist.

Abstract

Cross-Modal Representation Learning

Explore similar work

ZeroFold: Protein-RNA Binding Affinity Predictions from Pre-Structural Embeddings

Enhancing Protein-Protein Interaction Prediction with Hierarchical Motif-based Multimodal Protein Embedding

Multimodal Alignment and Preference Optimization for Zero-Shot Conditional RNA Generation