scHFBiMamba: a bidirectional Mamba autoencoder for cardiac disease-state classification from single-nucleus RNA-seq data
Authors: Ugo Lomoio, Pietro Hiram Guzzi, Pierangelo Veltri
Organizations: Department of Surgical and Medical Sciences University “Magna Græcia” of Catanzaro 88100 Catanzaro, Italy · DIMES University of Calabria 88063 Rende, Italy
To evaluate whether a compact bidirectional state-space autoencoder improves joint classification of species, cell type, and disease state from single-nucleus RNA-sequencing profiles in heart failure (HF), while yielding interpretable gene attributions. Building on the dense denoising autoencoder (DAE) and MLP framework of Ruz Jurado et al., we developed scHFBiMamba, combining a bidirectional Mamba autoencoder (BiMambaAE) with an MLP head. BiMambaAE maps 16,545-gene profiles to 350-dimensional latent representations using 13.2M parameters, about 14.6-fold fewer than the dense DAE (192.9M). We compared six encoder-classifier configurations on 38,275 held-out cardiac nuclei using McNemar tests with Benjamini-Hochberg correction, and assessed nine classifiers on frozen representations. Biological relevance was evaluated by comparing SHAP-derived differentially explained genes with differentially expressed genes against curated HF-related Gene Ontology keywords. BiMambaAE+MLP achieved the highest aggregate F1: pooled macro-F1 across 13 outputs was 0.966 versus 0.957 for DAE+MLP, and the mean category macro-F1 was 0.977 versus 0.970. It significantly improved cell-type and disease-state accuracy (adjusted p=7.1e-3 and p=4.8e-6, respectively), with species F1 of 1.000. SHAP-derived differentially explained genes achieved a pathway-enrichment benchmark F1 of 0.105 versus 0.053 for differentially expressed genes. Compact bidirectional state-space autoencoders can outperform substantially larger dense autoencoders for joint cardiac transcriptomic classification while reducing parameters. scHFBiMamba enables accurate multi-task prediction and hypothesis-generating gene-level interpretation. Code: https://github.com/UgoLomoio/scHFBiMamba.
Figures & tables
Figure 1: Architecture of the two main components. Left: the BiMambaAE autoencoder: a linear stem projects the 16,545-gene input to 350 dimensions, reshaped into 32 latent tokens of dimension 64; a bidirectional Mamba2 layer mixes the token sequence (the backward block processes the reversed sequence and its output is reversed back before summation); a projection maps the flattened tokens to the 350-dimensional latent vector; a two-layer MLP decoder reconstructs the input through a linear output layer. Right: the BiMamba classifier head: each latent dimension is embedded as a token with a learned positional embedding, processed by three bidirectional layers of two blocks each with a global residual connection, then mean- and max-pooled and mapped through a 128-unit head to 13 outputs (2 species, 7 cell types, 4 disease states).
Model
Encoder
Classifier
F1 cell type
F1 species
F1 disease
F1 mean (13)
F1 mean (cat.)
M0
Dense DAE
MLP [ 3 ]
0.9329
0.9996
0.9780
0.9570
0.9702
M1
Dense DAE
Mamba (uni)
0.9240
0.9994
0.9720
0.9504
0.9652
M2
Dense DAE
BiMamba
0.9246
0.9990
0.9667
0.9490
0.9634
M3
BiMambaAE
MLP
0.9463
1.0000
0.9833
0.9660
0.9765
M4
BiMambaAE
Mamba (uni)
0.9078
0.9998
0.9674
0.9403
0.9584
M5
BiMambaAE
BiMamba
0.9094
0.9997
0.9613
0.9393
0.9568
Table 1: Encoder–classifier comparison on the held-out test set of 38,275 nuclei. Category-level macro-F1 and both aggregate F1 statistics are shown: “F1 mean (13)” is the macro-F1 pooled over all 13 outputs (each class weighted equally); “F1 mean (cat.)” is the arithmetic mean of the three category macro-F1 scores. Both aggregates are computed at full precision from the stored per-class scores (Supplementary File S2); displayed category values are rounded, so their arithmetic mean can differ in the last digit. The highest value in each column is in bold ; M0 comparator values are underlined where shown. BH-adjusted McNemar tests compare nucleus-level prediction correctness, not F1: M3 exceeds M0 in all three categories (cell type p=7.1×10−3 , disease p=4.8×10−6 , species p=7.4×10−4 ). M1, M2, M4, and M5 have lower cell-type and disease-state accuracy than M0 (all adjusted p≤1.2×10−3 ); species differences are not significant for M4/M5.
Figure 2: scHFBiMamba workflow. Normalized snRNA-seq profiles (16,545 orthologous genes; human and mouse nuclei across CTRL, AS, HFrEF, and HFpEF) are compressed by either the dense DAE baseline or the proposed BiMambaAE into a 350-dimensional latent space. A classifier head predicts 13 outputs spanning species, cell type, and disease state. SHAP attributions through the composite model feed differential explanation (DXG) analysis and GSEA-based pathway evaluation against curated HF-related GO terms.
Figure 3: Macro-F1 per category (cell type, species, disease) for the six ablation models on the held-out test set. M3 (BiMambaAE+MLP) achieves the best cell-type (0.946) and disease (0.983) F1 and perfect species classification.
Figure 4: Classification performance versus model size. Total parameter count (log scale) is plotted against both aggregate F1 statistics for the six configurations (filled markers: pooled-13 macro-F1; open markers: mean-category F1). BiMambaAE configurations use a 13.2M-parameter autoencoder (orange shading), compared with 192.9M for the dense DAE (blue shading). M3 has the highest aggregate F1 under both definitions among these configurations.
Figure 5: Macro-averaged ROC curves on the held-out test set for cell-type (left) and disease-state (right) classification. M3 attains the highest macro ROC-AUC in both categories (0.9965 and 0.9981, respectively).
Classifier
F1 cell type
F1 species
F1 disease
Mean cat. F1
XGBoost
0.9381
0.9994
0.9787
0.9721
MLP
0.9362
0.9996
0.9748
0.9702
GCN ( k NN graph)
0.8094
0.9993
0.9309
0.9132
GAT ( k NN graph)
0.7721
0.9994
0.9194
0.8970
Mamba (plain BCE)
0.9240
0.9994
0.9720
0.9504
BiMamba (plain BCE)
0.9246
0.9990
0.9667
0.9490
Table 2: Classifier comparison using frozen 350-dimensional DAE representations. Category-level macro-F1 and the mean-category F1 (arithmetic mean of the three category macro-F1 scores) are shown for the held-out test set (single run, seed 1234). The pooled-13 macro-F1 for the main configurations (M0–M2) is reported in Table 1 . The highest value in each column is in bold .
Classifier
F1 cell type
F1 species
F1 disease
Mean cat. F1
MLP
0.9463
1.0000
0.9833
0.9765
XGBoost
0.9410
0.9998
0.9755
0.9721
GCN ( k NN graph)
0.9276
1.0000
0.9738
0.9671
GAT ( k NN graph)
0.9247
1.0000
0.9704
0.9650
Mamba (plain BCE)
0.9078
0.9998
0.9674
0.9584
BiMamba (plain BCE)
0.9094
0.9997
0.9613
0.9568
Table 3: Classifier comparison using frozen 350-dimensional BiMamba-AE representations. Category-level macro-F1 and the mean-category F1 (arithmetic mean of the three category macro-F1 scores) are shown for the held-out test set (single run, seed 1234). The pooled-13 macro-F1 for the main configurations (M3–M5) is reported in Table 1 . The highest value in each column is in bold .
Component
Inference time (ms)
Parameters
FLOPs
Autoencoders (encoder and decoder)
Dense DAE
7.01 ± 0.04
192.9M
385.81M
BiMambaAE
3.23 ± 0.95
13.2M
31.45M
Classifiers (350-d latent input)
MLP
0.15 ± 0.02
487.6K
0.98M
Mamba
98.04 ± 0.23
258.5K
181.0M
Table 4: Inference efficiency and model complexity. Inference time is mean ± SD per 1024-cell batch (50 batches after 10 warm-up, device-synchronized GPU iterations). XGBoost was benchmarked on CPU (parameter count reports total tree nodes; FLOPs not meaningful). FLOP conventions: dense layers estimated as 2× parameters; SSMs as 2× parameters × sequence length (32 latent tokens for BiMambaAE, 350 for the SSM classifier heads); GCN/GAT FLOPs scale with graph edges (full-graph transductive pass normalized per 1024 cells). GCN/GAT parameter counts refer to the trained hidden-64 configuration (23,309 and 23,386 parameters); their FLOPs and timings were measured with the hidden-128 benchmark configuration (46,605 and 46,746 parameters), a mismatch disclosed here and in the Limitations.
Figure 6: UMAP projections of the 350-dimensional test-set representations. Left: dense DAE (M0). Right: BiMambaAE (M3). Top: nuclei colored by cell type. Bottom: nuclei colored by disease state. Both representations show cell-type structure despite the approximately 14.6× difference in total autoencoder parameter count.
Figure 7: SHAP beeswarm plots for the 20 highest-ranked gene attributions to the HFrEF (left) and HFpEF (right) outputs of M3 (BiMambaAE+MLP). Point color represents normalized expression. Highly attributed HFrEF genes include MYH7 , TTN , RYR2 , ANKRD1 , ACTC1 , and MYL2 ; attribution magnitude reflects contribution to model predictions rather than experimental validation of disease mechanisms.
Figure 8: Overlap between M3 DXGs (SHAP-based) and DEGs (expression-based Wilcoxon test) for human cardiomyocytes in HFrEF vs. CTRL. The sets contain 1051 DXG-exclusive genes, 2807 DEG-exclusive genes, and 769 shared genes.
Figure 9: GSEA F1 scores against the curated, cell-type-specific GO keyword reference for HFpEF (left) and HFrEF (right). DXG-based rankings from each model are compared with DEG-based rankings. Scores describe agreement with the selected reference and vary across cell types and disease contrasts.
Figure 10: Supplementary Figure S1: Ablation heatmap across the six configurations (rows) and ten performance metrics (columns): macro-F1 per category, both aggregate F1 statistics (“F1 mean (13 outputs)” and “F1 mean (categories)”), mean precision and recall, and macro ROC-AUC per category, all on the held-out test set.
Figure 11: Supplementary Figure S2: Training and validation curves for the replicated baseline M0 (left) and the proposed M3 (BiMambaAE+MLP, right).
Autoencoders learn compressed representations by encoding data into latent codes and decoding them for reconstruction, generation, denoising, or anomaly detection.
Single-cell ribonucleic acid sequencing (scRNA-seq) is a foundational technology for precision-medicine workflows that contribute to United Nations Sustainable Development Goal 3 on Good Health and Well-being, and unsupervised clustering is the analytical step that turns raw expression matrices into interpretable cell populations. Practitioners therefore face a recurring engineering decision: is an additional deep representation stage worth its compute and tuning cost, or do classical principal component analysis (PCA) pipelines already suffice? We address this question with a diagnostic benchmark of nine clustering pipelines on ten real datasets (90-5,685 cells, 19,046-41,480 genes, 4-11 cell types), augmented by a partial scVI V2 specialized comparison on seven datasets. The protocol integrates Optuna hyperparameter search, repeated-run robustness, Friedman/Wilcoxon-Holm/TOST testing, and Sobol total-order sensitivity analysis. The contrastive autoencoder achieved the highest mean Adjusted Rand Index (0.7872), but Holm-corrected tests did not establish dominance over the strongest baselines. Per-dataset analysis reveals three reproducible regimes: probabilistic variational autoencoder (VAE) variants help on the smallest datasets, deep autoencoders win on mid-scale data with multi-batch or many-type structure, and classical PCA pipelines remain competitive when linear projection already captures the dominant variation. Sobol indices identify learning rate (ST=0.70) and latent dimensionality (ST=0.56) as the dominant variance contributors, indicating where limited tuning budgets should be allocated. The contribution is therefore a dataset-aware and compute-conscious decision framework for biomedical AI pipelines supporting sustainable healthcare analytics, rather than a universal superiority claim.
Nguyen Thanh Phong, Truong Viet Vu, Nguyen Ha Thu +4
Faculty of Information Technology, Van Lang School of Technology, Van Lang University, Ho Chi Minh City, Vietnam · Faculty of Law, Van Lang University, Ho Chi Minh City, Vietnam
Single-cell RNA sequencing (scRNA-seq) data exhibit strong and reproducible statistical structure. This has motivated the development of large-scale foundation models, such as TranscriptFormer, that use transformer-based architectures to learn a generative model for gene expression by embedding genes into a latent vector space. These embeddings have been used to obtain state-of-the-art (SOTA) performance on downstream tasks such as cell-type classification, disease-state prediction, and cross-species learning. Here, we ask whether similar performance can be achieved without utilizing computationally intensive deep learning-based representations. Using simple, interpretable pipelines that rely on careful normalization and linear methods, we obtain SOTA or near SOTA performance across multiple benchmarks commonly used to evaluate single-cell foundation models, including outperforming foundation models on out-of-distribution tasks involving novel cell types and organisms absent from the training data. Our findings highlight the need for rigorous benchmarking and suggest that the biology of cell identity can be captured by simple linear representations of single cell gene expression data.
Huan Souza, Pankaj Mehta
Department of Physics, Boston University, Boston, MA, 02215, USA · Faculty of Computing and Data Science, Boston University, Boston, MA, 02215, USA
RNA sequencing produces rich and diverse datasets of gene expression, offering compelling insights into cellular state and function that have many applications in drug discovery. Modeling such data is challenging due to inherent technical noise and experimental batch effects, as evidenced by many existing transcriptomic foundation models (FMs) underperforming relative to linear baselines. Such results raise the question of whether deep representation learning provides a distinct advantage over the direct use of raw transcript counts. Our work explores this by developing a new self-supervised model, TxFM, with a focus on inductive representation learning evaluations. TxFM employs a masked autoencoding approach tailored to diverse RNA-seq count data, and our ablation study empirically identifies crucial architecture configurations required for strong transfer performance. Additionally, we curate a public training corpus, DiverseRNA-1.4M, and find that TxFM trained on this curated dataset yields high-fidelity gene representations that outperform FMs trained on atlas-scale corpora over 100x larger. Overall, our results indicate that inductive self-supervised learning is a viable modeling approach for transcriptomics representation, provided a careful synthesis of model architecture and training data curation.