A Large-Scale Comparative Analysis of Imputation Methods for Single-Cell RNA Sequencing Data
Authors: Yuichiro Iwashita, Ahtisham Fazeel Abbasi, Koichi Kise, Andreas Dengel, Muhammad Nabeel Asim
Organizations: RPTU University Kaiserslautern-Landau, Department of Computer Science, 67663 Kaiserslautern, Germany. · German Research Center for Artificial Intelligence (DFKI GmbH), 67663 Kaiserslautern, Germany. · Graduate School of Informatics, Osaka Metropolitan University, Osaka 599-8531, Japan. · intelligentX GmbH (intelligentX.com), 67663 Kaiserslautern, Germany.
Background: Single-cell RNA sequencing (scRNA-seq) enables gene expression profiling at cellular resolution but is inherently affected by sparsity caused by dropout events, where expressed genes are recorded as zeros due to technical limitations. These artifacts distort gene expression distributions and compromise downstream analyses. Numerous imputation methods have been proposed to recover latent transcriptional signals. These methods range from traditional statistical models to deep learning (DL)-based methods. However, their comparative performance remains unclear, as existing benchmarks evaluate only a limited subset of methods, datasets, and downstream analyses. Results: We present a comprehensive benchmark of 15 scRNA-seq imputation methods spanning 7 methodological categories, including traditional and DL-based methods. Methods are evaluated across 30 datasets from 10 experimental protocols on 6 downstream analyses. Results show that traditional methods, such as model-based, smoothing-based, and low-rank matrix-based methods, generally outperform DL-based methods, including diffusion-based, GAN-based, GNN-based, and autoencoder-based methods. In addition, strong performance in numerical gene expression recovery does not necessarily translate into improved biological interpretability in downstream analyses, including cell clustering, differential expression analysis, marker gene analysis, trajectory analysis, and cell type annotation. Furthermore, method performance varies substantially across datasets, protocols, and downstream analyses, with no single method consistently outperforming others. Conclusions: Our findings provide practical guidance for selecting imputation methods tailored to specific analytical objectives and underscore the importance of task-specific evaluation when assessing imputation performance in scRNA-seq data analysis.
Figures & tables
Study
Methods
Datasets
Protocols
Tasks
Traditional
DL-based
Model- based
Smoothing- based
Low-rank Matrix-based
Diffusion- based
GAN- based
GNN- based
AE- based
Hou et al. (2020)
6
3
3
0
0
0
6
16
5
4
Dai et al. (2022)
2
3
2
0
1
1
3
8
1
4
Cheng et al. (2023)
2
4
1
0
0
0
4
16
3
3
This Study
2
3
3
2
2
1
2
30
10
6
Table 1: Summary of existing benchmarking studies on scRNA-seq data imputation methods
Figure 1: Distribution of LND between imputed and ground truth expression values for each imputation method. The x-axis represents LND values, and the y-axis represents different imputation methods.
Figure 2: Numerical gene expression recovery performance. a – b MAE and MedAE, respectively. The x-axis represents different imputation methods, and the y-axis represents error values in a log scale. c Protocol-wise total MAE and MedAE. The x-axis represents different imputation methods, and the y-axis represents total error values in a log scale. d – e PCC. The x-axis represents different imputation methods, and the y-axis represents PCC values. f MCC. The x-axis represents different cell lines, and the y-axis represents different imputation methods.
Figure 3: Cell clustering consistency and coherency performance. a ARI. Each plot shows ARI scores of different imputation methods. Each point in a plot represents different methods. The horizontal dashed line represents the masked baseline value. b SC. Each plot shows SC scores of different imputation methods. Each point in a plot represents different methods. The horizontal dashed line represents the masked baseline value.
Figure 4: UMAP visualization of the cell clustering using 4 real datasets with different protocols. Each plot shows the UMAP visualization of the method. Different colors represent different clusters.
Figure 5: UMAP visualization of the cell clustering using 4 simulated datasets with different dropout rates. Each plot shows the UMAP visualization of the method. Different colors represent different clusters.
Figure 6: The overview of 3 complementary analyses of DE analysis. a DE enrichment analysis, b null DE analysis, and c effect size analysis.
Figure 7: DE analysis performance. a – c DE enrichment analysis. IoU between DEGs identified from imputed scRNA-seq data and bulk RNA-seq data using MAST and the Wilcoxon rank-sum test. The dashed line represents equal performance. Each point represents an imputation method. d – f Null DE analysis. Average number of false positive DEGs across 6 different sample sizes for A549 ( d ), GM12878 ( e ), and monocyte ( f ). g – h Effect size analysis. IoU between DEGs identified from imputed scRNA-seq data and high- ( g ) or low- ( h ) LFC genes from bulk RNA-seq data.
Figure 8: Comparison of imputation methods for marker gene expression in hca_10x_tissue. Violin plots show the distribution of expression levels for 8 marker genes across 4 cell types, T cell, B cell, NK cell, and monocyte (shown as Mono). The y-axis represents gene expression values.
Figure 9: Marker gene expression performance on chu_cell_type. a UMAP visualizations with 7 cell type labels, namely DEC, EC, H1, H9, HFF, NPC, and TB, colored by cell type. Each plot represents an imputation method. b Heatmaps of 10 marker gene expression values across the 7 cell types. The x-axis represents individual cells ordered by cell type, and the y-axis represents marker genes.
Figure 10: Trajectory analysis performance. a – b POS and KRCC between the inferred pseudotime from imputed data and the true cellular development time label, respectively. The x-axis and y-axis represent the performance of petropoulos and chu_time_course, respectively. The dashed line represents equal performance across the 2 datasets. Each point in the plot represents an imputation method. c UMAP visualizations with inferred pseudotime trajectories for petropoulos (top) and chu_time_course (bottom). Cells are colored by inferred pseudotime.
Figure 11: Cell type annotation performance. Dumbbell plots comparing ACC, PR, RC, and F1 achieved by 1D-CNN (green circles) and scGPT Ding et al. (2025) (orange triangles) across 6 datasets, including sc_10x_5cl, hca_10x_tissue, chu_cell_type, baron, encode_fluidigm_5cl, and bladder. The x-axis represents the value of each evaluation measure, and the y-axis represents different imputation methods. Each horizontal line connects the 1D-CNN and scGPT performances for a given method.
Category
Method
Numerical gene expression recovery
Cell clustering
DE analysis
Marker gene analysis
Trajectory analysis
Cell type annotation
Model-based
PbImpute
△
△
△
△
★
△
scImpute
△
△
△
★
★
△
Smoothing- based
AcImpute
△
△
★
△
△
△
scTsI
★
△
△
△
△
△
MAGIC
△
★
△
★
△
★
Low-rank matrix-based
PBLR
★
×
×
×
△
△
Table 4: Summary of the performance of imputation methods. ★ , △ , and × indicate best, moderate, and worst performance, respectively.
Figure 12: Data Imputation Benchmarking Framework.
Dataset
Description
Source
Size ( Cells×Genes )
Sparsity Rates (%)
Protocol
ad_case Grubman et al. (2019)
Human brain cells with Alzheimer’s disease
GSE138852
10278×13214
94.93
10x Chromium
jurkat
Human Jurkat T-cell leukemia cell line scRNA-seq dataset
Single cells from three human lung adenocarcinoma cell lines
GSM3022245
902×16468
45.02
10x Chromium
sc_10x_5cl Tian et al. (2019) ; Tian et al. (2018)
Single cells from five human lung adenocarcinoma cell lines
GSM3618014
3913×11786
63.04
10x Chromium
Table 5: Details of the used scRNA-seq datasets
Figure 13: The overview of downstream tasks used for benchmarking imputation methods. a Numerical gene expression recovery, b cell clustering, c DE analysis, d marker gene analysis, e trajectory analysis, and f cell type annotation.
Single-cell RNA sequencing (scRNA-seq) data exhibit strong and reproducible statistical structure. This has motivated the development of large-scale foundation models, such as TranscriptFormer, that use transformer-based architectures to learn a generative model for gene expression by embedding genes into a latent vector space. These embeddings have been used to obtain state-of-the-art (SOTA) performance on downstream tasks such as cell-type classification, disease-state prediction, and cross-species learning. Here, we ask whether similar performance can be achieved without utilizing computationally intensive deep learning-based representations. Using simple, interpretable pipelines that rely on careful normalization and linear methods, we obtain SOTA or near SOTA performance across multiple benchmarks commonly used to evaluate single-cell foundation models, including outperforming foundation models on out-of-distribution tasks involving novel cell types and organisms absent from the training data. Our findings highlight the need for rigorous benchmarking and suggest that the biology of cell identity can be captured by simple linear representations of single cell gene expression data.
Huan Souza, Pankaj Mehta
Department of Physics, Boston University, Boston, MA, 02215, USA · Faculty of Computing and Data Science, Boston University, Boston, MA, 02215, USA
Single-cell RNA sequencing (scRNA-seq) serves a pivotal role in characterizing gene expression at the cellular level, enabling the identification of cell types and advancing the understanding of cellular heterogeneity. Despite the significant progress in scRNA-seq data clustering, we argue that current methods always ignore the sparsity and noise, as well as the complex intercellular structural information inherent in scRNA-seq data. Toward this end, in this paper, we propose a novel single-cell RNA-seq clustering framework via deep Siamese Graph Transformer Network (termed scGTN), which explicitly integrates gene expression profile and intercellular structural dependencies for cell clustering. In particular, we formulate scRNA-seq data as a graph and construct two augmented graph views that serve as dual views to capture complementary intercellular information. Then, a Siamese graph transformer network is employed to explicitly incorporate shortest-path information and node-wise distances for capturing richer structural relationships between cells. Finally, we employ an optimal transport strategy to guide the cell clustering in a self-supervised manner. Extensive experiments on multiple benchmark scRNA-seq datasets demonstrate that our scGTN consistently outperforms existing methods. Our code is available at https://github.com/W-RMSL/scGTN.
Jinke Wu, Yifan Wang, Siyu Yi +5
Sichuan University · University of International Business and Economics · Great Bay University +1
Despite theoretical advantages, causal methods for Gene Regulatory Network (GRN) inference from single-cell RNA-seq data consistently fail to match or outperform correlation-based baselines in many realistic benchmarks, a persistent puzzle which casts doubt on the value of causality for this task. We argue that existing benchmarks are insufficiently controlled to answer this question because they evaluate on real or semi-real data where multiple pathologies co-occur, confounding failure modes, and obscuring the specific conditions under which different inference methods excel or fail. To address this gap, we introduce a controlled diagnostic framework that isolates seven biologically motivated pathologies (dropout, latent confounders, cell-type mixing, feedback loops, network density, sample size, and pseudotime drift) and measure how six representative methods spanning three inference paradigms degrade as each pathology intensifies. Across 6,120 controlled experiments, we find that causal methods genuinely dominate in clean and structurally favorable regimes, but specific pathologies (notably dropout and latent confounders) selectively neutralize their advantages. We further introduce an error-type decomposition that reveals methods with similar aggregate accuracy commit qualitatively different errors. To probe whether single-pathology effects persist when multiple stressors co-occur, we perform an interaction sweep over the three most impactful pathologies and find that their joint effects are sub-additive, while also exposing density-conditional cross-overs invisible to single-dial analysis. Our findings offer a nuanced understanding of when and why different methods succeed or fail for GRN inference, providing actionable insights for method development and practical guidance for practitioners.
Miguel Fernandez-de-Retana, Ruben Sanchez-Corcuera, Unai Zulaika +2
Faculty of Engineering University of Deusto Bilbao, Spain