When Covariance Is Not Enough: Kernel-Based Regularization for Single-Cell Counterfactuals
Authors: Korbinian Traeuble, Edward de Brouwer, Archit Verma, Doron Haviv, Kipper Fletez-Brant
Organizations: Institute of Computational Biology, German Research Center for Environmental Health, Helmholtz Zentrum München, Neuherberg, Germany · Department of Computer Science, TUM School of Computation, Information and Technology, Technical University of Munich, Garching, Germany · Computational Sciences Center of Excellence, Genentech, South San Francisco, CA, USA
Counterfactual editing of single cells, predicting how an observed cell would look had one factor been different, enables in-silico screening where experiments are infeasible, but is useful only if the edit is faithful to the individual and not merely to the condition. Concept bottleneck generative models make the effects of interventions interpretable by decomposing a cell's representation into known and unknown factors. Previous work uses a linear cross-covariance penalty to disentangle known and unknown dimensions. We demonstrate that this first-order criterion does not fully separate information when non-linear interactions are present. We first demonstrate the inadequacy of linear cross-factor penalization with simulation experiments with controlled edits and known non-linear covariate effects. We then demonstrate that adding a nonlinear kernel criterion, the normalized Hilbert-Schmidt Independence Criterion, is able to recover the true intervention effects on simulation data. Finally, we apply the non-linear concept bottleneck model to a complex single-cell RNA-seq dataset and find that it improves the ability to generate counterfactual expression profiles of cells under different tissue environments.
Figures & tables
Figure 1: Structure of the synthetic donor-holdout dataset (240,000 cells, 150 donors, 20 of them held out). All panels show the same UMAP, fitted once on a uniform random subsample of 40,000 cells in the 128-dimensional PCA space the model is trained in, and recoloured: ( A ) the 4 simulated cell types; ( B ) donor identity, with 8 donors highlighted (donor_000, donor_001, donor_002, donor_003, donor_014, donor_015, donor_020, donor_021) and the remaining donors in gray; ( C ) the binary tissue concept is_ileum , which is what the counterfactual flips; ( D ) training donors versus the 20 held-out donors.
Regime
Orth. loss
λ⊥=0.5
λ⊥=1
λ⊥=5
Linear (mean shift)
cov
0.09
0.06
0.06
nHSIC
0.10
0.08
0.06
cov + nHSIC
0.03
0.24
2.02
Nonlinear (heteroskedastic)
cov
1.30
1.30
1.09
nHSIC
1.09
0.69
0.35
cov + nHSIC
0.15
0.21
2.16
Table 1: Median per-donor rMMD on held-out donors for the matched linear/heteroskedastic simulation pair. Bold = beats the baseline ( <1 ). A covariance penalty suffices in the linear regime but fails at every strength in the heteroskedastic one, where the condition enters through a conditional variance; adding nHSIC recovers a donor-specific edit.
Figure 2: Donor-holdout counterfactual on the matched linear / heteroskedastic simulation pair. Rows: (A) linear regime with a covariance penalty, (B) heteroskedastic regime with the same covariance penalty, (C) heteroskedastic regime with covariance + nHSIC; all at λ⊥=0.5 . Columns are held-out donors (donor_014, donor_015, donor_020, donor_024), never seen in training. Each panel shows all 4 simulated cell types of that donor: the retained is_ileum = non_ileum cells (blue), the withheld real is_ileum = ileum cells (green), and the model’s counterfactual edit of the blue cells (red). Rows B and C share one UMAP, fitted jointly on their real cells and both predictions, so they are directly comparable and only the red cloud moves; row A is a different dataset and therefore a different coordinate system. Each group is subsampled to at most 300 cells per donor and cell type, so the three groups are drawn at comparable density. The three groups are drawn in a single shuffled scatter, so neither is systematically painted over the other and a true overlap reads as a blend. Arrows run from each cell type’s real source cloud to the model’s counterfactual for that cloud (centroids), i.e. the edit the model actually made; the edit succeeds when the arrow head lands on the green held-out target. They are a reading guide in UMAP coordinates, not a displacement in any metric space. The gray backdrop is the real cells of all 20 held-out donors (20,000 of 32,769 drawn), so each gray blob is another donor. Panels are framed, and tinted by regime – rows B and C share both an embedding and a tint, row A neither. Panel labels give that donor’s median rMMD over the cell types scored for it (a cell type is scored only where the donor has enough cells on both sides); run-level medians over all 20 held-out donors: (A) 0.09, (B) 1.30, (C) 0.15 (lower is better; <1 beats mapping the cells to the closest existing population).
Orthogonality penalty
λ⊥
rMMD ↓
rSinkhorn ↓
Covariance
0.5
0.95
0.50
Covariance
10
0.96
0.46
Covariance + nHSIC
10
0.41
0.14
Table 2: Held-out counterfactual accuracy on the taurus stem holdout (Rectum → terminal ileum, stem_LGR5pos ). rMMD is the ratio metric of Eq. 20 : <1 means the edited cells are closer to the real held-out ileal cells than the closest existing cell population is. All three runs are below 1 , but the covariance-only numerators ( 0.231 and 0.234 ) are essentially the denominator ( 0.244 ), i.e. the prediction is no closer to the target than the un-edited source it was made from; only cov + nHSIC (bold) makes a complete edit (edit ratio 0.67 vs. 0.04 and 0.22 ).
Figure 3: Counterfactual edit of the held-out stem_LGR5pos subtype (rectum → terminal ileum) under three orthogonality penalties. Every is_ileum = ileum stem_LGR5pos cell ( n =4,100, green) was withheld from training; at inference the retained rectal cells of the same subtype ( n =6,342, blue) are edited to ileum (red) and compared against them. The gray backdrop is the rest of the epithelial lineage, shaded by fine subtype (largest subtype lightest); a uniform random sample of 30,000 of the 317,976 cells is drawn, to keep the UMAP tractable; sampling is uniform, so subtype proportions are preserved.
Table 3: Cell-type concept: mapping of the 22 major values onto seven lineages.
Symbol
Meaning
Value
N
cells
240,000
F
genes
3,000
B
donors (20 held out)
150
T
tissues ( is_ileum )
2
U
cell types
4
C
latent concepts
3
Table 4: Generator settings for the matched linear / heteroskedastic pair.
taurus (Rectum → ileum, stem)
Synthetic pair
cov baselines
selected
grid
Orthogonality penalty
cov
cov + nHSIC
cov / nHSIC / cov + nHSIC
λ⊥
0.5 , 10
10
0.5 , 1 , 5
Class-balanced concepts
is_ileum
is_ileum
none
Patience
50
25
50
Gradient clipping
–
–
1.0
Table 5: Training configurations. All models share one architecture and optimizer; the runs differ only in the orthogonality penalty and its weight (plus the stopping patience and gradient clipping noted above).
Understanding cellular phenotypes and how they respond to perturbations is critical for disease biology and therapeutic design. Single-cell RNA sequencing enables characterization at cellular resolution, yet the combinatorial space of conditions makes exhaustive experimental mapping infeasible. We introduce single-cell Concept Bottleneck Generative Models (scCBGM), a framework for interpretable and precise counterfactual editing of individual cells. scCBGM adapts concept bottleneck architectures for single-cell data through decoder skip connections and a cross-covariance penalty that promotes disentanglement without dimensional constraints. We extend the framework to flow matching models, enabling concept-guided editing in both encoding-decoding and generation regimes. To enable rigorous evaluation, we develop a synthetic benchmark with ground-truth counterfactuals. Across multiple real datasets, scCBGM demonstrates superior performance in combinatorial generalization and counterfactual prediction, supported by cell-level validation on synthetic data and population-level benchmarks on real datasets.
Alma Andersson, Aya Abdelsalam Ismail, Edward De Brouwer +6
Genentech · Guide Labs · Department of Computer Science, New York University +1
Tissue graph counterfactuals ask what a cell would express if its spatial neighborhood were altered, a question central to predicting cell behavior in tissues. Existing methods, however, present only specific instances of such queries. In this work, we first formalize tissue graph counterfactuals as a class of spatial interventions that either rewire connections between cells (edge perturbation) or modify the expression of their neighbors (node perturbation). We then introduce Cellina, a framework that uses supervised disentanglement to decompose a cell's intrinsic state from its spatial context, using the latter as a conditioning input for in-silico perturbation predictions. Across benchmarks spanning over 2.5 million spatially-resolved cells in observational and interventional screens, Cellina outperforms both spatially-informed and non-spatial competitors in in-silico perturbations, while training as fast as non-spatial methods. Moreover, we show that Cellina reveals cancer subdomains with distinct signaling programs in an unsupervised manner and enables targeted neighbor perturbation simulations.
Abdul Moeed, Stefan Schrod, Martin Rohbeck +4
Division of Computational Genomics and Systems Genetics, German Cancer Research Center (DKFZ), Heidelberg, Germany · Helmholtz Information & Data Science School for Health, Germany · Genome Biology Unit, European Molecular Biology Laboratory, Heidelberg, Germany +4
Predicting cellular responses to perturbations supports the study of gene function, disease mechanisms, and therapeutic strategies. Despite advances in single-cell perturbation modeling, existing models typically optimize surrogate losses that do not directly reflect the biological criteria used for evaluation, so better data fitting need not yield better biological predictions. To address this mismatch, we introduce CellRFT, a reinforcement fine-tuning framework that uses biological evaluation as direct training feedback. CellRFT uses policy-gradient optimization to learn from non-differentiable evaluations of generated cell populations and integrates multiple biological rewards through hierarchical reward aggregation. Comprehensive experiments demonstrate CellRFT's applicability across different pretrained models and effectiveness in improving perturbation prediction, reveal that optimizing one biological criterion can help or hinder others, and show that complementary rewards can improve criteria beyond those directly optimized, offering a way to probe how biological metrics shape model behavior, with the potential to inform evaluation design. Code will be made available.
Jie Yan, Li Liu, Hanze Guo +7
Chinese Academy of Sciences · Peking University · Renmin University of China +2