When Covariance Is Not Enough: Kernel-Based Regularization for Single-Cell Counterfactuals
Authors: Korbinian Traeuble, Edward de Brouwer, Archit Verma, Doron Haviv, Kipper Fletez-Brant
Organizations: Institute of Computational Biology, German Research Center for Environmental Health, Helmholtz Zentrum München, Neuherberg, Germany · Department of Computer Science, TUM School of Computation, Information and Technology, Technical University of Munich, Garching, Germany · Computational Sciences Center of Excellence, Genentech, South San Francisco, CA, USA
Counterfactual editing of single cells, predicting how an observed cell would look had one factor been different, enables in-silico screening where experiments are infeasible, but is useful only if the edit is faithful to the individual and not merely to the condition. Concept bottleneck generative models make the effects of interventions interpretable by decomposing a cell's representation into known and unknown factors. Previous work uses a linear cross-covariance penalty to disentangle known and unknown dimensions. We demonstrate that this first-order criterion does not fully separate information when non-linear interactions are present. We first demonstrate the inadequacy of linear cross-factor penalization with simulation experiments with controlled edits and known non-linear covariate effects. We then demonstrate that adding a nonlinear kernel criterion, the normalized Hilbert-Schmidt Independence Criterion, is able to recover the true intervention effects on simulation data. Finally, we apply the non-linear concept bottleneck model to a complex single-cell RNA-seq dataset and find that it improves the ability to generate counterfactual expression profiles of cells under different tissue environments.
Figures & tables
Figure 1: Structure of the synthetic donor-holdout dataset (240,000 cells, 150 donors, 20 of them held out). All panels show the same UMAP, fitted once on a uniform random subsample of 40,000 cells in the 128-dimensional PCA space the model is trained in, and recoloured: ( A ) the 4 simulated cell types; ( B ) donor identity, with 8 donors highlighted (donor_000, donor_001, donor_002, donor_003, donor_014, donor_015, donor_020, donor_021) and the remaining donors in gray; ( C ) the binary tissue concept is_ileum , which is what the counterfactual flips; ( D ) training donors versus the 20 held-out donors.
Regime
Orth. loss
λ⊥=0.5
λ⊥=1
λ⊥=5
Linear (mean shift)
cov
0.09
0.06
0.06
nHSIC
0.10
0.08
0.06
cov + nHSIC
0.03
0.24
2.02
Nonlinear (heteroskedastic)
cov
1.30
1.30
1.09
nHSIC
1.09
0.69
0.35
cov + nHSIC
0.15
0.21
2.16
Table 1: Median per-donor rMMD on held-out donors for the matched linear/heteroskedastic simulation pair. Bold = beats the baseline ( <1 ). A covariance penalty suffices in the linear regime but fails at every strength in the heteroskedastic one, where the condition enters through a conditional variance; adding nHSIC recovers a donor-specific edit.
Figure 2: Donor-holdout counterfactual on the matched linear / heteroskedastic simulation pair. Rows: (A) linear regime with a covariance penalty, (B) heteroskedastic regime with the same covariance penalty, (C) heteroskedastic regime with covariance + nHSIC; all at λ⊥=0.5 . Columns are held-out donors (donor_014, donor_015, donor_020, donor_024), never seen in training. Each panel shows all 4 simulated cell types of that donor: the retained is_ileum = non_ileum cells (blue), the withheld real is_ileum = ileum cells (green), and the model’s counterfactual edit of the blue cells (red). Rows B and C share one UMAP, fitted jointly on their real cells and both predictions, so they are directly comparable and only the red cloud moves; row A is a different dataset and therefore a different coordinate system. Each group is subsampled to at most 300 cells per donor and cell type, so the three groups are drawn at comparable density. The three groups are drawn in a single shuffled scatter, so neither is systematically painted over the other and a true overlap reads as a blend. Arrows run from each cell type’s real source cloud to the model’s counterfactual for that cloud (centroids), i.e. the edit the model actually made; the edit succeeds when the arrow head lands on the green held-out target. They are a reading guide in UMAP coordinates, not a displacement in any metric space. The gray backdrop is the real cells of all 20 held-out donors (20,000 of 32,769 drawn), so each gray blob is another donor. Panels are framed, and tinted by regime – rows B and C share both an embedding and a tint, row A neither. Panel labels give that donor’s median rMMD over the cell types scored for it (a cell type is scored only where the donor has enough cells on both sides); run-level medians over all 20 held-out donors: (A) 0.09, (B) 1.30, (C) 0.15 (lower is better; <1 beats mapping the cells to the closest existing population).
Orthogonality penalty
λ⊥
rMMD ↓
rSinkhorn ↓
Covariance
0.5
0.95
0.50
Covariance
10
0.96
0.46
Covariance + nHSIC
10
0.41
0.14
Table 2: Held-out counterfactual accuracy on the taurus stem holdout (Rectum → terminal ileum, stem_LGR5pos ). rMMD is the ratio metric of Eq. 20 : <1 means the edited cells are closer to the real held-out ileal cells than the closest existing cell population is. All three runs are below 1 , but the covariance-only numerators ( 0.231 and 0.234 ) are essentially the denominator ( 0.244 ), i.e. the prediction is no closer to the target than the un-edited source it was made from; only cov + nHSIC (bold) makes a complete edit (edit ratio 0.67 vs. 0.04 and 0.22 ).
Figure 3: Counterfactual edit of the held-out stem_LGR5pos subtype (rectum → terminal ileum) under three orthogonality penalties. Every is_ileum = ileum stem_LGR5pos cell ( n =4,100, green) was withheld from training; at inference the retained rectal cells of the same subtype ( n =6,342, blue) are edited to ileum (red) and compared against them. The gray backdrop is the rest of the epithelial lineage, shaded by fine subtype (largest subtype lightest); a uniform random sample of 30,000 of the 317,976 cells is drawn, to keep the UMAP tractable; sampling is uniform, so subtype proportions are preserved.
Table 3: Cell-type concept: mapping of the 22 major values onto seven lineages.
Symbol
Meaning
Value
N
cells
240,000
F
genes
3,000
B
donors (20 held out)
150
T
tissues ( is_ileum )
2
U
cell types
4
C
latent concepts
3
Table 4: Generator settings for the matched linear / heteroskedastic pair.
taurus (Rectum → ileum, stem)
Synthetic pair
cov baselines
selected
grid
Orthogonality penalty
cov
cov + nHSIC
cov / nHSIC / cov + nHSIC
λ⊥
0.5 , 10
10
0.5 , 1 , 5
Class-balanced concepts
is_ileum
is_ileum
none
Patience
50
25
50
Gradient clipping
–
–
1.0
Table 5: Training configurations. All models share one architecture and optimizer; the runs differ only in the orthogonality penalty and its weight (plus the stopping patience and gradient clipping noted above).
Division of Computational Genomics and Systems Genetics, German Cancer Research Center (DKFZ), Heidelberg, Germany · Helmholtz Information & Data Science School for Health, Germany · Genome Biology Unit, European Molecular Biology Laboratory, Heidelberg, Germany +4