Conditional Counterfactual Mean Embeddings: Doubly Robust Estimation and Learning Rates
Organizations: Department of Statistics, Chiang Mai University
Abstract
A complete understanding of heterogeneous treatment effects requires fully capturing the conditional distribution of the counterfactual outcomes. To this end, we propose the Conditional Counterfactual Mean Embeddings (CCME), a framework that embeds conditional distributions of counterfactual outcomes into a reproducing kernel Hilbert space (RKHS). Under this framework, we develop a two-stage meta-estimator for CCME that accommodates any RKHS-valued regression in each stage. Based on this meta-estimator, we develop three practical CCME estimators: (1) Ridge Regression estimator, (2) Deep Feature estimator that parameterizes the feature map by a neural network, and (3) Neural-Kernel estimator that performs RKHS-valued regression, with the coefficients parameterized by a neural network. We provide finite-sample convergence rates for all estimators, establishing that they possess the double robustness property. Our experiments demonstrate that our estimators accurately recover distributional features including multimodal structure of conditional counterfactual distributions. Our code is available at https://github.com/donlap/Conditional-Counterfactual-Mean-Embeddings.
Figures & tables
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| Hyperparameter | Ridge Regression | Deep Feature | Neural-Kernel |
|---|---|---|---|
| Model Parameters | |||
| Kernel Bandwidth ( ) | 2.0 | 2.0 | 2.0 |
| Regularization ( ) | 20.0 | 20.0 | N/A |
| Grid/Feature Dimension ( ) | N/A | 20 | 20 |
| Network Architecture | N/A | 2 layers 20 units, ReLU | |
| Training (SGD Optimizer) | |||
| Hyperparameter | Deep Feature Estimator |
|---|---|
| Feature Dimension ( ) | 1000 |
| Network Architecture | 2 layers 100 units, ReLU |
| Kernel Bandwidth ( ) | 1.0 |
| Regularization ( ) | 1.0 |
| Training (SGD Optimizer) | |
| Learning Rate (Stage 1) | |
| Test | Scenario 1 | Scenario 2 | Scenario 3 | Scenario 4 |
|---|---|---|---|---|
| CCME-DR | ||||
| One-Step [ 46 ] | ||||
| CPME [ 71 ] | ||||
| CATE-DR |
Kernel Mean Embeddings
Kernel mean embeddings map probability distributions into reproducing kernel Hilbert spaces for comparison, estimation, and learning.
See all 12 papers