Differentiable Structure Learning for Cyclic Linear Gaussian Models with Latent Confounders
Organizations: School of Computer and Communication Sciences, EPFL, Lausanne, Switzerland · Leiden Institute of Advanced Computer Science (LIACS), Leiden University, Leiden, The Netherlands · College of Management of Technology, EPFL, Lausanne, Switzerland
Abstract
We study causal structure learning from observational data in linear Gaussian structural causal models in the presence of directed cycles and an unknown number of exogenous latent confounders, bounded by a given maximum. We derive the covariance of the observed variables and introduce marginal quasi-equivalence, which characterizes when different causal models share a full-dimensional subset of the observational distributions they can generate. We formulate structure learning as minimization of the Gaussian negative log-likelihood with a logarithmically scaled complexity penalty that counts directed edges and latent variables. For a fixed number of observed variables and a fixed upper bound on latent variables, we establish consistency of global score minimizers up to marginal quasi-equivalence under algebraic faithfulness, structural minimality, and model-overlap assumptions. We parameterize the inclusion of directed edges and candidate latent variables using Bernoulli gates, whose continuous probabilities are optimized jointly with the structural coefficients. Averaging the penalized negative log-likelihood over these gates yields an objective with a closed-form differentiable complexity penalty. We prove that this expected objective has the same global infimum as the corresponding discrete structure-learning objective. Experimental results show that our approach achieves lower recovery error than previous methods in several experimental settings.
Figures & tables
| Latents | Method | Time (s) | |||
|---|---|---|---|---|---|
| 20 | Ghassami et al. (2020) | ||||
| DCCD-CONF | |||||
| Amendola et al. (2020) | |||||
| Ours | |||||
| 30 | Ghassami et al. (2020) | ||||
| DCCD-CONF |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| Setting | Value |
|---|---|
| Structure optimization | |
| Mask samples per update | |
| Adam learning rate | |
| Binary Concrete temperature | (fixed) |
| Structure-search initializations | per trial |
| Initial observed-edge probability | |