Heteroscedasticity -- where the variance of a variable changes with other variables -- is pervasive in real data, and elucidating why it arises from the perspective of statistical moments is crucial in scientific knowledge discovery and decision-making. However, standard causal discovery does not reveal which causes act on the mean versus the variance, as it returns a single moment-agnostic graph, limiting interpretability and downstream intervention design. We propose a Bayesian, moment-driven causal discovery framework that infers separate mean and variance causal graphs from observational heteroscedastic data. We first derive the identification results by establishing sufficient conditions under which these two graphs are separately identifiable. Building on this theory, we develop a variational inference method that learns a posterior distribution over both graphs, enabling principled uncertainty quantification of structural features (e.g., edges, paths, and subgraphs). To address the challenges of parameter optimization in heteroscedastic models with two graph structures, we take a curvature-aware optimization approach and develop a prior incorporation technique that leverages domain knowledge on node orderings, improving sample efficiency. Experiments on synthetic, semi-synthetic, and real data show that our approach accurately recovers mean and variance structures and outperforms state-of-the-art baselines.
Figures & tables
Figure 1. Local protein regulatory network example represented by (a) Moment-agnostic causal graph and (b) mean and variance causal graphs (shown in red and blue edges)
Figure 2. Mean and variance causal graph inference performance on synthetic and SERGIO datasets with sample size n=500 . Achieving both lower SHD rate (left) and higher F1 score (right) is better.
Figure 3. Mean and variance causal graph inference performance on synthetic datasets ( n=500 ) under dense graphs
d=2
d=3
d=4
TV
KL
TV
KL
TV
KL
MC3
0.01±0.00
0.01±0.00
0.10±0.01
0.09±0.01
0.42±0.10
6.52±0.91
DDS
0.31±0.10
1.94±0.28
0.32±0.08
1.81±0.21
0.45±0.11
5.52±0.73
Proposed
0.26±0.06
1.01±0.46
0.21±0.09
0.34±0.09
0.34±0.08
1.51±0.28
Table 1. Total variation (TV) distance and KL divergence (KL) of posteriors inferred from linear Gaussian ANM datasets ( n=500 ). Achieving both lower TV and KL values is better.
Figure 4. Moment-agnostic causal graph inference performance on non-Gaussian synthetic datasets ( n=500 ) with (a) Laplace noise and (b) student-t noise. Achieving both lower SHD (left) and lower SID (right) is better.
n=100
n=200
n=853
SHD ( ↓ )
F1 ( ↑ )
SHD ( ↓ )
F1 ( ↑ )
SHD ( ↓ )
F1 ( ↑ )
MC3
22.6 ± 1.1
0.19 ± 0.04
20.3 ± 1.3
0.20 ± 0.07
18.9 ± 0.7
0.14 ± 0.05
DDS
17.6 ± 0.7
0.20 ± 0.03
16.6 ± 0.6
0.20 ± 0.02
15.0 ± 0.7
0.28 ± 0.02
ICDH
20.0 ± 1.0
0.21 ± 0.01
17.5 ± 0.5
0.22 ± 0.01
14.5 ± 0.5
0.27 ± 0.03
HOST
16.1 ± 0.5
0.19 ± 0.01
15.0 ± 0.3
0.33 ± 0.02
13.5 ± 0.8
0.38 ± 0.03
Proposed
16.0 ± 0.3
0.20 ± 0.02
14.9 ± 0.4
0.31 ± 0.02
13.7 ± 0.5
0.36 ± 0.02
Table 2. Moment-agnostic graph inference performance on Sachs dataset. ↓ and ↑ denote "lower is better" and "higher is better".
Run time [sec]
DDS
367 ± 11
Proposed
733 ± 19
Proposed +25%
852 ± 38
Proposed +50%
1260 ± 51
Table 3. Run time on Sachs dataset ( n=853 ). Our Proposed needs about twice the run time of DDS due to two-graph inference and can use prior knowledge with small overhead.
n=100
n=200
n=853
Proposed
0.585±0.009
0.592±0.008
0.620±0.019
Table 4. Estimated posterior probability of MEK → ERK in inferred variance causal graph structure
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Mean ( d=5 )
Variance ( d=5 )
Mean ( d=10 )
Variance ( d=10 )
Proposed
2.34±0.20
3.33±0.30
15.5±1.2
19.3±1.1
ICDH
5.05±0.42
4.55±0.40
20.1±1.1
20.9±1.2
HOST
4.84±0.49
4.60±0.45
19.6±1.8
21.6±1.4
VarSort
5.00±1.00
6.00±1.66
18.5±6.2
21.9±4.4
Appendix
Table 5. SHD score comparison with VarSort on mean-variance HNM datasets ( n=500 ) with number of nodes d=5,10
Figure 5. Moment-agnostic causal graph inference performance on nonlinear Gaussian ANM datasets ( n=500 ) with number of nodes d=5,10,20,50 : ↓ and ↑ denote "lower is better" and "higher is better".
Figure 6. Moment-agnostic causal graph inference performance on nonlinear Gaussian HNM datasets ( n=500 ) with number of nodes d=5,10,20,50 : ↓ and ↑ denote "lower is better" and "higher is better".
Bayesian causal discovery seeks to determine the posterior distribution of causal theories, which are interpreted as directed acyclic graphs (DAGs) that explain the observed data. The resulting posterior allows systematic reasoning regarding epistemic uncertainty within these theories. Nonetheless, finding such graphs is difficult due to identifiability problems and limited observational data. Furthermore, precisely approximating posterior over graphs is challenging given vast range of potential DAGs. Recent Bayesian approaches have addressed some of these challenges, yet they remain limited as they fail to encode dependencies between edges, and lack principled ways to incorporate domain knowledge as inductive biases during the search process. To overcome these limitations, we propose SVI-DAG, a structured variational inference approach to Bayesian causal discovery using observational data and prior beliefs that uses normalizing flows to model dependencies between edges, supporting expressive and multimodal posterior learning over DAGs. To mitigate mode seeking behaviour in evidence lower bound optimization and promote mode coverage, we use stein variational gradient descent to update the node potentials using a kernel in acyclicity space. We evaluate SVI-DAG against 5 state-of-the-art Bayesian DAG learning methods and demonstrate superior performance in uncertainty quantification while remaining competitive in terms of structural accuracy.
Shrenik Zinage
Massachusetts Institute of Technology · This work was conducted while the author was a PhD student at Purdue University, independently of the author’s dissertation research.
Causal discovery seeks to uncover the causal dependencies among variables. For this purpose, we propose an algorithm called Tensor-based Second-order Causal Discovery (TSCD). Its input is a tensor obtained from the covariance matrices of observational and interventional data. Assuming the causal dependencies follow a linear structural equation model on a directed acyclic graph (DAG), TSCD outputs the DAG and the functions on its edges, requiring only that the noise variables are uncorrelated. We also implement a version of the approach for nonlinear models. Our focus on second-order statistics (via the covariance matrices) is motivated by their statistical and computational efficiency relative to higher-order moments, their identifiability relative to first-order statistics, and that they work regardless of whether the variables are Gaussian. We show that TSCD has identifiable causal order and parameters from a number of interventions that is logarithmic in the number of variables. Experiments show that TSCD is robust to noise, competitive with existing methods, and scales to hundreds of variables.
Causal discovery aims to understand the relationships between individual random variables. In many applications, such as brain imaging and climate modeling, it is more meaningful to consider interactions among groups of variables. Existing methods assume that knowledge of such groups or clusters is explicitly available when modeling interactions. However, in practice, these clusters as well as the causal relationships among them, are latent. In this paper, we present a novel approach based on variational inference to simultaneously infer both the latent clusters and causal structures. We learn an approximate posterior over clusters and graph-structure by considering variational distributions based on categorical and Bernoulli models respectively. We derive variational lower bounds and estimation techniques to learn variational and model parameters. The effectiveness of our proposed methods for cluster and causal discovery are demonstrated on both synthetic and real data sets.
Avni Rajpal, Anubhav Kumar, Rishabh Karnad +2
Department of Artificial Intelligence, Indian Institute of Technology Hyderabad, India · RIKEN Center for AI Project, Tokyo, Japan