Causal discovery from Bayesian symbolic regression
Authors: Roger Guimerà, Kai-Hendrik Cohrs, Sergio Pedreira, Marta Sales-Pardo, Gustau Camps-Valls
Organizations: Department of Chemical Engineering, Universitat Rovira i Virgili, Tarragona, Spain · Center for Computational Science and Applied Mathematics (ComSCIAM), Universitat Rovira i Virgili, Tarragona, Spain · ICREA, Barcelona, Spain · Image Processing Laboratory (IPL), Universitat de València, València, Spain
Causal discovery from observational data is central to scientific reasoning, yet established constraint-based and score-based methods often recover only a Markov equivalence class and provide no signal in bivariate or strongly autocorrelated regimes typical in some areas of science. Here, we recast causal discovery as a Bayesian symbolic regression problem. Given a set of candidate parents for each target variable, the Bayesian symbolic regression samples closed-form expressions from a posterior over symbolic models. The fraction of sampled expressions in which each candidate variable appears defines a posterior inclusion probability that we use as the causal score. This replaces pointwise model selection by an ensemble of plausible mechanisms. We evaluate the method on coupled nonlinear bivariate dynamics, on the CauseMe benchmark suite spanning linear, chaotic, nonlinear, higher-dimensional and high-noise settings, and on a four-variable biogeoscience structural causal model under Gaussian, heavy-tailed and mixed-noise regimes. Our approach is competitive with, and often outperforms, specialized causal discovery algorithms. Additionally, the method returns closed-form mechanisms leading to explicit causality and interpretability.
Figures & tables
Figure 1: Causal and functional recovery in unidirectionally coupled nonlinear dynamics. (a-d) Sample observed trajectories xt and yt for the system in Eq. ( 7 ). (e-h) Causal scores πk→i(τ) obtained by causalBMS from Eq. ( 4 ). (i-l) Minimum description length expressions sampled by causalBMS (green) reproduce the true self-influence (black). Grey points are the N=2000 observations. Equation labels in the legend are the closed-form expressions inferred by the chain. We consider four noise regimes: low noise ( σ=0.04 , first column); moderate noise ( σ=0.4 , second column); high noise ( σ=4 , third column); very high noise ( σ=10 , fourth column).
Figure 2: Performance of causalBMS on the CauseMe benchmark. For the experiments described in § III.2 , we show: (a) area under the ROC curve; (b) area under the precision–recall curve; and (c) F1-score. The green bars are the causalBMS scores; the grey ranges span the top-20 entries reported by CauseMe for each dataset. causalBMS sits at or near the top of every column and consistently falls within the upper range of competing methods, especially for the nonlinear and strong-noise variants.
Figure 3: Multi-method comparison on the biogeoscience SCM of Eq. ( 8 ). F1-score (left), AUC-PR (middle), and AUC-ROC (right) on the cross-influence entries of the lag-one summary graph, averaged over twenty replicates per noise regime; error bars are one standard deviation. Regimes (left to right inside each panel): Gaussian fluctuations at the reference scale, Gaussian fluctuations at twice the reference scale, Student- t fluctuations with ν=4 degrees of freedom and a mixed regime with an additional cubic term on the Tair fluctuations. causalBMS (green) is among the leading methods for F1-score and AUC-ROC and often ties for the best performer on these metrics. CCM (yellow) is the strongest method for AUC-PR throughout. In AUC-ROC, a linear Granger-causality test extended with contemporaneous terms (purple) matches or slightly exceeds every other method, including causalBMS, in all four regimes. VAR-LiNGAM (red) is consistently the weakest method in AUC-ROC and AUC-PR, while PCMCI + (blue) shows more erratic performance across noise regimes in F1-score.
Table 1: Equations recovered by causalBMS on the biogeoscience SCM. For each target, we show the ground truth model and the MDL expression identified by the causalBMS in the reference Gaussian-noise regime.
Figure 4: Properties of minimum description length models identified by the causalBMS for the biogeoscience model of Eq. ( 8 ). After obtaining the MDL model for each target with the causalBMS, we generate test data with the same SCM model of Eq. ( 8 ) and test the predictive abilities and correlation structure of the models. (a-c) Predicted versus observed values of the target variables for the MDL model (green) and the ground-truth model (grey). (d-f) Correlations between residuals and predictors for each target variable and each predictor identified by the MDL model. We report the BH-corrected p -values for a permutation test with HSIC assessing remaining correlation between residuals and chosen predictors.
Causal discovery aims to uncover the underlying causal relationships given data generated from a system. The goal, however, is not merely to predict causal edges given data, but also to be able to interpret and explain either observed or hypothesized phenomena, such as a particularly large causal effect. We consider this task of conditional causal discovery and cast it as a Bayesian inference problem, in which we target the posterior over causal graphs and parameters conditional on an event such as a causal-effect constraint. Unfortunately, this poses a computational challenge: existing approaches to Bayesian causal discovery struggle when the event has small posterior mass. To address this, we adapt rare-event estimation techniques to perform inference the joint graph-parameter space. Our method gradually drives a particle population toward the constrained region while maintaining samples that approximate the conditional posterior. Empirical evaluation on synthetic graphs validates the accuracy of our approach at small and large scales, and we show in a case study on the Sachs protein dataset how our method can be used to aid scientific exploration by providing pathway-level summaries.
Cixuan Zhang, Guy Van den Broeck, Benjie Wang
Computer Science Dept., Yale University, New Haven, Connecticut, USA · Computer Science Dept., University of California, Los Angeles, Los Angeles, California, USA
Causal discovery from observational data is a fundamental yet challenging task in scientific research. While existing approaches are primarily based on conditional independence tests, structure scores, or restrictive functional assumptions, we propose Decoupled Causal Discovery (DCD), a novel decoupling-based perspective that does not rely on these methodologies. DCD directly identifies the Markov boundary (MB) by decoupling non-target variables via weighting functions, such that only variables within the MB preserve dependence with the target under the decoupled distribution. Building on this, DCD iteratively constructs the Completed Partially Directed Acyclic Graph (CPDAG) by exploiting structural asymmetries within the MBs. We establish the theoretical identifiability, soundness, and completeness of DCD. Empirical evaluations demonstrate that DCD achieves strong performance, particularly excelling in challenging noise regimes.
Zhengkang Guan, Fei Wu, Kun Kuang
College of Computer Science and Technology Zhejiang University
Identifying latent variables and the causal structure involving them is essential across various scientific fields. While many existing works fall under the category of constraint-based methods (with e.g. conditional independence or rank deficiency tests), they may face empirical challenges such as testing-order dependency, error propagation, and choosing an appropriate significance level. These issues can potentially be mitigated by properly designed score-based methods, such as Greedy Equivalence Search (GES) (Chickering, 2002) in the specific setting without latent variables. Yet, formulating score-based methods with latent variables is highly challenging. In this work, we develop score-based methods that are capable of identifying causal structures containing causally-related latent variables with identifiability guarantees. Specifically, we show that a properly formulated scoring function can achieve score equivalence and consistency for structure learning of latent variable causal models. We further provide a characterization of the degrees of freedom for the marginal over the observed variables under multiple structural assumptions considered in the literature, and accordingly develop both exact and continuous score-based methods. This offers a unified view of several existing constraint-based methods with different structural assumptions. Experimental results validate the effectiveness of the proposed methods.
Ignavier Ng, Xinshuai Dong, Haoyue Dai +3
Carnegie Mellon University · University of California, San Diego · Mohamed bin Zayed University of Artificial Intelligence