Causal discovery from Bayesian symbolic regression
Authors: Roger Guimerà, Kai-Hendrik Cohrs, Sergio Pedreira, Marta Sales-Pardo, Gustau Camps-Valls
Organizations: Department of Chemical Engineering, Universitat Rovira i Virgili, Tarragona, Spain · Center for Computational Science and Applied Mathematics (ComSCIAM), Universitat Rovira i Virgili, Tarragona, Spain · ICREA, Barcelona, Spain · Image Processing Laboratory (IPL), Universitat de València, València, Spain
Causal discovery from observational data is central to scientific reasoning, yet established constraint-based and score-based methods often recover only a Markov equivalence class and provide no signal in bivariate or strongly autocorrelated regimes typical in some areas of science. Here, we recast causal discovery as a Bayesian symbolic regression problem. Given a set of candidate parents for each target variable, the Bayesian symbolic regression samples closed-form expressions from a posterior over symbolic models. The fraction of sampled expressions in which each candidate variable appears defines a posterior inclusion probability that we use as the causal score. This replaces pointwise model selection by an ensemble of plausible mechanisms. We evaluate the method on coupled nonlinear bivariate dynamics, on the CauseMe benchmark suite spanning linear, chaotic, nonlinear, higher-dimensional and high-noise settings, and on a four-variable biogeoscience structural causal model under Gaussian, heavy-tailed and mixed-noise regimes. Our approach is competitive with, and often outperforms, specialized causal discovery algorithms. Additionally, the method returns closed-form mechanisms leading to explicit causality and interpretability.
Figures & tables
Figure 1: Causal and functional recovery in unidirectionally coupled nonlinear dynamics. (a-d) Sample observed trajectories xt and yt for the system in Eq. ( 7 ). (e-h) Causal scores πk→i(τ) obtained by causalBMS from Eq. ( 4 ). (i-l) Minimum description length expressions sampled by causalBMS (green) reproduce the true self-influence (black). Grey points are the N=2000 observations. Equation labels in the legend are the closed-form expressions inferred by the chain. We consider four noise regimes: low noise ( σ=0.04 , first column); moderate noise ( σ=0.4 , second column); high noise ( σ=4 , third column); very high noise ( σ=10 , fourth column).
Figure 2: Performance of causalBMS on the CauseMe benchmark. For the experiments described in § III.2 , we show: (a) area under the ROC curve; (b) area under the precision–recall curve; and (c) F1-score. The green bars are the causalBMS scores; the grey ranges span the top-20 entries reported by CauseMe for each dataset. causalBMS sits at or near the top of every column and consistently falls within the upper range of competing methods, especially for the nonlinear and strong-noise variants.
Figure 3: Multi-method comparison on the biogeoscience SCM of Eq. ( 8 ). F1-score (left), AUC-PR (middle), and AUC-ROC (right) on the cross-influence entries of the lag-one summary graph, averaged over twenty replicates per noise regime; error bars are one standard deviation. Regimes (left to right inside each panel): Gaussian fluctuations at the reference scale, Gaussian fluctuations at twice the reference scale, Student- t fluctuations with ν=4 degrees of freedom and a mixed regime with an additional cubic term on the Tair fluctuations. causalBMS (green) is among the leading methods for F1-score and AUC-ROC and often ties for the best performer on these metrics. CCM (yellow) is the strongest method for AUC-PR throughout. In AUC-ROC, a linear Granger-causality test extended with contemporaneous terms (purple) matches or slightly exceeds every other method, including causalBMS, in all four regimes. VAR-LiNGAM (red) is consistently the weakest method in AUC-ROC and AUC-PR, while PCMCI + (blue) shows more erratic performance across noise regimes in F1-score.
Table 1: Equations recovered by causalBMS on the biogeoscience SCM. For each target, we show the ground truth model and the MDL expression identified by the causalBMS in the reference Gaussian-noise regime.
Figure 4: Properties of minimum description length models identified by the causalBMS for the biogeoscience model of Eq. ( 8 ). After obtaining the MDL model for each target with the causalBMS, we generate test data with the same SCM model of Eq. ( 8 ) and test the predictive abilities and correlation structure of the models. (a-c) Predicted versus observed values of the target variables for the MDL model (green) and the ground-truth model (grey). (d-f) Correlations between residuals and predictors for each target variable and each predictor identified by the MDL model. We report the BH-corrected p -values for a permutation test with HSIC assessing remaining correlation between residuals and chosen predictors.
Computer Science Dept., Yale University, New Haven, Connecticut, USA · Computer Science Dept., University of California, Los Angeles, Los Angeles, California, USA