cs.LGOct 9, 2026

Learning Non-Gaussian SEMs with Conditional Risk, and Why the Direction of Recursion Matters

Authors: Ujas Shah, Panagiota Birmpa, Rebecca Morrison

Organizations: University of Colorado Boulder · Heriot-Watt University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Variance-based methods recover a Gaussian linear structural equation model (SEM) with equal or suitably constrained error variances, either by recursively identifying the source nodes with smallest conditional variance (top-down method) or sink nodes with highest conditional variance (bottom-up method). We show that the use of conditional variance is incidental: replacing it with the conditional risk, i.e., the minimal expected loss of linear prediction under the negative log-likelihood of the noise distribution, extends both methods to non-Gaussian noise. Existing methods are the least-squares special case. This yields two main results. First, exploiting the noise distribution improves structure recovery: the matched-loss procedures recover the SEM more accurately than their least-squares counterparts and the LiNGAM methods, which are designed for non-Gaussianity. Second, the two methods, interchangeable in the Gaussian setting, separate under non-Gaussian noise: the top-down method is correct for every noise density with finite differential entropy, whereas the bottom-up method requires a regularity condition and can fail without it, for instance under mean-zero exponential or Cauchy noise. Simulations confirm both results.

Figures & tables

Figure 1: Representative classes of methods that recover the true DAG of a linear SEM, by noise distribution (rows) and noise-scale assumption (columns). The list is not exhaustive. The bottom-right assumptions are special cases of both neighboring cells, so their methods also apply there, provided the required moments exist.
Noise distributionρf(u)=−log⁡f(u)\rho_{f}(u)=-\log f(u)
Standard Normalu22+C\displaystyle\frac{u^{2}}{2}+C
Mean-zero Laplace with scale ss∣u∣s+log⁡(2s)\displaystyle\frac{|u|}{s}+\log(2s)
Student- tνt_{\nu}ν+12log⁡ ⁣(1+u2ν)+C\displaystyle\frac{\nu+1}{2}\log\!\left(1+\frac{u^{2}}{\nu}\right)+C
Mean-zero unit exponential{u+1,u≥−1,+∞,u<−1\displaystyle\begin{cases}u+1,&u\geq-1,\\ +\infty,&u<-1\end{cases}
Table 1: Noise distributions and their corresponding matched losses.
Figure 2: Structural Hamming distance between the true DAG and the learned DAG for different noise distributions, averaged over 50 replications of an ER-4 random graph. LS-BU and LS-TD are the existing least-squares bottom-up and top-down approaches. ρ\rho -BU and ρ\rho -TD refer to our matched-loss methods. ρ\rho -DirectLiNGAM uses knowledge of the noise density for both steps of the method. Error bars represent 95% confidence intervals.
Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Structural Hamming distance between the true DAG and the learned DAG for different noise distributions, averaged over 50 replications of the ER-2 random graph. Error bars represent 95% confidence intervals.
Figure 4: Structural Hamming distance between the true DAG and the learned DAG for different noise distributions, averaged over 50 replications of the SF-2 random graph. Error bars represent 95% confidence intervals.
Figure 5: Structural Hamming distance between the true DAG and the learned DAG for different noise distributions, averaged over 50 replications of the SF-4 random graph. Error bars represent 95% confidence intervals.
More in this topic

Causal Structure Learning

Causal structure learning infers directed relationships among variables from observational, interventional, or time-series data.

See all 124 papers

Explore similar work

CardsList
Sep 25, 2026

MARCEDES: Score-based causal discovery under non-Gaussianity with continuous optimization

We consider the problem of learning the underlying causal directed acyclic graph (DAG) structure corresponding to a structural equation model (SEM) with non-Gaussian errors. Motivated by an intentionally misspecified non-Gaussian SEM with all Laplace errors, we first introduce the mean absolute residual risk, defined over the space of all real matrices, and show that, asymptotically, the risk of the true weighted causal DAG matrix is strictly smaller than that of any other matrix. Nevertheless, to enhance generality and account for high-dimensional and finite-sample settings, we further incorporate row-specific sparsity penalties along with a soft DAG constraint to derive a continuous score function over the space of real matrices. Accordingly, we propose a score-based DAG learning method, named MARCEDES, formulated as an unconstrained score minimization problem, which can be efficiently solved using gradient-based optimization techniques, thereby circumventing the challenges associated with constrained optimization. Furthermore, we develop a computational algorithm to handle the non-smoothness of the score objective and to enable optimal tuning of row-specific sparsity penalties under a generalized Bayes framework. Finally, we demonstrate the efficiency and improved performance of the proposed method over existing approaches through an extensive simulation study.

Anamitra Chaudhuri, Anirban Bhattacharya, Yang Ni

Department of Statistics Florida State University Tallahassee, FL 32306 · Department of Statistics The University of Texas at Austin · Department of Statistics Texas A&M University College Station, TX 77843 +1

Causal DiscoveryStructural Causal Models
AskPDF
Aug 17, 2026

Causal Discovery in Equal Variance Linear Gaussian DAGs via SURE-Tuned Ridge Regression

Recovering the directed acyclic graph (DAG) of a structural equation model (SEM) from observational data is a central problem in causal discovery. The iterative gradient descent and per-problem hyperparameter tuning of continuous-optimization methods are poorly suited to two practically important regimes: the sample-limited regime, where the number of samples is comparable to or smaller than the number of nodes in the DAG, and the compute-limited regime. This work proposes SURE-Ridge, a non-iterative, closed-form estimator for equal variance linear Gaussian SEM. The method performs parallel node-wise regressions with regularization parameters chosen adaptively by Stein's unbiased risk estimate (SURE), and applies an adaptive thresholding procedure to extract a DAG from the resulting soft adjacency matrix. Numerical results show that SURE-Ridge achieves the lowest structural Hamming distance in the small-sample regime and the lowest run time across all sample sizes tested, compared with NOTEARS, DAGMA, and GBNSL baselines.

Sambit Mishra, Urbashi Mitra

Department of Electrical and Computer Engineering University of Southern California

Causal DiscoveryCausal Structure LearningRidge Regression
AskPDF
Nov 19, 2025

Recursive Entropic Variational Inference for Nonlinear State-Space Models

We present a class of algorithms for state estimation in nonlinear, non-Gaussian state-space models. Our approach is based on a variational Lagrangian formulation that casts Bayesian inference as a sequence of entropic trust-region updates subject to dynamic consistency constraints. This framework gives rise to a family of forward-backward algorithms whose structure is determined by the chosen factorization of the variational posterior. By focusing on Gauss--Markov approximations, we derive recursive schemes with favorable computational complexity. For general nonlinear, non-Gaussian models, we close the recursions using generalized statistical linear regression and Fourier--Hermite moment matching.

Hany Abdulsamad, Ángel F. García-Fernández, Simo Särkkä

Amsterdam Machine Learning Lab University of Amsterdam, Netherlands · Information Processing and Telecommunications Center Universidad Politécnica de Madrid, Spain · ELLIS Institute Finland Department of Electrical Engineering and Automation Aalto University, Finland

Bayesian FilteringVariational InferenceState Space Models
AskPDF