MARCEDES: Score-based causal discovery under non-Gaussianity with continuous optimization
Authors: Anamitra Chaudhuri, Anirban Bhattacharya, Yang Ni
Organizations: Department of Statistics Florida State University Tallahassee, FL 32306 · Department of Statistics The University of Texas at Austin · Department of Statistics Texas A&M University College Station, TX 77843 · Department of Statistics and Data Science The University of Texas at Austin Austin, TX 78705
We consider the problem of learning the underlying causal directed acyclic graph (DAG) structure corresponding to a structural equation model (SEM) with non-Gaussian errors. Motivated by an intentionally misspecified non-Gaussian SEM with all Laplace errors, we first introduce the mean absolute residual risk, defined over the space of all real matrices, and show that, asymptotically, the risk of the true weighted causal DAG matrix is strictly smaller than that of any other matrix. Nevertheless, to enhance generality and account for high-dimensional and finite-sample settings, we further incorporate row-specific sparsity penalties along with a soft DAG constraint to derive a continuous score function over the space of real matrices. Accordingly, we propose a score-based DAG learning method, named MARCEDES, formulated as an unconstrained score minimization problem, which can be efficiently solved using gradient-based optimization techniques, thereby circumventing the challenges associated with constrained optimization. Furthermore, we develop a computational algorithm to handle the non-smoothness of the score objective and to enable optimal tuning of row-specific sparsity penalties under a generalized Bayes framework. Finally, we demonstrate the efficiency and improved performance of the proposed method over existing approaches through an extensive simulation study.
Figures & tables
Figure 1: Performance comparison of different in terms of overall structure recovery.
Figure 2: Performance comparison of different in terms of edge recovery.
Recovering the directed acyclic graph (DAG) of a structural equation model (SEM) from observational data is a central problem in causal discovery. The iterative gradient descent and per-problem hyperparameter tuning of continuous-optimization methods are poorly suited to two practically important regimes: the sample-limited regime, where the number of samples is comparable to or smaller than the number of nodes in the DAG, and the compute-limited regime. This work proposes SURE-Ridge, a non-iterative, closed-form estimator for equal variance linear Gaussian SEM. The method performs parallel node-wise regressions with regularization parameters chosen adaptively by Stein's unbiased risk estimate (SURE), and applies an adaptive thresholding procedure to extract a DAG from the resulting soft adjacency matrix. Numerical results show that SURE-Ridge achieves the lowest structural Hamming distance in the small-sample regime and the lowest run time across all sample sizes tested, compared with NOTEARS, DAGMA, and GBNSL baselines.
Sambit Mishra, Urbashi Mitra
Department of Electrical and Computer Engineering University of Southern California
Continuous optimization methods for learning Directed Acyclic Graphs (DAGs) operate on weighted adjacency matrices and are therefore limited to pairwise causal relationships. We propose a framework for learning Directed Acyclic Hypergraphs (DAHGs) from observational data, capturing joint parental influences that pairwise models cannot represent. Our approach rests on three components: (i) a generalized linear structural equation model (SEM) with multiplicative interaction terms whose non-zero weights correspond one-to-one with directed hyperedges; (ii) a weighted adjacency tensor representation whose acyclicity is characterized via nilpotency under the tensor t-product; and (iii) a differentiable acyclicity constraint derived through the Fourier decomposition of the t-product, which reduces tensor nilpotency to slice-wise matrix nilpotency and enables least-squares learning via the augmented Lagrangian method.
Zhiyuan Dong, Carlos Mundo-Levano, Wei Qian +2
Institute for Financial Services Analytics, University of Delaware, Newark, DE 19716, USA · Department of Electrical and Computer Engineering, University of Delaware, Newark, DE 19716, USA · Department of Applied Economics AND Statistics, University of Delaware, Newark, DE 19716, USA +1
This work addresses the problem of learning directed acyclic graphs (DAGs) from nodal observations adhering to a linear structural equation model. DAG structure learning is a central task in signal processing, machine learning, and causal inference, but it remains challenging because acyclicity is a global combinatorial property and the search space grows superexponentially in the number of nodes. Continuous acyclicity characterizations have led to important algorithmic advances by replacing the discrete DAG restriction with smooth equality constraints on the candidate adjacency matrix. However, existing formulations still involve difficult non-convex optimization landscapes and may suffer from degenerate first-order optimality conditions. Here, we restrict attention to DAGs with non-negative edge weights and exploit this additional structure to obtain a simpler acyclicity characterization devoid of Hadamard products. We propose a new non-negative constrained DAG estimator and develop a tractable, locally convergent algorithm based on the method of multipliers. We further analyze the benign optimization landscape induced by the non-negativity constraints. In the large-sample regime, we show that the true DAG is the unique global minimizer of the population augmented Lagrangian. Moreover, the landscape contains no spurious interior stationary points, and the true DAG is the only acyclic Karush--Kuhn--Tucker (KKT) point. Critically, this allows us to guarantee local convergence to the true DAG given infinite samples. Comprehensive numerical experiments show that the proposed structure identification method improves over state-of-the-art continuous DAG-learning baselines.
Samuel Rey, Madeline navarro, Santiago Segarra +1
Dept. of Electrical and Computer Engineering, University of Rochester, Rochester, NY 14627, USA