cs.LGJul 13, 2026

DAG-FM: A Foundation Model for Causal Discovery under Heterogeneous Causal Mechanisms

Authors: Yikang ChenZhengkang GuanHaoyuan QianPeng CuiYi YangKun Kuang

Organizations: 1Zhejiang University · 2Tsinghua University

Abstract

Causal discovery from observational tabular data remains fundamentally challenging, primarily due to the heterogeneity of underlying causal mechanisms and the high-dimensional combinatorial search space of Directed Acyclic Graphs (DAGs). In this paper, we propose \textbf{DAG-FM}, a novel foundation model architecture that amortizes causal discovery. Unlike direct matrix prediction, DAG-FM decomposes the causal discovery process into two auto-regressive stages using two specialized Transformer-based sub-modules: a leaf-node predictor and a parent-node predictor. To effectively model complex row-column interactions, we adopt a robust tabular interaction block to output feature-wise representations. Crucially, to handle diverse and unknown Functional Causal Model (FCM) assumptions in real-world scenarios, we introduce Mixture-of-Leaf-Experts (MoLE), allowing the model to dynamically route and adapt to identifiable mechanism families. Through an iterative inference algorithm, DAG-FM seamlessly extracts causal orderings and constructs valid DAGs. Extensive experiments demonstrate that DAG-FM achieves state-of-the-art performance on both synthetic benchmarks and complex real-world datasets, significantly outperforming traditional classical algorithms and recent foundation models in both accuracy and scalability.

Explore similar work

CardsList
  1. Arrow: A Foundation Model for Causal Discovery

    May 8, 2026Ryan Thompson, He Zhao, Daniel M. Steinberg +1Causal Foundation ModelsCausal Attention

  2. FoundCause: Causal Discovery with Latent Confounders from Observational Data

    Jun 16, 2026Patrick Blöbaum, Krishnakumar Balasubramanian, Shiva Prasad KasiviswanathanCausal Foundation ModelsStructural Causal Model