KG-SoftMAP: Soft Knowledge-Graph Priors for Bayesian Network Structure Learning from Sparse Discrete Data

Guoliang Xu; James E. Corter

cs.LGJun 9, 2026

KG-SoftMAP: Soft Knowledge-Graph Priors for Bayesian Network Structure Learning from Sparse Discrete Data

Authors: Guoliang Xu, James E. Corter

Organizations: Columbia University

Abstract

Learning Bayesian network (BN) structure from sparse discrete data is hard: when each instance records only a few variables, most variable pairs lack the joint observations needed for reliable scoring, and data-only methods recover little structure. Imperfect domain knowledge, expressible as a weighted directed knowledge graph (KG), is often available. KG-SoftMAP encodes such a KG as a finite-strength, confidence-weighted edge prior and maximizes a MAP objective that adds this logit-form prior to the BDeu score. With an informative but imperfect KG, KG-SoftMAP recovers partial directed structure even at observation rate rho=0.05, with directed F1 (DF1) of 0.19-0.32 across benchmarks. At higher observation rates within this sparse grid, DF1 reaches 0.44-0.66 at rho=0.20 and 0.46-0.64 at rho=0.40. Across the same three rates, KG-SoftMAP without the KG prior averages DF1 0.00, 0.19, and 0.21. Stress tests that corrupt, remove, or blur the KG signal, together with checks on LLM-extracted graphs beyond canonical benchmarks, show that recovery rises and falls with KG quality. On three real sparse educational datasets without ground-truth DAGs, we evaluate prediction, calibration, and KG-consistency. On Short Answer Feedback (SAF), KG-SoftMAP+VE reaches Fail-class F1 0.75 versus 0.78 for logistic regression while also providing an inspectable concept graph, calibrated Fail probabilities, and posterior queries from partially observed concept evidence. The remaining datasets sharpen the operating picture: weak heuristic KG signal leaves prediction unchanged, while an independent expert ontology moves the learned graph toward expert relatedness.

Explore similar work

Jun 4, 2026cs.LG

Causal Atlases from Entropic Inference: Bayesian Networks beyond Optimal DAGs

Data-driven causal relationship identification is pertinent to advancing understanding of complex systems both within and beyond science. Bayesian networks offer a probabilistic method for modelling generic causal relationships via directed acyclic graphs (DAGs). However, typical techniques for constructing Bayesian networks rely on optimization, which can be ill-suited for learning causal relationships because the underlying data may admit multiple chains of causation. More data-faithful representations of causal relationships would provide frameworks for constructing multiple causal maps that are consistent with the variability that is inherent in underlying data. Here, we show that entropy-based inference generates atlases of plausible causal relationships that are consistent with underlying data. On simulated noisy data of 2- and 20-node linear structural equation models, we sample a maximum-entropy ensemble of graphs that allow us to quantify the inherent structural ambiguity in underlying causal relationships. Our method shows that "optimized" DAGs can contain causal artifacts are not consistent across equivalently accurate topologies.

Hazhir Aliahmadi, Irina Babayan, Greg van Anders

Bayesian Networks Directed Acyclic Graph Causal Reasoning

Ask PDF

Aug 5, 2026cs.LG

SVI-DAG: A Structured Variational Inference Approach to Bayesian Causal Discovery

Bayesian causal discovery seeks to determine the posterior distribution of causal theories, which are interpreted as directed acyclic graphs (DAGs) that explain the observed data. The resulting posterior allows systematic reasoning regarding epistemic uncertainty within these theories. Nonetheless, finding such graphs is difficult due to identifiability problems and limited observational data. Furthermore, precisely approximating posterior over graphs is challenging given vast range of potential DAGs. Recent Bayesian approaches have addressed some of these challenges, yet they remain limited as they fail to encode dependencies between edges, and lack principled ways to incorporate domain knowledge as inductive biases during the search process. To overcome these limitations, we propose SVI-DAG, a structured variational inference approach to Bayesian causal discovery using observational data and prior beliefs that uses normalizing flows to model dependencies between edges, supporting expressive and multimodal posterior learning over DAGs. To mitigate mode seeking behaviour in evidence lower bound optimization and promote mode coverage, we use stein variational gradient descent to update the node potentials using a kernel in acyclicity space. We evaluate SVI-DAG against 5 state-of-the-art Bayesian DAG learning methods and demonstrate superior performance in uncertainty quantification while remaining competitive in terms of structural accuracy.

Shrenik Zinage

Causal Foundation Models Bayesian Networks Variational Inference Epistemic Uncertainty

Ask PDF

May 5, 2026stat.ML

Heterogeneous Ordinal Structure Learning with Bayesian Nonparametric Complexity Discovery

Public attitudes toward artificial intelligence are heterogeneous, ordinally measured, and poorly captured by any single dependency graph. Existing ordinal structure learners assume a shared directed acyclic graph (DAG) across all respondents; recent heterogeneous ordinal graphical-model approaches focus on subgroup discovery rather than confirmatory cluster-specific DAG estimation; and latent profile analyses discard dependency structure entirely. We introduce a heterogeneous ordinal structure-learning framework combining monotone Gaussian score embedding, Bayesian nonparametric (BNP) complexity discovery via a truncated stick-breaking prior, and confirmatory fixed-K estimation with cluster-specific sparse DAG learning. The key methodological insight is a discovery-to-confirmation workflow: the nonparametric stage calibrates plausible archetype complexity, while inner-validated confirmatory refitting yields stable, interpretable structural estimates. On the 2024 Pew American Trends Panel AI attitudes survey, Wave 152 (W152) survey, (N = 4,788, 8 ordinal items), the confirmatory K*=5 model reduces holdout transformed-score mean squared error (MSE) by 25.8% over a single-graph baseline and by 4.6% over mixture-only clustering. A controlled tiered semi-synthetic benchmark calibrated to W152 structure validates recovery across difficulty regimes and transparently reveals failure modes under stress conditions.

Amir Rafe, Subasish Das

Structural Equation Models Directed Acyclic Graph Survey

Ask PDF