PRISM-BN: A Controlled Corpus and Benchmark for Text-to-Parameterized Bayesian Network Extraction

Amartya Bhattacharya; Nikhil Singh; Neeti Pokhriyal; Soroush Vosoughi

cs.CLSep 18, 2026

PRISM-BN: A Controlled Corpus and Benchmark for Text-to-Parameterized Bayesian Network Extraction

Authors: Amartya Bhattacharya, Nikhil Singh, Neeti Pokhriyal, Soroush Vosoughi

Organizations: Dartmouth College · RAND Corporation

Abstract

Probabilistic Graphical Models (PGMs), especially Bayesian Networks (BNs), expose directed structure and probabilistic parameters, making them natural symbolic targets for neurosymbolic AI. Yet training text-to-parameterized-BN systems requires paired text-to-BN resources unavailable at scale. We introduce PRISM-BN, a controlled corpus of 5054 BN-grounded descriptions paired with discrete reference BNs containing variables, states, directed edges, root priors, and full multi-parent CPDs across five domains. The instances are derived from 50 Wikipedia-seeded backbones, and their probabilities are internally constructed benchmark targets rather than externally validated causal estimates. PRISM-BN is built with PRISM, a marginal-first pipeline that elicits marginal and local joint distributions, analytically recovers normalized CPDs, and constructs locally reparameterized subgraphs. We define a benchmark with semantic node and state alignment, conditional structural scoring, and strict full-CPD evaluation. Across six LLM extractors, Node F1 ranges from 0.56 to 0.83, conditional Edge F1 from 0.90 to 0.97, and CPD-KL from 1.11 to 3.14. Conditional state and edge recovery remain consistently strong, whereas strict full-CPD agreement remains challenging. These trends persist with independently generated GPT-5.5 references, and a human pilot corroborates structural recoverability and similar probabilistic interpretations. PRISM-BN supports separate evaluation of structural recovery and probabilistic parameter estimation.

Figures & tables

Figure 1: The PRISM marginal-first construction pipeline. Five LLM passes elicit nodes and states, edges and DAG, unconditional marginals, pairwise joints, and multi-parent joints. Cached marginals and joints are combined to recover locally normalized CPDs analytically. Schema and DAG validation are applied between passes.

Figure 2: From backbone to released examples: (1) enumeration, (2) random sampling, (3) edge induction, (4) cut-node re-rooting using marginal priors, and (5) CPD recomputation via cached joint marginalization. Each resulting BN is paired with a fresh natural-language description.

Metric	Backbones	Released
# graphs	50	5,054
# node records	584	38,820
Mean nodes	11.68	7.68
Mean edges	14.26	7.50
Mean depth	5.04	3.63
Mean avg in-/out-deg.	1.16	0.97

Table 1: Structural comparison of the 50 source backbones and the 5,054 released paired examples (means).

Figure 3: Structural distributions across the 5,054 released subgraphs in PRISM-BN, generated by the subgraph extraction pipeline (Figure 2 ) applied to the 50 backbone networks. Panels show histograms with kernel density estimates for (a) number of nodes, (b) number of edges, (c) graph depth, (d) average in-degree, (e) average out-degree, and (f) average states per node.

Domain	#Ex.	Nodes	Edges	Depth	States
Economics	586	7.46	7.22	3.65	3.73
Environment	1,711	7.70	7.68	3.45	3.74
Geopolitics	2,043	7.85	7.69	3.86	3.99
Society	626	7.65	7.08	3.48	3.85
Technology	88	5.23	4.59	2.61	3.65

Table 2: Domain-level statistics for the 5,054 paired examples (mean values, computed from the JSON skeleton).

Figure 4: Per-metric performance versus reference graph size across six text-to-BN generators. Lines show per-bin means; shaded bands are confidence intervals. Panels (a)–(c) report Node, State, and Edge

F_{1}

; (d) reports extra predicted nodes; (e) reports symmetric CPD-KL; and (f) reports the percentage of predicted edges with no reference correspondence. State and edge metrics are conditional on Phase 1 node alignment. CPD metrics are conditional on the node, state, and structural alignments required to compare the complete multi-parent CPD of an eligible child.

Appendix figures & tables35 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Topological-level distribution across the 584 backbone node records (median 2, max 11). The longer tail relative to the subgraph corpus reflects backbone depths up to 11.

Figure 6: Average states per node across the 50 backbones. The distribution is concentrated near 4, similar to the subgraph corpus.

Figure 7: Total states per backbone (summed across all nodes). Total state counts span a substantially wider range than in the subgraph corpus, reflecting backbone sizes up to 20 nodes.

Figure 8: Backbone node-count distribution. Sizes span 5–20 nodes, substantially wider than the subgraph corpus.

Figure 9: Backbone depth distribution. Depth peaks at 4 with a tail reaching 11, motivating subgraph augmentation.

Figure 10: Joint (nodes, edges) distribution for the 50 backbones, coloured by domain. Geopolitics and Environment occupy the upper-right; Technology is concentrated in the lower-left.

Figure 11: Backbone structural means by domain with $\pm 1\sigma$ bars (nodes, edges, depth).

Figure 12: Backbone degree means by domain with $\pm 1\sigma$ bars (in-degree, out-degree). Backbone in-/out-degree is uniformly higher than at the subgraph level because the subgraph procedure re-roots orphaned parents and reduces per-node connectivity.

Figure 13: BN-level hyperparameter histograms across the 50 backbones. The wider distributions relative to the subgraph corpus motivate subgraph augmentation.

Figure 14: Node-level histograms across the 584 backbone records: in-degree, out-degree, and state cardinality. The high-fan-in tail motivates the multi-parent CPD treatment in § 3.2 .

Figure 15: Backbone BN-level distributions, faceted by domain. Geopolitics is the structural outlier on the upper end; Society and Technology sit at the lower end.

Figure 16: Backbone node-level distributions, faceted by domain. Degree distributions are nearly identical across domains; state cardinality is modal at 4 throughout.

Figure 17: Generated descriptions across the 5,054 examples. Top-left: overall text-length histogram (mean 349.3 words, median 342, range [243, 595]). Top-right: per-domain word-count distributions. Bottom-left: per-domain means with $\pm 1$ standard deviation. Bottom-right: fraction of texts containing numerical values, ranging from 84.0% (Society) to 95.1% (Economics) with corpus mean 88.7%.

Figure 18: Topological-level distribution across the 38,820 node records (median 1, max 8). The bulk of nodes occupy the upper causal layers, with a long tail extending into deeper layers driven mainly by larger Geopolitics subgraphs.

Figure 19: Average states per node across the 5,054 subgraphs. The distribution is sharply unimodal around 4.

Figure 20: Total states per subgraph (summed across all nodes). The distribution peaks sharply at 36, reflecting the dominant 9-node, 4-state subgraph class.

Figure 21: Subgraph node-count distribution. Subgraphs of 7–9 nodes account for 81.0% of the corpus.

Figure 22: Subgraph depth distribution. Depth peaks at 4, with mass between 2 and 6 covering the bulk of the corpus.

Figure 23: Joint (nodes, edges) distribution coloured by domain. Technology occupies the lower-left, while the four larger domains span the full range up to 9 nodes and 14 edges. The lower envelope follows the connectedness constraint $e\geq n-1$ .

Figure 24: Domain-level structural means with $\pm 1\sigma$ bars (nodes, edges, depth). The four largest domains are tightly clustered; Technology is the structural outlier on every axis.

Figure 25: Domain-level degree means with $\pm 1\sigma$ bars (in-degree, out-degree). Degree statistics are remarkably uniform across the four largest domains, with Technology slightly lower.

Figure 26: Node-level histograms across the 38,820 records: in-degree, out-degree, and state cardinality. Roughly 30% of nodes are roots and under 5% have three or more parents, which permits exact analytic CPD recovery for the overwhelming majority of nodes.

Figure 27: BN-level hyperparameter distributions, faceted by domain. The four largest domains are nearly indistinguishable on nodes, edges, and degree, with visible domain effects on depth and average states. Technology distributions are flatter and shifted toward smaller values throughout.

Figure 28: Node-level metric distributions, faceted by domain. Degree distributions are nearly identical across the four largest domains. State cardinality is most tightly modal at 4 for Environment and Geopolitics and most variable for Technology.

	Node			State			Edge
Model	P	R	$F_{1}$	P	R	$F_{1}$	P	R	$F_{1}$
DeepSeek-V3	0.711	0.985	0.826	0.861	0.982	0.918	0.972	0.963	0.966
Claude Haiku 4.5	0.699	0.978	0.816	0.883	0.983	0.931	0.965	0.953	0.959
Qwen3-30B-A3B	0.586	0.977	0.737	0.846	0.978	0.907	0.950	0.929	0.939
Llama 4 Maverick	0.451	0.987	0.618	0.809	0.979	0.886	0.964	0.957	0.960
GPT-4o-mini	0.408	0.972	0.573	0.845	0.970	0.903	0.889	0.918	0.902

Appendix

Table 3: Per-phase precision and recall on PRISM-BN. Node recall is near ceiling for all models; node precision drives most differences in node

F_{1}

. State and edge metrics are conditional on Phase 1 node alignment.

Panel of Figure 29 — Figure 29: Per-model node $F_{1}$ vs. reference node count. DeepSeek-V3 and Claude Haiku 4.5 maintain the tightest, highest distributions across the full range. Qwen3-30B-A3B sits a step below them with similar consistency. Llama 4 Maverick, GPT-4o-mini, and Gemma 3 12B exhibit wider IQRs and lower medians as node count grows, consistent with the precision-driven failure mode discussed in § 6 .

Panel of Figure 30 — Figure 30: Per-model state $F_{1}$ vs. reference node count. State $F_{1}$ is uniformly high across all models and sizes, supporting the observation that state enumeration is the easiest phase conditional on Phase 1 alignment.

Panel of Figure 31 — Figure 31: Per-model CPD-KL divergence vs. reference node count (lower is better). Claude Haiku 4.5 attains the lowest medians across the full range, while Gemma 3 12B exhibits the highest medians and heaviest upper tails.

Panel of Figure 32 — Figure 32: Per-model symmetric CPD-KL divergence vs. reference node count (lower is better). The patterns mirror the asymmetric CPD-KL plots, indicating that model differences are driven by overall calibration rather than directional bias.

Calibration references
Baseline / Model	Asym. KL	Sym. KL
Gaussian, $\sigma=0.1$	$0.34\pm 0.31$	$0.23\pm 0.18$
Marginal	$0.20\pm 0.12$	$0.30\pm 0.39$
Uniform	$0.27\pm 0.14$	$0.40\pm 0.47$
Dirichlet, $\alpha=1$	$0.71\pm 0.23$	$0.77\pm 0.50$
Gaussian, $\sigma=0.2$	$1.05\pm 0.74$	$0.67\pm 0.43$

Appendix

Table 4: CPD-KL calibration references and LLM generators evaluated on the same reference edge-CPDs. Lower is better. Calibration references make no use of LLM-elicited conditionals.

Model	Economics	Environment	Geopolitics	Society	Technology	ALL
DeepSeek-V3	0.818	0.829	0.816	0.838	0.885	0.826
Claude Haiku 4.5	0.807	0.817	0.808	0.836	0.874	0.816
Qwen3-30B-A3B	0.738	0.716	0.749	0.791	0.775	0.737
Llama 4 Maverick	0.707	0.590	0.541	0.738	0.757	0.618
GPT-4o-mini	0.622	0.562	0.509	0.630	0.604	0.573
Gemma 3 12B	0.620	0.535	0.486	0.631	0.640	0.566

Appendix

Table 5: Per-domain Node

F_{1}

\uparrow

for each generator. Bold marks the best model in each column.

Model	Economics	Environment	Geopolitics	Society	Technology	ALL
DeepSeek-V3	0.952	0.973	0.961	0.947	0.973	0.966
Claude Haiku 4.5	0.936	0.978	0.953	0.960	0.964	0.959
Qwen3-30B-A3B	0.930	0.958	0.918	0.945	0.895	0.939
Llama 4 Maverick	0.961	0.967	0.944	0.957	0.933	0.960
GPT-4o-mini	0.891	0.912	0.878	0.897	0.844	0.902
Gemma 3 12B	0.944	0.925	0.915	0.938	0.968	0.932

Appendix

Table 6: Per-domain Edge

F_{1}

\uparrow

for each generator. Bold marks the best model in each column.

Model	Economics	Environment	Geopolitics	Society	Technology	ALL
DeepSeek-V3	2.052	1.281	1.460	2.263	0.907	1.517
Claude Haiku 4.5	1.617	0.781	0.982	1.750	0.819	1.107
Qwen3-30B-A3B	2.310	1.162	1.851	2.599	1.358	1.722
Llama 4 Maverick	1.408	0.945	1.052	2.009	1.407	1.143
GPT-4o-mini	2.947	1.912	2.266	3.174	1.482	2.253
Gemma 3 12B	3.864	2.724	3.136	4.183	2.257	3.139

Appendix

Table 7: Per-domain CPD-KL

\downarrow

for each generator. Bold marks the lowest divergence in each column.

Model	Node $F_{1}$	State $F_{1}$	Edge $F_{1}$	CPD-KL	Sym. KL
Claude Haiku 4.5	0.929	0.993	0.998	0.481	0.362
DeepSeek-V3	0.908	0.993	0.998	1.133	0.734
Llama 4 Maverick	0.627	0.950	0.995	0.893	0.763
Qwen3-30B-A3B	0.781	0.953	0.977	2.112	1.176
GPT-4o-mini	0.550	0.964	0.962	1.552	1.245
Gemma 3 12B	0.480	0.958	0.970	3.307	2.245

Appendix

Table 8: Evaluation using the independently generated GPT-5.5 reference set. State and Edge

F_{1}

are conditional metrics. Lower KL is better.

Annotator	Node $F_{1}$	State $F_{1}$	Edge $F_{1}$	Full-CPD KL
Annotator 1	0.92	0.95	0.95	0.37
Annotator 2	0.95	0.98	0.95	0.53

Appendix

Table 9: Human extraction results on ten independently reconstructed BNs. State and Edge

F_{1}

are conditional on successful node alignment. Lower full-CPD KL is better.

Bayesian Networks

Bayesian networks represent joint probability distributions as graphs of variables and conditional dependencies, enabling inference under uncertainty.

See all 47 papers

Extracting Probabilistic Knowledge from Large Language Models for Bayesian Network Parameterization
May 21, 2025Aliakbar Nafar, Kristen Brent Venable, Zijun Cui +1Bayesian Networks Probabilistic Graphical Models
Michigan State University · Florida Institute for Human and Machine Cognition (IHMC) University of West Florida
ABSOL: Aggregated Bayesian Subsampling Orchestrated with LLMs
Sep 14, 2026Jackson Hassell, Chen Shen, Estevam HruschkaGraph Structure Learning Bayesian Network Structure Learning
Megagon Labs
Neural Bayesian Networks for Modular and Scalable Decision-Making Under Uncertainty
Oct 12, 2026Giovanni Briglia, Marco Lippi, Stefano Mariani +1Bayesian Networks Efficient Neural Network Inference
University of Modena and Reggio Emilia, Reggio Emilia, Italy · University of Florence, Florence, Italy

May 21, 2025J/K move · Enter open · S save

Extracting Probabilistic Knowledge from Large Language Models for Bayesian Network Parameterization

Aliakbar Nafar, Kristen Brent Venable, Zijun Cui, Parisa Kordjamshidi

Michigan State University · Florida Institute for Human and Machine Cognition (IHMC) University of West Florida

Bayesian Networks Probabilistic Graphical Models Probability Elicitation from Language Models

Open paper AskPDF arXiv

In this work, we evaluate the potential of Large Language Models (LLMs) in building Bayesian Networks (BNs) by approximating domain expert priors. LLMs have demonstrated potential as factual knowledge bases; however, their capability to generate probabilistic knowledge about real-world events remains understudied. We explore utilizing the probabilistic knowledge inherent in LLMs to derive probability estimates for statements regarding events and their relationships within a BN. Using LLMs in this context allows for the parameterization of BNs, enabling probabilistic modeling within specific domains. Our experiments on eighty publicly available Bayesian Networks, from healthcare to finance, demonstrate that querying LLMs about the conditional probabilities of events provides meaningful results when compared to baselines, including random and uniform distributions, as well as approaches based on next-token generation probabilities. We explore how these LLM-derived distributions can serve as expert priors to refine distributions extracted from data, especially when data is scarce. Overall, this work introduces a promising strategy for automatically constructing Bayesian Networks by combining probabilistic knowledge extracted from LLMs with real-world data. Additionally, we establish the first comprehensive baseline for assessing LLM performance in extracting probabilistic knowledge.

Abstract

Bayesian Networks

Explore similar work

Extracting Probabilistic Knowledge from Large Language Models for Bayesian Network Parameterization

ABSOL: Aggregated Bayesian Subsampling Orchestrated with LLMs

Neural Bayesian Networks for Modular and Scalable Decision-Making Under Uncertainty