Leveraging Large Language Models for Causal Discovery: a Constraint-based, Argumentation-driven Approach

Zihao Li; Fabrizio Russo

cs.AIFeb 18, 2026

Leveraging Large Language Models for Causal Discovery: a Constraint-based, Argumentation-driven Approach

Authors: Zihao Li, Fabrizio Russo

Organizations: Department of Computing, Imperial College London, UK

Abstract

Causal discovery seeks to uncover causal relations from data, typically represented as causal graphs, and is essential for predicting the effects of interventions. While expert knowledge is required to construct principled causal graphs, many statistical methods have been proposed to leverage observational data with varying formal guarantees. Causal Assumption-based Argumentation (ABA) is a framework that uses symbolic reasoning to ensure correspondence between input constraints and output graphs, while offering a principled way to combine data and expertise. We explore the use of large language models (LLMs) as imperfect experts for Causal ABA, eliciting semantic structural priors from variable names and descriptions and integrating them with conditional-independence evidence. Experiments on standard benchmarks and semantically grounded synthetic graphs demonstrate state-of-the-art performance, and we additionally introduce an evaluation protocol to mitigate memorisation bias when assessing LLMs for causal discovery.

Figures & tables

Figure 1: LLM Integration Pipeline: Given a set of variables with their names and (possibly) descriptions, we prompt an LLM to generate pairwise causal statements. These statements are then parsed into structured assumptions for Causal ABA, which combines them with data-derived independence or arrow constraints and background knowledge to infer a set of causal graphs. Expert knowledge can be injected in the LLM prompts or as defeasible facts. Variable descriptions and LLM parsing are optional components of the pipeline (dashed arrows) but enhance the quality of the generated assumptions. Detailed prompts and parsing rules are provided in App. A .

Panel of Figure 2 — Figure 2: Toy example graphs (true DAG vs. MPC output).

Figure 3: CauseNet synthetic benchmark protocol: 1. generate a random DAG, 2. ground it via induced sub-graph isomorphism in CauseNet , 3. rank candidate mappings with a three-term heuristic, 4. yielding a semantically grounded DAG with named variables for LLM elicitation.

	Method	SHD $\downarrow$	Precision $\uparrow$	Recall $\uparrow$	F1 $\uparrow$
$\|\mathbf{V}\|=5$	Random	$6.886\pm\pm 1.478$	$0.266\pm\pm 0.145$	$0.384\pm\pm 0.230$	$0.307\pm\pm 0.171$
	FGS	$5.684\pm\pm 1.079$	$0.273\pm\pm 0.198$	$0.181\pm\pm 0.133$	$0.209\pm\pm 0.151$
	NOTEARS-MLP	$5.332\pm\pm 0.008$	$0.001\pm\pm 0.008$	$0.000\pm\pm 0.002$	$0.001\pm\pm 0.003$
	GRaSP	$3.590\pm\pm 1.404$	$0.765\pm\pm 0.257$	$0.346\pm\pm 0.238$	$0.597\pm\pm 0.125$
	BOSS	$3.529\pm\pm 1.305$	$0.778\pm\pm 0.224$	$0.356\pm\pm 0.226$	$0.606\pm\pm 0.117$
	MPC	$3.294\pm\pm 0.643$	$0.859\pm\pm 0.101$	$0.371\pm\pm 0.117$	$0.602\pm\pm 0.060$

Table 1: Structural metrics on CauseNet synthetic datasets, grouped by node count

|\mathbf{V}|

. Among repeated-run methods, bold marks the best mean and statistically tied methods under BH-corrected Welch tests within each metric/column (Tabs. 10 - 11 , App. C.4.3 ). Significance levels for the

p

-values against the next non-bold method in the ranking are: ***

p<0.001

, **

p<0.01

, *

p<0.05

. LLM-BFS is shown from one saved run and is excluded from Welch tests and bolding.

Figure 4: Heatmap of the quality of LLM-derived and data-derived constraints in relation to changes in final graph reconstruction accuracy after integration ( $\Delta$ F1 against the true DAG; in brackets the number of constraints $n$ ).

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Examples of semantically grounded DAGs from our synthetic data pipeline, generated with different structural methods (ER, SF, LT) and semantic heuristics ( none , degrees , semantics ).

Dataset Name	$\|\mathbf{V}\|$	$\|E\|$	$\|E\|/\|\mathbf{V}\|$
CANCER	5	4	0.8
EARTHQUAKE	5	4	0.8
SURVEY	6	6	1
ASIA	8	8	1
SACHS	11	17	1.55
CHILD	20	25	1.25

Appendix

Table 2: Details of the bnlearn benchmark networks.

		Method	ER	SF	LT	Average
$\|\mathbf{V}\|=5$	SHD $\downarrow$	Random	$6.717\pm\pm 1.517$	$7.133\pm\pm 1.453$	$6.807\pm\pm 1.465$	$6.886\pm\pm 1.478$
		FGS	$6.570\pm\pm 0.972$	$4.580\pm\pm 0.907$	$5.903\pm\pm 1.358$	$5.684\pm\pm 1.079$
		NOTEARS-MLP	$6.000\pm\pm 0.000$	$3.997\pm\pm 0.023$	$6.000\pm\pm 0.000$	$5.332\pm\pm 0.008$
		GRaSP	$3.750\pm\pm 1.553$	$2.910\pm\pm 1.038$	$4.110\pm\pm 1.620$	$3.590\pm\pm 1.404$
		BOSS	$3.843\pm\pm 1.515$	$2.740\pm\pm 1.008$	$4.003\pm\pm 1.392$	$3.529\pm\pm 1.305$
		MPC	$2.727\pm\pm 0.658$	$3.130\pm\pm 0.468$	$4.027\pm\pm 0.802$	$3.294\pm\pm 0.643$

Appendix

Table 3: Structural metrics on CauseNet by graph type and

|\mathbf{V}|

, with marginal averages. Among repeated-run methods, bold marks the best mean and statistically tied methods under BH-corrected Welch tests within each metric/column. Significance levels for the

p

-values against the next non-bold method in the ranking are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

. LLM-BFS is shown from one saved run and is excluded from Welch tests and bolding. Tests: Table 12 ; tied bolding: Table 11 .

$\alpha$	Metric	Method	$\|\mathbf{V}\|=5$	$\|\mathbf{V}\|=10$	$\|\mathbf{V}\|=15$
$\alpha=0.01$	SHD $\downarrow$	Random	$6.886\pm\pm 1.478$	$28.247\pm\pm 6.281$	$63.766\pm\pm 19.364$
		FGS	$5.684\pm\pm 1.079$	$16.362\pm\pm 1.747$	$24.303\pm\pm 2.112$
		NOTEARS-MLP	$5.332\pm\pm 0.008$	$12.642\pm\pm 0.122$	$16.982\pm\pm 0.180$
		GRaSP	$3.590\pm\pm 1.404$	$8.148\pm\pm 2.752$	$10.422\pm\pm 3.014$
		BOSS	$3.529\pm\pm 1.305$	$7.391\pm\pm 2.627$	$9.994\pm\pm 3.010$
		MPC	$3.294\pm\pm 0.643$	$7.055\pm\pm 1.338$	$8.809\pm\pm 1.190$

Appendix

Table 4: Significance threshold ablation on CauseNet . The main runs use

\alpha=0.01

; the comparison uses

\alpha=0.05

runs. Among repeated-run methods, bold marks the best mean and statistically tied methods under BH-corrected Welch tests within each metric/column. Significance levels for the

p

-values against the next non-bold method in the ranking are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

. LLM-BFS is shown from one saved run and is excluded from Welch tests and bolding. Tests: Table 13 ; tied bolding: Table 11 .

Dataset	Method	SHD $\downarrow$	SID $\downarrow$	Precision $\uparrow$	Recall $\uparrow$	F1 $\uparrow$
cancer $\|\mathbf{V}\|=5$ $\|E\|=4$	Random	$5.600\pm\pm 1.650$	$11.260\pm\pm 4.400$	$0.200\pm\pm 0.220$	$0.200\pm\pm 0.220$	$0.360\pm\pm 0.160$
	FGS	$2.320\pm\pm 0.470$	$11.280\pm\pm 1.880$	$0.450\pm\pm 0.080$	$0.420\pm\pm 0.120$	$0.430\pm\pm 0.100$
	NOTEARS-MLP	$3.980\pm\pm 0.140$	$9.980\pm\pm 0.620$	$0.250\pm\pm 0.500$	$0.000\pm\pm 0.040$	$0.400$
	GRaSP	$3.800\pm\pm 0.760$	$9.740\pm\pm 1.880$	$0.580\pm\pm 0.370$	$0.050\pm\pm 0.190$	$0.660\pm\pm 0.280$
	BOSS	$3.600\pm\pm 1.110$	$9.160\pm\pm 2.620$	$0.860\pm\pm 0.240$	$0.100\pm\pm 0.280$	$0.770\pm\pm 0.300$
	MPC	$3.420\pm\pm 1.400$	$8.520\pm\pm 3.520$	$0.940\pm\pm 0.180$	$0.150\pm\pm 0.350$	$0.940\pm\pm 0.180$ **

Appendix

Table 5: Structural metrics on bnlearn benchmarks; dataset labels report

|\mathbf{V}|

and

|E|

. Among repeated-run methods, bold marks the best mean and statistically tied methods under BH-corrected Welch tests within each metric/column. Significance levels for the

p

-values against the next non-bold method in the ranking are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

. LLM-BFS is shown from one saved run and is excluded from Welch tests and bolding. Tests: Tables 14 , 15 , and 16 ; tied bolding: Table 17 .

Method	$\|\mathbf{V}\|=5$	$\|\mathbf{V}\|=10$	$\|\mathbf{V}\|=15$
Random	$0.00\pm\pm 0.00$	$0.00\pm\pm 0.00$	$0.00\pm\pm 0.00$
FGS	$0.11\pm\pm 0.03$	$0.14\pm\pm 0.04$	$0.15\pm\pm 0.04$
NOTEARS-MLP	$0.19\pm\pm 0.10$	$0.64\pm\pm 0.25$	$1.20\pm\pm 0.77$
GRaSP	$1.10\pm\pm 0.27$	$5.45\pm\pm 2.88$	$14.06\pm\pm 5.69$
BOSS	$1.29\pm\pm 0.32$	$7.22\pm\pm 3.41$	$18.63\pm\pm 7.64$
MPC	$0.01\pm\pm 0.00$	$0.04\pm\pm 0.03$	$0.05\pm\pm 0.03$

Appendix

Table 6: Runtime in seconds on CauseNet synthetic datasets (mean

\pm

std; LLM API latency excluded).

Forbidden constraints
	bnlearn				CauseNet
Metric	Average		Consensus		Average		Consensus
	w/o desc	with desc	w/o desc	with desc	w/o desc	with desc	w/o desc	with desc
# constraints	$19.80\pm\pm 13.90$	$25.47\pm\pm 24.39$	$8.00\pm\pm 6.60$	$8.17\pm\pm 1.72$	$17.59\pm\pm 13.02$	$20.64\pm\pm 16.60$	$4.85\pm\pm 4.83$	$4.07\pm\pm 3.42$
Precision	$0.927\pm\pm 0.186$	$0.973\pm\pm 0.041$ $\uparrow$	$0.833\pm\pm 0.408$	$0.948\pm\pm 0.085$ $\uparrow$	$0.952\pm\pm 0.132$	$0.942\pm\pm 0.155$ $\downarrow$	$0.855\pm\pm 0.339$	$0.877\pm\pm 0.317$ $\uparrow$
Recall	$0.409\pm\pm 0.234$	$0.472\pm\pm 0.219$ $\uparrow$	$0.241\pm\pm 0.240$	$0.275\pm\pm 0.216$ $\uparrow$	$0.293\pm\pm 0.216$	$0.312\pm\pm 0.221$ $\uparrow$	$0.123\pm\pm 0.164$	$0.115\pm\pm 0.168$ $\downarrow$

Appendix

Table 7: LLM-derived constraint quality by dataset, consensus strategy, and description use. Bold compares Average vs. Consensus within each setting; green marks the row best; arrows show the description effect.

Forbidden constraints
	bnlearn				CauseNet
Metric	Average		Consensus		Average		Consensus
	w/o desc	with desc	w/o desc	with desc	w/o desc	with desc	w/o desc	with desc
# constraints	$20.48\pm\pm 13.63$	$25.47\pm\pm 24.39$	$9.60\pm\pm 5.94$	$8.17\pm\pm 1.72$	$17.86\pm\pm 12.94$	$20.79\pm\pm 16.57$	$5.57\pm\pm 4.78$	$4.58\pm\pm 3.29$
Precision	$0.959\pm\pm 0.062$	$0.973\pm\pm 0.041$ $\uparrow$	$1.000\pm\pm 0.000$	$0.948\pm\pm 0.085$ $\downarrow$	$0.966\pm\pm 0.061$	$0.949\pm\pm 0.132$ $\downarrow$	$0.982\pm\pm 0.069$	$0.986\pm\pm 0.053$ $\uparrow$
Recall	$0.423\pm\pm 0.225$	$0.472\pm\pm 0.219$ $\uparrow$	$0.289\pm\pm 0.234$	$0.275\pm\pm 0.216$ $\downarrow$	$0.297\pm\pm 0.215$	$0.314\pm\pm 0.220$ $\uparrow$	$0.141\pm\pm 0.168$	$0.129\pm\pm 0.172$ $\downarrow$

Appendix

Table 8: LLM-derived constraint quality excluding zero-constraint cases. Formatting follows Table 7 : bold compares Average vs. Consensus within each setting; green marks the row best; arrows show the description effect.

	Method	SHD $\downarrow$	Precision $\uparrow$	Recall $\uparrow$	F1 $\uparrow$
$\|\mathbf{V}\|=5$	Random	$6.857\pm\pm 1.470$	$0.273\pm\pm 0.145$	$0.385\pm\pm 0.225$	$0.311\pm\pm 0.168$
	FGS	$5.610\pm\pm 1.101$	$0.295\pm\pm 0.199$	$0.192\pm\pm 0.130$	$0.224\pm\pm 0.149$
	NOTEARS-MLP	$5.467\pm\pm 0.000$	$0.000\pm\pm 0.000$	$0.000\pm\pm 0.000$	$0.000\pm\pm 0.000$
	GRaSP	$3.677\pm\pm 1.465$	$0.737\pm\pm 0.266$	$0.347\pm\pm 0.242$	$0.591\pm\pm 0.137$
	BOSS	$3.633\pm\pm 1.367$	$0.746\pm\pm 0.232$	$0.355\pm\pm 0.229$	$0.601\pm\pm 0.127$
	MPC	$3.395\pm\pm 0.643$	$0.836\pm\pm 0.107$	$0.367\pm\pm 0.111$	$0.592\pm\pm 0.065$

Appendix

Table 9: Structural metrics on the 43 matched CauseNet datasets with Gemini and GPT-5-mini constraints, grouped by

|\mathbf{V}|

. LLM-constraint variants are labelled by source. Among repeated-run methods, bold marks the best mean and statistically tied methods under BH-corrected Welch tests within each metric/column. Significance levels for the

p

-values against the next non-bold method in the ranking are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

. LLM-BFS is shown from one saved run and is excluded from Welch tests and bolding. Tests: Table 18 ; tied bolding: Table 11 .

Panel of Figure 6 — Figure 6: With variable descriptions: interaction between LLM-constraint F1 and retained-CI F1. Cell values are mean $\Delta$ DAG F1 after adding semantic constraints to ABAPC; counts are in parentheses.

Panel of Figure 7 — Figure 7: Without variable descriptions: interaction between LLM-constraint F1 and retained-CI F1. Names-only metadata weakens the negative CauseNet cells but also removes useful contextual information for some bnlearn labels.

Figure 8: CauseNet : interaction heatmaps split by node count, using the same global bins as Fig. 4 .

Figure 9: CauseNet : interaction heatmaps split by graph type, using the same global bins as Fig. 4 .

Figure 10: bnlearn : per-dataset interaction heatmaps using the same global bins as Fig. 6 , allowing absolute LLM-constraint-quality comparisons across benchmarks.

Figure 11: bnlearn : per-dataset interaction heatmaps using within-dataset quantile bins, highlighting relative variation inside each benchmark.

Group	Metric	Methods	Means $\pm$ Std	t	p-value	$p_{\mathrm{BH}}$
$\|\mathbf{V}\|=5$	SHD	ABAPC-LLM vs MPC-LLM	$1.230\pm\pm 0.369$ vs $1.500\pm\pm 0.318$	$-5.60$	$2.19\text{e-}8$	$2.19\text{e-}8$
$\|\mathbf{V}\|=5$	Precision	MPC-LLM vs ABAPC-LLM	$0.958\pm\pm 0.031$ vs $0.882\pm\pm 0.046$	$12.94$	$2.57\text{e-}38$	$2.57\text{e-}38$
$\|\mathbf{V}\|=5$	Recall	ABAPC-LLM vs MPC-LLM	$0.763\pm\pm 0.062$ vs $0.710\pm\pm 0.050$	$5.12$	$3.13\text{e-}7$	$3.13\text{e-}7$
$\|\mathbf{V}\|=5$	F1	ABAPC-LLM vs MPC-LLM	$0.812\pm\pm 0.049$ vs $0.795\pm\pm 0.037$	$1.98$	$0.048$	$0.048$
$\|\mathbf{V}\|=10$	SHD	ABAPC-LLM vs ABAPC	$5.626\pm\pm 1.176$ vs $6.166\pm\pm 1.442$	$-5.03$	$4.89\text{e-}7$	$4.89\text{e-}7$
$\|\mathbf{V}\|=10$	Precision	BOSS vs MPC-LLM	$0.880\pm\pm 0.197$ vs $0.856\pm\pm 0.088$	$4.34$	$1.40\text{e-}5$	$1.40\text{e-}5$

Appendix

Table 10: Welch tests for Table 1 . Each row compares a bolded top entry with the next non-bold method in that column. Significance levels for both reported

p

-values are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

Table	Group	Metric	Methods	Means $\pm$ Std	t	p-value		$p_{\mathrm{BH}}$
1	$\|\mathbf{V}\|=15$	Precision	MPC vs MPC-LLM	$0.936\pm\pm 0.071$ vs $0.929\pm\pm 0.061$	$1.91$	$0.056$	.	$0.056$	.
3	SF $\|\mathbf{V}\|=5$	F1	ABAPC-LLM vs MPC-LLM	$0.734\pm\pm 0.046$ vs $0.697\pm\pm 0.055$	$1.81$	$0.071$	.	$0.071$	.
3	LT $\|\mathbf{V}\|=5$	SHD	MPC-LLM vs ABAPC-LLM	$1.217\pm\pm 0.345$ vs $1.335\pm\pm 0.457$	$-1.20$	$0.231$	ns	$0.231$	ns
3	ER $\|\mathbf{V}\|=10$	F1	MPC-LLM vs ABAPC-LLM	$0.663\pm\pm 0.057$ vs $0.656\pm\pm 0.057$	$0.57$	$0.570$	ns	$0.570$	ns
3	SF $\|\mathbf{V}\|=10$	SHD	ABAPC-LLM vs MPC-LLM	$5.711\pm\pm 1.302$ vs $6.112\pm\pm 1.198$	$-1.95$	$0.051$	.	$0.051$	.
3	LT $\|\mathbf{V}\|=10$	F1	BOSS vs ABAPC-LLM	$0.638\pm\pm 0.192$ vs $0.623\pm\pm 0.111$	$1.67$	$0.095$	.	$0.095$	.

Appendix

Table 11: Non-significant best-vs-comparator tests that justify additional bolded methods in Tables 1 , 3 , 4 , and 9 . Significance levels for both reported

p

-values are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

Group	Metric	Methods	Means $\pm$ Std	t	p-value		$p_{\mathrm{BH}}$
ER, $\|\mathbf{V}\|=5$	SHD	ABAPC-LLM vs ABAPC	$1.054\pm\pm 0.372$ vs $1.515\pm\pm 0.542$	$-7.91$	$2.63\text{e-}15$		$2.63\text{e-}15$
ER, $\|\mathbf{V}\|=5$	F1	ABAPC-LLM vs MPC-LLM	$0.843\pm\pm 0.044$ vs $0.805\pm\pm 0.025$	$3.94$	$8.25\text{e-}5$		$8.25\text{e-}5$
SF, $\|\mathbf{V}\|=5$	SHD	ABAPC-LLM vs MPC-LLM	$1.302\pm\pm 0.278$ vs $1.603\pm\pm 0.305$	$-3.06$	$0.002$	**	$0.002$	**
SF, $\|\mathbf{V}\|=5$	F1	ABAPC-LLM vs ABAPC	$0.734\pm\pm 0.046$ vs $0.632\pm\pm 0.075$	$7.05$	$1.79\text{e-}12$		$3.59\text{e-}12$
SF, $\|\mathbf{V}\|=5$	F1	MPC-LLM vs ABAPC	$0.697\pm\pm 0.055$ vs $0.632\pm\pm 0.075$	$3.91$	$9.07\text{e-}5$		$9.07\text{e-}5$
LT, $\|\mathbf{V}\|=5$	SHD	MPC-LLM vs ABAPC	$1.217\pm\pm 0.345$ vs $2.709\pm\pm 0.722$	$-13.57$	$5.71\text{e-}42$		$1.14\text{e-}41$

Appendix

Table 12: Welch tests for Table 3 , including marginal averages. Significance levels for both reported

p

-values are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

Group	Metric	Methods	Means $\pm$ Std	t	p-value		$p_{\mathrm{BH}}$
$\alpha=0.01$ , $\|\mathbf{V}\|=5$	SHD	ABAPC-LLM vs MPC-LLM	$1.230\pm\pm 0.369$ vs $1.500\pm\pm 0.318$	$-5.60$	$2.19\text{e-}8$		$2.19\text{e-}8$
$\alpha=0.01$ , $\|\mathbf{V}\|=5$	F1	ABAPC-LLM vs MPC-LLM	$0.812\pm\pm 0.049$ vs $0.795\pm\pm 0.037$	$1.98$	$0.048$		$0.048$
$\alpha=0.01$ , $\|\mathbf{V}\|=10$	SHD	ABAPC-LLM vs ABAPC	$5.626\pm\pm 1.176$ vs $6.166\pm\pm 1.442$	$-5.03$	$4.89\text{e-}7$		$4.89\text{e-}7$
$\alpha=0.01$ , $\|\mathbf{V}\|=10$	F1	ABAPC-LLM vs MPC-LLM	$0.653\pm\pm 0.084$ vs $0.627\pm\pm 0.077$	$4.38$	$1.17\text{e-}5$		$1.17\text{e-}5$
$\alpha=0.01$ , $\|\mathbf{V}\|=15$	SHD	ABAPC-LLM vs MPC-LLM	$7.550\pm\pm 1.281$ vs $7.874\pm\pm 1.038$	$-2.94$	$0.003$	**	$0.003$	**
$\alpha=0.01$ , $\|\mathbf{V}\|=15$	F1	MPC-LLM vs ABAPC-LLM	$0.667\pm\pm 0.058$ vs $0.654\pm\pm 0.072$	$2.52$	$0.012$		$0.012$

Appendix

Table 13: Welch tests for Table 4 . Significance levels for both reported

p

-values are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

Group	Metric	Methods	Means $\pm$ Std	t	p-value	$p_{\mathrm{BH}}$
earthquake	SHD	MPC vs NOTEARS-MLP	$0.040\pm\pm 0.200$ vs $2.120\pm\pm 1.170$	$-12.39$	$2.92\text{e-}35$	$8.77\text{e-}35$
earthquake	SHD	MPC-LLM vs NOTEARS-MLP	$0.040\pm\pm 0.200$ vs $2.120\pm\pm 1.170$	$-12.39$	$2.92\text{e-}35$	$8.77\text{e-}35$
earthquake	SHD	ABAPC-LLM vs NOTEARS-MLP	$0.080\pm\pm 0.340$ vs $2.120\pm\pm 1.170$	$-11.84$	$2.48\text{e-}32$	$4.97\text{e-}32$
earthquake	SHD	ABAPC vs NOTEARS-MLP	$0.100\pm\pm 0.463$ vs $2.120\pm\pm 1.170$	$-11.35$	$7.25\text{e-}30$	$1.09\text{e-}29$
earthquake	SHD	GRaSP vs NOTEARS-MLP	$0.160\pm\pm 0.790$ vs $2.120\pm\pm 1.170$	$-9.82$	$9.49\text{e-}23$	$1.14\text{e-}22$
earthquake	SHD	BOSS vs NOTEARS-MLP	$0.400\pm\pm 1.210$ vs $2.120\pm\pm 1.170$	$-7.23$	$4.98\text{e-}13$	$4.98\text{e-}13$

Appendix

Table 14: Welch tests for the earthquake rows in Table 5 . Significance levels for both reported

p

-values are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

Group	Metric	Methods	Means $\pm$ Std	t	p-value	$p_{\mathrm{BH}}$
asia	SHD	MPC-LLM vs ABAPC-LLM	$2.840\pm\pm 0.470$ vs $3.580\pm\pm 0.710$	$-6.15$	$7.89\text{e-}10$	$7.89\text{e-}10$
asia	Precision	ABAPC-LLM vs ABAPC	$0.992\pm\pm 0.040$ vs $0.684\pm\pm 0.207$	$10.31$	$6.65\text{e-}25$	$9.97\text{e-}25$
asia	Precision	MPC-LLM vs ABAPC	$0.990\pm\pm 0.030$ vs $0.684\pm\pm 0.207$	$10.32$	$5.96\text{e-}25$	$9.97\text{e-}25$
asia	Precision	MPC vs ABAPC	$0.980\pm\pm 0.080$ vs $0.684\pm\pm 0.207$	$9.41$	$5.11\text{e-}21$	$5.11\text{e-}21$
asia	Recall	MPC-LLM vs ABAPC-LLM	$0.650\pm\pm 0.050$ vs $0.557\pm\pm 0.082$	$6.83$	$8.49\text{e-}12$	$8.49\text{e-}12$
asia	F1	MPC-LLM vs ABAPC-LLM	$0.780\pm\pm 0.040$ vs $0.710\pm\pm 0.068$	$6.22$	$5.00\text{e-}10$	$5.00\text{e-}10$

Appendix

Table 15: Welch tests for Table 5 , excluding the earthquake and child rows. Significance levels for both reported

p

-values are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

Group	Metric	Methods	Means $\pm$ Std	t	p-value	$p_{\mathrm{BH}}$
child	SHD	ABAPC-LLM vs MPC	$4.417\pm\pm 2.009$ vs $9.160\pm\pm 4.130$	$-7.30$	$2.82\text{e-}13$	$8.45\text{e-}13$
child	SHD	MPC-LLM vs MPC	$5.060\pm\pm 1.800$ vs $9.160\pm\pm 4.130$	$-6.44$	$1.23\text{e-}10$	$1.85\text{e-}10$
child	SHD	ABAPC vs MPC	$5.372\pm\pm 3.250$ vs $9.160\pm\pm 4.130$	$-5.10$	$3.46\text{e-}7$	$3.46\text{e-}7$
child	Precision	BOSS vs ABAPC-LLM	$0.960\pm\pm 0.080$ vs $0.894\pm\pm 0.050$	$4.94$	$7.67\text{e-}7$	$7.67\text{e-}7$
child	Recall	ABAPC-LLM vs MPC	$0.824\pm\pm 0.080$ vs $0.630\pm\pm 0.170$	$7.31$	$2.59\text{e-}13$	$7.76\text{e-}13$
child	Recall	MPC-LLM vs MPC	$0.800\pm\pm 0.070$ vs $0.630\pm\pm 0.170$	$6.54$	$6.22\text{e-}11$	$9.32\text{e-}11$

Appendix

Table 16: Welch tests for the child rows in Table 5 . Significance levels for both reported

p

-values are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

Group	Metric	Methods	Means $\pm$ Std	t	p-value		$p_{\mathrm{BH}}$
asia	Precision	ABAPC-LLM vs MPC	$0.992\pm\pm 0.040$ vs $0.980\pm\pm 0.080$	$0.95$	$0.342$	ns	$0.391$	ns
asia	Precision	ABAPC-LLM vs MPC-LLM	$0.992\pm\pm 0.040$ vs $0.990\pm\pm 0.030$	$0.28$	$0.776$	ns	$0.776$	ns
cancer	SHD	MPC-LLM vs ABAPC-LLM	$0.680\pm\pm 0.590$ vs $0.680\pm\pm 0.587$	$0.00$	$1.000$	ns	$1.000$	ns
cancer	Precision	ABAPC-LLM vs MPC-LLM	$0.996\pm\pm 0.028$ vs $0.990\pm\pm 0.050$	$0.74$	$0.460$	ns	$0.460$	ns
cancer	Recall	MPC-LLM vs ABAPC-LLM	$0.840\pm\pm 0.130$ vs $0.835\pm\pm 0.148$	$0.18$	$0.858$	ns	$0.858$	ns
cancer	F1	MPC vs MPC-LLM	$0.940\pm\pm 0.180$ vs $0.900\pm\pm 0.080$	$1.44$	$0.151$	ns	$0.167$	ns

Appendix

Table 17: Non-significant best-vs-comparator tests that justify additional bolded methods in Table 5 . Significance levels for both reported

p

-values are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

Group	Metric	Methods	Means $\pm$ Std	t	p-value	$p_{\mathrm{BH}}$
$\|\mathbf{V}\|=5$	SHD	ABAPC-LLM (Gemini) vs MPC-LLM (Gemini)	$1.186\pm\pm 0.375$ vs $1.328\pm\pm 0.331$	$-2.57$	$0.010$	$0.010$
$\|\mathbf{V}\|=5$	Precision	MPC-LLM (Gemini) vs ABAPC-LLM (Gemini)	$0.952\pm\pm 0.027$ vs $0.902\pm\pm 0.043$	$7.39$	$1.50\text{e-}13$	$1.50\text{e-}13$
$\|\mathbf{V}\|=5$	Recall	ABAPC-LLM (Gemini) vs MPC-LLM (Gemini)	$0.778\pm\pm 0.063$ vs $0.753\pm\pm 0.053$	$2.20$	$0.028$	$0.028$
$\|\mathbf{V}\|=5$	F1	ABAPC-LLM (Gemini) vs ABAPC-LLM (GPT)	$0.828\pm\pm 0.048$ vs $0.755\pm\pm 0.084$	$7.82$	$5.42\text{e-}15$	$1.08\text{e-}14$
$\|\mathbf{V}\|=5$	F1	MPC-LLM (Gemini) vs ABAPC-LLM (GPT)	$0.822\pm\pm 0.041$ vs $0.755\pm\pm 0.084$	$6.56$	$5.37\text{e-}11$	$5.37\text{e-}11$
$\|\mathbf{V}\|=10$	SHD	ABAPC-LLM (GPT) vs MPC-LLM (Gemini)	$5.755\pm\pm 1.417$ vs $6.197\pm\pm 1.095$	$-3.63$	$2.81\text{e-}4$	$5.62\text{e-}4$

Appendix

Table 18: Welch tests for Table 9 . Significance levels for both reported

p

-values are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

Causal Discovery

Causal discovery infers cause-and-effect structures from observational and interventional data, distinguishing causal relationships from correlation.

See all 220 papers

May 26, 2026

Why LLMs Fail at Causal Discovery and How Interventional Agents Escape

Causal discovery is a cornerstone of scientific reasoning, yet whether large language models can perform it reliably remains an open question. Recent benchmarks show that even fine-tuned models plateau on simple causal graphs and degrade as complexity grows, but why they fail has not been established. We prove the failure is fundamental: supervised fine-tuning, direct preference optimization, and in-context learning all produce predictors that cannot distinguish between causal graphs generating similar observational data, and any attempt to do so requires the model's internal representations to grow unboundedly, violating the very conditions under which these methods work. We formalize this as a kernel obstruction theorem, establishing that the limitation is intrinsic to the learning paradigm, not any particular model or dataset. We propose Agentic Causal Bayesian Optimization (A-CBO), wherein a frozen language model serves as an interventional oracle answering targeted queries about intervention effects, while an external Bayesian loop concentrates beliefs over candidate graphs in logarithmically many rounds. Because the decision operates outside the space where the obstruction applies, A-CBO provably converges while the underlying model remains unchanged. On Corr2Cause, A-CBO matches fine-tuned baselines without any training. On Extended Corr2Cause, a new benchmark scaling to 24 variables with 18K test samples, A-CBO significantly outperforms both fine-tuning and preference optimization, with the advantage growing

Amartya Roy, Sonali Parbhoo

SIRE, IIT Delhi and Robert Bosch GmbH, India · Imperial College London

Causal Discovery Causal Reasoning in Language Models Causal Identifiability Causal Interventions in Language Models

Ask PDF

Jun 22, 2026

Causal Discovery in the Era of Agents

Recent attempts to combine large language models (LLMs) with causal discovery ask models to infer pairwise directions, propose graph structures, or inject language-model outputs as priors and constraints. These approaches promise faster analysis, but they also obscure whether a causal evidence is supported by data and assumptions or by textual associations, prompt artifacts and hallucinated mechanisms. We argue for a different role for agents in causal discovery. Agents should inspect data, retrieve context, explain method assumptions and clarify graph outputs, but they should not supply edges, orientations, priors, constraints or causal conclusions. We propose the principle that agents assist the workflow, while causal claims remain grounded in data, explicit assumptions, formal algorithms, diagnostics and user or domain-expert decisions. We instantiate this principle in causal-learn+, an online platform that coordinates data analysis, preprocessing, method recommendation, expert-knowledge incorporation, formal discovery and interpretation around the algorithmic ecosystem of causal-learn. A case study on Big Five personality data illustrates agent-assisted pipeline of causal discovery without turning language-model unreliability into causal evidence. The platform is available at causallearn.com.

Yujia Zheng, Vishal Verma, Mantej Gill +3

1Carnegie Mellon University. · 2Mohamed bin Zayed University of Artificial Intelligence.

AI Agents for Scientific Discovery Causal Discovery

Ask PDF

Jun 2, 2026

CauTion: Knowing When to Trust LLMs for Ensemble Causal Discovery

Causal discovery from observational data remains challenging due to the fundamental limitations of purely statistical methods, such as statistical distinguishability within equivalence classes and sensitivity to finite sample sizes. While large language models (LLMs) offer a promising source of domain knowledge to complement statistical inference, existing LLM-augmented methods are vulnerable to LLM errors and incur high token costs. Moreover, reliance on a single data-centric algorithm can make results sensitive to algorithm-specific biases. To address these limitations, we propose CauTion, a framework that reliably integrates LLM domain knowledge into an ensemble of statistical causal discovery algorithms through consensus filtering and LLM reliability estimation. CauTion proceeds in three stages. First, an algorithm ensemble utilizes a consensus voting to resolve up to 96% of edges on which algorithms agree, achieving near-perfect accuracy on the filtered consensus edges. Second, a trust-calibrated arbitration mechanism estimates the relative reliability of the LLM and the algorithms via an annotation-free trust calibration procedure, which is then utilized to govern a trust-weighted voting process that restricts LLM arbitration exclusively to edges with unreliable algorithmic evidence. Third, a cycle repair step is applied to guarantee the final causal graph is validly acyclic. Experiments on six datasets demonstrate that CauTion consistently outperforms both data-centric and LLM-augmented baselines, with larger gains on larger graphs and strong robustness to LLM errors. Code is available at https://github.com/OpenCausaLab/CauTion.

Bo Peng, Kaiwen Wu, Sirui Chen +3

Shanghai AI Laboratory · Shanghai Innovation Institute · Shanghai Jiao Tong University +2

Causal Discovery

Ask PDF

Abstract

Causal Discovery

Explore similar work

Why LLMs Fail at Causal Discovery and How Interventional Agents Escape

Causal Discovery in the Era of Agents

CauTion: Knowing When to Trust LLMs for Ensemble Causal Discovery