Leveraging Large Language Models for Causal Discovery: a Constraint-based, Argumentation-driven Approach

Zihao Li; Fabrizio Russo

cs.AIFeb 18, 2026

Leveraging Large Language Models for Causal Discovery: a Constraint-based, Argumentation-driven Approach

Authors: Zihao Li, Fabrizio Russo

Organizations: Department of Computing, Imperial College London, UK

Abstract

Causal discovery seeks to uncover causal relations from data, typically represented as causal graphs, and is essential for predicting the effects of interventions. While expert knowledge is required to construct principled causal graphs, many statistical methods have been proposed to leverage observational data with varying formal guarantees. Causal Assumption-based Argumentation (ABA) is a framework that uses symbolic reasoning to ensure correspondence between input constraints and output graphs, while offering a principled way to combine data and expertise. We explore the use of large language models (LLMs) as imperfect experts for Causal ABA, eliciting semantic structural priors from variable names and descriptions and integrating them with conditional-independence evidence. Experiments on standard benchmarks and semantically grounded synthetic graphs demonstrate state-of-the-art performance, and we additionally introduce an evaluation protocol to mitigate memorisation bias when assessing LLMs for causal discovery.

Figures & tables

Figure 1: LLM Integration Pipeline: Given a set of variables with their names and (possibly) descriptions, we prompt an LLM to generate pairwise causal statements. These statements are then parsed into structured assumptions for Causal ABA, which combines them with data-derived independence or arrow constraints and background knowledge to infer a set of causal graphs. Expert knowledge can be injected in the LLM prompts or as defeasible facts. Variable descriptions and LLM parsing are optional components of the pipeline (dashed arrows) but enhance the quality of the generated assumptions. Detailed prompts and parsing rules are provided in App. A .

Panel of Figure 2 — Figure 2: Toy example graphs (true DAG vs. MPC output).

Figure 3: CauseNet synthetic benchmark protocol: 1. generate a random DAG, 2. ground it via induced sub-graph isomorphism in CauseNet , 3. rank candidate mappings with a three-term heuristic, 4. yielding a semantically grounded DAG with named variables for LLM elicitation.

	Method	SHD $\downarrow$	Precision $\uparrow$	Recall $\uparrow$	F1 $\uparrow$
$\|\mathbf{V}\|=5$	Random	$6.886\pm\pm 1.478$	$0.266\pm\pm 0.145$	$0.384\pm\pm 0.230$	$0.307\pm\pm 0.171$
	FGS	$5.684\pm\pm 1.079$	$0.273\pm\pm 0.198$	$0.181\pm\pm 0.133$	$0.209\pm\pm 0.151$
	NOTEARS-MLP	$5.332\pm\pm 0.008$	$0.001\pm\pm 0.008$	$0.000\pm\pm 0.002$	$0.001\pm\pm 0.003$
	GRaSP	$3.590\pm\pm 1.404$	$0.765\pm\pm 0.257$	$0.346\pm\pm 0.238$	$0.597\pm\pm 0.125$
	BOSS	$3.529\pm\pm 1.305$	$0.778\pm\pm 0.224$	$0.356\pm\pm 0.226$	$0.606\pm\pm 0.117$
	MPC	$3.294\pm\pm 0.643$	$0.859\pm\pm 0.101$	$0.371\pm\pm 0.117$	$0.602\pm\pm 0.060$

Table 1: Structural metrics on CauseNet synthetic datasets, grouped by node count

|\mathbf{V}|

. Among repeated-run methods, bold marks the best mean and statistically tied methods under BH-corrected Welch tests within each metric/column (Tabs. 10 - 11 , App. C.4.3 ). Significance levels for the

p

-values against the next non-bold method in the ranking are: ***

p<0.001

, **

p<0.01

, *

p<0.05

. LLM-BFS is shown from one saved run and is excluded from Welch tests and bolding.

Figure 4: Heatmap of the quality of LLM-derived and data-derived constraints in relation to changes in final graph reconstruction accuracy after integration ( $\Delta$ F1 against the true DAG; in brackets the number of constraints $n$ ).

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Examples of semantically grounded DAGs from our synthetic data pipeline, generated with different structural methods (ER, SF, LT) and semantic heuristics ( none , degrees , semantics ).

Dataset Name	$\|\mathbf{V}\|$	$\|E\|$	$\|E\|/\|\mathbf{V}\|$
CANCER	5	4	0.8
EARTHQUAKE	5	4	0.8
SURVEY	6	6	1
ASIA	8	8	1
SACHS	11	17	1.55
CHILD	20	25	1.25

Appendix

Table 2: Details of the bnlearn benchmark networks.

		Method	ER	SF	LT	Average
$\|\mathbf{V}\|=5$	SHD $\downarrow$	Random	$6.717\pm\pm 1.517$	$7.133\pm\pm 1.453$	$6.807\pm\pm 1.465$	$6.886\pm\pm 1.478$
		FGS	$6.570\pm\pm 0.972$	$4.580\pm\pm 0.907$	$5.903\pm\pm 1.358$	$5.684\pm\pm 1.079$
		NOTEARS-MLP	$6.000\pm\pm 0.000$	$3.997\pm\pm 0.023$	$6.000\pm\pm 0.000$	$5.332\pm\pm 0.008$
		GRaSP	$3.750\pm\pm 1.553$	$2.910\pm\pm 1.038$	$4.110\pm\pm 1.620$	$3.590\pm\pm 1.404$
		BOSS	$3.843\pm\pm 1.515$	$2.740\pm\pm 1.008$	$4.003\pm\pm 1.392$	$3.529\pm\pm 1.305$
		MPC	$2.727\pm\pm 0.658$	$3.130\pm\pm 0.468$	$4.027\pm\pm 0.802$	$3.294\pm\pm 0.643$

Appendix

Table 3: Structural metrics on CauseNet by graph type and

|\mathbf{V}|

, with marginal averages. Among repeated-run methods, bold marks the best mean and statistically tied methods under BH-corrected Welch tests within each metric/column. Significance levels for the

p

-values against the next non-bold method in the ranking are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

. LLM-BFS is shown from one saved run and is excluded from Welch tests and bolding. Tests: Table 12 ; tied bolding: Table 11 .

$\alpha$	Metric	Method	$\|\mathbf{V}\|=5$	$\|\mathbf{V}\|=10$	$\|\mathbf{V}\|=15$
$\alpha=0.01$	SHD $\downarrow$	Random	$6.886\pm\pm 1.478$	$28.247\pm\pm 6.281$	$63.766\pm\pm 19.364$
		FGS	$5.684\pm\pm 1.079$	$16.362\pm\pm 1.747$	$24.303\pm\pm 2.112$
		NOTEARS-MLP	$5.332\pm\pm 0.008$	$12.642\pm\pm 0.122$	$16.982\pm\pm 0.180$
		GRaSP	$3.590\pm\pm 1.404$	$8.148\pm\pm 2.752$	$10.422\pm\pm 3.014$
		BOSS	$3.529\pm\pm 1.305$	$7.391\pm\pm 2.627$	$9.994\pm\pm 3.010$
		MPC	$3.294\pm\pm 0.643$	$7.055\pm\pm 1.338$	$8.809\pm\pm 1.190$

Appendix

Table 4: Significance threshold ablation on CauseNet . The main runs use

\alpha=0.01

; the comparison uses

\alpha=0.05

runs. Among repeated-run methods, bold marks the best mean and statistically tied methods under BH-corrected Welch tests within each metric/column. Significance levels for the

p

-values against the next non-bold method in the ranking are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

. LLM-BFS is shown from one saved run and is excluded from Welch tests and bolding. Tests: Table 13 ; tied bolding: Table 11 .

Dataset	Method	SHD $\downarrow$	SID $\downarrow$	Precision $\uparrow$	Recall $\uparrow$	F1 $\uparrow$
cancer $\|\mathbf{V}\|=5$ $\|E\|=4$	Random	$5.600\pm\pm 1.650$	$11.260\pm\pm 4.400$	$0.200\pm\pm 0.220$	$0.200\pm\pm 0.220$	$0.360\pm\pm 0.160$
	FGS	$2.320\pm\pm 0.470$	$11.280\pm\pm 1.880$	$0.450\pm\pm 0.080$	$0.420\pm\pm 0.120$	$0.430\pm\pm 0.100$
	NOTEARS-MLP	$3.980\pm\pm 0.140$	$9.980\pm\pm 0.620$	$0.250\pm\pm 0.500$	$0.000\pm\pm 0.040$	$0.400$
	GRaSP	$3.800\pm\pm 0.760$	$9.740\pm\pm 1.880$	$0.580\pm\pm 0.370$	$0.050\pm\pm 0.190$	$0.660\pm\pm 0.280$
	BOSS	$3.600\pm\pm 1.110$	$9.160\pm\pm 2.620$	$0.860\pm\pm 0.240$	$0.100\pm\pm 0.280$	$0.770\pm\pm 0.300$
	MPC	$3.420\pm\pm 1.400$	$8.520\pm\pm 3.520$	$0.940\pm\pm 0.180$	$0.150\pm\pm 0.350$	$0.940\pm\pm 0.180$ **

Appendix

Table 5: Structural metrics on bnlearn benchmarks; dataset labels report

|\mathbf{V}|

and

|E|

. Among repeated-run methods, bold marks the best mean and statistically tied methods under BH-corrected Welch tests within each metric/column. Significance levels for the

p

-values against the next non-bold method in the ranking are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

. LLM-BFS is shown from one saved run and is excluded from Welch tests and bolding. Tests: Tables 14 , 15 , and 16 ; tied bolding: Table 17 .

Method	$\|\mathbf{V}\|=5$	$\|\mathbf{V}\|=10$	$\|\mathbf{V}\|=15$
Random	$0.00\pm\pm 0.00$	$0.00\pm\pm 0.00$	$0.00\pm\pm 0.00$
FGS	$0.11\pm\pm 0.03$	$0.14\pm\pm 0.04$	$0.15\pm\pm 0.04$
NOTEARS-MLP	$0.19\pm\pm 0.10$	$0.64\pm\pm 0.25$	$1.20\pm\pm 0.77$
GRaSP	$1.10\pm\pm 0.27$	$5.45\pm\pm 2.88$	$14.06\pm\pm 5.69$
BOSS	$1.29\pm\pm 0.32$	$7.22\pm\pm 3.41$	$18.63\pm\pm 7.64$
MPC	$0.01\pm\pm 0.00$	$0.04\pm\pm 0.03$	$0.05\pm\pm 0.03$

Appendix

Table 6: Runtime in seconds on CauseNet synthetic datasets (mean

\pm

std; LLM API latency excluded).

Forbidden constraints
	bnlearn				CauseNet
Metric	Average		Consensus		Average		Consensus
	w/o desc	with desc	w/o desc	with desc	w/o desc	with desc	w/o desc	with desc
# constraints	$19.80\pm\pm 13.90$	$25.47\pm\pm 24.39$	$8.00\pm\pm 6.60$	$8.17\pm\pm 1.72$	$17.59\pm\pm 13.02$	$20.64\pm\pm 16.60$	$4.85\pm\pm 4.83$	$4.07\pm\pm 3.42$
Precision	$0.927\pm\pm 0.186$	$0.973\pm\pm 0.041$ $\uparrow$	$0.833\pm\pm 0.408$	$0.948\pm\pm 0.085$ $\uparrow$	$0.952\pm\pm 0.132$	$0.942\pm\pm 0.155$ $\downarrow$	$0.855\pm\pm 0.339$	$0.877\pm\pm 0.317$ $\uparrow$
Recall	$0.409\pm\pm 0.234$	$0.472\pm\pm 0.219$ $\uparrow$	$0.241\pm\pm 0.240$	$0.275\pm\pm 0.216$ $\uparrow$	$0.293\pm\pm 0.216$	$0.312\pm\pm 0.221$ $\uparrow$	$0.123\pm\pm 0.164$	$0.115\pm\pm 0.168$ $\downarrow$

Appendix

Table 7: LLM-derived constraint quality by dataset, consensus strategy, and description use. Bold compares Average vs. Consensus within each setting; green marks the row best; arrows show the description effect.

Forbidden constraints
	bnlearn				CauseNet
Metric	Average		Consensus		Average		Consensus
	w/o desc	with desc	w/o desc	with desc	w/o desc	with desc	w/o desc	with desc
# constraints	$20.48\pm\pm 13.63$	$25.47\pm\pm 24.39$	$9.60\pm\pm 5.94$	$8.17\pm\pm 1.72$	$17.86\pm\pm 12.94$	$20.79\pm\pm 16.57$	$5.57\pm\pm 4.78$	$4.58\pm\pm 3.29$
Precision	$0.959\pm\pm 0.062$	$0.973\pm\pm 0.041$ $\uparrow$	$1.000\pm\pm 0.000$	$0.948\pm\pm 0.085$ $\downarrow$	$0.966\pm\pm 0.061$	$0.949\pm\pm 0.132$ $\downarrow$	$0.982\pm\pm 0.069$	$0.986\pm\pm 0.053$ $\uparrow$
Recall	$0.423\pm\pm 0.225$	$0.472\pm\pm 0.219$ $\uparrow$	$0.289\pm\pm 0.234$	$0.275\pm\pm 0.216$ $\downarrow$	$0.297\pm\pm 0.215$	$0.314\pm\pm 0.220$ $\uparrow$	$0.141\pm\pm 0.168$	$0.129\pm\pm 0.172$ $\downarrow$

Appendix

Table 8: LLM-derived constraint quality excluding zero-constraint cases. Formatting follows Table 7 : bold compares Average vs. Consensus within each setting; green marks the row best; arrows show the description effect.

	Method	SHD $\downarrow$	Precision $\uparrow$	Recall $\uparrow$	F1 $\uparrow$
$\|\mathbf{V}\|=5$	Random	$6.857\pm\pm 1.470$	$0.273\pm\pm 0.145$	$0.385\pm\pm 0.225$	$0.311\pm\pm 0.168$
	FGS	$5.610\pm\pm 1.101$	$0.295\pm\pm 0.199$	$0.192\pm\pm 0.130$	$0.224\pm\pm 0.149$
	NOTEARS-MLP	$5.467\pm\pm 0.000$	$0.000\pm\pm 0.000$	$0.000\pm\pm 0.000$	$0.000\pm\pm 0.000$
	GRaSP	$3.677\pm\pm 1.465$	$0.737\pm\pm 0.266$	$0.347\pm\pm 0.242$	$0.591\pm\pm 0.137$
	BOSS	$3.633\pm\pm 1.367$	$0.746\pm\pm 0.232$	$0.355\pm\pm 0.229$	$0.601\pm\pm 0.127$
	MPC	$3.395\pm\pm 0.643$	$0.836\pm\pm 0.107$	$0.367\pm\pm 0.111$	$0.592\pm\pm 0.065$

Appendix

Table 9: Structural metrics on the 43 matched CauseNet datasets with Gemini and GPT-5-mini constraints, grouped by

|\mathbf{V}|

. LLM-constraint variants are labelled by source. Among repeated-run methods, bold marks the best mean and statistically tied methods under BH-corrected Welch tests within each metric/column. Significance levels for the

p

-values against the next non-bold method in the ranking are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

. LLM-BFS is shown from one saved run and is excluded from Welch tests and bolding. Tests: Table 18 ; tied bolding: Table 11 .

Panel of Figure 6 — Figure 6: With variable descriptions: interaction between LLM-constraint F1 and retained-CI F1. Cell values are mean $\Delta$ DAG F1 after adding semantic constraints to ABAPC; counts are in parentheses.

Panel of Figure 7 — Figure 7: Without variable descriptions: interaction between LLM-constraint F1 and retained-CI F1. Names-only metadata weakens the negative CauseNet cells but also removes useful contextual information for some bnlearn labels.

Figure 8: CauseNet : interaction heatmaps split by node count, using the same global bins as Fig. 4 .

Figure 9: CauseNet : interaction heatmaps split by graph type, using the same global bins as Fig. 4 .

Figure 10: bnlearn : per-dataset interaction heatmaps using the same global bins as Fig. 6 , allowing absolute LLM-constraint-quality comparisons across benchmarks.

Figure 11: bnlearn : per-dataset interaction heatmaps using within-dataset quantile bins, highlighting relative variation inside each benchmark.

Group	Metric	Methods	Means $\pm$ Std	t	p-value	$p_{\mathrm{BH}}$
$\|\mathbf{V}\|=5$	SHD	ABAPC-LLM vs MPC-LLM	$1.230\pm\pm 0.369$ vs $1.500\pm\pm 0.318$	$-5.60$	$2.19\text{e-}8$	$2.19\text{e-}8$
$\|\mathbf{V}\|=5$	Precision	MPC-LLM vs ABAPC-LLM	$0.958\pm\pm 0.031$ vs $0.882\pm\pm 0.046$	$12.94$	$2.57\text{e-}38$	$2.57\text{e-}38$
$\|\mathbf{V}\|=5$	Recall	ABAPC-LLM vs MPC-LLM	$0.763\pm\pm 0.062$ vs $0.710\pm\pm 0.050$	$5.12$	$3.13\text{e-}7$	$3.13\text{e-}7$
$\|\mathbf{V}\|=5$	F1	ABAPC-LLM vs MPC-LLM	$0.812\pm\pm 0.049$ vs $0.795\pm\pm 0.037$	$1.98$	$0.048$	$0.048$
$\|\mathbf{V}\|=10$	SHD	ABAPC-LLM vs ABAPC	$5.626\pm\pm 1.176$ vs $6.166\pm\pm 1.442$	$-5.03$	$4.89\text{e-}7$	$4.89\text{e-}7$
$\|\mathbf{V}\|=10$	Precision	BOSS vs MPC-LLM	$0.880\pm\pm 0.197$ vs $0.856\pm\pm 0.088$	$4.34$	$1.40\text{e-}5$	$1.40\text{e-}5$

Appendix

Table 10: Welch tests for Table 1 . Each row compares a bolded top entry with the next non-bold method in that column. Significance levels for both reported

p

-values are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

Table	Group	Metric	Methods	Means $\pm$ Std	t	p-value		$p_{\mathrm{BH}}$
1	$\|\mathbf{V}\|=15$	Precision	MPC vs MPC-LLM	$0.936\pm\pm 0.071$ vs $0.929\pm\pm 0.061$	$1.91$	$0.056$	.	$0.056$	.
3	SF $\|\mathbf{V}\|=5$	F1	ABAPC-LLM vs MPC-LLM	$0.734\pm\pm 0.046$ vs $0.697\pm\pm 0.055$	$1.81$	$0.071$	.	$0.071$	.
3	LT $\|\mathbf{V}\|=5$	SHD	MPC-LLM vs ABAPC-LLM	$1.217\pm\pm 0.345$ vs $1.335\pm\pm 0.457$	$-1.20$	$0.231$	ns	$0.231$	ns
3	ER $\|\mathbf{V}\|=10$	F1	MPC-LLM vs ABAPC-LLM	$0.663\pm\pm 0.057$ vs $0.656\pm\pm 0.057$	$0.57$	$0.570$	ns	$0.570$	ns
3	SF $\|\mathbf{V}\|=10$	SHD	ABAPC-LLM vs MPC-LLM	$5.711\pm\pm 1.302$ vs $6.112\pm\pm 1.198$	$-1.95$	$0.051$	.	$0.051$	.
3	LT $\|\mathbf{V}\|=10$	F1	BOSS vs ABAPC-LLM	$0.638\pm\pm 0.192$ vs $0.623\pm\pm 0.111$	$1.67$	$0.095$	.	$0.095$	.

Appendix

Table 11: Non-significant best-vs-comparator tests that justify additional bolded methods in Tables 1 , 3 , 4 , and 9 . Significance levels for both reported

p

-values are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

Group	Metric	Methods	Means $\pm$ Std	t	p-value		$p_{\mathrm{BH}}$
ER, $\|\mathbf{V}\|=5$	SHD	ABAPC-LLM vs ABAPC	$1.054\pm\pm 0.372$ vs $1.515\pm\pm 0.542$	$-7.91$	$2.63\text{e-}15$		$2.63\text{e-}15$
ER, $\|\mathbf{V}\|=5$	F1	ABAPC-LLM vs MPC-LLM	$0.843\pm\pm 0.044$ vs $0.805\pm\pm 0.025$	$3.94$	$8.25\text{e-}5$		$8.25\text{e-}5$
SF, $\|\mathbf{V}\|=5$	SHD	ABAPC-LLM vs MPC-LLM	$1.302\pm\pm 0.278$ vs $1.603\pm\pm 0.305$	$-3.06$	$0.002$	**	$0.002$	**
SF, $\|\mathbf{V}\|=5$	F1	ABAPC-LLM vs ABAPC	$0.734\pm\pm 0.046$ vs $0.632\pm\pm 0.075$	$7.05$	$1.79\text{e-}12$		$3.59\text{e-}12$
SF, $\|\mathbf{V}\|=5$	F1	MPC-LLM vs ABAPC	$0.697\pm\pm 0.055$ vs $0.632\pm\pm 0.075$	$3.91$	$9.07\text{e-}5$		$9.07\text{e-}5$
LT, $\|\mathbf{V}\|=5$	SHD	MPC-LLM vs ABAPC	$1.217\pm\pm 0.345$ vs $2.709\pm\pm 0.722$	$-13.57$	$5.71\text{e-}42$		$1.14\text{e-}41$

Appendix

Table 12: Welch tests for Table 3 , including marginal averages. Significance levels for both reported

p

-values are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

Group	Metric	Methods	Means $\pm$ Std	t	p-value		$p_{\mathrm{BH}}$
$\alpha=0.01$ , $\|\mathbf{V}\|=5$	SHD	ABAPC-LLM vs MPC-LLM	$1.230\pm\pm 0.369$ vs $1.500\pm\pm 0.318$	$-5.60$	$2.19\text{e-}8$		$2.19\text{e-}8$
$\alpha=0.01$ , $\|\mathbf{V}\|=5$	F1	ABAPC-LLM vs MPC-LLM	$0.812\pm\pm 0.049$ vs $0.795\pm\pm 0.037$	$1.98$	$0.048$		$0.048$
$\alpha=0.01$ , $\|\mathbf{V}\|=10$	SHD	ABAPC-LLM vs ABAPC	$5.626\pm\pm 1.176$ vs $6.166\pm\pm 1.442$	$-5.03$	$4.89\text{e-}7$		$4.89\text{e-}7$
$\alpha=0.01$ , $\|\mathbf{V}\|=10$	F1	ABAPC-LLM vs MPC-LLM	$0.653\pm\pm 0.084$ vs $0.627\pm\pm 0.077$	$4.38$	$1.17\text{e-}5$		$1.17\text{e-}5$
$\alpha=0.01$ , $\|\mathbf{V}\|=15$	SHD	ABAPC-LLM vs MPC-LLM	$7.550\pm\pm 1.281$ vs $7.874\pm\pm 1.038$	$-2.94$	$0.003$	**	$0.003$	**
$\alpha=0.01$ , $\|\mathbf{V}\|=15$	F1	MPC-LLM vs ABAPC-LLM	$0.667\pm\pm 0.058$ vs $0.654\pm\pm 0.072$	$2.52$	$0.012$		$0.012$

Appendix

Table 13: Welch tests for Table 4 . Significance levels for both reported

p

-values are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

Group	Metric	Methods	Means $\pm$ Std	t	p-value	$p_{\mathrm{BH}}$
earthquake	SHD	MPC vs NOTEARS-MLP	$0.040\pm\pm 0.200$ vs $2.120\pm\pm 1.170$	$-12.39$	$2.92\text{e-}35$	$8.77\text{e-}35$
earthquake	SHD	MPC-LLM vs NOTEARS-MLP	$0.040\pm\pm 0.200$ vs $2.120\pm\pm 1.170$	$-12.39$	$2.92\text{e-}35$	$8.77\text{e-}35$
earthquake	SHD	ABAPC-LLM vs NOTEARS-MLP	$0.080\pm\pm 0.340$ vs $2.120\pm\pm 1.170$	$-11.84$	$2.48\text{e-}32$	$4.97\text{e-}32$
earthquake	SHD	ABAPC vs NOTEARS-MLP	$0.100\pm\pm 0.463$ vs $2.120\pm\pm 1.170$	$-11.35$	$7.25\text{e-}30$	$1.09\text{e-}29$
earthquake	SHD	GRaSP vs NOTEARS-MLP	$0.160\pm\pm 0.790$ vs $2.120\pm\pm 1.170$	$-9.82$	$9.49\text{e-}23$	$1.14\text{e-}22$
earthquake	SHD	BOSS vs NOTEARS-MLP	$0.400\pm\pm 1.210$ vs $2.120\pm\pm 1.170$	$-7.23$	$4.98\text{e-}13$	$4.98\text{e-}13$

Appendix

Table 14: Welch tests for the earthquake rows in Table 5 . Significance levels for both reported

p

-values are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

Group	Metric	Methods	Means $\pm$ Std	t	p-value	$p_{\mathrm{BH}}$
asia	SHD	MPC-LLM vs ABAPC-LLM	$2.840\pm\pm 0.470$ vs $3.580\pm\pm 0.710$	$-6.15$	$7.89\text{e-}10$	$7.89\text{e-}10$
asia	Precision	ABAPC-LLM vs ABAPC	$0.992\pm\pm 0.040$ vs $0.684\pm\pm 0.207$	$10.31$	$6.65\text{e-}25$	$9.97\text{e-}25$
asia	Precision	MPC-LLM vs ABAPC	$0.990\pm\pm 0.030$ vs $0.684\pm\pm 0.207$	$10.32$	$5.96\text{e-}25$	$9.97\text{e-}25$
asia	Precision	MPC vs ABAPC	$0.980\pm\pm 0.080$ vs $0.684\pm\pm 0.207$	$9.41$	$5.11\text{e-}21$	$5.11\text{e-}21$
asia	Recall	MPC-LLM vs ABAPC-LLM	$0.650\pm\pm 0.050$ vs $0.557\pm\pm 0.082$	$6.83$	$8.49\text{e-}12$	$8.49\text{e-}12$
asia	F1	MPC-LLM vs ABAPC-LLM	$0.780\pm\pm 0.040$ vs $0.710\pm\pm 0.068$	$6.22$	$5.00\text{e-}10$	$5.00\text{e-}10$

Appendix

Table 15: Welch tests for Table 5 , excluding the earthquake and child rows. Significance levels for both reported

p

-values are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

Group	Metric	Methods	Means $\pm$ Std	t	p-value	$p_{\mathrm{BH}}$
child	SHD	ABAPC-LLM vs MPC	$4.417\pm\pm 2.009$ vs $9.160\pm\pm 4.130$	$-7.30$	$2.82\text{e-}13$	$8.45\text{e-}13$
child	SHD	MPC-LLM vs MPC	$5.060\pm\pm 1.800$ vs $9.160\pm\pm 4.130$	$-6.44$	$1.23\text{e-}10$	$1.85\text{e-}10$
child	SHD	ABAPC vs MPC	$5.372\pm\pm 3.250$ vs $9.160\pm\pm 4.130$	$-5.10$	$3.46\text{e-}7$	$3.46\text{e-}7$
child	Precision	BOSS vs ABAPC-LLM	$0.960\pm\pm 0.080$ vs $0.894\pm\pm 0.050$	$4.94$	$7.67\text{e-}7$	$7.67\text{e-}7$
child	Recall	ABAPC-LLM vs MPC	$0.824\pm\pm 0.080$ vs $0.630\pm\pm 0.170$	$7.31$	$2.59\text{e-}13$	$7.76\text{e-}13$
child	Recall	MPC-LLM vs MPC	$0.800\pm\pm 0.070$ vs $0.630\pm\pm 0.170$	$6.54$	$6.22\text{e-}11$	$9.32\text{e-}11$

Appendix

Table 16: Welch tests for the child rows in Table 5 . Significance levels for both reported

p

-values are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

Group	Metric	Methods	Means $\pm$ Std	t	p-value		$p_{\mathrm{BH}}$
asia	Precision	ABAPC-LLM vs MPC	$0.992\pm\pm 0.040$ vs $0.980\pm\pm 0.080$	$0.95$	$0.342$	ns	$0.391$	ns
asia	Precision	ABAPC-LLM vs MPC-LLM	$0.992\pm\pm 0.040$ vs $0.990\pm\pm 0.030$	$0.28$	$0.776$	ns	$0.776$	ns
cancer	SHD	MPC-LLM vs ABAPC-LLM	$0.680\pm\pm 0.590$ vs $0.680\pm\pm 0.587$	$0.00$	$1.000$	ns	$1.000$	ns
cancer	Precision	ABAPC-LLM vs MPC-LLM	$0.996\pm\pm 0.028$ vs $0.990\pm\pm 0.050$	$0.74$	$0.460$	ns	$0.460$	ns
cancer	Recall	MPC-LLM vs ABAPC-LLM	$0.840\pm\pm 0.130$ vs $0.835\pm\pm 0.148$	$0.18$	$0.858$	ns	$0.858$	ns
cancer	F1	MPC vs MPC-LLM	$0.940\pm\pm 0.180$ vs $0.900\pm\pm 0.080$	$1.44$	$0.151$	ns	$0.167$	ns

Appendix

Table 17: Non-significant best-vs-comparator tests that justify additional bolded methods in Table 5 . Significance levels for both reported

p

-values are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

Group	Metric	Methods	Means $\pm$ Std	t	p-value	$p_{\mathrm{BH}}$
$\|\mathbf{V}\|=5$	SHD	ABAPC-LLM (Gemini) vs MPC-LLM (Gemini)	$1.186\pm\pm 0.375$ vs $1.328\pm\pm 0.331$	$-2.57$	$0.010$	$0.010$
$\|\mathbf{V}\|=5$	Precision	MPC-LLM (Gemini) vs ABAPC-LLM (Gemini)	$0.952\pm\pm 0.027$ vs $0.902\pm\pm 0.043$	$7.39$	$1.50\text{e-}13$	$1.50\text{e-}13$
$\|\mathbf{V}\|=5$	Recall	ABAPC-LLM (Gemini) vs MPC-LLM (Gemini)	$0.778\pm\pm 0.063$ vs $0.753\pm\pm 0.053$	$2.20$	$0.028$	$0.028$
$\|\mathbf{V}\|=5$	F1	ABAPC-LLM (Gemini) vs ABAPC-LLM (GPT)	$0.828\pm\pm 0.048$ vs $0.755\pm\pm 0.084$	$7.82$	$5.42\text{e-}15$	$1.08\text{e-}14$
$\|\mathbf{V}\|=5$	F1	MPC-LLM (Gemini) vs ABAPC-LLM (GPT)	$0.822\pm\pm 0.041$ vs $0.755\pm\pm 0.084$	$6.56$	$5.37\text{e-}11$	$5.37\text{e-}11$
$\|\mathbf{V}\|=10$	SHD	ABAPC-LLM (GPT) vs MPC-LLM (Gemini)	$5.755\pm\pm 1.417$ vs $6.197\pm\pm 1.095$	$-3.63$	$2.81\text{e-}4$	$5.62\text{e-}4$

Appendix

Table 18: Welch tests for Table 9 . Significance levels for both reported

p

-values are: ***

p<0.001

, **

p<0.01

, *

p<0.05

, .

p<0.1

Causal Discovery

Causal discovery infers cause-and-effect structures from observational and interventional data, distinguishing causal relationships from correlation.

See all 221 papers

Why LLMs Fail at Causal Discovery and How Interventional Agents Escape
May 26, 2026Amartya Roy, Sonali ParbhooCausal Discovery Causal Reasoning in Language Models
SIRE, IIT Delhi and Robert Bosch GmbH, India · Imperial College London
Causal Discovery in the Era of Agents
Jun 22, 2026Yujia Zheng, Vishal Verma, Mantej Gill +3AI Agents for Scientific Discovery Causal Discovery
1Carnegie Mellon University. · 2Mohamed bin Zayed University of Artificial Intelligence.
CauTion: Knowing When to Trust LLMs for Ensemble Causal Discovery
Jun 2, 2026Bo Peng, Kaiwen Wu, Sirui Chen +3Causal Discovery
Shanghai AI Laboratory · Shanghai Innovation Institute · Shanghai Jiao Tong University +2

May 26, 2026J/K move · Enter open · S save

Why LLMs Fail at Causal Discovery and How Interventional Agents Escape

Amartya Roy, Sonali Parbhoo

SIRE, IIT Delhi and Robert Bosch GmbH, India · Imperial College London

Causal Discovery Causal Reasoning in Language Models Causal Identifiability Causal Interventions in Language Models

Open paper AskPDF arXiv

Causal discovery is a cornerstone of scientific reasoning, yet whether large language models can perform it reliably remains an open question. Recent benchmarks show that even fine-tuned models plateau on simple causal graphs and degrade as complexity grows, but why they fail has not been established. We prove the failure is fundamental: supervised fine-tuning, direct preference optimization, and in-context learning all produce predictors that cannot distinguish between causal graphs generating similar observational data, and any attempt to do so requires the model's internal representations to grow unboundedly, violating the very conditions under which these methods work. We formalize this as a kernel obstruction theorem, establishing that the limitation is intrinsic to the learning paradigm, not any particular model or dataset. We propose Agentic Causal Bayesian Optimization (A-CBO), wherein a frozen language model serves as an interventional oracle answering targeted queries about intervention effects, while an external Bayesian loop concentrates beliefs over candidate graphs in logarithmically many rounds. Because the decision operates outside the space where the obstruction applies, A-CBO provably converges while the underlying model remains unchanged. On Corr2Cause, A-CBO matches fine-tuned baselines without any training. On Extended Corr2Cause, a new benchmark scaling to 24 variables with 18K test samples, A-CBO significantly outperforms both fine-tuning and preference optimization, with the advantage growing

Abstract

Causal Discovery

Explore similar work

Why LLMs Fail at Causal Discovery and How Interventional Agents Escape

Causal Discovery in the Era of Agents

CauTion: Knowing When to Trust LLMs for Ensemble Causal Discovery