Discovering and Reasoning of Causality in the Hidden World with Large Language Models

Chenxi Liu; Yongqiang Chen; Tongliang Liu; Mingming Gong; James Cheng; Bo Han; Kun Zhang

cs.LGFeb 6, 2024

Discovering and Reasoning of Causality in the Hidden World with Large Language Models

Authors: Chenxi Liu, Yongqiang Chen, Tongliang Liu, Mingming Gong, James Cheng, Bo Han, Kun Zhang

Organizations: TMLR Group, Hong Kong Baptist University, Kowloon, Hong Kong SAR · Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · Department of Philosophy, Carnegie Mellon University, Pittsburgh, PA 15213, USA · School of Computer Science, The University of Sydney, NSW 2008, Australia ·

Abstract

Revealing hidden causal variables alongside the underlying causal mechanisms is essential to the development of science. Despite the progress in the past decades, existing practice in causal discovery (CD) heavily relies on high-quality measured variables, which are usually given by human experts. In fact, the lack of well-defined high-level variables behind unstructured data has been a longstanding roadblock to a broader real-world application of CD. This procedure can naturally benefit from an automated process that can suggest potential hidden variables in the system. Interestingly, Large language models (LLMs) are trained on massive observations of the world and have demonstrated great capability in processing unstructured data. To leverage the power of LLMs, we develop a new framework termed Causal representatiOn AssistanT (COAT) that incorporates the rich world knowledge of LLMs to propose useful measured variables for CD with respect to high-value target variables on their paired unstructured data. Instead of directly inferring causality with LLMs, COAT constructs feedback from intermediate CD results to LLMs to refine the proposed variables. Given the target variable and the paired unstructured data, we first develop COAT-MB that leverages the predictivity of the proposed variables to iteratively uncover the Markov Blanket of the target variable. Built upon COAT-MB, COAT-PAG further extends to uncover a more complete causal graph, i.e., Partial Ancestral Graph, by iterating over the target variables and actively seeking new high-level variables. Moreover, the reliable CD capabilities of COAT also extend the debiased causal inference to unstructured data by discovering an adjustment set. We establish theoretical guarantees for the CD results and verify their efficiency and reliability across realistic benchmarks and real-world case studies.

Figures & tables

	Content	Clarification
Method	COAT-MB (Algorithm 1 ; Section 3.3 )	inherited
	COAT-PAG (Algorithm 2 ; Section 3.4 )	new
	COAT-ADJ (Algorithm 3 ; Section 3.5 )	new
Theoretical Analysis	Identifiability on Markov Blanket (Section 4.1 )	inherited
	Identifiability on Partial Ancestral Graphs (Section 4.2 )	new
	Identifiability on Adjustment Sets (Section 4.3 )	new

Table 1: Comparison of main results with conference version ( 2024 ).

Panel of Figure 1 — Figure 1: An example where PAG is less informative when limited in $\text{MB}(Y)$ . (a) is the true causal structure indicating causal directions; (b) and (c) are the partial ancestral graphs. The circle mark ( $\circ$ ) means it is undetermined to be arrow head or tail. The node out of $\text{MB}(Y)$ is colored.

Panel of Figure 2 — Figure 2: Examples of Partial Ancestral Graph and two Markov-equivalent mixed graphs.

Figure 3: Illustration of COAT framework in eliciting the Markov Blanket of the rating score from the paired text reviews. COAT first (a) adopts an LLM to read, comprehend, and relate the rich knowledge about reviews on apples. The LLM needs to propose a series of candidate factors such as apple sizes and smells, along with some meta-information such as annotation guidelines. Based on the candidate factors, COAT then (b) prompts another LLM to annotate the unstructured review into structured data. (c) The CD algorithm then finds causal relations among the factors, and constructs feedback based on samples where the ratings can not be well explained by the existing factors. By looking into the new samples, the LLM is expected to use more related knowledge to uncover the desired causal factors.

Figure 4: Illustration of the prompt template for factor proposal in COAT .

Panel of Figure 5 — Figure 5: Illustration of variables that could be discovered with COAT . Let $Y$ be the target variable, and $W$ be a factor that has been discovered, and also assume a latent variable $U=\widehat{{\bm{w}}}({\bm{X}})\in\text{MB}(Y)$ . Conditioning on $W$ facilitates the discovery of $U$ .

Panel of Figure 6 — Figure 6: An example where arrow heads cannot be revealed within $\text{MB}^{(N)}(Y)$ .

Panel of Figure 7 — Figure 7: An example where COAT-PAG is conducted in several loops. $\text{S}_{\text{target}}$ are colored.

Figure 8: An illustration of COAT-MB with uncertainty estimation. Deeper colors of discovered factors indicate lower uncertainty.

Assumption	Notes
I. LLM Capability Assumptions
Propose valid factors, i.e., $p_{\Psi}>0$ and $C_{\Psi}>0$ (Definition 3 ).	Validated in Figure 10 (c) and Table 15
Accurate annotation with independent noise (Assumption 1 ).	Validated in Figure 10 (a-b) and Table 5
II. Causal Discovery Assumptions ( Spirtes et al., 1995 )
Faithfulness and Markov condition	Depends on the specific algorithm (e.g., FCI, LiNGAM)
Validity of conditional independence tests

Table 2: Summary of Core Assumptions for COAT Methods.

Panel of Figure 9 — Figure 9: The discovered causal graphs in AppleGastronome . Compared to the ground truth results, directly adopting LLMs to reason the causal relations can easily elicit many false positive edges. In contrast, the relations recovered by COAT have a high precision and recall. The directed edge between “taste” and “juiciness” can not be recovered by COAT because of the limitations of FCI.

LLMs	Method	MB	NMB	OT	Recall	Precision	F1
GPT-4o	META	3.00 $\pm$ 0.82	1.00 $\pm$ 0.00	4.00 $\pm$ 0.82	0.60 $\pm$ 0.16	0.37 $\pm$ 0.09	0.46 $\pm$ 0.12
	DATA	3.00 $\pm$ 0.00	0.67 $\pm$ 0.47	0.00 $\pm$ 0.00	0.60 $\pm$ 0.00	0.83 $\pm$ 0.12	0.69 $\pm$ 0.04
	DATA+CoT	4.67 $\pm$ 0.58	1.00 $\pm$ 0.00	0.33 $\pm$ 0.58	0.93 $\pm$ 0.12	0.78 $\pm$ 0.10	0.85 $\pm$ 0.10
	COAT	4.00 $\pm$ 0.82	0.00 $\pm$ 0.00	0.00 $\pm$ 0.00	0.80 $\pm$ 0.16	1.00 $\pm$ 0.00	0.88 $\pm$ 0.10
GPT-4	META	2.67 $\pm$ 0.94	0.67 $\pm$ 0.47	2.33 $\pm$ 0.47	0.53 $\pm$ 0.19	0.46 $\pm$ 0.08	0.49 $\pm$ 0.13
	DATA	3.00 $\pm$ 0.00	0.33 $\pm$ 0.47	0.00 $\pm$ 0.00	0.60 $\pm$ 0.00	0.92 $\pm$ 0.12	0.72 $\pm$ 0.04

Table 3: Full Results on the Apple Gastronome Benchmark.

LLM	Method	SHD	SID	Recall	Precision	F1
GPT-4o	pairwise	5.33 $\pm$ 2.62	0.00 $\pm$ 0.00	1.00 $\pm$ 0.00	0.45 $\pm$ 0.07	0.62 $\pm$ 0.07
	COAT	1.00 $\pm$ 0.00	0.00 $\pm$ 0.00	1.00 $\pm$ 0.00	0.79 $\pm$ 0.03	0.89 $\pm$ 0.02
GPT-4	pairwise	8.67 $\pm$ 2.05	1.33 $\pm$ 1.89	0.93 $\pm$ 0.09	0.35 $\pm$ 0.05	0.51 $\pm$ 0.06
	COAT	2.33 $\pm$ 0.47	1.00 $\pm$ 1.41	0.93 $\pm$ 0.09	0.70 $\pm$ 0.09	0.79 $\pm$ 0.04
GPT-3.5	pairwise	5.67 $\pm$ 4.03	3.00 $\pm$ 2.94	0.67 $\pm$ 0.24	0.49 $\pm$ 0.36	0.55 $\pm$ 0.32
	COAT	2.00 $\pm$ 0.82	2.67 $\pm$ 2.05	0.75 $\pm$ 0.20	0.63 $\pm$ 0.21	0.68 $\pm$ 0.21

Table 4: Causal relation extraction results in AppleGastronome .

Panel of Figure 10 — Figure 10: Quantitative evaluation of the causal capabilities of LLMs in COAT .

LLM	Test Object	T	P-value
GPT-4	Feature	0.2828	0.9997
GPT-4	Noise	0.0327	0.9962
GPT-3.5	Feature	0.4803	0.0325
GPT-3.5	Noise	0.0446	0.9962

Table 5: Independence tests of the annotation noises with annotated features and other noises in AppleGastronome .

	V1	V2	X1	X2	FCI Orientation Rules
graph 1	Taste	Score	Nutrition	Market Potential	${\mathcal{R}}$ 0, ${\mathcal{R}}$ 1
graph 2	Taste	Score	Juiciness	Size	${\mathcal{R}}$ 0
graph 3	Smell	Score	Market Potential	Nutrition	${\mathcal{R}}$ 0, ${\mathcal{R}}$ 4
graph 4	Juiciness	Score	Nutrition	Smell	${\mathcal{R}}$ 0, ${\mathcal{R}}$ 2, ${\mathcal{R}}$ 4
graph 5	Market Potential	Score	Taste	Juiciness	${\mathcal{R}}$ 3

Table 6: The variable specification for different PAGs.