Contextual Causality with Large Language Models: A Survey

Yiheng Zhao; Jun Yan; Chengming Hu

cs.CLSep 18, 2026

Contextual Causality with Large Language Models: A Survey

Authors: Yiheng Zhao, Jun Yan, Chengming Hu

Organizations: Concordia University, Montreal, Canada · Mcgill University, Montreal, Canada

Abstract

Understanding contextual causality is critical for large language models (LLMs), as it enables them to accurately identify causal relations in specific situations and support more reliable decision-making. Despite its significance, a systematic exploration of contextual causality with LLMs is still lacking. To fill this gap, we present a comprehensive survey on this topic. In this survey, we first propose a taxonomy of contextual causality, consisting of semantic, intervention, and counterfactual causality, and characterize each category by its core causal question, required model capabilities, representative tasks, and practical uses in causality analysis. We then analyze existing studies and discuss their key limitations. Finally, we examine the gaps between current benchmarks and real-world needs and outline promising directions for future research. Our goal is to clarify the research landscape of contextual causality with LLMs, emphasize its importance, and highlight promising future directions.

Figures & tables

Panel of Table 1 — Table 1: Comparison with methodology-oriented causality surveys. ✓ indicates systematic coverage, indicates coverage of only a few relevant studies, and ✗ indicates no systematic coverage. ( Wang et al., 2024b ) mainly covers counterfactual generation (CG), whereas ours covers broader counterfactual reasoning (CR).

Core Causal Question	Capability	Task	Practical Use
Semantic Causality
What causal information is expressed or implied in context?	Identification	Event Causality Identification	Causal Graph Construction
		Causal Judgment	Causal Link Refinement
		Causal Attribution	Responsibility Assignment
		Causal Detection and Span Extraction	Causal Filtering and Evidence Grounding
	Explanation	Causal Explanation Generation	Causal Mechanism Explanation

Table 2: A taxonomy-driven overview of contextual causality with LLMs. The “Core Question” column specifies the causal question addressed by each category. The “Capability” column summarizes the causal ability required of LLMs for each category. The “Tasks” column lists representative tasks corresponding to each capability. The “Practical Use” column describes how each task can be employed in practical causality analysis.

Benchmark	Source	Focus
Semantic Causality
Event Causal Identification
MAVEN-ERE Wang et al. (2022a)	Wiki	Intrinsic
MECI Lai et al. (2022)	Wiki	Intrinsic
ESL Caselli and Vossen (2017)	News	Intrinsic
CTB Mirza et al. (2014)	News	Intrinsic

Table 3: Representative benchmarks for contextual causality. “Focus” indicates whether a benchmark measures causality ability or how much the causality ability of current LLMs improves decision-making for real-world needs. “Intrinsic” denotes the former, while “Extrinsic” denotes the latter. “Wiki”, “W&N”, “Human”, and “Agents” denote Wikipedia-derived data, mixed Wikipedia and news sources, human-constructed based on world or commonsense knowledge, and agent-system traces or execution logs, respectively.

Causal Reasoning in Language Models

Causal reasoning in language models studies how models identify cause-and-effect structure, predict interventions, and answer counterfactual questions.

See all 102 papers

METER: Evaluating Multi-Level Contextual Causal Reasoning in Large Language Models
Apr 13, 2026Pengfeng Li, Chen Huang, Chaoqun Hao +4Causal Reasoning in Language Models LLM Evaluation
College of Computer Science, Sichuan University · Engineering Research Center of Machine Learning and Industry Intelligence, Ministry of Education, China · Institute of Data Science, National University of Singapore
CausalBN-Bench: A Comprehensive Benchmark for Causal Learning Capability of LLMs
Apr 9, 2024Yu Zhou, Xingyu Wu, Jibin Wu +2LLM Evaluation Causal Discovery
Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University, Hong Kong SAR 999077, China · College of Computer Science, Chongqing University, Chongqing 400044, China
Causality Elicitation from Large Language Models
Mar 4, 2026Takashi Kameyama, Masahiro Kato, Yasuko Hio +2Causal Reasoning in Language Models
Mizuho-DL Financial Technology Co., Ltd.

Apr 13, 2026J/K move · Enter open · S save

METER: Evaluating Multi-Level Contextual Causal Reasoning in Large Language Models

Pengfeng Li, Chen Huang, Chaoqun Hao, Hongyao Chen+3

College of Computer Science, Sichuan University · Engineering Research Center of Machine Learning and Industry Intelligence, Ministry of Education, China · Institute of Data Science, National University of Singapore

Causal Reasoning in Language Models LLM Evaluation LLM Interpretability

Open paper AskPDF arXiv

Contextual causal reasoning is a critical yet challenging capability for Large Language Models (LLMs). Existing benchmarks, however, often evaluate this skill in fragmented settings, failing to ensure context consistency or cover the full causal hierarchy. To address this, we pioneer METER to systematically benchmark LLMs across all three levels of the causal ladder under a unified context setting. Our extensive evaluation of various LLMs reveals a significant decline in proficiency as tasks ascend the causal hierarchy. To diagnose this degradation, we conduct a deep mechanistic analysis via both error pattern identification and internal information flow tracing. Our analysis reveals two primary failure modes: (1) LLMs are susceptible to distraction by causally irrelevant but factually correct information at lower level of causality; and (2) as tasks ascend the causal hierarchy, faithfulness to the provided context degrades, leading to a reduced performance. We belive our work advances our understanding of the mechanisms behind LLM contextual causal reasoning and establishes a critical foundation for future research. Our code and dataset are available at https://github.com/SCUNLP/METER .

Abstract

Causal Reasoning in Language Models

Explore similar work

METER: Evaluating Multi-Level Contextual Causal Reasoning in Large Language Models

CausalBN-Bench: A Comprehensive Benchmark for Causal Learning Capability of LLMs

Causality Elicitation from Large Language Models