Causal Reasoning in Language Models

Follow this topic Ask about this topic

The field asks whether language models can go beyond patterns of correlation to identify causes, predict what would happen under interventions, and reason about what might have happened otherwise. Researchers test these abilities with causal graphs, counterfactual questions, context-sensitive tasks in areas such as medicine and policy, and interactive settings where models must choose experiments. Approaches include prompting with explicit graphs, retrieval and tool use, multi-agent experiment design, and training with simulated interventions or step-level feedback. Models can handle some familiar causal patterns, but often confuse association with cause, miss context changes, or give fluent answers unsupported by a valid causal path; a central challenge is distinguishing genuine causal competence from shortcuts in language or benchmark performance.

Latest papers

102 papers13 papers in the last four weeks, up 30% on the four weeks before. 0.1% of all new papers.

InterveneBench: Benchmarking LLMs for Intervention Reasoning and Causal Study Design in Real Social Systems
Mar 16, 2026Shaojie Shi, Zhengyu Shi, Lingran Zheng +15Causal Reasoning in Language Models Causal Inference
Fudan University · Shanghai Innovation Institute
OOD-MMSafe: Advancing MLLM Safety from Harmful Intent to Hidden Consequences
Mar 10, 2026Ming Wen, Kun Yang, Jingyu Zhang +4MLLM Safety LLM Safety
Fudan University, Shanghai, China · Shanghai Innovation Institute, Shanghai, China · Ant Group, Hangzhou, China +1
HART: Data-Driven Hallucination Attribution and Evidence-Based Tracing for Large Language Models
Mar 6, 2026Shize Liang, Hongzhi WangHallucination in Language Models LLM Interpretability
Faculty of Computing, Harbin Institute of Technology, Harbin, China
AILS-NTUA at SemEval-2026 Task 12: Graph-Based Retrieval and Reflective Prompting for Abductive Event Reasoning
Mar 4, 2026Nikolas Karafyllis, Maria Lymperaiou, Giorgos Filandrianos +2Abductive Reasoning Causal Reasoning in Language Models
School of Electrical and Computer Engineering, AILS Laboratory, National Technical University of Athens
Causality Elicitation from Large Language Models
Mar 4, 2026Takashi Kameyama, Masahiro Kato, Yasuko Hio +2Causal Reasoning in Language Models
Mizuho-DL Financial Technology Co., Ltd.
CausalFlip: A Benchmark for LLM Causal Judgment Beyond Semantic Matching
Feb 23, 2026Yuzhe Wang, Yaochen Zhu, Jundong LiLLM Evaluation Causal Reasoning in Language Models
University of Virginia Charlottesville, Virginia, USA
Better Think Thrice: Learning to Reason Causally with Double Counterfactual Consistency
Feb 18, 2026Victoria Lin, Xinnuo Xu, Rachel Lawrence +4Counterfactual Reasoning in Language Models Causal Reasoning in Language Models
Carnegie Mellon University · Microsoft Research Cambridge · Microsoft Research India
ReBeCA: Causality-Inspired Behavior Analysis for Understanding the Self-Reflection of Language Models
Feb 6, 2026Tianqiang Yan, Lizhen Qu, Sihan Shang +6LLM Self-Refinement Causal Interpretability
Monash University · Harbin Institute of Technology, Shenzhen · Hainan University +2
Retell, Reward, Repeat: Reinforcement Learning for Narrative Theory-Informed Story Retelling
Jan 23, 2026David Y. Liu, Xanthe Muston, Dipankar Srirag +2Narrative Generation Text Generation
University of New South Wales, Sydney, Australia
EconCausal: A Context-Aware Economic Reasoning Benchmark for Large Language Models
Oct 8, 2025Donggyu Lee, Hyeok Yun, Meeyoung Cha +3Causal Reasoning in Language Models LLM Evaluation
Graduate School of Data Science, KAIST · College of Business, KAIST · Data Science for Humanity Group, MPI-SP +2
Synergizing Large Language Models and Theory for Human-like Causal Reasoning
May 13, 2025Yanxi Zhang, Xin Cong, Zhong Zhang +3Causal Reasoning in Language Models Causal Reasoning
Center for Data Science, AAIS, Peking University · Wangxuan Institute of Computer Technology, Peking University · Tsinghua University +1
CausalBN-Bench: A Comprehensive Benchmark for Causal Learning Capability of LLMs
Apr 9, 2024Yu Zhou, Xingyu Wu, Jibin Wu +2LLM Evaluation Causal Discovery
Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University, Hong Kong SAR 999077, China · College of Computer Science, Chongqing University, Chongqing 400044, China

Mar 16, 2026J/K move · Enter open · S save

InterveneBench: Benchmarking LLMs for Intervention Reasoning and Causal Study Design in Real Social Systems

Shaojie Shi, Zhengyu Shi, Lingran Zheng, Xinyu Su+14

Fudan University · Shanghai Innovation Institute

Causal Reasoning in Language Models Causal Inference Computational Social Science

Open paper AskPDF arXiv

Causal inference in social science relies on end-to-end, intervention-centered research-design reasoning grounded in real-world policy interventions, but current benchmarks fail to evaluate this capability of large language models (LLMs). We present InterveneBench, a benchmark designed to assess such reasoning in realistic social settings. Each instance in InterveneBench is derived from an empirical social science study and requires models to reason about policy interventions and identification assumptions without access to predefined causal graphs or structural equations. InterveneBench comprises 744 peer-reviewed studies across diverse policy domains. Experimental results show that state-of-the-art LLMs struggle under this setting. To address this limitation, we further propose a multi-agent framework, STRIDES. It achieves significant performance improvements over state-of-the-art reasoning models. Our code and data are available at https://github.com/Sii-yuning/STRIDES.