Counterfactual Reasoning in Language Models

Follow this topic Ask about this topic

Counterfactual reasoning asks whether a language model can predict what would happen if a relevant fact or action were different, and whether its answer actually depends on that change. Researchers test this with paired-world benchmarks, controlled prompt or evidence edits, causal graphs, and counterfactual explanations, while training methods include supervised fine-tuning, preference optimization, reinforcement learning, and multi-agent critique. Applications range from clinical decisions and legal analysis to strategic games, multimodal question answering, and explanations of model behavior. A central challenge is that fluent answers and high accuracy can hide weak causal reasoning: performance often varies across kinds of interventions, and robustness or faithfulness on one benchmark may not transfer to new settings.

Latest papers

45 papers9 papers in the last four weeks, up 80% on the four weeks before. 0.1% of all new papers.

Did the Model Use That? Counterfactual Training Boosts Self-Attribution in Language Models
Oct 26, 2026Zikai Zhou, Jiacheng Sang, Yilin Xu +2Data Provenance in Language Models Counterfactual Learning
Stanford University · Duke University · Emory University +1
From Document Layout to Causal Topology: An End-to-End Architecture for Temporal Causal Reasoning
Oct 25, 2026Zhifeng Yang, Wei Luo, Yongjian Chen +1Causal Reasoning in Language Models Temporal Reasoning in Language Models
School of Future Technology, South China University of Technology · Guangdong Artificial Intelligence and Digital Economy Laboratory (Guangzhou)
When Does Pre-Trained Capacity Help Causal Learning? From Scalar Prediction to Counterfactual Post-Training
Oct 23, 2026Qi Xu, Zijie Lei, Zhifei Deng +2Causal Inference Causal Effect Estimation
Meta Platforms, Inc., Sunnyvale, CA, USA
Interpreting Model Reasoning through Behavioral Counterfactuals
Oct 22, 2026Shawn Im, James Oldfield, Adel Bibi +1Causal Interventions in Language Models Faithfulness of Language Model Reasoning
University of Wisconsin–Madison · University of Oxford
ConCI: A Controlled Counterfactual Benchmark for Contextual Integrity in LLMs
Oct 17, 2026Juhyeon Park, Sumin Yu, Taesup MoonContextual Integrity LLM Safety Benchmarks
IPAI, Seoul National University · ECE, Seoul National University · ASRI / INMC / AIIS, Seoul National University
Watts If? A Benchmark for Counterfactual Reasoning About the AI Data Center Buildout
Oct 15, 2026Pankaj Chouhan, Rajarshi Das, Aditya Dubey +5Counterfactual Reasoning in Language Models Generative AI Evaluation
AlphaInfra · New York University · Envana Software Solutions
CivWorldBench: Evaluating Language World Models in Structured Multi-Agent Sandbox Worlds
Oct 12, 2026Ang Li, Haofei Yu, Zhengwei Tao +7LLM World Models World Model Evaluation
The Chinese University of Hong Kong · University of Illinois at Urbana-Champaign · UniPat AI +3
CoBe: A Benchmark For Conversational Counterfactual Text Editing
Oct 9, 2026Arvind Raghavan, Inwoo Hwang, Shreyas Havaldar +6Causal Counterfactual Generation Counterfactual Reasoning in Language Models
Causal Artificial Intelligence Lab Columbia University
Cited but Not Consulted: A Counterfactual Audit of Legal Chain-of-Thought Faithfulness
Oct 8, 2026Saisab Sadhu, Shreeyans Arora, Pratinav SethFaithfulness of Language Model Explanations LLM Auditing
Lexsi Labs
A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined
Oct 1, 2026Zhangshu Joshua Jiang, Zina Ibrahim, James T. TeoFaithfulness of Language Model Explanations Temporal Reasoning in Language Models
DRIVE-Health CDT Department of Biostatistics and Health Informatics Institute of Psychiatry, Psychology and Neuroscience King’s College London London, UK · Cleveland Clinic London London, UK
Counterfactual Self-Evolving Agents for Evidence-Grounded Reasoning
Sep 26, 2026Xing Han, Yuxin Wang, Chen Chen +8Self-Evolving Agents Evidence-Grounded Reasoning
Johns Hopkins University · MIT · University of Texas at Austin +1
Beyond Accuracy: Counterfactual Fragility and Demographic Bias in Clinical Evaluation of LLMs
Sep 26, 2026Chaitali Deb Purkayastha, Bharath Kumar Bolla, Vishnu Surya Reddy NandiClinical Language Model Evaluation Language Model Robustness
Independent Researcher, Bengaluru · The Institute of Product Leadership, Bengaluru, India · Amazon, Seattle, US
Contextual Causality with Large Language Models: A Survey
Sep 18, 2026Yiheng Zhao, Jun Yan, Chengming HuCausal Reasoning in Language Models Causal Interventions in Language Models
Concordia University, Montreal, Canada · Mcgill University, Montreal, Canada
An Empirical Study of Counterfactual Self-Explanations in LLMs
Sep 15, 2026Giannis Kalyvas, Giorgos Filandrianos, Orfeas Menis Mastromichalakis +2Counterfactual Explanations Faithfulness of Language Model Explanations
National Technical University of Athens, Athens, Greece · Instituto de Telecomunicações, Lisbon, Portugal
InfluenceField: A Differentiable Field with Interventionally Identifiable Causal Structure for Multimodal World Modeling
Sep 7, 2026Zihao Yang, Zijia Wang, Zhiqiu HuangCausal Representation Learning Causal World Models
Mathematical Institute, University of Oxford · Department of Computer Science, University of Oxford · School of Mathematical Sciences, University of Nottingham Ningbo China
The Illusion of $\textit{What If}$ : Evaluating the Breakdown of Counterfactual Reasoning in LLMs
Aug 28, 2026Yucheng Wang, Yuetian Du, Zhengyi Liu +8Counterfactual Reasoning in Language Models Counterfactual Reasoning
Zhejiang University · Alibaba Group · City University of Hong Kong
SCIT: Testing Causal Cache Carriers in Latent Chain-of-Thought Models
Aug 27, 2026Yi Ding, Lijun Huang, Menglin YangTransformer Interpretability Causal Interventions in Language Models
The Hong Kong University of Science and Technology (Guangzhou) Guangzhou, China
SDoH-Aware Narrative Anchoring Bias in Medical LLMs for Trustworthy Clinical Decision Support
Aug 24, 2026Ahnaf Atef Choudhury, Ramkrishna SahaMedical LLMs Language Model Bias Evaluation
Department of Information Sciences and Technology George Mason University, USA · Department of Computer Science The University of Texas at Dallas, USA
No Judgment Without a Reason: Counterfactual Receipts for Versioned AI Evaluators
Aug 21, 2026Ye Chen, Weining ZhangLLM Evaluation Counterfactual Evaluation
Alibaba Group · Cheung Kong Graduate School of Business
Would this change your answer? Evaluating Explanations of LLM Behavior In The Wild with Counterfactual Experiments
Aug 17, 2026Adam Karvonen, Euan Ong, Subhash Kantamneni +1Faithfulness of Language Model Explanations LLM Interpretability
Anthropic Fellows Program · Anthropic
What Would Fix This RAG Failure? Auditing Counterfactual Response with Paired Evidence Interventions
Aug 9, 2026Wenzhang DuRetrieval-Augmented Generation Counterfactual Reasoning in Language Models
Independent Researcher China
Counterfactual Benchmarking and Training for Factuality Consistency and Order-Robust Grounded Reasoning in LLMs over Heterogeneous Knowledge
Aug 8, 2026Shibo Chu, Yuze Liu, Tiehua Zhang +4LLM Evaluation LLM Grounding
Tongji University, Shanghai, China · Swinburne University of Technology, Melbourne, Australia · Wuhan University of Technology, Wuhan, China
Counterfactual Analysis via Large Language Models
Aug 5, 2026Zonghao YangFinancial Services Counterfactual Reasoning in Language Models
Stevens Institute of Technology
An Explicit World Model Based on Data-First Ontology: DaoQL Multimodal Storage Validation and Counterfactual Reasoning Evaluation
Jul 19, 2026Zhanbo Li, Shifeng Wu, Xiangjin Meng +1LLM Grounding Counterfactual Reasoning in Language Models
Graduate Students, School of Mathematics and Statistics, Chongqing University · DaoQL Project Group · Guangzhou Polytechnic Normal University
CRiT-QA: Evaluating Multi-hop Reasoning with Counterfactual Chains and Distractor Traps
Jul 12, 2026JungMin Yun, JuneHyoung Kwon, YoungBin KimMulti-Hop QA LLM Evaluation
Department of Artificial Intelligence, Chung-Ang University · Graduate School of Advanced Imaging Sciences, Multimedia and Film, Chung-Ang University
NormWorlds-CF: Solver-Verified Counterfactual Normative Reasoning with Metamorphic-Relation GRPO
Jul 4, 2026Xinqi ZhangCounterfactual Evaluation Moral Reasoning in Language Models
Tsinghua University
Testing Frontier Large Language Models' Physics Literacy in Parallel Physical Worlds
Jun 30, 2026Dong ZhangLLM Evaluation Physical Reasoning
PragReST: Self-Reinforcing Counterfactual Reasoning for Pragmatic Language Understanding
Jun 17, 2026Jihyung Park, Minchao Huang, Leqi Liu +1Pragmatic Reasoning in Language Models Counterfactual Reasoning in Language Models
The University of Texas at Austin
Re-feeding Is Not Replaying: Measuring Replay Noise in Counterfactual Token-Credit Estimation
Jun 14, 2026Nils MattesonCounterfactual Evaluation Token-Level Credit Assignment
Northeastern University
DMC-CF: Dynamic Multimodal CounterFactual QA benchmark for Causal Reasoning
May 28, 2026Junzhe Zhang, Huixuan Zhang, Guirong Wang +5Counterfactual Evaluation Counterfactual Reasoning in Language Models
Wangxuan Institute of Computer Technology, Peking University · Alibaba Group

Oct 26, 2026J/K move · Enter open · S save

Did the Model Use That? Counterfactual Training Boosts Self-Attribution in Language Models

Zikai Zhou, Jiacheng Sang, Yilin Xu, Yufei Jin+1

Stanford University · Duke University · Emory University+1

Data Provenance in Language Models Counterfactual Learning Counterfactual Reasoning in Language Models

Open paper AskPDF arXiv

A fundamental challenge for explainable and accountable language models is revealing how external information influences their answers. This paper examines whether models are self-attributive: that is, whether they indicate when -- and only when -- their responses to questions are influenced by information provided alongside the question. Furthermore, we explore how self-attribution can be improved through counterfactual training, which explicitly teaches models to distinguish between answers that do and do not depend on supplied information. We found that most of the eleven frontier models rarely attribute influential information in their answers, and the rest attribute non-influential information. Training acknowledgment into reasoning does not fix this: the acknowledgment does not reach the final answer. We introduce CREDIT, which directly supervises final-answer self-attribution with counterfactual labels, without the need for human annotation. CREDIT substantially improves attribution in four open models, including two distilled reasoning models, and outperforms reasoning-level fine-tuning and prompting baselines. Among answers that agree with a user's suggestion, CREDIT distinguishes those changed by the suggestion from those the model would have given anyway. Gains transfer partly to unseen information types and prompts containing sources that support conflicting answers.