Counterfactual Reasoning in Language Models

Overview

Counterfactual reasoning asks whether a language model can predict what would happen if a relevant fact or action were different, and whether its answer actually depends on that change. Researchers test this with paired-world benchmarks, controlled prompt or evidence edits, causal graphs, and counterfactual explanations, while training methods include supervised fine-tuning, preference optimization, reinforcement learning, and multi-agent critique. Applications range from clinical decisions and legal analysis to strategic games, multimodal question answering, and explanations of model behavior. A central challenge is that fluent answers and high accuracy can hide weak causal reasoning: performance often varies across kinds of interventions, and robustness or faithfulness on one benchmark may not transfer to new settings.

Latest papers

CardsList

50 papers8 papers in the last four weeks, level with the four weeks before. 0.1% of all new papers.

  1. PROSE: Preservation-Constrained Selective Editing for Guilty Bias in Legal Judgment Prediction

    Nov 3, 2026Yimeng Wang, Zeyuan Tian, Xinyu Wang +5Legal Judgment PredictionLanguage Model Editing

  2. Did the Model Use That? Counterfactual Training Boosts Self-Attribution in Language Models

    Oct 26, 2026Zikai Zhou, Jiacheng Sang, Yilin Xu +2Data Provenance in Language ModelsCounterfactual Learning

  3. From Document Layout to Causal Topology: An End-to-End Architecture for Temporal Causal Reasoning

    Oct 25, 2026Zhifeng Yang, Wei Luo, Yongjian Chen +1Causal Reasoning in Language ModelsTemporal Reasoning in Language Models

  4. Interpreting Model Reasoning through Behavioral Counterfactuals

    Oct 22, 2026Shawn Im, James Oldfield, Adel Bibi +1Causal Interventions in Language ModelsFaithfulness of Language Model Reasoning

  5. ConCI: A Controlled Counterfactual Benchmark for Contextual Integrity in LLMs

    Oct 17, 2026Juhyeon Park, Sumin Yu, Taesup MoonContextual IntegrityLLM Safety Benchmarks

  6. Watts If? A Benchmark for Counterfactual Reasoning About the AI Data Center Buildout

    Oct 15, 2026Pankaj Chouhan, Rajarshi Das, Aditya Dubey +5Counterfactual Reasoning in Language ModelsGenerative AI Evaluation

  7. CivWorldBench: Evaluating Language World Models in Structured Multi-Agent Sandbox Worlds

    Oct 12, 2026Ang Li, Haofei Yu, Zhengwei Tao +7LLM World ModelsWorld Model Evaluation

  8. CoBe: A Benchmark For Conversational Counterfactual Text Editing

    Oct 9, 2026Arvind Raghavan, Inwoo Hwang, Shreyas Havaldar +6Causal Counterfactual GenerationCounterfactual Reasoning in Language Models

  9. A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined

    Oct 1, 2026Zhangshu Joshua Jiang, Zina Ibrahim, James T. TeoFaithfulness of Language Model ExplanationsTemporal Reasoning in Language Models

  10. Counterfactual Self-Evolving Agents for Evidence-Grounded Reasoning

    Sep 26, 2026Xing Han, Yuxin Wang, Chen Chen +8Self-Evolving AgentsEvidence-Grounded Reasoning

  11. Beyond Accuracy: Counterfactual Fragility and Demographic Bias in Clinical Evaluation of LLMs

    Sep 26, 2026Chaitali Deb Purkayastha, Bharath Kumar Bolla, Vishnu Surya Reddy NandiClinical Language Model EvaluationLanguage Model Robustness

  12. Contextual Causality with Large Language Models: A Survey

    Sep 18, 2026Yiheng Zhao, Jun Yan, Chengming HuCausal Reasoning in Language ModelsCausal Interventions in Language Models

  13. An Empirical Study of Counterfactual Self-Explanations in LLMs

    Sep 15, 2026Giannis Kalyvas, Giorgos Filandrianos, Orfeas Menis Mastromichalakis +2Counterfactual ExplanationsFaithfulness of Language Model Explanations

  14. InfluenceField: A Differentiable Field with Interventionally Identifiable Causal Structure for Multimodal World Modeling

    Sep 7, 2026Zihao Yang, Zijia Wang, Zhiqiu HuangCausal Representation LearningCausal World Models

  15. The Illusion of What If\textit{What If}: Evaluating the Breakdown of Counterfactual Reasoning in LLMs

    Aug 28, 2026Yucheng Wang, Yuetian Du, Zhengyi Liu +8Counterfactual Reasoning in Language ModelsCounterfactual Reasoning

  16. SCIT: Testing Causal Cache Carriers in Latent Chain-of-Thought Models

    Aug 27, 2026Yi Ding, Lijun Huang, Menglin YangTransformer InterpretabilityCausal Interventions in Language Models

  17. SDoH-Aware Narrative Anchoring Bias in Medical LLMs for Trustworthy Clinical Decision Support

    Aug 24, 2026Ahnaf Atef Choudhury, Ramkrishna SahaMedical LLMsLanguage Model Bias Evaluation

  18. No Judgment Without a Reason: Counterfactual Receipts for Versioned AI Evaluators

    Aug 21, 2026Ye Chen, Weining ZhangLLM EvaluationCounterfactual Evaluation

  19. Counterfactual Benchmarking and Training for Factuality Consistency and Order-Robust Grounded Reasoning in LLMs over Heterogeneous Knowledge

    Aug 8, 2026Shibo Chu, Yuze Liu, Tiehua Zhang +4LLM EvaluationLLM Grounding

  20. An Explicit World Model Based on Data-First Ontology: DaoQL Multimodal Storage Validation and Counterfactual Reasoning Evaluation

    Jul 19, 2026Zhanbo Li, Shifeng Wu, Xiangjin Meng +1LLM GroundingCounterfactual Reasoning in Language Models

  21. CRiT-QA: Evaluating Multi-hop Reasoning with Counterfactual Chains and Distractor Traps

    Jul 12, 2026JungMin Yun, JuneHyoung Kwon, YoungBin KimMulti-Hop QALLM Evaluation