Counterfactual Reasoning in Language Models

Follow this topic Ask about this topic

Momentum

3 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 19

All topics

A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined
Oct 1, 2026Zhangshu Joshua Jiang, Zina Ibrahim, James T. TeoFaithfulness of Language Model Explanations Temporal Reasoning in Language Models
DRIVE-Health CDT Department of Biostatistics and Health Informatics Institute of Psychiatry, Psychology and Neuroscience King’s College London London, UK
An Empirical Study of Counterfactual Self-Explanations in LLMs
Sep 15, 2026Giannis Kalyvas, Giorgos Filandrianos, Orfeas Menis Mastromichalakis +2Counterfactual Explanations Faithfulness of Language Model Explanations
National Technical University of Athens, Athens, Greece · Instituto de Telecomunicações, Lisbon, Portugal
InfluenceField: A Differentiable Field with Interventionally Identifiable Causal Structure for Multimodal World Modeling
Sep 7, 2026Zihao Yang, Zijia Wang, Zhiqiu HuangCausal Representation Learning Causal World Models
Mathematical Institute, University of Oxford · Department of Computer Science, University of Oxford · School of Mathematical Sciences, University of Nottingham Ningbo China
What Would Fix This RAG Failure? Auditing Counterfactual Response with Paired Evidence Interventions
Aug 9, 2026Wenzhang DuRetrieval-Augmented Generation Counterfactual Reasoning in Language Models
Independent Researcher China
Counterfactual Benchmarking and Training for Factuality Consistency and Order-Robust Grounded Reasoning in LLMs over Heterogeneous Knowledge
Aug 8, 2026Shibo Chu, Yuze Liu, Tiehua Zhang +4LLM Evaluation LLM Grounding
Tongji University, Shanghai, China · Swinburne University of Technology, Melbourne, Australia · Wuhan University of Technology, Wuhan, China
Counterfactual Analysis via Large Language Models
Aug 5, 2026Zonghao YangFinancial Services Counterfactual Reasoning in Language Models
Stevens Institute of Technology
An Explicit World Model Based on Data-First Ontology: DaoQL Multimodal Storage Validation and Counterfactual Reasoning Evaluation
Jul 19, 2026Zhanbo Li, Shifeng Wu, Xiangjin Meng +1LLM Grounding Counterfactual Reasoning in Language Models
Graduate Students, School of Mathematics and Statistics, Chongqing University · DaoQL Project Group · Guangzhou Polytechnic Normal University
CRiT-QA: Evaluating Multi-hop Reasoning with Counterfactual Chains and Distractor Traps
Jul 12, 2026JungMin Yun, JuneHyoung Kwon, YoungBin KimMulti-Hop QA LLM Evaluation
Department of Artificial Intelligence, Chung-Ang University · Graduate School of Advanced Imaging Sciences, Multimedia and Film, Chung-Ang University
NormWorlds-CF: Solver-Verified Counterfactual Normative Reasoning with Metamorphic-Relation GRPO
Jul 4, 2026Xinqi ZhangCounterfactual Evaluation Moral Reasoning in Language Models
Tsinghua University
Testing Frontier Large Language Models' Physics Literacy in Parallel Physical Worlds
Jun 30, 2026Dong ZhangLLM Evaluation Physical Reasoning
PragReST: Self-Reinforcing Counterfactual Reasoning for Pragmatic Language Understanding
Jun 17, 2026Jihyung Park, Minchao Huang, Leqi Liu +1Pragmatic Reasoning in Language Models Counterfactual Reasoning in Language Models
The University of Texas at Austin
Re-feeding Is Not Replaying: Measuring Replay Noise in Counterfactual Token-Credit Estimation
Jun 14, 2026Nils MattesonCounterfactual Evaluation Token-Level Credit Assignment
Northeastern University
DMC-CF: Dynamic Multimodal CounterFactual QA benchmark for Causal Reasoning
May 28, 2026Junzhe Zhang, Huixuan Zhang, Guirong Wang +5Counterfactual Evaluation Counterfactual Reasoning in Language Models
Wangxuan Institute of Computer Technology, Peking University · Alibaba Group
The Point of No Return: Counterfactual Localization of Deceptive Commitment in Language-Model Reasoning
May 16, 2026Scott Merrill, Shashank SrivastavaLanguage Model Safety Evaluation LLM Interpretability
University of North Carolina at Chapel Hill
Conditional Attribute Estimation with Autoregressive Sequence Models
May 13, 2026Erica Stutz, Giacomo Marino, Daniella Meeker +2Counterfactual Reasoning in Language Models Controllable Text Generation
Department of Biomedical Informatics and Data Science Yale University New Haven, CT 06510 · Department of Biostatistics Yale University New Haven, CT 06510 · Department of Biomedical Informatics and Data Science, Department of Pediatrics Yale University New Haven, CT 06510
Macro: Enhancing Multilingual Counterfactual Explanations through Alignment-as-Preference Optimization
May 12, 2026Yilong Wang, Qianli Wang, Bohao Chu +3Multilingual Language Models Counterfactual Explanations
1Technische Universität Berlin · 2German Research Center for Artificial Intelligence (DFKI) · University of Duisburg-Essen +5
DeepImagine: Learning Biomedical Reasoning via Successive Counterfactual Imagining
Apr 24, 2026Youze Zheng, Jianyou Wang, Yuhan Chen +8LLM Fine-Tuning Counterfactual Reasoning in Language Models
Laboratory for Emerging Intelligence, University of California, San Diego · University of Chicago · Elsevier +1
Exposing the Unsaid: Visualizing Hidden LLM Bias through Stochastic Path Aggregation
Apr 24, 2026Matteo Pelossi, Rita Sevastjanova, Thilo Spinner +1LLM Auditing Social Bias in Language Models
ETH Zurich
Counterfactual Modeling with Fine-Tuned LLMs for Health Intervention Design and Sensor Data Augmentation
Jan 21, 2026Shovito Barua Soumma, Asiful Arefeen, Stephanie M. Carpenter +2Counterfactual Explanations LLM Fine-Tuning
College of Health Solutions, Arizona State University, Phoenix, AZ 85004, USA. · School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ 85281, USA. · School of Nutritional Sciences and Wellness, University of Arizona, Tucson, AZ 85721, USA.

Oct 1, 2026, cs.CLJ/K move · Enter open · S save

A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined

Zhangshu Joshua Jiang, Zina Ibrahim, James T. Teo

DRIVE-Health CDT Department of Biostatistics and Health Informatics Institute of Psychiatry, Psychology and Neuroscience King’s College London London, UK

Faithfulness of Language Model Explanations Temporal Reasoning in Language Models Counterfactual Reasoning in Language Models Clinical Reasoning Clinical Language Model Evaluation

Open paper AskPDF arXiv

Exam-style accuracy does not establish whether large language models (LLMs) reason well over clinical records. We define clinical reasoning as integrating and updating evidence across time and sources to form, revise and justify a patient's problem representation and a defensible plan. This structured narrative review maps three literatures: medical education assessment instruments, clinical LLM benchmarks published from 2023 onwards, and general-domain methods for evaluating long-form generation. We examine six dimensions: problem representation, temporal synthesis, differential and management reasoning, counterfactual reasoning, calibrated uncertainty, and reasoning faithfulness. Preprints are included and flagged. No single instrument covers all six dimensions. Problem representation and differential or management reasoning are reasonably covered, although reliability varies by instrument and setting. TIMER-Eval targets temporal synthesis, and ER-Reason assesses sequential diagnostic belief updating. Dedicated uncertainty and counterfactual evaluations are emerging, but their applicability to longitudinal free-text reasoning remains limited. Factual completeness is well theorised in general-domain evaluation, with early clinical evidence of important omissions. Faithfulness remains the weakest dimension, with one identified clinical causal-ablation study on multiple-choice questions. Existing tools should be combined through binary rubric items, separate completeness and correctness scores, case-specific importance weighting with non-compensable safety caps, temporal order-consistency checks, and chance-corrected reliability reporting. Further design work is needed for calibrated uncertainty, counterfactual reasoning and faithfulness over longitudinal free-text records. This review provides a design rationale, not a validated instrument.

Momentum

Related topics

Latest papers 19