Faithfulness of Language Model Explanations

Follow this topic Ask about this topic

This topic asks whether a model’s explanations identify the evidence and reasoning that actually influence its answers, rather than merely offering plausible justifications afterward. Researchers test this with counterfactual prompt and evidence edits, feature ablations, activation probes and patching, and citation or verifier checks across reasoning, medical, classification, and retrieval-augmented tasks. To improve faithfulness, methods include evidence-grounded generation, structured or executable reasoning, and training with faithfulness rewards. A central challenge is that correct answers and convincing explanations do not prove causal faithfulness, while metrics often disagree and improvements can trade off against accuracy, completeness, or brevity.

Latest papers

107 papers25 papers in the last four weeks, up 39% on the four weeks before. 0.1% of all new papers.

Surrogate Fidelity: When Can Open LLMs Explain Closed Ones?
Jun 30, 2026Philippe Chlenski, Zachariah Carmichael, Ayush Warikoo +5Faithfulness of Language Model Explanations LLM Interpretability
1Meta.
Decodable but Not Faithful: Coupling Natural-Language Rationales to Programmatic Verifiers
Jun 19, 2026Vatsal Ananthula, Adarsh KumarappanFaithfulness of Language Model Explanations LLM Auditing
University of California, Berkeley · 2California Institute of Technology
Self-CTRL: Self-Consistency Training with Reinforcement Learning
Jun 16, 2026Itamar Pres, Laura Ruis, Melat Ghebreselassie +2Self-Consistency in Language Models Faithfulness of Language Model Explanations
MIT CSAIL
CAREF: Calibration-Aware Regularization for Explanation Faithfulness Without Rationale Supervision
May 27, 2026Naphat Nithisopa, Teerapong PanboonyuenFaithfulness of Language Model Explanations LLM Interpretability
†MARSAIL · ‡Chulalongkorn University · §PBYAIL (Panboonyuen AI Lab)
Do Models Know Why They Changed Their Mind? Interpretability and Faithfulness of Chain-of-Thought Under Knowledge Conflict
May 26, 2026Pruthvinath Jeripity VenkataLLM Uncertainty Estimation Faithfulness of Language Model Explanations
Independent Researcher
Faithfulness Metrics Don't Measure Faithfulness: A Meta-Evaluation with Ground Truth
May 24, 2026Yoav Gur-Arieh, Ana Marasović, Mor GevaCoT Faithfulness LLM Evaluation
Tel Aviv University · University of Utah
Lost in Interpretation: The Plausibility-Faithfulness Trade-off in Cross-Lingual Explanations
May 19, 2026Somnath Banerjee, Pranav Jha, Rima Hazra +1Language Model Auditing Cross-Lingual Reasoning in Language Models
Indian Institute of Technology Kharagpur · TCG Crest · National University of Singapore
Faithful or Fabricated? A Causal Framework for Rationalization Bias in LLM Judges
May 13, 2026Riya Tapwal, Abhishek Kumar, Carsten MapleLLM-as-a-Judge Faithfulness of Language Model Explanations
Indian Institute of Technology (IIT) Mandi · The Alan Turing Institute, London, U.K. · Warwick Manufacturing Group University of Warwick, U.K.
Drop the Act: Probe-Filtered RL for Faithful Chain-of-Thought Reasoning
May 12, 2026Swapnil Parekh, Naman GoyalRL for Language Model Reasoning Faithfulness of Language Model Explanations
Intuit
RSAT: Structured Attribution Makes Small Language Models Faithful Table Reasoners
Apr 30, 2026Jugal Gajjar, Kamalasankari SubramaniakuppusamyLLM Reasoning LLM Grounding
Department of Computer Science, The George Washington University, USA
Faithfulness Serum: Mitigating the Faithfulness Gap in Textual Explanations of LLM Decisions via Attribution Guidance
Apr 15, 2026Bar Alon, Itamar Zimerman, Lior WolfFaithfulness of Language Model Explanations LLM Interpretability
Blavatnik School of Computer Science, Tel Aviv University
Applied Explainability for Large Language Models: A Comparative Study
Apr 15, 2026Venkata Abhinandan KancharlaLLM Interpretability Explainability Evaluation
NRI Institute of Technology
ReSS: Learning Reasoning Models for Tabular Data Prediction via Symbolic Scaffold
Apr 15, 2026Chenlang Yi, Gang Li, Zizhan Xiong +4Tabular Prediction Neuro-Symbolic Reasoning
Department of Computer Science & Engineering, Texas A&M University · Department of Computer Science, University of Florida
Hessian-Enhanced Token Attribution (HETA): Interpreting Autoregressive LLMs
Apr 14, 2026Vishal Pramanik, Maisha Maliha, Nathaniel D. Bastian +1Feature Attribution LLM Interpretability
Department of Computer & Information Science & Engineering, University of Florida, Gainesville, FL 32611, USA · School of Computer Science, University of Oklahoma, Norman, OK 73019, USA · Department of Electrical Engineering and Computer Science, United States Military Academy, West Point, NY 10996, USA
A Two-Stage LLM Framework for Accessible and Verified XAI Explanations
Apr 14, 2026Georgios Mermigkis, Dimitris Metaxakis, Marios Tyrovolas +4Explainable Artificial Intelligence Faithfulness of Language Model Explanations
Department of Computer Engineering and Informatics, University of Patras, Patras, Greece · Archimedes Unit, Athena Research Center, Athens, Greece · Department of Informatics and Telecommunications, University of Ioannina, Arta, Greece +1
CLSGen: A Dual-Head Fine-Tuning Framework for Joint Probabilistic Classification and Verbalized Explanation
Apr 13, 2026WonJin Yoon, Kangyu Zhu, Ian Bulovic +5LLM Fine-Tuning Confidence Estimation in Language Models
Boston Children’s Hospital · Harvard Medical School · University of Colorado Anschutz Medical Campus +2
FACT-E: Causality-Inspired Evaluation for Trustworthy Chain-of-Thought Reasoning
Apr 12, 2026Yuxi Sun, Aoqi Zuo, Haotian Xie +3CoT Evaluation Faithfulness of Language Model Explanations
Hong Kong Baptist University · The University of Melbourne · Singapore Management University
CARE-ECG: Causal Agent-based Reasoning for Explainable and Counterfactual ECG Interpretation
Apr 12, 2026Elahe Khatibi, Ziyu Wang, Ankita Sharma +4Electrocardiogram Classification Causal Reasoning in Language Models
University of California, Irvine; Irvine, California, United States · Arizona State University; Tempe, Arizona, United States · California State University, Dominguez Hills; Carson, California, United States
Coherent Without Grounding, Grounded Without Success: Observability and Epistemic Failure
Mar 30, 2026Camilo Chacón SartoriLLM Grounding Explainability Evaluation
Institut Català de Nanociència i Nanotecnologia (ICN2). · Artificial Intelligence Research Institute (IIIA-CSIC), Barcelona, Spain.
Culturally Adaptive Explainable LLM Assessment for Multilingual Information Disorder: A Human-in-the-Loop Approach
Mar 28, 2026Maziar Kianimoghadam JouneghaniCross-Cultural Language Model Evaluation Faithfulness of Language Model Explanations
University of Turin
Why Models Know But Don't Say: Chain-of-Thought Faithfulness Divergence Between Thinking Tokens and Answers in Open-Weight Reasoning Models
Mar 27, 2026Richard J. YoungCoT Faithfulness Large Reasoning Models
University of Nevada, Las Vegas, Department of Management, Entrepreneurship and Technology, Lee Business School, Las Vegas, NV, USA · DeepNeuro AI, Las Vegas, NV, USA
I Came, I Saw, I Explained: Benchmarking Multimodal LLMs on Figurative Meaning in Memes
Mar 24, 2026Shijia Zhou, Saif M. Mohammad, Barbara Plank +1Figurative Language Understanding Multimodal Large Language Models
MaiNLP, Center for Information and Language Processing, LMU Munich, Germany · Munich Center for Machine Learning (MCML), Munich, Germany · National Research Council Canada, Ottawa, Canada
Measuring and curing reasoning rigidity: from decorative chain-of-thought to genuine faithfulness
Mar 24, 2026Abhinaba Basu, Pavan ChakrabortyLanguage Model Reasoning Evaluation Structured Reasoning in Language Models
Indian Institute of Information Technology Allahabad (IIITA) · National Institute of Electronics and Information Technology (NIELIT)
Lie to Me: How Faithful Is Chain-of-Thought Reasoning in Reasoning Models?
Mar 23, 2026Richard J. YoungCoT Faithfulness Faithfulness of Language Model Explanations
University of Nevada, Las Vegas, Department of Management, Entrepreneurship and Technology, Lee Business School, Las Vegas, NV, USA · DeepNeuro AI, Las Vegas, NV, USA
Explainable AML Triage with LLMs: Evidence Retrieval and Counterfactual Checks
Mar 22, 2026Dorothy Torres, Wei Cheng, Ke HuAnti-Money Laundering Anti-Money Laundering Detection
School of Science, Technology, Engineering and Mathematics 263 Academy Street, Room S204 Jersey City, NJ 07306 · School of Electrical Engineering and Computer Science, The Pennsylvania State University, 207 Electrical Engineering West, University Park, PA 16802, USA
Reasoning Traces Shape Outputs but Models Won't Say So
Mar 21, 2026Yijie Hao, Lingjie Chen, Ali Emami +1Large Reasoning Models Faithfulness of Language Model Explanations
Emory University · University of Illinois Urbana-Champaign
Measuring Faithfulness Depends on How You Measure: Classifier Sensitivity in LLM Chain-of-Thought Evaluation
Mar 20, 2026Richard J. YoungCoT Faithfulness LLM Evaluation
University of Nevada, Las Vegas, Department of Management, Entrepreneurship and Technology, Lee Business School, Las Vegas, NV, USA · DeepNeuro AI, Las Vegas, NV, USA
An Agentic Approach to Generating XAI-Narratives
Mar 20, 2026Yifan He, David MartensExplainable Artificial Intelligence Faithfulness of Language Model Explanations
University of Antwerp, Belgium
EvidenceRL: Reinforcing Evidence Consistency for Trustworthy Language Models
Mar 20, 2026J. Ben Tamo, Yuxing Lu, Benoit L. Marteau +2LLM Hallucination Mitigation LLM Grounding
Georgia Institute of Technology · Pekin University
MLLM-based Textual Explanations for Face Comparison
Mar 17, 2026Redwan Sony, Anil K Jain, Arun RossFaithfulness of Language Model Explanations Face Recognition
Department of Computer Science and Engineering, Michigan State University, East Lansing, MI, USA

Jun 30, 2026J/K move · Enter open · S save

Surrogate Fidelity: When Can Open LLMs Explain Closed Ones?

Philippe Chlenski, Zachariah Carmichael, Ayush Warikoo, Chia-Tse Shao+4

1Meta.

Faithfulness of Language Model Explanations LLM Interpretability Mechanistic Interpretability Causal Interpretability Explainability Evaluation

Open paper AskPDF arXiv

Mechanistic interpretability (MI) requires full access to model internals, yet the APIs for most widely deployed language models at best expose log-probabilities over output tokens. This creates a surrogate problem: when do measurements made on open models allow us to make claims about a closed model? We evaluate surrogate fidelity at the prediction, attribution, and representation levels. For binary classification tasks, log-odds provide an API-compatible scalar readout of the model's representation space, and leave-one-out attributions provide insight into model behavior. Across eleven models spanning four families (Llama, Qwen, GPT, and Gemini), we find that prediction fidelity substantially overstates attribution fidelity: models that agree on what the answer is often disagree on why. We document an access-validity inversion: white-box signals like attention patterns and perturbation magnitudes are highly stable across models but only weakly predictive of causal attributions, which black-box input ablations capture by design. Mechanistic insight does not automatically transfer to closed targets, and prediction-level agreement is insufficient to warrant such transfer. Code and results are available at https://github.com/facebookresearch/surrogate.