cs.AIOct 23, 2026

Do Language Models Trust Their Own Justifications? A Study on Functional Consistency

Authors: Alisson Rosa Pereira, Bruno Iochins Grisci

Organizations: Institute of Informatics Universidade Federal do Rio Grande do Sul Porto Alegre, Brazil

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Large Language Models (LLMs) have been widely adopted in text classification tasks, where they not only output class predictions but also generate explanations that highlight the tokens deemed most relevant for reaching the predicted label. Yet it remains unclear whether these highlighted elements are behaviorally consistent with the model's own predictions under targeted interventions, which we refer to as functional auto-consistency. While much of the literature evaluates the textual plausibility of such explanations, few studies assess their functional consistency with the model's actual behavior. In this work, we propose an experimental framework based on the principle of auto-consistency: if a model identifies certain tokens as decisive, then isolating, removing, or semantically inverting them should produce systematic and interpretable changes in its predictions. We operationalize this evaluation through sufficiency, comprehensiveness, and counterfactuality metrics, and conduct experiments on IMDB and Steam reviews across both closed-source (GPT-4o-mini) and open-source LLMs (Gemma3, Granite8B, DeepSeek). Results show that GPT-4o-mini follows the expected progression across all metrics, Gemma3 and Granite8B maintain coherence under sufficiency but lose consistency under more demanding interventions, while DeepSeek variants display structural deviations, either failing to preserve sufficiency or overreacting under comprehensiveness and counterfactuality. These findings show that explanation reliability varies across LLM families and scales, with smaller models displaying contradictions and larger ones exhibiting over-sensitivity. By combining sufficiency, comprehensiveness, and counterfactuality, our approach provides a systematic methodology for assessing the functional consistency of LLM self-explanations.

Figures & tables

Fig. 1 : Sentence Analysis Workflow
Fig. 2 : Workflow of interventions and re-evaluations.
 IMDBSteam
ModelΔR‾\overline{\Delta R}σΔR\sigma_{\Delta R}πalt\pi_{\text{alt}} [%]πred\pi_{\text{red}} [%]ΔR‾\overline{\Delta R}σΔR\sigma_{\Delta R}πalt\pi_{\text{alt}} [%]πred\pi_{\text{red}} [%]
Sufficiency
Gemma3:4B0.021.483.814.20.271.846.916.4
GPT-4o-mini-0.280.870.910.5-0.091.333.513.3
Granite8B-0.411.423.89.90.131.896.721.0
DeepSeek-1.5B0.662.8429.143.60.562.9130.942.9
TABLE I : Summary of auto-consistency metrics on the IMDB and Steam datasets. Selected results are discussed in section VI .
More in this topic

Faithfulness of Language Model Explanations

Faithfulness of language model explanations measures whether stated reasons reflect the evidence and processes driving model outputs.

See all 106 papers

Explore similar work

CardsList
Sep 15, 2026

An Empirical Study of Counterfactual Self-Explanations in LLMs

Large language models can easily generate explanations for their own outputs, but such self-explanations are not necessarily faithful to the model's behavior. We study this issue through counterfactual self-explanations, where a model minimally edits an input so that its own prediction changes. Across sentiment analysis and natural language inference, we evaluate ten instruction-tuned models from the LLaMA-3 and Qwen-2.5 families, measuring faithfulness, minimality, and alignment with human-annotated rationales. Our results show that model scale is the strongest determinant of explanation quality: larger models are substantially more likely to generate counterfactuals that flip their own predictions and target decision-relevant evidence. In contrast, the rationale-guided condition produces edit-minimal counterfactuals that are also more human-aligned. However, it does not consistently improve faithfulness. Overall, counterfactual self-explanations can provide useful behavioral evidence about model decisions, but their reliability depends strongly on model capacity and should be empirically validated rather than assumed.

Giannis Kalyvas, Giorgos Filandrianos, Orfeas Menis Mastromichalakis +2

National Technical University of Athens, Athens, Greece · Instituto de Telecomunicações, Lisbon, Portugal

Counterfactual ExplanationsFaithfulness of Language Model ExplanationsLLM InterpretabilityCounterfactual Reasoning in Language ModelsLLM Reliability
AskPDF
Feb 2, 2026

A Positive Case for Faithfulness: LLM Self-Explanations Help Predict Model Behavior

LLM self-explanations are often presented as a promising tool for AI oversight, yet their faithfulness to the model's true reasoning process is poorly understood. Existing faithfulness metrics have critical limitations, typically relying on identifying unfaithfulness via adversarial prompting or detecting reasoning errors. These methods overlook the predictive value of explanations. We introduce Normalized Simulatability Gain (NSG), a general and scalable metric based on the idea that a faithful explanation should allow an observer to learn a model's decision-making criteria, and thus better predict its behavior on related inputs. We evaluate 18 frontier proprietary and open-weight models, e.g., Gemini 3, GPT-5.2, and Claude 4.5, on 7,000 counterfactuals from popular datasets covering health, business, and ethics. We find self-explanations substantially improve prediction of model behavior (11-37% NSG). Self-explanations also provide more predictive information than explanations generated by external models, even when those models are stronger. This implies an advantage from self-knowledge that external explanation methods cannot replicate. Our approach also reveals that, across models, 5-15% of self-explanations are highly misleading. Despite their imperfections, we show a positive case for self-explanations: they encode information that helps predict model behavior.

Harry Mayne, Justin Singh Kang, Dewi Gould +3

University of Oxford · University of California, Berkeley · Independent +2

Faithfulness of Language Model ExplanationsLLM InterpretabilityExplainability Evaluation
AskPDF
Sep 25, 2026

Enhancing Assessment of Self-Consistency in LLM Explanations using Perturbation Strength

Prior work has examined the self-consistency of LLM-generated explanations using surface-level perturbation methods. However, the strength of these perturbations is not explicitly measured and controlled. In this work, we propose an LLM-as-a-judge approach to measure perturbation strength in a unified manner across input and CoT perturbations. We then evaluate the self-consistency in explanations generated from various LLMs under controlled strength conditions, ensuring a fair comparison across perturbation types. Experiments show that our proposed LLM-based perturbation strength measure outperforms other embedding- and probability-based approaches and that input perturbations generally affect LLMs more strongly than CoT perturbations. Our work suggests that judgments about a model's self-consistency is fair only within the same perturbation type.

Phuong Q. Le, Kemal Kurniawan, Jey Han Lau

School of Computing and Information Systems The University of Melbourne, Melbourne, Australia · School of Computer Science and Engineering University of New South Wales, Sydney, Australia · The University of Melbourne

Self-Consistency in Language ModelsLanguage Model Robustness
AskPDF