Enhancing Assessment of Self-Consistency in LLM Explanations using Perturbation Strength

Phuong Q. Le; Kemal Kurniawan; Jey Han Lau

cs.CLSep 25, 2026

Enhancing Assessment of Self-Consistency in LLM Explanations using Perturbation Strength

Authors: Phuong Q. Le, Kemal Kurniawan, Jey Han Lau

Organizations: School of Computing and Information Systems The University of Melbourne, Melbourne, Australia · School of Computer Science and Engineering University of New South Wales, Sydney, Australia · The University of Melbourne

Abstract

Prior work has examined the self-consistency of LLM-generated explanations using surface-level perturbation methods. However, the strength of these perturbations is not explicitly measured and controlled. In this work, we propose an LLM-as-a-judge approach to measure perturbation strength in a unified manner across input and CoT perturbations. We then evaluate the self-consistency in explanations generated from various LLMs under controlled strength conditions, ensuring a fair comparison across perturbation types. Experiments show that our proposed LLM-based perturbation strength measure outperforms other embedding- and probability-based approaches and that input perturbations generally affect LLMs more strongly than CoT perturbations. Our work suggests that judgments about a model's self-consistency is fair only within the same perturbation type.

Figures & tables

Panel of Figure 1 — Figure 1: A high semantic change between the original and perturbed text does not necessarily interfere with the reasoning path. Changing "bacon" from "meat" to "vegetable" is a high semantic change on its own; however, when being aware of the question context, this change does not interfere with the reasoning path.

Panel of Figure 3 — Figure 3: Example of CoT steps where the second step is perturbed.

Strength level	Category	Description
0	No/Minimal change	Meaning is preserved (e.g., paraphrasing, synonyms, rewording) OR the change(s) is/are not enough to interfere with the reasoning path
1	Small change	Minor semantic shift that might interfere with the reasoning path but not enough to remove or reverse support for an answer option
2	Moderately strong change	The meaning changes enough to notably weaken, remove, or reverse support for an answer option, without clearly shifting support to another option (including cases where it shifts support to an answer that is not in the answer options)
3	Strong change	Clearly shift support from one answer option to another that is in the option list

Table 1: Key perturbation strength levels and criteria. The full guidelines are shown in Figure 11 .

Panel of Figure 4 — Figure 4: Example of a question and its perturbed version.

Table 2: Correlations and Kappa scores of different perturbation strength measures. Kappa scores are only computed for LLM because it is designed for categorical data, which matches LLM’s outputs for perturbation strength.

Figure 5: Proportion of perturbed CoTs at each perturbation strength for datasets described in Section 3.1 .

Figure 6: ARC-Challenge flip rates of input and CoT perturbations across 4 CoT models

CoT Perturbation
	ARC-Challenge	OpenBook QA	Sports	Strategy QA
LLaMA-3B	70.7	45.2	96.0	37.5
Mistral-2	36.2	45.9	23.2	55.2
Phi-3	34.5	53.7	36.1	53.2
LLaMA-8B	24.7	34.7	47.2	50.1
Input Perturbation

Table 3: Indicator for overall self-consistency of 4 CoT models across all datasets under CoT and input perturbations, measured by

\mathrm{FR(3)-FR(0)}

. The bold fields indicate highest value for that dataset.

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 7 — Figure 7: Prompt for gpt-4o-mini to perturb CoT step ( Lanham et al., 2023 ; Tutek et al., 2025 )

Panel of Figure 8 — Figure 8: Prompt for gpt-4o-mini to perturb the input (i.e. question text).

Panel of Figure 9 — Figure 9: Sampling procedure for annotation in StrategyQA dataset. This is repeated for all 4 datasets.

Panel of Figure 10 — Figure 10: Prompt for gpt-4o-mini to paraphrase CoT step ( Lanham et al., 2023 )

Panel of Figure 11 — Figure 11: Full Perturbation Strength Assessment Guideline for Human Annotators

Panel of Figure 12 — Figure 12: Example of perturbed question with strength of 2 and the justification

Panel of Figure 13 — Figure 13: Example of perturbed CoT with strength of 1 and the justification

	All	Score 0 & 3	Score 1 & 2
CoT	0.649	0.830	0.192
Input	0.855	0.840	0.750

Appendix

Table 4: Linear Kappa scores between primary and second annotators

Figure 14: Change in surprisals versus True perturbation strength for CoT perturbations

Panel of Figure 15 — Figure 15: Prompt template for gemini-3-flash to assess input perturbation strength.

Panel of Figure 16 — Figure 16: Prompt template for gemini-3-flash to assess CoT perturbation strength.

Panel of Figure 17 — Figure 17: Perturbation Strength Guide as system instruction for LLM. Modifications from the guidelines for human annotators ( Figure 11 ) are in bold.

Panel of Figure 18 — Figure 18: Prompt to get CoT model’s answer to a question ( Tutek et al., 2025 )

Figure 19: OpenBookQA flip rates of input and CoT perturbations across 4 CoT models

Figure 20: Sports flip rates of input and CoT perturbations across 4 CoT models

Figure 21: StrategyQA flip rates of input and CoT perturbations across 4 CoT models

Panel of Figure 22 — Figure 22: Example of model changing answer under zero-strength input perturbation

Panel of Figure 23 — Figure 23: Example of model not changing answer under high-strength CoT perturbation

Figure 24: ARC-Challenge flip rates of four CoT models under 3 settings: CoT perturbation, input perturbation using direct prompting and input perturbation using CoT prompting

Figure 25: OpenBookQA flip rates of four CoT models under 3 settings: CoT perturbation, input perturbation using direct prompting and input perturbation using CoT prompting

Figure 26: Sports flip rates of four CoT models under 3 settings: CoT perturbation, input perturbation using direct prompting and input perturbation using CoT prompting

Figure 27: StrategyQA flip rates of four CoT models under 3 settings: CoT perturbation, input perturbation using direct prompting and input perturbation using CoT prompting

Self-Consistency in Language Models

Self-consistency in language models studies whether repeated outputs agree and how agreement can guide reasoning, confidence, and behavior.

See all 49 papers

Do Language Models Trust Their Own Justifications? A Study on Functional Consistency
Oct 23, 2026Alisson Rosa Pereira, Bruno Iochins GrisciFaithfulness of Language Model Explanations LLM Evaluation
Institute of Informatics Universidade Federal do Rio Grande do Sul Porto Alegre, Brazil
An Empirical Study of Counterfactual Self-Explanations in LLMs
Sep 15, 2026Giannis Kalyvas, Giorgos Filandrianos, Orfeas Menis Mastromichalakis +2Counterfactual Explanations Faithfulness of Language Model Explanations
National Technical University of Athens, Athens, Greece · Instituto de Telecomunicações, Lisbon, Portugal
What LLMs explain is not what they believe: Evaluating explanation sufficiency under models' own input beliefs
Jun 26, 2026Nhi Nguyen, Shauli Ravfogel, Rajesh RanganathLLM Evaluation LLM Interpretability
New York University

Oct 23, 2026J/K move · Enter open · S save

Do Language Models Trust Their Own Justifications? A Study on Functional Consistency

Alisson Rosa Pereira, Bruno Iochins Grisci

Institute of Informatics Universidade Federal do Rio Grande do Sul Porto Alegre, Brazil

Faithfulness of Language Model Explanations LLM Evaluation

Open paper AskPDF arXiv

Large Language Models (LLMs) have been widely adopted in text classification tasks, where they not only output class predictions but also generate explanations that highlight the tokens deemed most relevant for reaching the predicted label. Yet it remains unclear whether these highlighted elements are behaviorally consistent with the model's own predictions under targeted interventions, which we refer to as functional auto-consistency. While much of the literature evaluates the textual plausibility of such explanations, few studies assess their functional consistency with the model's actual behavior. In this work, we propose an experimental framework based on the principle of auto-consistency: if a model identifies certain tokens as decisive, then isolating, removing, or semantically inverting them should produce systematic and interpretable changes in its predictions. We operationalize this evaluation through sufficiency, comprehensiveness, and counterfactuality metrics, and conduct experiments on IMDB and Steam reviews across both closed-source (GPT-4o-mini) and open-source LLMs (Gemma3, Granite8B, DeepSeek). Results show that GPT-4o-mini follows the expected progression across all metrics, Gemma3 and Granite8B maintain coherence under sufficiency but lose consistency under more demanding interventions, while DeepSeek variants display structural deviations, either failing to preserve sufficiency or overreacting under comprehensiveness and counterfactuality. These findings show that explanation reliability varies across LLM families and scales, with smaller models displaying contradictions and larger ones exhibiting over-sensitivity. By combining sufficiency, comprehensiveness, and counterfactuality, our approach provides a systematic methodology for assessing the functional consistency of LLM self-explanations.

Abstract

Self-Consistency in Language Models

Explore similar work

Do Language Models Trust Their Own Justifications? A Study on Functional Consistency

An Empirical Study of Counterfactual Self-Explanations in LLMs

What LLMs explain is not what they believe: Evaluating explanation sufficiency under models' own input beliefs