Faithfulness Serum: Mitigating the Faithfulness Gap in Textual Explanations of LLM Decisions via Attribution Guidance

Bar Alon; Itamar Zimerman; Lior Wolf

cs.CLApr 15, 2026

Faithfulness Serum: Mitigating the Faithfulness Gap in Textual Explanations of LLM Decisions via Attribution Guidance

Authors: Bar Alon, Itamar Zimerman, Lior Wolf

Organizations: Blavatnik School of Computer Science, Tel Aviv University

Abstract

Large language models (LLMs) achieve strong performance and have revolutionized NLP, but their lack of explainability keeps them treated as black boxes, limiting their use in domains that demand transparency and trust. A promising direction to address this issue is post-hoc text-based explanations, which aim to explain model decisions in natural language. Prior work has focused on generating convincing rationales that appear to be subjectively faithful, but it remains unclear whether these explanations are epistemically faithful, whether they reflect the internal evidence the model actually relied on for its decision. In this paper, we first assess the epistemic faithfulness of LLM-generated explanations via counterfactuals and show that they are often unfaithful. We then introduce a training-free method that enhances faithfulness by guiding explanation generation through attention-level interventions, informed by token-level heatmaps extracted via a faithful attribution method. This method significantly improves epistemic faithfulness across multiple models, benchmarks, and prompts.

Figures & tables

Figure 1: Faithfulness Serum . A hidden hint (professor indication) creates a counterfactual scenario where the model’s answer changes. Without Faithfulness Serum, the explanation is epistemically unfaithful, ignoring the hint. With Faithfulness Serum, the explanation becomes faithful, attributing the answer to the injected hint.

Figure 2: Faithfulness Serum. To nudge the model toward generating faithful explanations, we compute the question ( $Q_{1},..,Q_{T})$ token-level relevance scores $\mathcal{R}$ using LRP (blue), scale them by an intervention strength $\alpha$ (purple), and inject them into the attention scores of the question ( $A_{1},..,A_{T})$ at specific layers.

Figure 3: Distribution of logits for the chosen answer option across layers of Llama 3.1 8B, evaluated on 100 random MMLU auxiliary training samples.

Figure 4: Intervention scaling factor selection for Llama 3.1 8B. The plot shows the percentage of completions with distinct-1 < 40% versus intervention strength $\alpha$ . The vertical line marks the selected maximum non-destructive intervention threshold.

	Llama 3.1 8B				Qwen 2.5 7B
	General		Protocol-Specific		General		Protocol-Specific
Hint Type	LLM	CT	LLM	CT	LLM	CT	LLM	CT
Unethical Info	8.6 $\to$ 15.9	12.1 $\to$ 19.3	48.3 $\to$ 62.3	38.7 $\to$ 44.8	14.1 $\to$ 29.6	28.8 $\to$ 32.4	12.5 $\to$ 30.3	25.6 $\to$ 27.7
Metadata	3.0 $\to$ 10.0	2.9 $\to$ 6.9	7.2 $\to$ 13.4	15.0 $\to$ 30.2	11.6 $\to$ 22.3	12.8 $\to$ 25.3	13.4 $\to$ 20.6	15.6 $\to$ 27.2
Sycophancy	36.1 $\to$ 47.7	47.0 $\to$ 65.0	59.4 $\to$ 65.6	89.8 $\to$ 94.4	66.3 $\to$ 69.1	74.5 $\to$ 75.0	82.9 $\to$ 83.2	93.7 $\to$ 94.3
Grader Hack	2.8 $\to$ 10.7	7.7 $\to$ 17.7	24.5 $\to$ 25.8	35.8 $\to$ 58.6	14.4 $\to$ 17.3	20.9 $\to$ 19.6	9.8 $\to$ 17.1	20.2 $\to$ 26.7

Table 1: Epistemic faithfulness under intervention (open-source models). General and protocol-specific results using LLM judge and Counterfactual Test CT . Values (%) over 1000 MMLU examples with successful hint injection. Format: Original

\to

Intervention. Bold :

>1.5\%

improvement.

Hint Type	LLM	CT	LLM	CT	LLM	CT	LLM	CT
	CommonsenseQA				SciQ
	General		Protocol-Specific		General		Protocol-Specific
Unethical.	3.9 $\to$ 22.5	7.8 $\to$ 25.2	56 $\to$ 74.3	56 $\to$ 74.3	15.7 $\to$ 32	19.4 $\to$ 35.7	55.3 $\to$ 78.9	65.2 $\to$ 57.9
Meta.	3.0 $\to$ 35.0	2.6 $\to$ 28.6	13.2 $\to$ 30.1	39.8 $\to$ 55.3	5.4 $\to$ 27.7	6.2 $\to$ 27.7	5.5 $\to$ 26.0	16.5 $\to$ 55.9
Sycoph.	57.7 $\to$ 76.2	61.4 $\to$ 82.4	82.0 $\to$ 74.6	96.9 $\to$ 97.7	47.6 $\to$ 60.9	63.9 $\to$ 80.4	92.6 $\to$ 96.4	65.6 $\to$ 65.2
Grader.	3.2 $\to$ 9.1	4.1 $\to$ 14.2	41.1 $\to$ 41.1	67.6 $\to$ 90.4	2.0 $\to$ 9.2	11.2 $\to$ 28.6	15.6 $\to$ 41.7	35.4 $\to$ 79.2

Table 2: Epistemic faithfulness results on full datasets. Results for Llama 3.1 8B shown for both General and Protocol-Specific prompts, evaluated using LLM as a judge and Counterfactual Test ( CT ). Format: Original

\to

Intervention.

	GPT-4o				Gemini 2.0 Flash
	G		PS		G		PS
Hint Type	LLM	CT	LLM	CT	LLM	CT	LLM	CT
Unethical.	26	20	22	24	54	40	72	82
Metadata	0	12	2	4	0	0	0	0
Sycoph.	54	48	42	40	68	72	72	82
Grader.	14	28	20	34	4	10	2	6

Table 3: Closed-source models evaluation. Values are (%) out of 50 examples with successful hint injection from MMLU. G = General, PS = Protocol-Specific.

	General		Protocol-Specific
Hint	LLM	CT	LLM	CT
Unet.(7.7)	15.9 $\to$ 32.0	19.3 $\to$ 38.4	62.3 $\to$ 63.5	44.8 $\to$ 52.5
Meta.(5.5)	10.0 $\to$ 15.2	6.9 $\to$ 7.2	13.4 $\to$ 16.7	30.2 $\to$ 41.7
Syco.(8.1)	47.7 $\to$ 62.3	65.0 $\to$ 68.5	65.6 $\to$ 64.4	94.4 $\to$ 96.1
Grad.(7.5)	10.7 $\to$ 34.0	17.7 $\to$ 41.0	25.8 $\to$ 44.7	58.6 $\to$ 70.9

Table 4: Test-Time Adaptive Intervention. Faithfulness over 1000 MMLU benchmark examples. Values in parentheses denote average convergence steps per hint type. Format: Static

\to

Adaptive. Bold : improvement

>1.5\%

	General		Protocol-Specific
Hint	LLM	CT	LLM	CT
Unethic.	11.2 $\to$ 13.0	11.1 $\to$ 35.2	29.6 $\to$ 58.3	22.2 $\to$ 43.1
Meta.	5.9 $\to$ 34.1	4.6 $\to$ 31.9	15.2 $\to$ 15.8	23.0 $\to$ 35.8
Sycoph.	37.8 $\to$ 48.9	51.3 $\to$ 63.4	31.7 $\to$ 33.1	67.4 $\to$ 76.9
Grader.	4.2 $\to$ 11.9	14.5 $\to$ 23.2	6.3 $\to$ 20.4	10.0 $\to$ 34.1

Table 5: Open-question epistemic evaluation. Faithfulness percentages out of 1000 MMLU benchmark examples. Format: Original

\to

Intervention. Bold : improvement

>1.5\%

Role	#Explored	Faithfulness
Stanford professor	1327	21.2% $\rightarrow$ 43.8 %
Neurosurgeon	1554	20.1% $\rightarrow$ 31.3 %
Taxi driver	2627	2.3% $\rightarrow$ 13.0 %
Street vendor	2395	10.7% $\rightarrow$ 22.7 %

Table 6: Credit Bias by Occupation. Prompts required to reach 1000 examples for MMLU and faithfulness rates (LLM), reported as Original

\to

Intervention, using the general prompt on Llama 3.1 8B Instruct.

Figure 5: The effect of layer choice on intervention effectiveness across four hint types. Solid black lines indicate baseline performance without intervention. Results are based on Llama 3.1 with successful hint injection from MMLU

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

	Point to the Question		Important Information
Hint Type	LLM	CT	LLM	CT
Unethical Info	6.0 $\to$ 13.8	3.4 $\to$ 8.0	12.3 $\to$ 18.2	11.5 $\to$ 14.6
Metadata	5.1 $\to$ 13.5	5.2 $\to$ 11.8	7.4 $\to$ 14.7	6.4 $\to$ 9.0
Sycophancy	15.0 $\to$ 37.5	17.5 $\to$ 52.0	27.0 $\to$ 44.7	23.6 $\to$ 3 4.2
Grader Hack	2.6 $\to$ 11.5	4.8 $\to$ 14.2	5.5 $\to$ 9.6	3.5 $\to$ 7.4

Appendix

Table 7: Prompt engineering orthogonality. Faithfulness scores (Prompt

\to

Prompt + Our Method) on MMLU, evaluated using LLM-as-a-Judge (LLM) and the Counterfactual Test (CT). Results show consistent improvements across hint types and prompts, indicating orthogonality to prompt design.

Figure 6: Intervention scaling factor selection for Qwen 2.5 7B Instruct. The vertical line marks the selected $\alpha=0.1$ threshold based on a sharp increase in distinct-1 under 40%.

Dataset	Total	Syco.	Meta.	Uneth.	Grader
ARC-Challenge	1172	900	353	691	249
OpenBookQA	500	452	194	352	135
CommonsenseQA	1121	835	266	591	219
SciQ	1000	443	130	356	98

Appendix

Table 8: Dataset sizes and number of changed examples per hint type.

	ARC-Challenge				OpenBookQA
	General		Protocol-Specific		General		Protocol-Specific
Hint Type	LLM	CT	LLM	CT	LLM	CT	LLM	CT
Unethical.	8.4 $\to$ 17.8	15.5 $\to$ 23.9	52.5 $\to$ 74.5	62.1 $\to$ 51.2	9.1 $\to$ 23.9	12.2 $\to$ 25.6	52.6 $\to$ 78.7	50.0 $\to$ 52.8
Meta.	2.0 $\to$ 9.9	2.0 $\to$ 7.1	2.0 $\to$ 13.3	14.7 $\to$ 39.9	5.2 $\to$ 36.6	6.2 $\to$ 31.4	4.1 $\to$ 33.7	24.9 $\to$ 54.4
Sycoph.	44.0 $\to$ 56.1	57.1 $\to$ 74.7	65.6 $\to$ 65.9	97.1 $\to$ 97.6	56 $\to$ 66.2	67 $\to$ 80.5	77.7 $\to$ 67.7	97.1 $\to$ 96.9
Grader.	0.4 $\to$ 6.0	4.0 $\to$ 13.3	25.7 $\to$ 22.9	35.7 $\to$ 71.5	5.9 $\to$ 13.3	10.4 $\to$ 20.7	31.1 $\to$ 38.5	46.7 $\to$ 77.8

Appendix

Table 9: Epistemic faithfulness results on full datasets for Llama. Results shown for both General and Protocol-Specific prompts, evaluated using LLM as a judge and Counterfactual Test (CT). Format: Original

\to

Intervention.

Method	Sycophancy	Grader Hack	Unethical Information	Metadata
Ours (LRP)	49%	10%	16%	10%
Attention Rollout attribution	45%	4%	7%	5%
Layer-to-layer attribution	36%	4%	4%	3%
No Intervention	34%	3%	4%	2%

Appendix

Table 10: Faithfulness scores (LLM as judge) for 100 MMLU samples under four hint types, comparing different intervention strategies (Llama 3.1 8B Instruct; General prompt).

Hint Type	General LLM	General CT	Protocol LLM	Protocol CT
Unethical Info	8.6 $\to$ 8.2/9.3/ 15.9	12.1 $\to$ 14.2/17.4/ 19.3	48.3 $\to$ 51.0/50.2/ 62.3	38.7 $\to$ 37.9/39.1/ 44.8
Metadata	3.0 $\to$ 3.1/3.0/ 10.0	2.9 $\to$ 2.1/2.6/ 6.9	7.2 $\to$ 7.2/8.1/ 13.4	15.0 $\to$ 21.0/14.3/ 30.2
Sycophancy	36.1 $\to$ 38.2/37.1/ 47.7	47.0 $\to$ 49.3/57.0/ 65.0	59.4 $\to$ 59.3/60.1/ 65.6	89.8 $\to$ 88.9/89.6/ 94.4
Grader Hack	2.8 $\to$ 3.3/4.1/ 10.7	7.7 $\to$ 7.7/7.9/ 17.7	24.5 $\to$ 26.1 /25.8/25.8	35.8 $\to$ 34.3/34.9/ 58.6

Appendix

Table 11: Faithfulness scores (%) over 1000 MMLU examples comparing intervention across decoding steps (Llama 3.1 8B). Format: Original

\to

first-token/first-sentence/full-decoding.

Hint Type	General LLM	General CT	Protocol LLM	Protocol CT
Unethical Info	8.6 $\to$ 15.9 /13.5	12.1 $\to$ 19.3 /19.2	48.3 $\to$ 62.3/ 65.7	38.7 $\to$ 44.8/ 49.3
Metadata	3.0 $\to$ 10.0/ 11.1	2.9 $\to$ 6.9/ 7.1	7.2 $\to$ 13.4 /12.0	15.0 $\to$ 30.2/ 33.4
Sycophancy	36.1 $\to$ 47.7 /44.2	47.0 $\to$ 65.0 /61.3	59.4 $\to$ 65.6/ 68.4	89.8 $\to$ 94.4 /93.2
Grader Hack	2.8 $\to$ 10.7/ 13.5	7.7 $\to$ 17.7 /16.2	24.5 $\to$ 25.8 /23.1	35.8 $\to$ 58.6 /55.1

Appendix

Table 12: Faithfulness scores (%) over 1000 MMLU examples comparing PE-LRP computed on the first token vs. the answer token (Llama 3.1 8B). Format: Original

\to

first-token/answer-token.

Explore similar work

A Removal Based Approach to Improve LLM Faithfulness at Test-Time
Sep 3, 2026Qinglan Luo, S M A Nahian, John Guttag +2Faithfulness of Language Model Explanations LLM Auditing
MIT CSAIL · MGH, HMS
Verbosity Tradeoffs and the Impact of Scale on the Faithfulness of LLM Self-Explanations
Mar 17, 2025Noah Y. Siegel, Nicolas Heess, Maria Perez-Ortiz +1Faithfulness of Language Model Explanations LLM Interpretability
Google DeepMind · Centre for AI, University College London · Imperial College London +1
An Empirical Study of Counterfactual Self-Explanations in LLMs
Sep 15, 2026Giannis Kalyvas, Giorgos Filandrianos, Orfeas Menis Mastromichalakis +2Counterfactual Explanations Faithfulness of Language Model Explanations
National Technical University of Athens, Athens, Greece · Instituto de Telecomunicações, Lisbon, Portugal

Sep 3, 2026J/K move · Enter open · S save

A Removal Based Approach to Improve LLM Faithfulness at Test-Time

Qinglan Luo, S M A Nahian, John Guttag, S. Mazdak Abulnaga+1

MIT CSAIL · MGH, HMS

Faithfulness of Language Model Explanations LLM Auditing LLM Interpretability LLM Reliability

Open paper AskPDF arXiv

Large language models (LLMs) are increasingly used for consequential decisions, making their explanations an important tool for auditing model behavior. Unfortunately, these explanations can be unfaithful, failing to reflect the actual reasoning underlying the model's decisions. We consider a setting in which an LLM provides both an answer and an explanation in response to a question. We identify two distinct dimensions of unfaithful explanations: incompleteness, meaning that the explanation omits factors that influence the answer, and unsoundness, meaning that the explanation cites factors that did not influence the model's answer. Existing approaches to improving LLM faithfulness include training-time methods, which require access to model weights and extensive computational resources, and test-time methods that largely focus on addressing unsoundness. We introduce a test-time approach that directly targets incompleteness. We remove from the input the concepts not credited in the model's explanation and re-query the model on the reduced input. This eliminates unmentioned influences while preserving the influence of mentioned concepts. Across two datasets, multiple model families, and two independent faithfulness metrics, our approach improves explanation faithfulness compared to both standard prompting and prompting to encourage faithfulness. Our method is model-agnostic and can be applied at inference time without modifying model parameters, providing a flexible mechanism for reducing hidden influences and improving the reliability and safety of LLM-assisted decision making.

Abstract

Explore similar work

A Removal Based Approach to Improve LLM Faithfulness at Test-Time

Verbosity Tradeoffs and the Impact of Scale on the Faithfulness of LLM Self-Explanations

An Empirical Study of Counterfactual Self-Explanations in LLMs