When Do Explanations Help In-Context Learning? A Comparative Study of Natural Language Explanation Types and Faithfulness

Mahdi Dhaini; Adam Dejl; Juraj Vladika; Volkan Özer; Barbara Plank; Gjergji Kasneci

cs.CLAug 17, 2026

When Do Explanations Help In-Context Learning? A Comparative Study of Natural Language Explanation Types and Faithfulness

Authors: Mahdi Dhaini, Adam Dejl, Juraj Vladika, Volkan Özer, Barbara Plank, Gjergji Kasneci

Organizations: MaiNLP lab, CIS, LMU Munich Munich Center for Machine Learning (MCML) · Department of Computing, Imperial College London · Technical University of Munich · Technical University of Munich, Munich Center for Machine Learning (MCML)

Abstract

Natural language explanations (NLEs) are increasingly used as inputs, for example, as few-shot rationales that influence model behavior in in-context learning (ICL). However, it remains unclear how different types of NLEs compare in their effects on downstream model performance in explanation-augmented prompting. Therefore, we provide a comparative evaluation across six benchmarks and four instruction-tuned models, studying how NLE source (human-written when available, self-generated explanations, generated by an external LLM) and NLE selection (random vs faithfulness-based filtering) affect downstream utility of NLEs when used in ICL settings. Our extensive evaluation shows that, on classification-style benchmarks, adding NLEs to few-shot prompts often improves accuracy over few-shot prompting without explanations; among NLE sources, externally generated LLM-NLEs often provide strong downstream utility and remain competitive with human rationales where both are available, whereas self-NLEs are more sensitive to the selection strategy. On math reasoning, the effects are more model- and source-dependent. We further show that faithfulness-based selection of self-NLEs yields small average gains overall, but can improve or reduce performance depending on the metric, task, and model. Different faithfulness metrics can disagree substantially, affecting which self-NLE examples are selected and their downstream predictive utility. Robustness tests with randomly swapped and out-of-distribution rationales indicate partial robustness, suggesting that semantic alignment contributes to performance gains. Overall, our results provide insights for selecting and reporting explanations that influence model behavior in practical prompting pipelines.

Figures & tables

Figure 1: Overview of Experimental Setup.

	Baselines			Random
Dataset	ZS	FS-R	CoT	Self-NLE	Human-NLE	LLM-4o	LLM-o3
ecqa	$0.760_{0.011}$	$0.791_{0.014}$	$0.756_{0.015}$	$0.753_{0.045}$	$0.773_{0.029}$	$\mathbf{0.799}_{0.004}$	$0.780_{0.007}$
e- snli	$0.593_{0.008}$	$0.724_{0.017}$	$0.591_{0.012}$	$0.711_{0.026}$	$\mathbf{0.726}_{0.038}$	$0.701_{0.009}$	$0.608_{0.021}$
snarks	$0.566_{0.014}$	$0.587_{0.021}$	$0.752_{0.019}$	$0.556_{0.014}$	–	$0.801_{0.009}$	$\mathbf{0.814}_{0.016}$
boolean	$0.516_{0.009}$	$0.671_{0.011}$	$0.806_{0.011}$	$0.877_{0.033}$	–	$0.900_{0.003}$	$\mathbf{0.916}_{0.011}$
cj	$0.587_{0.013}$	$0.619_{0.018}$	$0.648_{0.016}$	$0.650_{0.023}$	–	$0.650_{0.015}$	$\mathbf{0.654}_{0.008}$

Table 1: NLE Type comparison under random selection (RQ1). Values represent performance (

mean_{\text{standard deviation}}

) over five runs for baseline methods and random-selection explanation setup, averaged over the four evaluated models for each dataset. Bold indicates the best-performing method per dataset.

Dataset	Setting 1	Setting 2 (Most-Faithful)		Setting 3 (Lowest-Faithful)
	Random	$fm_{1}$	$fm_{2}$	$fm_{1}$	$fm_{2}$
ecqa	LLM-4o (0.799)	Human-NLE (0.774)	Human-NLE (0.783)	LLM-o3 (0.801)	LLM-o3 (0.806)
e- snli	Human-NLE (0.726)	Human-NLE (0.762)	Human-NLE (0.741)	Human-NLE (0.756)	Human-NLE (0.763)
snarks	LLM-o3 (0.814)	LLM-o3 (0.816)	LLM-o3 (0.818)	LLM-o3 (0.798)	LLM-o3 (0.794)
boolean	LLM-o3 (0.916)	LLM-4o (0.926)	LLM-o3 (0.910)	LLM-o3 (0.911)	LLM-4o (0.919)
cj	LLM-o3 (0.654)	LLM-o3 (0.634)	LLM-o3 (0.654)	LLM-o3 (0.647)	Self-NLE (0.646)

Table 2: Best-performing NLE source under each selection setting (RQ1). Each cell reports the NLE source with the highest mean absolute accuracy, averaged over the four evaluation models, with the accuracy shown in parentheses. Cell colors indicate the NLE type: Self-NLE , Human-NLE , and LLM-NLE .

Dataset	Models	Random	Faithful-fm1	Faithful-fm2	Lowest-Faithful-fm1	Lowest-Faithful-fm2
Dataset	Models	Self-exp.	$\Delta$ vs Rand.	$\Delta$ vs Rand.	$\Delta$ vs Rand.	$\Delta$ vs Rand.
ecqa	4o-mini	0.828	-0.004	-0.008	-0.004	+0.001 †
	llama-70B	0.745	+0.037	+0.068	-0.086	+0.117 †
	llama-8B	0.780	-0.021	0.000	-0.550	+0.002
	mistral-7B	0.660	+0.036	+0.026	+0.018	+0.064
	Avg	0.753	+0.012	+0.022	-0.155	+0.046

Table 3: Self-NLE performance under random and faithfulness-based selection (RQ2). Random Self-exp. reports accuracy. Other columns report

\Delta

vs. random self-NLE selection ((

+

): improvement, (

-

): degradation). Bold marks the best strategy per dataset–model pair.

{\dagger}

indicates that faithfulness-based self-NLE selection also outperforms the strongest non-NLE baseline for the same pair.

Dataset	Random				OOD
Dataset	Self-exp.	Human	4o-mini	o3-mini	OOD
ecqa	-0.062	+0.002	-0.124	-0.002	-0.081
e- snli	-0.053	-0.106	-0.162	-0.128	-0.113
snarks	+0.063	–	-0.033	-0.061	-0.101
bool	-0.098	–	-0.181	-0.070	-0.090
cj	-0.054	–	-0.086	-0.074	-0.029

Table 4: The difference (

\Delta

) of average performance across five runs of models per dataset for Random and OOD settings vs the faithful selection (setting 2 with

fm_{1}

) across datasets and explanation types (RQ3).

Appendix figures & tables34 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Size
ecqa	310
e- snli	168
snarks	181
boolean	250
Causal Judgment	190
gsm8k	250

Appendix

Table 5: Datasets and their sizes used in experiments

Dataset	Faithful - Random				OOD
Dataset	Self-exp.	Human	4o-mini	o3-mini	OOD
ecqa	0.011	0.010	0.013	0.009	0.043
e- snli	0.023	0.037	0.022	0.031	0.027
snarks	0.019	–	0.015	0.027	0.015
bool	0.065	–	0.044	0.008	0.008
cj	0.022	–	0.020	0.015	0.018

Appendix

Table 6: Standard deviations across five runs for the average performance of models per dataset under Random explanations (Setting 4) and OOD explanations (Setting 5).

Dataset	Models	Step B - Step A
		Random	Faithful-fm1	Faithful-fm2	Lowest-Faithful-fm1	Lowest-Faithful-fm2
		Self-exp.	Self-exp.	Self-exp.	Self-exp.	Self-exp.
ecqa	4o-mini	+0.011	+0.012	+0.001	0.000	0.000
	llama-70B	-0.098	-0.059	0.000	-0.156	-0.002
	llama-8B	+0.003	-0.038	-0.014	-0.542	0.000
	mistral-7B	-0.054	-0.047	-0.032	-0.037	-0.004

Appendix

Table 7: Comparison of two self-explanation variants in the Ph-CoT setup. Step A generates post-hoc self-explanations after prediction, while Step B regenerates explanations for Step A errors conditioned on the gold label. The reported values show the performance delta (Step B

-

Step A) across datasets and models, with positive values indicating improvement.

	Self-NLEs				Human-NLE	LLM-NLEs
Dataset	4o-mini	llama-70b	llama-8b	mistral-7b	Human	4o-mini	o3-mini
ecqa	$42.18_{7.09}$	$70.28_{8.81}$	$54.37_{9.48}$	$35.17_{10.56}$	$44.14_{13.53}$	$39.06_{9.19}$	$29.95_{7.45}$
e- snli	$50.76_{9.56}$	$76.53_{11.51}$	$62.33_{8.76}$	$48.83_{10.62}$	$10.98_{4.73}$	$23.17_{4.56}$	$26.55_{5.38}$
snarks	$50.78_{9.74}$	$81.67_{10.33}$	$61.10_{9.81}$	$49.64_{13.31}$	—	$42.03_{8.74}$	$38.91_{7.90}$
boolean	$43.53_{8.50}$	$77.87_{11.22}$	$67.16_{11.41}$	$50.95_{19.72}$	—	$44.14_{10.98}$	$34.02_{8.05}$
cj	$53.59_{9.12}$	$96.85_{14.85}$	$73.97_{13.90}$	$48.94_{14.70}$	—	$60.74_{13.65}$	$43.96_{9.76}$

Appendix

Table 8: Average length of NLEs across different datasets, models, and NLE types. Values report mean word counts with standard deviations shown as subscripts.

Figure 2: Performance comparison of explanation selection strategies using small and large LMs from the same model family (Llama- 8B and 70B) and from the different family in Mistral 7B. Results are averaged across all datasets to highlight the impact of model scale on strategy effectiveness.

Dataset	Model	Disagreement	Rate	Dis. Dir.
Dataset	Model	Disagreement	Rate	fm1 ✓ fm2 ✗	fm1 ✗ fm2 ✓
ecqa	4o-mini	158	0.510	95	63
	llama-70B	168	0.542	131	37
	llama-8B	162	0.523	117	45
	mistral-7B	111	0.358	74	37
	Avg	150	0.483	104	46

Appendix

Table 9: Disagreements between Faithfulness Metrics

fm1

and

fm2

. The table details the total count, rate, and direction of disagreements across all datasets and models.

Figure 3: Binary heatmap visualization of ecqa test samples comparing two faithfulness metrics using the 4o-mini model. Each column represents a test sample, while rows correspond to the two faithfulness metrics. Cells are color-coded to indicate binary outcomes: green = True, blue = False.

Figure 4: Binary heatmap visualization of ecqa test samples comparing two faithfulness metrics using the Llama-70B model. Each column represents a test sample, while rows correspond to the two faithfulness metrics. Cells are color-coded to indicate binary outcomes: green = True, blue = False.

Figure 5: Binary heatmap visualization of ecqa test samples comparing two faithfulness metrics using the Llama-8B model. Each column represents a test sample, while rows correspond to the two faithfulness metrics. Cells are color-coded to indicate binary outcomes: green = True, blue = False.

Figure 6: Binary heatmap visualization of ecqa test samples comparing two faithfulness metrics using the Mistral-7B model. Each column represents a test sample, while rows correspond to the two faithfulness metrics. Cells are color-coded to indicate binary outcomes: green = True, blue = False.

Figure 7: NLE type comparison under random selection (RQ1). Each panel reports results for one dataset. Bars present mean accuracy across five runs, averaged over the four evaluation models. Error bars indicate the standard deviation across runs. Baselines include ZS, FS-R, and CoT, while random-selection NLE conditions include Self-NLEs, Human-NLEs where available, LLM-4o, and LLM-o3. Bar fill patterns distinguish the prompting conditions: solid fill (ZS), diagonal hatching (FS-R), circular hatching (CoT), horizontal hatching (Self-NLE), vertical hatching (Human-NLE), dotted hatching (LLM-4o), and cross-hatching (LLM-o3).

Dataset	Models	Zero-shot	Few-shot	CoT	Random
Dataset	Models	Zero-shot	Rand.	CoT	Self-exp.	Human	4o-mini	o3-mini
ecqa	4o-mini	0.778	0.826	0.784	0.828	0.814	0.832	0.831
	llama-70B	0.826	0.832	0.816	0.745	0.812	0.859	0.856
	llama-8B	0.751	0.792	0.759	0.780	0.806	0.831	0.794
	mistral-7B	0.684	0.715	0.665	0.660	0.659	0.676	0.638
	Avg	0.760	0.791	0.756	0.753	0.773	0.799	0.780

Appendix

Table 10: Model Specific results for the Error sample selection setup, showing Random Selection performance. We report self-explanation ( Self-exp. ) and LLM explanation two variants ( 4o-mini and o3-mini ). Dashes (–) indicate unavailable human explanations for non-explainable datasets. Highest values in each row are highlighted in bold .

Dataset	Models	Few-shot			Random	Faithful-fm1	Faithful-fm2	Lowest-Faithful-fm1	Lowest-Faithful-fm2
Dataset	Models	Rand. (Ref)	Faith.-1	Faith.-2	Self-NLE	Self-NLE	Self-NLE	Self-NLE	Self-NLE
ecqa	4o-mini	0.826	0.820	0.823	0.828 ***	$0.824^{***}$	$0.820^{***}$	$0.824^{***}$	0.829
	llama-70B	0.832	0.847	0.838	$0.745^{***}$	$0.782^{***}$	$0.813^{***}$	$0.659^{***}$	0.864 ***
	llama-8B	0.792	0.799	0.794	0.780	$0.759^{***}$	$0.780^{**}$	$0.230^{***}$	0.782
	mistral-7B	0.715	0.749	0.730	$0.660^{**}$	$0.696^{**}$	$0.686^{**}$	0.678	$0.724^{***}$
	Avg	0.791	0.804	0.796	0.753	0.765	0.775	0.598	0.799

Appendix

Table 11: Results for the Error sample-selection setup. We report mean performance over five runs for each model (4o-mini, Llama-70B , Llama-8B , and Mistral-7B ) on five datasets across the following prompting and explanation-selection conditions: Few-shot Rand., Few-shot Faith.-1, Few-shot Faith.-2, Random, Faithful-

fm_{1}

, Faithful-

fm_{2}

, Lowest-Faithful-

fm_{1}

, and Lowest-Faithful-

fm_{2}

. The highest value in each row is shown in bold . Statistical significance is assessed using the Stuart–Maxwell test, where

{}^{*}p<0.10

{}^{**}p<0.05

, and

{}^{***}p<0.01

. The Few-shot Random baseline ( FS-R ) serves as the reference for statistical tests involving self-NLEs ( Self-exp. ). FS-R follows Setup 1, using randomly selected few-shot input-output pairs without rationales. Few-shot Faith.-1 ( FS-

fm_{1}

) and Few-shot Faith.-2 ( FS-

fm_{2}

) are matched no-rationale baselines: they use the same selected input-output pairs

(x,y)

as the corresponding

fm_{1}

- and

fm_{2}

-based conditions, respectively, while omitting the rationales.

Dataset	Models	Zero-shot	Few-shot			CoT	Random				Faithful-fm1				Faithful-fm2				Lowest-Faithful-fm1				Lowest-Faithful-fm2
Dataset	Models	Zero-shot	Rand.	Faith.-1	Faith.-2	CoT	Self-exp.	Human	4o-mini	o3-mini	Self-exp.	Human	4o-mini	o3-mini	Self-exp.	Human	4o-mini	o3-mini	Self-exp.	Human	4o-mini	o3-mini	Self-exp.	Human	4o-mini	o3-mini
ecqa	4o-mini	0.778	0.826	0.820	0.823	0.784	0.828	0.814	0.832	0.831	0.824	0.823	0.822	0.823	0.820	0.810	0.814	0.812	0.824	0.816	0.816	0.829	0.829	0.829	0.838	0.834
	llama-70B	0.826	0.832	0.847	0.838	0.816	0.745	0.812	0.859	0.856	0.782	0.861	0.847	0.858	0.813	0.860	0.859	0.864	0.659	0.854	0.854	0.859	0.862	0.848	0.848	0.855
	llama-8B	0.751	0.792	0.799	0.794	0.759	0.780	0.806	0.831	0.794	0.759	0.775	0.791	0.791	0.780	0.745	0.583	0.774	0.230	0.786	0.786	0.785	0.782	0.781	0.792	0.801
	mistral-7B	0.684	0.715	0.749	0.730	0.665	0.660	0.659	0.676	0.638	0.696	0.635	0.621	0.575	0.686	0.717	0.695	0.673	0.678	0.744	0.665	0.731	0.724	0.719	0.692	0.735
	Avg	0.760	0.791	0.804	0.796	0.756	0.753	0.773	0.799	0.780	0.765	0.774	0.770	0.762	0.775	0.783	0.738	0.781	0.598	0.800	0.780	0.801	0.799	0.794	0.793	0.806

Appendix

Table 12: Results for the Error sample selection setup, showing the performance of average of five runs of each model (4o-mini, Llama-70B , Llama-8B , and Mistral-7B ) on each dataset across explanation selection setups (Random, Faithful-1, Lowest-Faithful-1, and Lowest-Faithful-2). For each selection setup, we report self-explanations ( Self-exp. ), along with human and LLM-generated explanations ( 4o-mini and o3-mini ). Dashes (–) denote unavailable human explanations for non-explainable datasets. Highest values in each row are highlighted in bold .

Dataset	Zero-shot	Few-shot			CoT	Random				Faithful- $fm_{1}$				Faithful- $fm_{2}$				Lowest-Faithful- $fm_{1}$				Lowest-Faithful- $fm_{2}$
Dataset	Zero-shot	Rand.	Faith.-1	Faith.-2	CoT	Self-exp.	Human	4o-mini	o3-mini	Self-exp.	Human	4o-mini	o3-mini	Self-exp.	Human	4o-mini	o3-mini	Self-exp.	Human	4o-mini	o3-mini	Self-exp.	Human	4o-mini	o3-mini
ecqa	0.760	0.791	0.804	0.796	0.756	0.753	0.773	0.799	0.780	0.765	0.774	0.770	0.762	0.775	0.783	0.738	0.781	0.598	0.800	0.780	0.801	0.799	0.794	0.793	0.806
e- snli	0.593	0.724	0.737	0.749	0.591	0.711	0.726	0.701	0.608	0.713	0.762	0.743	0.721	0.716	0.741	0.717	0.701	0.735	0.756	0.731	0.704	0.675	0.763	0.729	0.696
snarks	0.566	0.587	0.614	0.551	0.752	0.556	–	0.801	0.814	0.552	–	0.792	0.816	0.595	–	0.805	0.818	0.564	–	0.775	0.798	0.665	–	0.788	0.794
boolean	0.516	0.671	0.732	0.786	0.806	0.877	–	0.900	0.916	0.869	–	0.926	0.910	0.879	–	0.886	0.910	0.845	–	0.904	0.911	0.900	–	0.919	0.894
cj	0.587	0.619	0.626	0.619	0.648	0.650	–	0.650	0.654	0.633	–	0.626	0.634	0.637	–	0.634	0.654	0.643	–	0.630	0.647	0.646	–	0.638	0.638

Appendix

Table 13: Average results for the Error sample selection setup. Values are averaged over the four evaluation models and five runs. We report baseline prompting, random NLE selection, faithful selection using

fm_{1}

and

fm_{2}

, and lowest-faithful selection using

fm_{1}

and

fm_{2}

. Dashes (–) denote unavailable human explanations for datasets without human rationales.

		Setting 1	Setting 2		Setting 3
Dataset	NLE source	Random	$\Delta$ Faithful- $fm_{1}$	$\Delta$ Faithful- $fm_{2}$	$\Delta$ Lowest- $fm_{1}$	$\Delta$ Lowest- $fm_{2}$
ecqa	Self-NLE	0.753	$+0.012$ (0.765)	$+0.022$ (0.775)	$-0.155$ (0.598)	$+0.046$ (0.799)
	Human-NLE	0.773	$+0.001$ ( 0.774 )	$+0.010$ ( 0.783 )	$+0.027$ (0.800)	$+0.021$ (0.794)
	LLM-4o	0.799	$-0.029$ (0.770)	$-0.061$ (0.738)	$-0.019$ (0.780)	$-0.006$ (0.793)
	LLM-o3	0.780	$-0.018$ (0.762)	$+0.001$ (0.781)	$+0.021$ ( 0.801 )	$+0.026$ ( 0.806 )
e- snli	Self-NLE	0.711	$+0.002$ (0.713)	$+0.005$ (0.716)	$+0.024$ (0.735)	$-0.036$ (0.675)

Appendix

Table 14: Dataset-level change in accuracy relative to random selection by NLE source. Setting 1 corresponds to random selection, Setting 2 to most-faithful selection using

fm_{1}

fm_{2}

, and Setting 3 to lowest-faithful selection using

fm_{1}

fm_{2}

. The Random column reports mean accuracy under random selection, averaged over the four evaluation models and five runs. The remaining columns report

\Delta

accuracy relative to Random, with the absolute accuracy under that setup shown in parentheses. Positive

\Delta

values indicate improvement over random selection. Bold values indicate the highest absolute accuracy within each dataset and selection column . Human-NLE rows are shown only for ecqa and e- snli , where human rationales are available.

Dataset	Models	Zero-shot	Few-shot			CoT	Random				Faithful-fm1				Faithful-fm2				Lowest-Faithful-fm1				Lowest-Faithful-fm2
Dataset	Models	Zero-shot	Rand.	Faith.-1	Faith.-2	CoT	Self-exp.	Human	4o-mini	o3-mini	Self-exp.	Human	4o-mini	o3-mini	Self-exp.	Human	4o-mini	o3-mini	Self-exp.	Human	4o-mini	o3-mini	Self-exp.	Human	4o-mini	o3-mini
ecqa	4o-mini	0.007	0.006	0.009	0.013	0.005	0.010	0.004	0.004	0.012	0.008	0.005	0.005	0.008	0.006	0.008	0.006	0.008	0.005	0.004	0.006	0.006	0.008	0.005	0.008	0.008
	llama-70B	0.007	0.008	0.002	0.004	0.006	0.172	0.122	0.006	0.006	0.126	0.005	0.003	0.005	0.009	0.006	0.008	0.005	0.236	0.022	0.207	0.007	0.004	0.005	0.006	0.005
	llama-8B	0.015	0.003	0.005	0.004	0.006	0.009	0.006	0.006	0.021	0.032	0.018	0.021	0.007	0.009	0.139	0.152	0.004	0.059	0.012	0.007	0.011	0.004	0.007	0.011	0.011
	mistral-7B	0.003	0.008	0.002	0.000	0.010	0.040	0.008	0.005	0.005	0.029	0.207	0.174	0.244	0.012	0.008	0.012	0.006	0.018	0.001	0.010	0.006	0.007	0.006	0.007	0.003
	Avg	0.008	0.006	0.005	0.005	0.007	0.058	0.035	0.005	0.011	0.049	0.059	0.051	0.066	0.009	0.040	0.045	0.006	0.080	0.010	0.058	0.008	0.006	0.006	0.008	0.007

Appendix

Table 15: Results for the Error sample selection setup, reporting the standard deviations across five runs of each model (4o-mini, Llama-70B , Llama-8B , and Mistral-7B ) on all datasets under different explanation selection strategies (Random, Faithful-1, Faithful-2, Lowest-Faithful-1, and Lowest-Faithful-2). For each selection setup, we report self-explanations ( Self-exp. ), along with human and LLM-generated explanations ( 4o-mini and o3-mini ). Dashes (–) indicate unavailable human explanations for non-explainable datasets. The highest standard deviation in each row is highlighted in bold .

Figure 8: Accuracy values across explanation selection strategies: Baseline , Random Selection , Faithful Selection-

fm1

, Faithful Selection-

fm2

, Lowest-Faithful-

fm1

Selection , and Lowest-Faithful-

fm2

Selection ; averaged over the four considered models and evaluated on six datasets: ecqa , e- snli , snarks , boolean , Causal Judgment, and gsm8k . Baselines include Zero-shot, Few-shot with random or faithful inputs and outputs only (no explanations), and CoT. Other strategies involve Self-NLEs, Human-provided, or LLM-NLEs (from 4o-mini and o3-mini ).

(a) ecqa , e- snli & snarks

Figure 10: Model-specific accuracy results across explanation selection strategies, Baseline (red), Random Selection (blue), Faithful Selection 1 (yellow), Faithful Selection 2 (purple), and Lowest-Faithful Selection (green) evaluated using 4o-mini (left), Llama-70B (center-left), Llama-8B (center-right), and Mistral-7B (right). Baselines include Zero-shot, Few-shot with random or faithful inputs and outputs only (no explanations), and CoT. Other strategies involve Self-generated, Human-provided, or LLM-generated explanations (from 4o-mini and o3-mini ).

Panel of Figure 11 — Figure 11: Illustration of an in-context learning prompt using Random Rationales (Setting 4) on the Boolean dataset with 4o-mini as the evaluation model. The prompt includes an instruction followed by six few-shot examples (top), with explanations generated by 4o-mini. The final input (bottom) is the test question. The model’s prediction is correct . This example reflects an error case under the LLM-explanation strategy.

Panel of Figure 12 — Figure 12: Illustration of an in-context learning prompt using Random Rationales (Setting 4) on the Boolean dataset with 4o-mini as the evaluation model. The prompt includes an instruction followed by six few-shot examples (top), with explanations generated by o3-mini . The final input (bottom) is the test question. The model’s prediction is correct . This example reflects an error case under the LLM-explanation strategy .

Panel of Figure 13 — Figure 13: Illustration of an in-context learning prompt using Random Rationales (Setting 4) on the Boolean dataset with Llama-8B as the evaluation model. The prompt includes an instruction followed by six few-shot examples (top), with explanations generated by the evaluation model itself. The final input (bottom) is the test question. The model’s prediction is correct . This example reflects an error case under the Self-explanation strategy.

Panel of Figure 14 — Figure 14: Illustration of an in-context learning prompt using Random Rationales (Setting 4) on the ECQA dataset with Llama-70B as the evaluation model. The prompt includes an instruction followed by six few-shot examples (top), with explanations generated by the evaluation model itself. The final input (bottom) is the test question. The model’s prediction is correct . This example reflects an error case under the Self-explanation strategy.

Panel of Figure 15 — Figure 15: Illustration of an in-context learning prompt using OOD Rationales (Setting 5) on the Boolean dataset with Llama-8B as the evaluation model. Snarks dataset LLM-explanations generated by o3-mini are used as OOD-random explanations from a different domain. The prompt includes an instruction followed by six few-shot examples (top). The final input (bottom) is the test question. The model’s prediction is wrong . This example reflects an error case under the LLM-explanation strategy.

Panel of Figure 16 — Figure 16: Illustration of an in-context learning prompt using OOD Rationales (Setting 5) on the Boolean dataset with Llama-8B as the evaluation model. Snarks dataset LLM-explanations generated by o3-mini are used as OOD-random explanations from a different domain. The prompt includes an instruction followed by six few-shot examples (top). The final input (bottom) is the test question. The model’s prediction is correct . This example reflects an error case under the LLM-explanation strategy.

Panel of Figure 17 — Figure 17: Illustration of an in-context learning prompt using OOD Rationales (Setting 5) on the ECQA dataset with Llama-70B as the evaluation model. e-SNLI dataset LLM-explanations generated by 4o-mini are used as OOD-random explanations from a different domain. The prompt includes an instruction followed by six few-shot examples (top). The final input (bottom) is the test question. The model’s prediction is wrong . This example reflects an error case under the LLM-explanation strategy.

Panel of Figure 18 — Figure 18: Illustration of an in-context learning prompt using OOD Rationales (Setting 5) on the Snarks dataset with Llama-70B as the evaluation model. Boolean dataset LLM-explanations generated by 4o-mini are used as OOD-random explanations from a different domain. The prompt includes an instruction followed by six few-shot examples (top). The final input (bottom) is the test question. The model’s prediction is correct . This example reflects an error case under the LLM-explanation strategy.

Panel of Figure 19 — Figure 19: Illustration of an in-context learning prompt using OOD Rationales (Setting 5) on the Snarks dataset with Llama-70B as the evaluation model. Boolean dataset LLM-explanations generated by 4o-mini are used as OOD-random explanations from a different domain. The prompt includes an instruction followed by six few-shot examples (top). The final input (bottom) is the test question. The model’s prediction is correct . This example reflects an error case under the LLM-explanation strategy.

Panel of Figure 20 — Figure 20: Explanations generated for a single ECQA input across six different setups: self-generated explanations from 4o-mini, Llama-70B , and Llama-8B ; a human-written explanation; and LLM-generated explanations from 4o-mini and o3-mini . This comparison highlights differences in explanation style, detail, and reasoning approach depending on model size, explanation strategy, and human vs. model generation.

Panel of Figure 21 — Figure 21: Explanations generated for a single e-SNLI input across six different setups: self-generated explanations from 4o-mini, Llama-70B , and Llama-8B ; a human-written explanation; and LLM-generated explanations from 4o-mini and o3-mini . This comparison highlights differences in explanation style, detail, and reasoning approach depending on model size, explanation strategy, and human vs. model generation.

Panel of Figure 22 — Figure 22: Explanations generated for a single Snarks input across six different setups: self-generated explanations from 4o-mini, Llama-70B , and Llama-8B ; and LLM-generated explanations from 4o-mini and o3-mini . This comparison highlights differences in explanation style, detail, and reasoning approach depending on model size, explanation strategy, and human vs. model generation.

Panel of Figure 23 — Figure 23: Explanations generated for a single Boolean input across six different setups: self-generated explanations from 4o-mini, Llama-70B , and Llama-8B ; and LLM-generated explanations from 4o-mini and o3-mini . This comparison highlights differences in explanation style, detail, and reasoning approach depending on model size, explanation strategy, and human vs. model generation.

Panel of Figure 24 — Figure 24: Explanations generated for a single Causal Judgment input across six different setups: self-generated explanations from 4o-mini, Llama-70B , and Llama-8B ; and LLM-generated explanations from 4o-mini and o3-mini . This comparison highlights differences in explanation style, detail, and reasoning approach depending on model size, explanation strategy, and human vs. model generation.

In-Context Learning

In-context learning studies how models adapt to new tasks from examples or instructions supplied at inference time, without changing their weights.

See all 404 papers

Sep 15, 2026

An Empirical Study of Counterfactual Self-Explanations in LLMs

Large language models can easily generate explanations for their own outputs, but such self-explanations are not necessarily faithful to the model's behavior. We study this issue through counterfactual self-explanations, where a model minimally edits an input so that its own prediction changes. Across sentiment analysis and natural language inference, we evaluate ten instruction-tuned models from the LLaMA-3 and Qwen-2.5 families, measuring faithfulness, minimality, and alignment with human-annotated rationales. Our results show that model scale is the strongest determinant of explanation quality: larger models are substantially more likely to generate counterfactuals that flip their own predictions and target decision-relevant evidence. In contrast, the rationale-guided condition produces edit-minimal counterfactuals that are also more human-aligned. However, it does not consistently improve faithfulness. Overall, counterfactual self-explanations can provide useful behavioral evidence about model decisions, but their reliability depends strongly on model capacity and should be empirically validated rather than assumed.

Giannis Kalyvas, Giorgos Filandrianos, Orfeas Menis Mastromichalakis +2

National Technical University of Athens, Athens, Greece · Instituto de Telecomunicações, Lisbon, Portugal

Counterfactual Explanations Faithfulness of Language Model Explanations LLM Interpretability Counterfactual Reasoning in Language Models LLM Reliability

Ask PDF

May 31, 2026

Not All Explanations Simulate Equally: Comparing Verbalized Feature Attributions and Self-Generated Rationales

Natural-language explanations are often treated as a unified interface for understanding model behavior, but different explanation sources may support simulation in different ways. This paper compares two families of explanations for question answering models: verbalized feature attributions and self-generated rationales. We evaluate them under a shared counterfactual simulation setting, using an LLM judge as predictor and measuring whether it can better predict a model's answers to follow-up questions when given its explanation. Across multiple instruction-tuned models, we analyze how explanation source, verbalization strategy, and feature granularity affect the simulatability of explanations. Our results show that explanation format and granularity affect simulatability: attribution-based explanations and self-generated rationales differ in how much they improve counterfactual prediction, with effects that vary across models and formats.

Pingjun Hong, Benjamin Roth

Faculty of Computer Science, University of Vienna, Vienna, Austria · UniVie Doctoral School Computer Science, University of Vienna, Vienna, Austria · Faculty of Philological and Cultural Studies, University of Vienna, Vienna, Austria

Feature Attribution LLM Interpretability Explainability Evaluation

Ask PDF

Feb 2, 2026

A Positive Case for Faithfulness: LLM Self-Explanations Help Predict Model Behavior

LLM self-explanations are often presented as a promising tool for AI oversight, yet their faithfulness to the model's true reasoning process is poorly understood. Existing faithfulness metrics have critical limitations, typically relying on identifying unfaithfulness via adversarial prompting or detecting reasoning errors. These methods overlook the predictive value of explanations. We introduce Normalized Simulatability Gain (NSG), a general and scalable metric based on the idea that a faithful explanation should allow an observer to learn a model's decision-making criteria, and thus better predict its behavior on related inputs. We evaluate 18 frontier proprietary and open-weight models, e.g., Gemini 3, GPT-5.2, and Claude 4.5, on 7,000 counterfactuals from popular datasets covering health, business, and ethics. We find self-explanations substantially improve prediction of model behavior (11-37% NSG). Self-explanations also provide more predictive information than explanations generated by external models, even when those models are stronger. This implies an advantage from self-knowledge that external explanation methods cannot replicate. Our approach also reveals that, across models, 5-15% of self-explanations are highly misleading. Despite their imperfections, we show a positive case for self-explanations: they encode information that helps predict model behavior.

Harry Mayne, Justin Singh Kang, Dewi Gould +3

University of Oxford · University of California, Berkeley · Independent +2

Faithfulness of Language Model Explanations LLM Interpretability Explainability Evaluation

Ask PDF

Abstract

In-Context Learning

Explore similar work

An Empirical Study of Counterfactual Self-Explanations in LLMs

Not All Explanations Simulate Equally: Comparing Verbalized Feature Attributions and Self-Generated Rationales

A Positive Case for Faithfulness: LLM Self-Explanations Help Predict Model Behavior