Breaking the Chain: A Causal Analysis of LLM Faithfulness to Intermediate Structures

Oleg Somov; Mikhail Chaichuk; Gleb Ershov; Karim Vafin; Mikhail Seleznyov; Alexander Panchenko; Elena Tutubalina

cs.AIMar 17, 2026

Breaking the Chain: A Causal Analysis of LLM Faithfulness to Intermediate Structures

Authors: Oleg Somov, Mikhail Chaichuk, Gleb Ershov, Karim Vafin, Mikhail Seleznyov, Alexander Panchenko, Elena Tutubalina

Organizations: AIRI · MIPT · HSE University · Avito AI Lab · Skoltech

Abstract

In schema-guided reasoning (SGR) pipelines, LLMs produce explicit intermediate structures -- rubrics, checklists, or verification queries -- before committing to a final decision. SGR is increasingly adopted because it promises controllability: practitioners expect to inspect, edit, and override these structures to steer the outcome. But does the promise hold? We introduce a causal evaluation protocol to measure it: by selecting tasks where a deterministic function maps intermediate structures to decisions, every controlled edit implies a unique correct output. Across 12 models and 4 benchmarks, models appear self-consistent with their own intermediate structures but fail to update predictions after intervention -- revealing that apparent faithfulness is fragile once the intermediate structure changes. When derivation of the final decision from the structure is delegated to an external tool, this fragility largely disappears; stronger prompting yields only limited improvements, while preference optimization substantially improves intervention faithfulness. Overall, intermediate structures in schema-guided pipelines function as influential context rather than stable causal mediators.

Figures & tables

Figure 1: Causal faithfulness in schema - guided reasoning. In high - stakes applications, domain experts (e.g., doctors, teachers) may edit the intermediate structure produced by an LLM such as a checklist or rubric. If the model is faithful, the final decision should update to reflect the edited mediator. This paper evaluates whether such interventions reliably change the model’s output.

Figure 2: Causal framing with explicit intervention. $X\rightarrow M\rightarrow M^{\star}\rightarrow Y$ shows mediator replacement via $\mathrm{do}(M{=}M^{\star})$ .

Dataset	Input $X$	Mediator $M$	Intervention	$M$ Complexity	Target $\tilde{Y}$	$N$
RiceChem	Student answer	True/False rubric + weights	Flip rubric entries	6–8 entries	Recomputed score	1233
AVeriTeC	Claim text	Binary sub-question answers	Flip sub-answers	1–7 sub-qs.	Flipped verdict	178
TabFact	Statement + table	Verification query	Swap columns / operators	1–14 attrs.	Entailed or Refuted	1252
CRUXEval	Function + input	Execution trace	Flip variable value	3–15 steps	Recomputed output	788

Table 1: Overview of benchmark datasets. Each dataset provides a structured mediator

M

that admits deterministic interventions with known counterfactual targets

\tilde{Y}

	RiceChem				AVeriTeC				TabFact				CRUXEval
Model	$F_{\scriptscriptstyle\mathrm{ID}}$	$F_{\scriptscriptstyle\mathrm{OOD}}$	$F_{\scriptscriptstyle\mathrm{Strong}}$	$\Delta$	$F_{\scriptscriptstyle\mathrm{ID}}$	$F_{\scriptscriptstyle\mathrm{OOD}}$	$F_{\scriptscriptstyle\mathrm{Strong}}$	$\Delta$	$F_{\scriptscriptstyle\mathrm{ID}}$	$F_{\scriptscriptstyle\mathrm{OOD}}$	$F_{\scriptscriptstyle\mathrm{Strong}}$	$\Delta$	$F_{\scriptscriptstyle\mathrm{ID}}$	$F_{\scriptscriptstyle\mathrm{OOD}}$	$F_{\scriptscriptstyle\mathrm{Strong}}$	$\Delta$
Qwen-3 1.7B	0.44	0.26	0.17	0.27	0.77	0.73	0.60	0.17	0.11	0.35	0.08	0.03	0.26	0.26	0.13	0.13
Gemma-2 2B	0.58	0.36	0.22	0.36	0.72	0.50	0.41	0.31	0.04	0.31	0.02	0.02	0.30	0.35	0.18	0.12
Falcon-3 3B	0.43	0.40	0.25	0.18	0.85	0.85	0.75	0.10	0.19	0.42	0.09	0.10	0.42	0.41	0.26	0.16
Llama-3.2 3B	0.28	0.21	0.05	0.23	0.13	0.11	0.08	0.05	0.00	0.00	0.00	0.00	0.23	0.24	0.14	0.09
Qwen-3 4B	0.92	0.74	0.68	0.24	0.84	0.78	0.65	0.19	0.29	0.59	0.21	0.08	0.48	0.22	0.16	0.32

Table 2: Overall faithfulness results.

F_{\mathrm{ID}}

reports in-distribution consistency,

F_{\mathrm{OOD}}

reports post-intervention faithfulness, and

F_{\mathrm{Strong}}

requires both original and intervened consistency.

\Delta=F_{\mathrm{ID}}-F_{\mathrm{Strong}}

highlights cases where apparent faithfulness is fragile under intervention.

Figure 3: Symmetry analysis. The X-axis shows faithfulness under Correction interventions (where an incorrect mediator is replaced with a correct one), and the Y-axis shows faithfulness under Counterfactual interventions (and vice versa). Models with fewer than 10 generations in either subset are excluded due to noisy estimates.

Figure 4: The bar plot shows the measured faithfulness gap for each model on the four datasets before and after tool use. The green arrows highlight this reduction, indicating the drop from the original gap to the post–tool-use gap for each model.

	RiceChem			AVeriTeC			TabFact			CRUXEval
Model	Acc.	$F_{\scriptscriptstyle\mathrm{ID}}$	$F_{\scriptscriptstyle\mathrm{Strong}}$	Acc.	$F_{\scriptscriptstyle\mathrm{ID}}$	$F_{\scriptscriptstyle\mathrm{Strong}}$	Acc.	$F_{\scriptscriptstyle\mathrm{ID}}$	$F_{\scriptscriptstyle\mathrm{Strong}}$	Acc.	$F_{\scriptscriptstyle\mathrm{ID}}$	$F_{\scriptscriptstyle\mathrm{Strong}}$
Qwen-3 8B (S)	0.30	0.63	0.52	0.63	0.72	0.31	0.46	0.28	0.19	0.46	0.55	0.30
Qwen-3 8B (D)	0.34	0.73	0.62	0.56	0.64	0.31	0.41	0.29	0.18	0.46	0.54	0.30
Qwen-3 8B (M)	0.34	0.74	0.63	0.39	0.44	0.21	0.42	0.30	0.19	0.48	0.54	0.30
Qwen-3 8B (DPO)	0.30	0.86	0.73	0.92	0.93	0.29	0.79	0.33	0.26	0.59	0.60	0.37
Llama-3.1 8B (S)	0.17	0.35	0.27	0.72	0.51	0.26	0.46	0.20	0.17	0.24	0.32	0.22

Table 3: Faithfulness and accuracy across prompting and fine-tuning regimes. Green indicates improvement over the corresponding Standard (S) regime, while red indicates degradation. Detailed (D) and Max Detailed (M) denote stronger prompting regimes, and FT denotes DPO fine-tuning.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

	RiceChem									AVeriTeC
	$F_{\mathrm{ID}}$			$F_{\mathrm{OOD}}$			$F_{\mathrm{Strong}}$			$F_{\mathrm{ID}}$			$F_{\mathrm{OOD}}$			$F_{\mathrm{Strong}}$
Model	S	D	M	S	D	M	S	D	M	S	D	M	S	D	M	S	D	M
Qwen-3 1.7B	0.44	0.43	0.45	0.26	0.38	0.42	0.17	0.23	0.24	0.77	0.79	0.81	0.73	0.74	0.79	0.60	0.60	0.66
Gemma-2 2B	0.58	0.45	0.34	0.36	0.43	0.42	0.22	0.19	0.12	0.72	0.74	0.78	0.50	0.66	0.64	0.41	0.60	0.57
Falcon-3 3B	0.43	0.49	0.50	0.40	0.52	0.51	0.25	0.31	0.31	0.85	0.83	0.81	0.85	0.85	0.85	0.75	0.74	0.72
Llama-3.2 3B	0.28	0.16	0.18	0.21	0.56	0.57	0.05	0.11	0.11	0.13	0.19	0.17	0.11	0.15	0.12	0.08	0.12	0.10

Appendix

Table 4: Influence of prompt strength on faithfulness metrics across datasets. Each panel contains two datasets.

F_{\mathrm{ID}}

reports in-distribution consistency,

F_{\mathrm{OOD}}

reports post-intervention consistency, and

F_{\mathrm{Strong}}

requires both. Prompt regimes are Standard (S), Detailed (D), and Max Detailed (M).

	RiceChem					TabFact					AVeriTeC					CRUXEval
Model	Base	Std.	Det.	Max	Tool	Base	Std.	Det.	Max	Tool	Base	Std.	Det.	Max	Tool	Std.	Det.	Max	Tool
Qwen-3 1.7B	0.21	0.20	0.12	0.12	0.23	0.11	0.38	0.28	0.28	0.11	0.79	0.63	0.67	0.66	0.53	0.26	0.24	0.25	0.15
Gemma-2 2B	0.30	0.25	0.20	0.14	0.17	0.15	0.30	0.16	0.18	0.06	0.60	0.41	0.47	0.46	0.34	0.24	0.24	0.23	0.19
Falcon-3 3B	0.28	0.22	0.16	0.16	0.28	0.43	0.41	0.35	0.35	0.07	0.83	0.47	0.47	0.49	0.49	0.31	0.30	0.29	0.16
Llama-3.2 3B	0.20	0.22	0.24	0.25	0.21	0.28	0.00	0.00	0.00	0.01	0.80	0.52	0.60	0.51	0.09	0.20	0.21	0.21	0.21
Qwen-3 4B	0.32	0.33	0.39	0.38	0.33	0.47	0.56	0.39	0.42	0.24	0.90	0.67	0.63	0.56	0.58	0.41	0.42	0.43	0.22

Appendix

Table 5: Accuracy across datasets and prompting regimes for all evaluated models. Base denotes the no-mediator baseline. Std denotes the generation with mediator. The last row reports averages across displayed models; Llama-3.2 1B is excluded.

Dataset	Model	Acc.	$F_{\scriptscriptstyle\mathrm{ID}}$	$F_{\scriptscriptstyle\mathrm{Strong}}$
RiceChem	Qwen-3 8B	0.30 $\pm$ 0.00	0.86 $\pm$ 0.01	0.73 $\pm$ 0.00
RiceChem	Llama-3.1 8B	0.34 $\pm$ 0.01	0.31 $\pm$ 0.01	0.29 $\pm$ 0.00
AVeriTeC	Qwen-3 8B	0.92 $\pm$ 0.00	0.93 $\pm$ 0.00	0.29 $\pm$ 0.00
AVeriTeC	Llama-3.1 8B	0.71 $\pm$ 0.02	0.50 $\pm$ 0.01	0.29 $\pm$ 0.00
TabFact	Qwen-3 8B	0.79 $\pm$ 0.01	0.33 $\pm$ 0.01	0.26 $\pm$ 0.01
TabFact	Llama-3.1 8B	0.76 $\pm$ 0.07	0.22 $\pm$ 0.01	0.18 $\pm$ 0.01

Appendix

Table 6: DPO fine-tuning results averaged across 3 seed runs. Reported values are mean

\pm

standard deviation.

Hyperparameter	Value
DPO objective
DPO $\beta$	$0.1$
Optimization
Epochs	$1$
Per-device batch size	$2$
Gradient accumulation steps	$8$

Appendix

Table 7: DPO fine-tuning hyperparameters.

Explore similar work

LIFT: Likelihood-Based Intervention Faithfulness Training
Sep 10, 2025Anand Swaroop, Maheep Chaudhary, Kevin Zhu +2CoT Faithfulness Faithfulness of Language Model Explanations
Algoverse AI Research
Structure Tax: How Structured Output affects LLMs Performance
Oct 8, 2026Vineet Kumar, Kanishka, Bhuvanesh MandoraStructured Output Generation LLM Reliability
PayPal Artificial Intelligence PayPal, Bengaluru India
From Concept Alignment to Causal Grounding: An Intervention Test of Chain-of-Thought Faithfulness
Sep 19, 2026Qianli Wang, Yilong Wang, Dennis Wei +5CoT Faithfulness LLM Interpretability
Technische Universität Berlin · German Research Center for Artificial Intelligence (DFKI) · IBM Research +4

Sep 10, 2025J/K move · Enter open · S save

LIFT: Likelihood-Based Intervention Faithfulness Training

Anand Swaroop, Maheep Chaudhary, Kevin Zhu, Sunishchal Dev+1

Algoverse AI Research

CoT Faithfulness Faithfulness of Language Model Explanations Causal Interventions in Language Models

Open paper AskPDF arXiv

Chain-of-thought (CoT) reasoning improves LLM accuracy on complex tasks, but CoTs are often unfaithful; that is, the model does not rely on the stated reasoning steps to produce its answer. Existing methods to improve faithfulness require human annotations. We propose LIFT, which automatically scores each reasoning step by its causal influence: we replace it with a domain-matched but irrelevant fact and measure the drop in answer log-probability. We fine-tune on high-scoring CoT traces where interventions successfully disrupted reasoning, jointly optimizing for faithfulness and accuracy. LIFT improves faithfulness across three models (Mistral-7B-Instruct, Qwen3-8B, and Qwen3-4B-Instruct) and three benchmarks, with accuracy comparable to a model fine-tuned solely for accuracy.

Abstract

Explore similar work

LIFT: Likelihood-Based Intervention Faithfulness Training

Structure Tax: How Structured Output affects LLMs Performance

From Concept Alignment to Causal Grounding: An Intervention Test of Chain-of-Thought Faithfulness