Fragile Reasoning: A Mechanistic Analysis of LLM Sensitivity to Meaning-Preserving Perturbations

Shou-Tzu Han; Rodrigue Rizk; KC Santosh

cs.CLApr 2, 2026

Fragile Reasoning: A Mechanistic Analysis of LLM Sensitivity to Meaning-Preserving Perturbations

Authors: Shou-Tzu Han, Rodrigue Rizk, KC Santosh

Organizations: Department of Computer Science, University of South Dakota

Abstract

Large language models demonstrate strong performance on mathematical reasoning benchmarks, yet remain surprisingly fragile to meaning-preserving surface perturbations. We systematically evaluate three open-weight LLMs, Mistral-7B, Llama-3-8B, and Qwen2.5-7B, on 677 GSM8K problems paired with semantically equivalent variants generated through name substitution and number format paraphrasing. All three models exhibit substantial answer-flip rates (28.8%-45.1%), with number paraphrasing consistently more disruptive than name swaps. To trace the mechanistic basis of these failures, we introduce the Mechanistic Perturbation Diagnostics (MPD) framework, combining logit lens analysis, activation patching, component ablation, and the Cascading Amplification Index (CAI) into a unified diagnostic pipeline. CAI, a novel metric quantifying layer-wise divergence amplification, outperforms first divergence layer as a failure predictor for two of three architectures (AUC up to 0.679). Logit lens reveals that flipped samples diverge from correct predictions at significantly earlier layers than stable samples. Activation patching reveals a stark architectural divide in failure localizability: Llama-3 failures are recoverable by patching at specific layers (43/60 samples), while Mistral and Qwen failures are broadly distributed (3/60 and 0/60). Based on these diagnostic signals, we propose a mechanistic failure taxonomy (localized, distributed, and entangled) and validate it through targeted repair experiments: steering vectors and layer fine-tuning recover 12.2% of localized failures (Llama-3) but only 7.2% of entangled (Qwen) and 5.2% of distributed (Mistral) failures.

Figures & tables

Work	Behavioral Eval	Mechanistic Diagnosis	Failure Taxonomy	Repair Validation
Stolfo et al. (2023)	Yes	No	No	No
Shi et al. (2023)	Yes	No	No	No
Zhou et al. (2023)	Yes	No	No	No
Levy et al. (2024)	Yes	No	No	No
Wu et al. (2024)	Yes	No	No	No
Ours	Yes	Yes	Yes	Yes

Table 1: Comparison with prior work on LLM reasoning robustness. Prior studies mainly evaluate behavioral sensitivity under perturbations, whereas our work adds mechanistic diagnosis, failure taxonomy, and repair validation.

Figure 1: Overview of MPD. We construct meaning-preserving perturbation pairs from GSM8K, evaluate paired robustness across models, and diagnose failures with logit lens, activation patching, component ablation, and the Cascading Amplification Index (CAI).

Figure 2: Logit lens applied to a flipped sample and a stable sample from Mistral-7B ( Jiang et al., 2023 ) . JSD rises sharply for the flip case but remains near zero for the stable case.

Model	Overall	Name	Number
Mistral-7B	45.1%	27.3%	63.1%
Llama-3-8B	33.5%	23.5%	43.8%
Qwen2.5-7B	28.8%	18.2%	39.6%

Table 2: Flip rates by model and perturbation type.

Model	Flip	No-Flip	$p$	$r$
Mistral-7B	9.1	11.6	2.07e-04	0.157
Llama-3-8B	13.0	14.7	2.14e-02	0.451
Qwen2.5-7B	17.2	19.5	9.23e-06	0.395

Table 3: Mean first divergence layer for flipped vs. non-flipped samples. All comparisons significant at

p<0.05

Figure 3: Distribution of first divergence layers for flipped vs. non-flipped samples.

Model	Recovery	Rate
Mistral-7B	3/60	5.0%
Llama-3-8B	43/60	71.7%
Qwen2.5-7B	0/60	0.0%

Table 4: Activation patching recovery rates.

Figure 4: Activation patching recovery rate by layer.

Model	Attn Recov.	MLP Recov.	Attn Mean	MLP Mean	Attn First	MLP First
Mistral-7B	27/60 (45%)	21/60 (35%)	1.13	0.78	11.8	14.9
Llama-3-8B	45/60 (75%)	46/60 (77%)	5.78	4.95	6.4	8.0
Qwen2.5-7B	24/60 (40%)	29/60 (48%)	1.75	2.15	5.2	3.9

Table 5: Component ablation results. Recov. = samples with at least one recovering layer. Mean = mean recovering layers per sample. First = mean first recovery layer.

Figure 5: Per-layer recovery under component ablation. Mistral and Llama-3 show attention dominance, while Qwen shows MLP dominance.

Figure 6: Mean layer-wise JSD trajectory for flipped vs. stable samples. Flipped samples exhibit cascading amplification; stable samples show attenuation.

Model	Flip	Stable	$p$	$r$
Mistral-7B	0.616	0.545	2.86e-15	0.357
Llama-3-8B	0.548	0.511	7.39e-05	0.186
Qwen2.5-7B	0.500	0.472	1.06e-02	0.121

Table 6: CAI for flipped vs. stable samples.

Property	Mistral	Llama-3	Qwen
Flip rate	45.1%	33.5%	28.8%
Patch recovery	5.0%	71.7%	0.0%
Dominant comp.	Attn	Attn	MLP
First diverge	9.1	13.0	17.2
Failure type	Distributed	Localized	Entangled

Table 7: Architecture-specific failure profiles.

Model	Strategy	Recovery	Rate
Llama-3	Layer FT (L3–5)	25/222	11.3%
Llama-3	Steering ( $\alpha$ =0.5)	27/222	12.2%
Qwen	Layer FT (L0,2,5)	14/195	7.2%
	Steering ( $\alpha$ =1.0)	14/195	7.2%
	ROME MLP edit	$-$ 2/180	$-$ 1.1%
Mistral	Layer FT (L12,14,27)	15/305	4.9%

Table 8: Repair results. Recovery = previously-flipped samples now correct. Negative recovery indicates regression.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Pert. Type	FT	Steering
Llama-3	Name swap	18.2%	13.0%
Llama-3	Num. para.	7.6%	11.7%
Qwen	Name swap	8.1%	8.1%
Qwen	Num. para.	6.8%	6.8%
Mistral	Name swap	6.5%	6.5%
Mistral	Num. para.	4.2%	4.7%

Appendix

Table 9: Repair recovery rate by perturbation type.

Model	Pert. Type	Attn	MLP
Mistral-7B	Name swap	0.82	0.41
Mistral-7B	Num. para.	1.26	0.93
Llama-3-8B	Name swap	7.00	6.17
Llama-3-8B	Num. para.	5.03	4.19
Qwen2.5-7B	Name swap	0.93	2.33
Qwen2.5-7B	Num. para.	2.02	2.09

Appendix

Table 10: Mean recovering layers per sample by perturbation type and component.

Language Model Robustness

Language model robustness studies how reliably language models preserve correct, safe behavior under input changes, distribution shifts, and adversarial pressure.

See all 493 papers

Uncovering Why LLMs Fail: From Internal Mechanisms to Adversarial Reasoning Trajectories
Oct 27, 2026Zirui Ding, Xinxin Fan, Hao Yuan +3LLM Auditing Mechanistic Interpretability
Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · University of the Chinese Academy of Sciences +3
Robust Reasoning Benchmark
Mar 26, 2026Pavel Golikov, Evgenii Opryshko, Gennady Pekhimenko +1Language Model Robustness LLM Evaluation
University of Toronto · Vector Institute
Lexical Perturbations Disrupt LLM Reasoning: An Empirical Study of Attention Diversion
Aug 22, 2026Jiaqian Zhu, Yang Zhang, Junhua Ding +1Language Model Robustness Causal Interventions in Language Models
Missouri University of Science and Technology, Rolla, MO, USA · University of North Texas, Denton, TX, USA

Oct 27, 2026J/K move · Enter open · S save

Uncovering Why LLMs Fail: From Internal Mechanisms to Adversarial Reasoning Trajectories

Zirui Ding, Xinxin Fan, Hao Yuan, Shaoye Luo+2

Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · University of the Chinese Academy of Sciences+3

LLM Auditing Mechanistic Interpretability Adversarial Robustness of Language Models

Open paper AskPDF arXiv

Large language models (LLMs) are usually evaluated by final-output correctness or safety, yet an endpoint label does not reveal how a failure develops throughout the reasoning process. We therefore ask whether ordinary reasoning failures and adversarially induced failures can be characterized using the same internal coordinates. Towards this end, we in this paper study a white-box diagnostic framework CauTrace to disclose the root-cause of why LLM fails from three fundamental inspections: Functional States, Propagation of Representation, Temporal-State Coordination and Reasoning Trajectory. First, seven functional states are creatively defined to represent the support and constraint processing in LLMs. Second, the propagation of representation change measures how the changes among the seven states persist across subsequent Transformer blocks. Third, temporal-state coordination and trajectory topology characterize when the commitment and verification occur relative to the support, and how the dominant state sequence evolves over the full inference process. Finally, we evaluate CauTrace on both clean reasoning failures and paired adversarial perturbations. Across Qwen and LLaMA-family models, the failed traces show weaker Attention-MLP agreement on support-state updates and less verification activity relative to commitment, while the exact representation-level signatures and timing shifts vary by architecture. Held-out repair and steering tests demonstrate consistent operational signals that were in line with our claims for certain models and specific settings, with aligned steering producing the predicted directional changes in process measures. Overall, CauTrace provides a unified diagnostic framework for analyzing failure dynamics through changes in support, commitment, and verification, which would shed light on the in-depth research on LLM security.

Abstract

Language Model Robustness

Explore similar work

Uncovering Why LLMs Fail: From Internal Mechanisms to Adversarial Reasoning Trajectories

Robust Reasoning Benchmark

Lexical Perturbations Disrupt LLM Reasoning: An Empirical Study of Attention Diversion