Uncovering Why LLMs Fail: From Internal Mechanisms to Adversarial Reasoning Trajectories

Zirui Ding; Xinxin Fan; Hao Yuan; Shaoye Luo; Men Niu; Yunfeng Lu

cs.AIOct 27, 2026

Uncovering Why LLMs Fail: From Internal Mechanisms to Adversarial Reasoning Trajectories

Authors: Zirui Ding, Xinxin Fan, Hao Yuan, Shaoye Luo, Men Niu, Yunfeng Lu

Organizations: Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · University of the Chinese Academy of Sciences · State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences ·

Abstract

Large language models (LLMs) are usually evaluated by final-output correctness or safety, yet an endpoint label does not reveal how a failure develops throughout the reasoning process. We therefore ask whether ordinary reasoning failures and adversarially induced failures can be characterized using the same internal coordinates. Towards this end, we in this paper study a white-box diagnostic framework CauTrace to disclose the root-cause of why LLM fails from three fundamental inspections: Functional States, Propagation of Representation, Temporal-State Coordination and Reasoning Trajectory. First, seven functional states are creatively defined to represent the support and constraint processing in LLMs. Second, the propagation of representation change measures how the changes among the seven states persist across subsequent Transformer blocks. Third, temporal-state coordination and trajectory topology characterize when the commitment and verification occur relative to the support, and how the dominant state sequence evolves over the full inference process. Finally, we evaluate CauTrace on both clean reasoning failures and paired adversarial perturbations. Across Qwen and LLaMA-family models, the failed traces show weaker Attention-MLP agreement on support-state updates and less verification activity relative to commitment, while the exact representation-level signatures and timing shifts vary by architecture. Held-out repair and steering tests demonstrate consistent operational signals that were in line with our claims for certain models and specific settings, with aligned steering producing the predicted directional changes in process measures. Overall, CauTrace provides a unified diagnostic framework for analyzing failure dynamics through changes in support, commitment, and verification, which would shed light on the in-depth research on LLM security.

Figures & tables

Figure 1: CauTrace overview. CauTrace maps clean/adversarial traces to seven functional-state coordinates and analyzes (1) Propagation of Representation across Transformer blocks; (2) Temporal-state Coordination among support, commitment, and verification; and (3) Reasoning Trajectory over the full generation trace.

Seven States	Functional Roles	Questions
Premise Binding ( $\mathit{PB}$ )	Bind entities, conditions, and prompt constraints	Que.I
Knowledge Retrieval ( $\mathit{KR}$ )	Activate relevant factual or commonsense knowledge	Que. I
Reasoning Composition ( $\mathit{RC}$ )	Compose evidence across hops or intermediate claims	Que. I
Operational Computation ( $\mathit{OC}$ )	Execute numerical or symbolic operations	Que. I
Formal Rule Tracking ( $\mathit{FRT}$ )	Maintain task rules, labels, and output constraints	Que. I
Answer Commitment ( $\mathit{AC}$ )	Select and stabilize a candidate answer	Que. II

Table 1: Seven functional-state coordinates organized by three diagnostic questions.

Model	Label	$\Delta^{attn}_{sup}$	$\Delta^{mlp}_{sup}$	$\bar{A}_{\mathrm{sup}}$
Qwen2.5-14B	Correct	0.076	-0.015	-0.560
Qwen2.5-14B	Full Hall.	0.125	-0.160	-0.600
Qwen2.5-7B	Correct	-0.009	0.009	-0.568
Qwen2.5-7B	Full Hall.	0.019	-0.056	-0.625
Qwen2.5-3B	Correct	0.023	0.029	-0.500
Qwen2.5-3B	Full Hall.	0.041	-0.050	-0.950

Table 2: Representation-level indicators. Lower

\bar{A}_{\mathrm{sup}}

denotes lower Attention and MLP updates to support states.

Panel of Figure 2 — Figure 2: Representation-level diagnostics. Panel (a) reports value-projected Attention spectral concentration $\rho^{V}_{t}$ ; Panel (b) reports step-to-step change in the normalized value spectrum, $D^{V}_{\mathrm{KL}}(t)$ ; Panel (c) reports alignment $\Gamma_{\mathrm{ac},V}(t)$ between the dominant value-space direction and the $\mathit{AC}$ probe direction.

Panel of Figure 3 — Figure 3: Temporal-state coordination diagnostics: Panel (a) compares $R_{\mathrm{proc}}$ and the verification-onset indicator $I_{\mathrm{ver}}$ ; Panel (b) decomposes $R_{\mathrm{proc}}$ into early commitment onset, support delay, oscillation, late verification, and low verification coverage.

Model	$\boldsymbol{g_{cs}}$ : support–commitment gap negative = early commitment; shift = change	$\boldsymbol{g_{vc}}$ : verification–commitment gap large shift / sign flip = disrupted checking	$\boldsymbol{R_{vc}}$ : verification/commitment ratio lower ratio = less relative verification activity
All models	$14.025\rightarrow 0.489$ $\Delta=\textbf{-13.536}$ , $q<0.001$	$-0.274\rightarrow 16.311$ $\Delta=\textbf{16.585}$ , $q<0.001$	$4.172\rightarrow 1.065$ $\Delta=\textbf{-3.107}$ , $q<0.001$
Qwen2.5-1.5B	$14.182\rightarrow-1.432$ $\Delta=\textbf{-15.614}$ , $q=0.034$	not significant after FDR correction	$9.064\rightarrow 1.281$ $\Delta=\textbf{-7.783}$ , $q=0.011$
Qwen2.5-3B	$24.283\rightarrow 3.125$ $\Delta=\textbf{-21.159}$ , $q<0.001$	$4.951\rightarrow 20.616$ $\Delta=\textbf{15.664}$ , $q=0.011$	$6.473\rightarrow 1.181$ $\Delta=\textbf{-5.292}$ , $q<0.001$
Qwen2.5-7B	$3.750\rightarrow-0.857$ $\Delta=\textbf{-4.607}$ , $q=0.042$	not significant after FDR correction	$1.784\rightarrow 0.592$ $\Delta=\textbf{-1.192}$ , $q=0.013$
Qwen2.5-14B	$9.319\rightarrow 0.333$ $\Delta=\textbf{-8.986}$ , $q<0.001$	$-3.940\rightarrow 21.437$ $\Delta=\textbf{25.376}$ , $q<0.001$	$2.474\rightarrow 0.722$ $\Delta=\textbf{-1.752}$ , $q=0.018$

Table 3: Timing comparisons between Correct and Full Hallucination traces. Each cell reports Correct

\rightarrow

Full Hallucination with

\Delta=

Full Hallucination

-

Correct. For

g_{cs}

and

g_{vc}

, the measured quantity is a shift in relative temporal position; negative

g_{cs}

denotes the early-commitment risk direction. Lower

R_{vc}

indicates less verification activity relative to answer commitment.

Panel of Figure 4 — Figure 4: State-transition matrices for correct and failed trajectories: Panels (a) and (b) report transition probabilities for correct and failed generations, respectively; Panel (c) reports the failure-minus-correct difference.

Panel of Figure 5 — Figure 5: Trajectory topology in the trajectory-feature space: Panel (a) visualizes trajectory-level features with approximate type neighborhoods; Panel (b) gives the row-normalized confusion matrix of a lightweight rule-derived trajectory predictor constructed directly from the diagnostic definitions in Eq. ( 20 ).

Panel of Figure 6 — Figure 6: Time-domain state-score trajectories for one normal and five failure-prone trajectory types. The curves show how each trajectory pattern differs in support formation, answer commitment, and self-verification over the generation process.

Figure 7: Representative state-transition chains for the five failure-prone trajectory types.

Dataset	Model	Pairs	BSR	AFR	PFR	Cond.PFR	TAR@1	TAR@3
Reasoning Interfered	Qwen2.5-1.5B	500	23.8%	34.0%	6.2%	26.1%	16.6%	49.8%
	Qwen2.5-3B	500	18.0%	14.0%	1.8%	10.0%	15.2%	41.8%
	Qwen2.5-7B	500	26.6%	21.0%	4.8%	18.0%	16.6%	48.4%
AdvBench/HarmBench	Qwen2.5-1.5B	300	73.3%	37.0%	22.3%	30.5%	–	–
	Qwen2.5-3B	300	69.3%	33.3%	14.7%	21.2%	–	–
	Qwen2.5-7B	300	86.0%	18.0%	12.7%	14.7%	–	–

Table 4: Paired adversarial test results. BSR denotes clean-side success, AFR is adversarial failure rate, PFR is pair flip rate, and conditional PFR measures flips among clean-successful pairs. Top-k Target Accuracy Rate (TAR) is reported for process reasoning perturbations, where each perturbation defines an explicit seven-state target.

Dataset	Model	Dominant State Shift	Strongest Flip-Associated Signal	Strongest Paired QKV Signal
Reasoning Interfered	Qwen2.5-1.5B	Premise Binding $-0.133$	$\mathit{OC}$ cascade ( $d=-3.180$ )	K pairwise cosine $+0.003$ ( $d_{z}=0.352$ )
	Qwen2.5-3B	Reasoning composition $+0.115$	Formal Rule Tracking onset delay ( $d=2.980$ )	Q stable rank $-0.038$ ( $d_{z}=-0.975$ )
	Qwen2.5-7B	Reasoning Composition $-0.058$	$\mathit{OC}$ state ( $d=-3.134$ )	V–O norm correlation $+0.063$ ( $d_{z}=0.315$ )
AdvBench /HarmBench	Qwen2.5-1.5B	Answer commitment $+0.057$	Commitment cascade strength ( $d=0.232$ )	O consecutive cosine $-0.078$ ( $d_{z}=-1.289$ )
	Qwen2.5-3B	Answer commitment $-0.001$	Commitment peak-layer shift ( $d=0.642$ )	Q pairwise-cosine dispersion $+0.006$ ( $d_{z}=0.485$ )
	Qwen2.5-7B	Answer commitment $+0.002$	Commitment peak-layer shift ( $d=0.636$ )	O row-norm dispersion $-0.031$ ( $d_{z}=-0.661$ )

Table 5: Internal changes under paired adversarial perturbations.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Size	Judge	Dataset	Size	Judge	Dataset	Size	Judge
TruthfulQA	120	semantic	HotpotQA	7,400	support	MuSiQue	242	support
GSM8K	880	numeric	MATH	1,250	math	FOLIO	150	label
ProofWriter	600	label	StrategyQA	250	yes/no	CommonsenseQA	1,200	mcq

Appendix

Table 6: Clean benchmark composition used in the main reliability evaluation.

judge_type	Output Form	Constraint
label_match	Label	entailment/contradiction/unknown
yesno_match	Binary answer	yes/no
mcq_match	Option	A/B/C/D/E
numeric_match	Number	Final number only
math_answer_match	Mathematical answer	Final expression or result only
semantic	Short answer	Concise, comparable answer

Appendix

Table 7: Output constraints by judge_type .

Model	w1	w2	w4	w8	w16	w32	w64	fused
Qwen2.5-1.5B	0.9006	0.9253	0.9501	0.9715	0.9843	0.9901	0.9918	0.9950
Qwen2.5-3B	0.9038	0.9311	0.9570	0.9793	0.9902	0.9939	0.9952	0.9972
Qwen2.5-7B	0.9557	0.9752	0.9880	0.9950	0.9973	0.9977	0.9979	0.9957
Qwen2.5-14B	0.9794	0.9916	0.9974	0.9985	0.9992	0.9990	0.9989	0.9979

Appendix

Table 8: Probe validation AUC across pooling-window scales.

Parameter	Setting
Epochs	40
Batch size	4096
Learning rate	$1\times 10^{-2}$
Weight decay	$1\times 10^{-4}$
Patience	6
Train/Val	0.9 / 0.1

Appendix

Table 9: Multi-scale probe training hyperparameters.

Dimension	Concise	Analysis	Potential Failure Mode
Output length	Short	2–4 steps	May induce Verbose Unsupported Continuation outputs
Observability	Low	High	Rationale text is not always faithful verification
Commitment risk	Medium	Potentially higher	Early Commitment or Verification without Overwrite
Parsing stability	High	Medium	Invalid parsing if final-line format is violated

Appendix

Table 10: Differences between concise and analysis tracks.

Target State	Process Role	Perturbation Focus
Premise Binding	Bind entities, roles, context, and constraints.	False premises, role confusion, conflicting context bindings.
Knowledge Retrieval	Retrieve relevant facts or supporting evidence.	Plausible but wrong facts, source bias, retrieval decoys.
Reasoning Composition	Organize multi-step inference and intermediate claims.	Shortcut reasoning, chain reordering, incompatible intermediate claims.
Operational Computation	Execute numerical or symbolic operations.	Wrong scratch work, expression substitution, arithmetic interference.
Formal Rule Tracking	Maintain formal rules, safety policies, and task boundaries.	Rule-priority inversion, forged exceptions, constraint override.
Answer Commitment	Stabilize an answer after sufficient support.	Premature conclusion, forced single answer, uncertainty compression.

Appendix

Table 11: CauTrace state-targeted adversarial perturbations.

Score Variant	Perturbation	Rank Stability	Separation Delta	Decision Stability
$R_{\mathrm{str}}$ default	none; equal weights	1.000	+0.014	1.000
$R_{\mathrm{str}}$ ablation	support/value components leave-one-out	1.000 [1.000, 1.000]	+0.014 [+0.000, +0.018]	0.812 [0.250, 1.000]
$R_{\mathrm{str}}$ random weights	Dirichlet random weights	1.000 [1.000, 1.000]	+0.014 [+0.000, +0.050]	1.000 [1.000, 1.000]
$R_{\mathrm{proc}}$ default	none; equal weights	1.000	-0.022	1.000
$R_{\mathrm{proc}}$ ablation	process components leave-one-out	0.790 [0.633, 0.887]	-0.022 [-0.046, +0.006]	0.827 [0.767, 0.933]
$R_{\mathrm{proc}}$ random weights	Dirichlet random weights	0.696 [0.022, 0.987]	-0.022 [-0.107, +0.061]	0.811 [0.567, 1.000]

Appendix

Table 12: Aggregation sensitivity and component-ablation analysis for

R_{\mathrm{str}}

and

R_{\mathrm{proc}}

. Rank stability denotes Spearman correlation with the default equal-weight score. Separation delta denotes the failure-minus-correct contrast under the perturbed score. Decision stability denotes preservation of the default threshold decision.

Model	$\Delta\bar{A}_{\mathrm{sup}}$	$\Delta\rho_{V}$	$\Delta D^{V}_{KL}$	$\Delta\Gamma_{\mathrm{ac},V}$	$\Delta R_{vc}$
LLaMA-3.2-1B	-0.052	-0.000	+0.001	-0.058	-13.085
LLaMA-3.2-3B	-0.022	+0.075	+0.002	+0.050	-13.558
LLaMA-3.1-8B	-0.042	+0.073	+0.007	-0.018	-15.519
Qwen1.5-MoE	-0.012	+0.000	-0.000	+0.057	-1.102

Appendix

Table 13: Cross-family validation of representation-level and temporal diagnostics.

Model	Track	Correct	Partial	Full	Abstain	Invalid	Len.
1.5B	Concise	19.7	0.5	54.4	24.7	0.7	16.0
1.5B	Analysis	32.9	1.2	43.7	21.9	0.2	76.5
3B	Concise	12.1	0.3	20.1	65.1	2.3	25.5
3B	Analysis	32.2	1.2	26.2	40.1	0.3	67.5
7B	Concise	15.8	0.4	12.7	71.1	0.0	6.0
7B	Analysis	34.4	1.3	20.3	44.0	0.0	41.3

Appendix

Table 14: Outcome distributions across Qwen2.5 models and prompt tracks. Len. denotes mean generation length.

Task group	Size	Correct	Failure	Abstain	Invalid
Calculation	7032	0.445	0.380	0.090	0.085
Formal math	10000	0.240	0.718	0.042	0.000
Multi-hop QA	61176	0.137	0.185	0.678	0.000
Knowledge QA	656	0.110	0.494	0.396	0.000
Logic/rule	5952	0.252	0.177	0.486	0.086
Commonsense MCQ	9680	0.721	0.263	0.013	0.003

Appendix

Table 15: Task-group outcome composition, aggregated across Qwen2.5 scales and prompt tracks.

Figure 8: Task-group outcome distributions for Qwen2.5 generations. The horizontal bars visualize the proportions reported in Table 15 ; they are not independent estimates.

Trajectory	Metric Signature	Diagnostic Interpretation
Early Commitment	$\tau_{\mathrm{com}}<\tau_{\mathrm{sup}}$ or $g_{\mathrm{cs}}\downarrow$	Answer commitment becomes dominant before support is sufficiently established.
Fragmented Support	$\Omega\uparrow$ , $\bar{S}\downarrow$ , unstable $\tau_{\mathrm{sup}}$	Support-related states activate but do not form a stable, continuous chain.
Rule Failure	$L_{\mathrm{rule}}\uparrow$ with weak $\mathit{FRT}$ / $\mathit{OC}$ state scores	The trace approaches an answer without preserving formal rules, output constraints, or symbolic/numerical consistency.
Verification without Overwrite	$g_{\mathrm{vc}}\uparrow$ , $R_{\mathrm{vc}}\downarrow$	Verification follows commitment but remains too weak to revise the stabilized answer.
Verbose Unsupported Continuation	$T\uparrow$ with $\bar{S}\downarrow$ and $R_{\mathrm{vc}}\downarrow$	Generation continues without a corresponding increase in support or effective verification.
Normal Convergence	$\tau_{\mathrm{sup}}\leq\tau_{\mathrm{com}}$ , bounded $\Omega$ , $R_{\mathrm{vc}}\uparrow$	Support precedes commitment, constraints remain active, and verification can confirm or revise the candidate.

Appendix

Table 16: Operational signatures of the Trajectory Topology regions. Arrows indicate the expected direction relative to Normal Convergence. A sample may satisfy more than one signature.

Condition	Adv. fail	Clean good	$\Delta$ Fail
Baseline	76.0	26.0	–
Unified CauTrace	71.0	31.0	-5.0
Generic review	72.0	32.0	-4.0
Topology-conditioned	73.0	29.5	-3.0
Wrong topology	73.5	30.5	-2.5

Appendix

Table 17: Held-out Qwen2.5-1.5B reasoning repair on 200 test pairs. Endpoint rates are percentages. Clean good denotes the proportion of clean responses that remain correct, and

\Delta

Fail is the adversarial-failure change relative to the baseline in percentage points.

Condition	Adv. fail	Clean	$\Delta R_{\mathrm{proc}}$	$\Delta\mathrm{Rule}_{\mathrm{cov}}$	$\Delta V_{\mathrm{cov}}$
Baseline	100.0	100.0	0	0	0
Aligned	66.7	91.7	-0.008	+0.011	+0.021
Random	66.7	100.0	0.000	+0.002	0.000
Reversed	66.7	91.7	+0.015	-0.014	-0.019
Wrong state	66.7	91.7	+0.010	-0.007	-0.026

Appendix

Table 18: Held-out Qwen2.5-7B safety steering on 12 test pairs. Process deltas pool the clean and adversarial member of each pair; lower

R_{\mathrm{proc}}

and higher rule and verification coverage are better.

LLM Auditing

Large language model auditing tests models and deployed systems for bias, safety, privacy, reliability, and provenance using behavioral and internal evidence.

See all 639 papers

Aug 4, 2026

The Tell-Tale Trace: Detecting Reasoning Failures in LLMs Using Chain-of-Thought Dynamics

Chain-of-thought (CoT) reasoning improves large language model (LLM) performance while also providing an observable interface to the model's reasoning process. Existing approaches that leverage verbalized CoTs to monitor reasoning correctness, however, largely evaluate the semantic correctness or consistency of individual intermediate steps, rather than how the reasoning process evolves across the trace. As a result, failures distributed across the reasoning trajectory, rather than those localized to a single incorrect step, remain comparatively underexplored. Furthermore, verbalized CoTs need not faithfully reflect the model's internal reasoning, motivating analyses that do not treat individual statements as literal accounts of internal computation. In this work, we therefore ask whether the dynamics of visible CoT can be leveraged to systematically distinguish successful from failed reasoning without assuming such semantic faithfulness. We study a range of LLMs on verifiable Boolean satisfiability tasks with variable complexity, enabling controlled comparisons near each model's capability frontier. Tagging CoT sentences by reasoning function reveals premature verification collapse on SAT problems: incorrect traces enter clause checking earlier, repeat similar operations, and finalize sooner. On UNSAT problems, models presumptuously move towards incorrect SAT conclusions, checking candidate assignments rather than deriving contradictions across constructed cases. Subsequently, a targeted proof-search prompt intervention raises Llama3-70B accuracy from 13.3% to 85%, correcting 84.6% of these errors. These results show that capability failures can manifest as distributed, task-dependent changes in the structure of visible reasoning, and that CoT dynamics agnostic to whether the verbalized trace reflects the model's internal computations can help diagnose and correct failures.

Shashwat Sourav, Aishwarya Balwani

Department of Physics, Washington University in St. Louis · Department of Developmental Neurobiology, St. Jude Children’s Research Hospital

LLM Reliability Boolean Satisfiability Reasoning Trace Analysis

Ask PDF

Apr 16, 2026

Dissecting Failure Dynamics in Large Language Model Reasoning

Large Language Models (LLMs) achieve strong performance through extended inference-time deliberation, yet how their reasoning failures arise remains poorly understood. By analyzing model-generated reasoning trajectories, we find that errors are not uniformly distributed but often originate from a small number of early transition points, after which reasoning remains locally coherent but globally incorrect. These transitions coincide with localized spikes in token-level entropy, and alternative continuations from the same intermediate state can still lead to correct solutions. Based on these observations, we introduce GUARD, a targeted inference-time framework that probes and redirects critical transitions using uncertainty signals. Empirical evaluations across multiple benchmarks confirm that interventions guided by these failure dynamics lead to more reliable reasoning outcomes. Our findings highlight the importance of understanding when and how reasoning first deviates, complementing existing approaches that focus on scaling inference-time computation.

Wei Zhu, Jian Zhang, Lixing Yu +2

School of Information Science and Engineering, Yunnan University, Kunming, China · Yunnan Key Laboratory of Intelligent Systems and Computing, Kunming, China

Inference-Time Guidance LLM Reasoning

Ask PDF

Apr 2, 2026

Fragile Reasoning: A Mechanistic Analysis of LLM Sensitivity to Meaning-Preserving Perturbations

Large language models demonstrate strong performance on mathematical reasoning benchmarks, yet remain surprisingly fragile to meaning-preserving surface perturbations. We systematically evaluate three open-weight LLMs, Mistral-7B, Llama-3-8B, and Qwen2.5-7B, on 677 GSM8K problems paired with semantically equivalent variants generated through name substitution and number format paraphrasing. All three models exhibit substantial answer-flip rates (28.8%-45.1%), with number paraphrasing consistently more disruptive than name swaps. To trace the mechanistic basis of these failures, we introduce the Mechanistic Perturbation Diagnostics (MPD) framework, combining logit lens analysis, activation patching, component ablation, and the Cascading Amplification Index (CAI) into a unified diagnostic pipeline. CAI, a novel metric quantifying layer-wise divergence amplification, outperforms first divergence layer as a failure predictor for two of three architectures (AUC up to 0.679). Logit lens reveals that flipped samples diverge from correct predictions at significantly earlier layers than stable samples. Activation patching reveals a stark architectural divide in failure localizability: Llama-3 failures are recoverable by patching at specific layers (43/60 samples), while Mistral and Qwen failures are broadly distributed (3/60 and 0/60). Based on these diagnostic signals, we propose a mechanistic failure taxonomy (localized, distributed, and entangled) and validate it through targeted repair experiments: steering vectors and layer fine-tuning recover 12.2% of localized failures (Llama-3) but only 7.2% of entangled (Qwen) and 5.2% of distributed (Mistral) failures.

Shou-Tzu Han, Rodrigue Rizk, KC Santosh

Department of Computer Science, University of South Dakota

Language Model Robustness LLM Reasoning Adversarial Robustness of Language Models LLM Interpretability Causal Interventions in Language Models

Ask PDF

Abstract

LLM Auditing

Explore similar work

The Tell-Tale Trace: Detecting Reasoning Failures in LLMs Using Chain-of-Thought Dynamics

Dissecting Failure Dynamics in Large Language Model Reasoning

Fragile Reasoning: A Mechanistic Analysis of LLM Sensitivity to Meaning-Preserving Perturbations