Directional Reasoning Trajectory Change (DRTC): Identifying Critical Trace Segments in Reasoning Models

Waldemar Chang

cs.LGFeb 17, 2026

Directional Reasoning Trajectory Change (DRTC): Identifying Critical Trace Segments in Reasoning Models

Authors: Waldemar Chang

Organizations: Johns Hopkins University

Abstract

Understanding how language models carry out long-horizon reasoning remains an open challenge. Existing interpretability methods often highlight tokens correlated with an answer, but rarely reveal where consequential reasoning turns occur, which earlier context triggers them under causal intervention, or whether highlighted text actually steers the rollout. We introduce Directional Reasoning Trajectory Change (DRTC), a process-causal method that (i) detects pivot decision points via uncertainty and distribution-shift signals and (ii) applies receiver-side interventions that preserve the realized continuation without resampling while blocking information flow from selected earlier chunks only at a pivot. DRTC measures how each intervention redirects the log-probability trajectory relative to the realized rollout direction, yielding signed per-chunk attributions; we also compute logit-space curvature changes and curvature signatures as a complementary geometric diagnostic. Across four reasoning models, influence is sharply concentrated (Gini approximately 0.50-0.58, top-5% mass approximately 0.23-0.28), and learned pivots induce stronger effects than matched random spans. In a 500-problem MATH scaling study with R1-Distill-Qwen-1.5B, learned spans continue to outperform matched random spans (median Delta=0.409, 355/500 positive; p=2.3e-21), and curvature-impact co-localizes with DRTC within traces as a diagnostic. We benchmark against gradient- and perturbation-based chunk attributions and show graded outcome linkage: under embedding-interpolation edits, top-ranked DRTC chunks reduce teacher-forced gold-answer log-probability more than strict position-matched random chunks on a stability-filtered subset. Overall, DRTC provides a causally grounded view of how specific context elements steer on-policy reasoning trajectories.

Figures & tables

Figure 1: DRTC pipeline overview. Curvature is diagnostic only and is not used to define pivots or scores.

Figure 2: Curvature invariance under diagnostic logging (representative model: R1-Distill-Qwen-1.5B ). Per-chunk DRTC scores from C0 and C8 lie on the identity line ( $\rho=1.000$ ), confirming curvature computation is purely diagnostic.

Figure 3: Cross-model comparison of median per-example mean pivot-local intervention magnitude. Learned pivots (C8) induce stronger interventions than matched random spans (C9) across all four models. Error bars denote 95% bootstrap confidence intervals.

Appendix figures & tables50 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 4 — Figure 4: C0 vs. C8 invariance across models. Each panel shows per-chunk DRTC scores under C0 (baseline) versus C8 (curvature-enabled). Points lie exactly on the identity line, confirming that curvature computation is strictly diagnostic and does not alter attribution.

Figure 5: Direction-estimation robustness and meander diagnostics (A2).

Comparison	Median $\rho$	Min $\rho$	Notes
PCA vs endpoints	1.00	1.00	24/24 identical
Robust PCA vs endpoints	1.00	1.00	24/24 identical
Piecewise vs endpoints	$\approx$ 1.00	0.50	short-trace edge case
Piecewise-signed vs endpoints	$\approx$ 1.00	0.50	same edge case

Appendix

Table 1: Ranking invariance across

g

estimators on the 24-example slice (model: R1-Distill-Qwen-1.5B).

Model	Gini (C0) [95% CI]
LFM2.5-1.2B	0.499 [0.479, 0.557]
Ministral-3-3B	0.574 [0.519, 0.625]
Phi-4-Mini	0.539 [0.491, 0.549]
R1-Distill-Qwen-1.5B	0.584 [0.556, 0.670]

Appendix

Table 2: Per-example concentration of

|\mathrm{DRTC}(i)|

summarized by median Gini coefficient (C0)

Model	Top-1 mass [95% CI]	Top-5% mass [95% CI]	Mean $\|\mathrm{DRTC}\|$
LFM2.5-1.2B	0.076 [0.066, 0.092]	0.229 [0.217, 0.258]	0.492
Ministral-3-3B	0.086 [0.076, 0.097]	0.262 [0.244, 0.286]	0.455
Phi-4-Mini	0.071 [0.062, 0.084]	0.241 [0.217, 0.254]	0.493
R1-Distill-Qwen-1.5B	0.096 [0.080, 0.118]	0.284 [0.264, 0.336]	0.494

Appendix

Table 3: Top-

k

mass fractions for per-example normalized

|\mathrm{DRTC}|

Panel of Figure 7 — Figure 7: Per-example DRTC attribution concentration across models. Each panel shows the fraction of total per-example $|\mathrm{DRTC}|$ mass captured by the top-ranked chunk(s) within an example. Across architectures, a small number of chunks accounts for a disproportionate share of directional influence, consistent with moderate but robust sparsity (see Tabs. 2 – 3 ).

Model	C8 median $\mathrm{std}_{d}$ [95% CI]	C9 median $\mathrm{std}_{d}$ [95% CI]	Gap
LFM2.5-1.2B	1.086 [1.007, 1.290]	0.907 [0.852, 0.976]	+0.178
Ministral-3-3B	1.065 [0.982, 1.119]	1.026 [0.957, 1.179]	+0.039
Phi-4-Mini	1.089 [0.971, 1.162]	0.926 [0.862, 1.012]	+0.163
R1-Distill-Qwen-1.5B	1.290 [1.193, 1.367]	1.208 [1.131, 1.254]	+0.081

Appendix

Table 4: Primary falsification metric: per-example mean absolute pivot-local intervention magnitude (

\mathrm{std}_{d}

)

Model	C8 median $\rho$ [95% CI]	C9 median $\rho$ [95% CI]	$\Delta$
LFM2.5-1.2B	0.863 [0.829, 0.910]	0.802 [0.777, 0.842]	+0.061
Ministral-3-3B	0.832 [0.818, 0.889]	0.924 [0.896, 0.949]	-0.092
Phi-4-Mini	0.877 [0.866, 0.900]	0.918 [0.867, 0.933]	-0.041
R1-Distill-Qwen-1.5B	0.892 [0.854, 0.943]	0.869 [0.805, 0.896]	+0.023

Appendix

Table 5: Secondary falsification diagnostic:

\delta

-vs-full Spearman rank agreement

Figure 8: Cross-model falsification summary. Top : primary falsification (median per-example intervention magnitude; std_d), showing C8

>

C9 for all four models. Bottom :

\delta

-vs-full Spearman rank-stability diagnostic; two models show C9

>

C8, which we interpret as rank-preservation effects rather than stronger causal interventions. Error bars denote 95% bootstrap confidence intervals.

Metric	Mean	Median	Min	Max
Wall time per example (s)	11.8	12.28	3.72	17.03
Peak GPU memory (MB)	4835	4864	4164	5333

Appendix

Table 6: Compute cost summary from main_text_table_cost.csv .

Masking variant	$\rho$ vs all-heads (range)	Notes
Head subsets (fixed layers)	0.22 – 1.00	agreement can degrade sharply

Appendix

Table 7: Head-subset masking can substantially alter rankings, motivating the default all-head intervention.

Figure 9: Representative ranking agreement for head-subset masking variants.

Knob	Values	Median $\rho$ range
window_size	3, 5, 8	0.93 – 1.00
min_pivot_spacing	2, 4, 8	0.90 – 1.00
(others)	(see CSV)	$\geq 0.96$ typical

Appendix

Table 8: Sensitivity grid summary (full table in the exported CSV).

Figure 10: Sensitivity heatmap over knob sweeps.

Method	Median $\rho$ vs DRTC	Top-5 overlap	Top-10 overlap
Grad $\times$ Input	0.53	0.67	0.82
Smooth masking	0.45	0.43	0.82
Chunk occlusion	0.30	0.43	0.67
Activation patching	0.04	0.43	0.67

Appendix

Table 9: Baseline agreement summary (R1-Distill-Qwen-1.5B, 24-example slice).

Figure 11: Distribution of per-example Spearman agreement between DRTC and baseline methods.

Statistic (within-trace, N=423)	Value
Median $\rho(\mathrm{CurvImpact},\mathrm{DRTC})$	0.77
Bootstrap 95% CI	[0.70, 0.80]
Fraction $p<0.05$	48%

Appendix

Table 10: Within-trace curvature-impact co-localization at scale (competition-math scaling run).

Figure 12: Distribution of within-trace Spearman correlations for curvature-impact vs DRTC.

Panel of Figure 13 — Figure 13: Curvature diagnostic across models: relationship between curvature magnitude (CurvImpact) and attribution magnitude (DRTC), shown for both pivot probes (C8) and random-span control (C9). Curvature is used diagnostically to stratify how chunks bend the trajectory under intervention; it is not interpreted as outcome-causal evidence.

Figure 14: Representative curvature diagnostic visualization. Left: Scatter of $\mathrm{DRTC}(i)$ vs. $\mathrm{CurvImpact}(i)$ for a single example under C8 (pivot spans) and C9 (matched random spans). Signed association is near zero, while magnitude association can be positive, consistent with curvature tracking reorientation intensity rather than directional steering. Right: Per-example maxima of $|\mathrm{CurvImpact}|$ under C8 and C9 (same masking logic, random-span replacement in C9 ), shown as an auxiliary diagnostic of geometric response magnitude.

Model	Adjacent mean $\|\mathrm{DRTC}\|$	Distant mean $\|\mathrm{DRTC}\|$	Ratio	p-value
LFM2.5-1.2B	0.2718	0.0971	2.80 $\times$	$1.65\times 10^{-69}$
Ministral-3-3B	0.2822	0.0917	3.08 $\times$	$1.13\times 10^{-95}$
Phi-4-Mini	0.2234	0.1047	2.13 $\times$	$2.78\times 10^{-40}$
R1-Distill-Qwen-1.5B	0.2749	0.0873	3.15 $\times$	$4.68\times 10^{-92}$

Appendix

Table 11: Adjacency analysis based on chunk-index distance

Panel of Figure 15 — Figure 15: Shuffle control across models. Randomly permuting curvature signatures destroys role structure, supporting the claim that curvature similarity is not a trivial artifact of the clustering mechanism.

Model	Real median $\rho$	Null median $\rho$	$p$ -value
LFM2.5-1.2B	0.243	0.003	$1.4\times 10^{-5}$
Ministral-3-3B	0.218	-0.002	$9.7\times 10^{-9}$
Phi-4-Mini	0.112	-0.001	$1.4\times 10^{-2}$
R1-Distill-Qwen-1.5B	0.243	0.000	$1.5\times 10^{-9}$

Appendix

Table 12: Shuffle control for curvature–DRTC correlation. Per-example Spearman correlation

\rho(\mathrm{CurvImpact},|\delta|)

compared against a null obtained by randomly permuting curvature signatures.

Model	Median max( $u_{k}$ )	Median entropy( $u_{k}$ )
LFM2.5-1.2B	0.162	2.067
Ministral-3-3B	0.172	2.053
Phi-4-Mini	0.169	2.080
R1-Distill-Qwen-1.5B	0.148	2.072

Appendix

Table 13: Pivot-weight distribution summary across models

Model	Median Spearman $\rho$ [95% CI]
LFM2.5-1.2B	0.990 [0.966, 0.994]
Ministral-3B	0.975 [0.962, 0.986]
Phi-4-Mini	0.988 [0.981, 0.991]
R1-Distill-Qwen-1.5B	0.988 [0.979, 0.995]

Appendix

Table 14: Pivot-weighting ablation: Spearman rank correlation between learned-weight DRTC and post-hoc uniform aggregation

Panel of Figure 16 — Figure 16: Pivot-weighting ablation diagnostic across models. Points compare DRTC(i) computed with learned pivot weights $u_{k}$ to post-hoc uniform aggregation ( $u_{k}=1/K$ ) using identical per-pivot effects $(\delta_{k,i},w_{k,i})$ . Because $u_{k}$ enters only at final aggregation and does not influence $\delta_{k,i}$ or $w_{k,i}$ , this is mathematically equivalent to rerunning the method with uniform pivot weights. Spearman correlations are high but strictly less than 1, indicating that pivot importance modulates attribution rankings without collapsing them to a single dominant pivot.

Cohort	Regime $(cs,\alpha)$	$N$	Median paired diff	95% CI	Sign frac
Stable subset ( $\|\mathrm{orig\_margin}\|>1$ )	(4, 0.2)	19	0.34	[0.03, 1.19]	0.74
All usable examples	(8, 0.7)	22	0.44	[0.04, 1.53]	0.73

Appendix

Table 15: U1b outcome linkage using teacher-forced

\Delta\log p(\text{gold})

. Paired diff is defined as

\mathbb{E}[\Delta\log p(\text{gold})]_{\text{random}}-\Delta\log p(\text{gold})_{\text{top}}

, so positive values indicate the top chunk is more damaging than matched random.

Figure 17: U1b regime scan over $(cs,\alpha)$ showing discrimination on $\Delta\log p(\text{gold})$ .

Figure 18: Best-regime effect sizes by chunk type (top/low/random) with bootstrap CIs.

Figure 19: Distribution of paired differences for

\Delta\log p(\text{gold})

under strict quartile matching.

Panel of Figure 20 — Figure 20: Case-study visual summary for math_rollout_2312 (LFM2.5-1.2B).

Rank	Chunk idx	DRTC	$\|\mathrm{DRTC}\|$	CurvImpact	Excerpt
1	6	+3.4020	3.4020	0.0080	polynomial has rational coefficients, then any irrational root must come with …
2	0	-1.9895	1.9895	0.0189	<think> Okay, let me try to tackle this problem. So the question is
3	52	+1.4424	1.4424	0.0154	1000 of these quadratic factors. But wait, maybe some of these

Appendix

Table 16: LFM2.5-1.2B, math_rollout_2312 : top contributing chunks by DRTC. Positive values align with the realized rollout direction; negative values oppose it. CurvImpact is a geometric diagnostic.

Panel of Figure 21 — Figure 21: Case-study visual summary for math_rollout_2757 (LFM2.5-1.2B).

Rank	Chunk idx	DRTC	$\|\mathrm{DRTC}\|$	CurvImpact	Excerpt
1	4	+3.0206	3.0206	0.0813	)(z - cs)(z - ct). We need to find the number
2	0	+2.2185	2.2185	0.0106	<think> Okay, let me try to tackle this problem. So the question says
3	1	+0.7457	0.7457	0.0141	Let c be a complex number. Suppose there exist distinct complex numbers r,
4	5	+0.4803	0.4803	0.1017	of distinct possible values of c, and then give the final answer starting with ’
5	3	-0.4079	0.4079	-0.0106	product (z - r)(z - s)(z - t) equals (
6	2	-0.0450	0.0450	-0.0060	numbers r, s, and t such that for every complex number z, the

Appendix

Table 17: LFM2.5-1.2B, math_rollout_2757 : top contributing chunks by DRTC. Positive values align with the realized rollout direction; negative values oppose it. CurvImpact is a geometric diagnostic.

Panel of Figure 22 — Figure 22: Case-study visual summary for math_rollout_3295 (Ministral-3-3B). Left: pivot importance weights $u_{k}$ across the $K$ detected pivots. Right: top-ranked chunks by signed DRTC (process-causal directional steering), shown for qualitative auditing.

Rank	Chunk idx	DRTC	$\|\mathrm{DRTC}\|$	CurvImpact	Excerpt
1	9	-3.5838	3.5838	-0.0238	at some point, maybe x=3, but maybe it’s better
2	0	-2.6963	2.6963	0.0026	Okay, let’s try to solve this problem. We have a function f
3	10	-2.1506	2.1506	0.0609	to compute the general form of f_n(x) first. Let
4	2	-1.8071	1.8071	0.0018	find f_1993(3), where f
5	37	-1.3859	1.3859	-0.0013	{1 + x - 1/1 + 3x}{
6	6	+1.2177	1.2177	-0.0156	maybe I should compute the first few iterates to see if there’s a

Appendix

Table 18: Ministral-3-3B, math_rollout_3295 : top contributing chunks by DRTC. Positive values align with the realized rollout direction; negative values oppose it. CurvImpact is a geometric diagnostic.

Panel of Figure 23 — Figure 23: Case-study visual summary for math_rollout_3374 (Ministral-3-3B). Left: pivot importance weights $u_{k}$ across the $K$ detected pivots. Right: top-ranked chunks by signed DRTC (process-causal directional steering), shown for qualitative auditing.

Rank	Chunk idx	DRTC	$\|\mathrm{DRTC}\|$	CurvImpact	Excerpt
1	0	+3.6400	3.6400	-0.0593	Okay, let’s see. The problem is asking how many base-2 digits
2	6	+2.2631	2.2631	-0.0151	bits. First, let’s confirm that. Yes, in base-16
3	5	+2.0559	2.0559	0.0039	$16=2^{4}$ . So, each hex digit is 4
4	8	+2.0423	2.0423	-0.0045	to convert $66666_{16}$ to base-2
5	3	+1.9297	1.9297	-0.0538	Hmm, first, I need to remember how to convert from base-1
6	13	+1.6693	1.6693	0.0007	1010 to 1111. So, 6

Appendix

Table 19: Ministral-3-3B, math_rollout_3374 : top contributing chunks by DRTC. Positive values align with the realized rollout direction; negative values oppose it. CurvImpact is a geometric diagnostic.

Panel of Figure 24 — Figure 24: Case-study visual summary for math_rollout_2757 (Phi-4-Mini). Left: pivot importance weights $u_{k}$ across the $K$ detected pivots. Right: top-ranked chunks by signed DRTC (process-causal directional steering), shown for qualitative auditing.

Rank	Chunk idx	DRTC	$\|\mathrm{DRTC}\|$	CurvImpact	Excerpt
1	6	-3.4363	3.4363	-0.0218	down step by step. First, both sides of the equation are cubic polynomials
2	10	-3.0111	3.0111	0.1074	and set the coefficients equal, I can find conditions on c, r, s
3	16	-2.3690	2.3690	-0.0138	+ (cs r + ctr + crt)z - crt 3 . Wait,
4	19	-1.9486	1.9486	0.0502	st) with c factored in, so (cr * cs + cr *
5	3	-1.6787	1.6787	0.0036	)(z - s)(z - t) is equal to (z - cr
6	17	-1.6302	1.6302	0.0105	actually, let me check that again. The coefficient for the cubic term is

Appendix

Table 20: Phi-4-Mini, math_rollout_2757 : top contributing chunks by DRTC. Positive values align with the realized rollout direction; negative values oppose it. CurvImpact is a geometric diagnostic.

Panel of Figure 25 — Figure 25: Case-study visual summary for math_rollout_3759 (Phi-4-Mini). Left: pivot importance weights $u_{k}$ across the $K$ detected pivots. Right: top-ranked chunks by signed DRTC (process-causal directional steering), shown for qualitative auditing.

Rank	Chunk idx	DRTC	$\|\mathrm{DRTC}\|$	CurvImpact	Excerpt
1	27	+3.4374	3.4374	0.1217	) and up to (0,2) and (2,2). So
2	23	+3.3994	3.3994	-0.0892	Wait, the coordinates in the Asymptote code might be scaled. Because
3	20	+2.2437	2.2437	0.0044	12,10), which is probably connecting point B to point A. There’s also
4	37	+1.6648	1.6648	0.1015	the Asymptote code is 0.5 cm in reality. Wait
5	24	-1.3774	1.3774	0.0098	might be scaled. Because the smaller square has a perimeter of 4 cm,
6	1	+1.3597	1.3597	-0.0041	find the distance from point A to point B in the given figure. The problem

Appendix

Table 21: Phi-4-Mini, math_rollout_3759 : top contributing chunks by DRTC. Positive values align with the realized rollout direction; negative values oppose it. CurvImpact is a geometric diagnostic.

Panel of Figure 26 — Figure 26: Case-study visual summary for math_rollout_3295 (R1-Distill-Qwen-1.5B). Left: pivot importance weights $u_{k}$ across the $K$ detected pivots. Right: top-ranked chunks by signed DRTC (process-causal directional steering), shown for qualitative auditing.

Rank	Chunk idx	DRTC	$\|\mathrm{DRTC}\|$	CurvImpact	Excerpt
1	5	+3.7780	3.7780	-0.0409	So, f_n(x) is just f composed n times with itself.
2	2	+2.9065	2.9065	0.0277	+ x)/(1 - 3x). Then, f_1(x)
3	12	+2.1814	2.1814	0.2078	). Wait, no, the denominator is 1 - 3x, so
4	0	+1.8452	1.8452	-0.0007	problem where I need to find f_1993(3). The
5	4	+1.4766	1.4766	-0.0255	subsequent function being f composed with the previous one. So, f_n(x)
6	8	-1.4092	1.4092	0.0008	what f_1993(3) is without having to compute all

Appendix

Table 22: R1-Distill-Qwen-1.5B, math_rollout_3295 : top contributing chunks by DRTC. Positive values align with the realized rollout direction; negative values oppose it. CurvImpact is a geometric diagnostic.

Panel of Figure 27 — Figure 27: Case-study visual summary for math_rollout_3374 (R1-Distill-Qwen-1.5B). Left: pivot importance weights $u_{k}$ across the $K$ detected pivots. Right: top-ranked chunks by signed DRTC (process-causal directional steering), shown for qualitative auditing.

Rank	Chunk idx	DRTC	$\|\mathrm{DRTC}\|$	CurvImpact	Excerpt
1	20	+3.6469	3.6469	0.1574	0-5, but here we have 6. Wait, that can’t
2	24	+3.3646	3.3646	0.0441	Wait, hold on, maybe I misread the problem. Wait, the number
3	30	-2.1259	2.1259	0.0195	6, which is hexadecimal, but the problem says base-6.number.6
4	37	+1.9587	1.9587	0.0612	but the number is 66666_16. So
5	32	+1.6817	1.6817	0.0359	the problem says base-16, but the number is 66
6	0	+1.6305	1.6305	-0.0005	) does it have? Okay, so I have this problem here: I need

Appendix

Table 23: R1-Distill-Qwen-1.5B, math_rollout_3374 : top contributing chunks by DRTC. Positive DRTC aligns with the realized rollout direction; negative DRTC opposes it. CurvImpact is reported as a geometric diagnostic only.

Framework family	Representative works	Primary explanatory target	Intervention / comparison style	Why DRTC is distinct
Step faithfulness & resampling	Zhao et al. [2026] ; Macar et al. [2025]	Whether specific CoT steps are causally used vs. decorative; resilience/necessity of semantic content for the final answer	Stochastic perturbation of steps; on-policy resampling to compare distributions over alternative traces	Targets necessity or faithfulness (often outcome- or decision-centric). DRTC instead localizes decision pivots and measures signed directional steering in log-probability space under a method-faithful counterfactual that does not resample a new trace .
Recursive attribution graphs	Walker and Ewetz [2025]	Inter-generational influence pathways across prompt and prior generations	Constructs an attribution graph using a base attribution method (e.g., saliency) and marginalizes path contributions	Provides a global dependency structure over full traces. DRTC uses sparse pivot-local causal probes with receiver-side masking and produces directional (signed) steering rather than path-marginal dependence scores.
Geometry-first trajectory analyses	Zhou et al. [2025] ; Manson [2025]	Intrinsic geometric structure of reasoning dynamics (e.g., flow/shape); logic as a carrier-invariant skeleton; concern-sensitive residual-stream geometry	Primarily geometry-as-signal: representation/residual trajectories, pullback metrics, curvature/flow descriptors (typically not intervention-grounded causal attribution)	Treats geometry as an intrinsic signature . DRTC uses geometry as an intervention-response diagnostic : curvature is computed in logit space to summarize how targeted causal interventions reorient the trajectory, while attribution remains defined by signed directional redirection.
Layer-wise dynamics & decoding-time predictors	He et al. [2025] ; Yan [2025]	Vertical evolution across transformer depth; decoding improvements via layer-wise logit trajectory prediction; stage-wise information progression (probes/logit lens)	Cross-layer prediction/probing; layer-wise diagnostics and decoding-time modifications (not token-time causal tests)	Operates over depth (layers) and often targets decoding quality. DRTC operates over token-time rollout depth with pivot-local counterfactuals, explicitly attributing which earlier chunks steer the unfolding reasoning trajectory.
Chunk-level attribution baselines	Li et al. [2017] ; Sundararajan et al. [2017] ; Fong and Vedaldi [2017] ; Zhang and Nanda [2024]	Chunk importance for a chosen scalar (e.g., logit/probability/directional score) at a target position	Input perturbations (remove/replace chunks), gradient saliency, optimized masking, or activation patching at selected layers/positions	Provides non-pivoted chunk attributions that do not explicitly localize decision pivots or enforce pivot-local counterfactuals. DRTC differs by (i) discovering pivots, (ii) applying receiver-side masking only at pivots while holding the realized prefix fixed, and (iii) scoring directional redirection relative to the realized trajectory.
Directional causal attribution (DRTC)	This work	Causal attribution of long-horizon reasoning: which earlier context redirects the rollout and in what direction	Receiver-side pivot masking (block information flow from a selected chunk only at a pivot while holding the realized prefix fixed and evaluating pivot-local counterfactual logits, without generating a new continuation); directional projection in log-probability space	Unique combination: (i) explicit pivot discovery to localize decision points; (ii) method-faithful, pivot-local counterfactuals that avoid off-policy resampling; (iii) signed directional attribution via alignment with the realized trajectory direction; (iv) curvature used only to organize intervention-response geometry, not as an intrinsic importance score.

Appendix

Table 24: Comparison of reasoning-interpretability frameworks and their explanatory targets and intervention styles.

Figure 28: Left panel of the qualitative inspection interface. The model’s self-generated reasoning rollout is shown with text spans corresponding to high-magnitude DRTC scores highlighted. Highlight intensity reflects attribution strength. Prompt and ground-truth information are separated and collapsed to ensure inspection focuses on the model’s own reasoning rather than the problem statement or reference solution.

Panel of Figure 29 — Figure 29: Center and right panels of the qualitative inspection interface. ( a ) The reasoning process is visualized as a trajectory in representation space using a three-dimensional PCA projection of hidden states, with segments corresponding to selected text spans highlighted. High-curvature turning points (computed in logit space) indicate sharp local reorientations of the reasoning trajectory under targeted interventions. ( b ) Reasoning chunks are ranked by signed DRTC score. Positive scores indicate chunks whose removal disrupts the realized reasoning path, while negative scores indicate anti-aligned influence. Curvature diagnostics are geometric descriptors and are not interpreted as causal importance scores.

LLM Interpretability

Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.

See all 778 papers

Oct 19, 2026

DeltaTrace: Efficient Signed Attribution for Reasoning Language Models

We study how words in the context support or oppose a language model's response and present DeltaTrace, a novel attribution framework that explains the effect of input tokens on a fixed model response. DeltaTrace measures how the response's log-probability changes when the context tokens being explained are masked. To distribute this difference among source tokens, it constructs operator-specific local rules for finite changes and composes them backward through the computational graph using effective secant slopes. Each token is assigned a signed attribution as the inner product of its embedding displacement and the corresponding backward-propagated slope. Local conservation makes these contributions sum to the measured score difference and preserves their meaning when aggregated over sentences or passages. DeltaTrace supports both attention and linear-attention models, with auxiliary memory growing linearly with sequence length at fixed model dimensions and chunk size. On the evaluated attention model, DeltaTrace achieves the lowest deletion-faithfulness scores (RISE and MAS) on most retrieval and reasoning tasks and the highest needle-in-a-haystack Recall, exceeding FlashTrace by 5.67-24.33 percentage points. On the evaluated hybrid attention and linear-attention model, DeltaTrace lowers RISE on 9 tasks and MAS on 12 relative to FlashTrace.

Chen Liang, Daoping Wu, Donghua Yang +4

Harbin Institute of Technology · Zhejiang Lab

Feature Attribution LLM Interpretability Faithfulness of Language Model Explanations

Ask PDF

Sep 8, 2026

Answer-Distribution Trajectories: A Stochastic-Dynamics View of LLM Reasoning

Chain-of-thought reasoning provides a structured computation between a model's input and final answer. Yet it is often evaluated through endpoint accuracy, which ignores the path taken to reach that answer. An emerging line of work addresses this limitation using entropy profiles, which track how uncertainty evolves over the reasoning process but do not reveal which competing hypotheses account for that uncertainty. We introduce answer-distribution trajectories, a stochastic-dynamics-inspired representation that tracks the model's full predictive distribution over answers as reasoning unfolds. As a strictly finer representation than endpoint and entropy summaries, answer-distribution trajectories enable us to characterize a trace through a dynamical reasoning profile spanning exploration, revision, motion, and commitment, and to distinguish different dynamical mechanisms of reasoning success and failure. Across sixteen open-weight language models and four reasoning benchmarks, we show that traces with the same endpoint and similar entropy profiles can exhibit substantially different reasoning dynamics. We further find substantial variation in these dynamics both within and across models and tasks, with different objectives favoring different dynamical profiles. Additionally, we show that training and inference choices systematically reshape these profiles. Our results suggest that answer-distribution trajectories provide a rich framework for analysing and evaluating the dynamics of LLM reasoning.

Mar Gonzàlez I Català, Haitz Sáez de Ocáriz Borde, Davide Murari +3

University of Cambridge · Harvey Mudd College

LLM Uncertainty Estimation LLM Reasoning Reasoning Trace Analysis

Ask PDF

Jun 28, 2026

Invariant Reasoning Directions in Latent Trajectories of Language Models

Latent reasoning models perform multi-step inference directly in hidden-state space, yet the structure of these latent reasoning trajectories remains poorly understood. We show that contrastive refinement signals between stronger and weaker reasoning trajectories exhibit a highly concentrated low-rank structure, while unconstrained latent updates remain sensitive to paraphrases, checkpoint choice, and trajectory perturbations. These observations suggest that latent reasoning trajectories contain stable invariant directions mixed with unstable instance-specific variation. We introduce Trajectory-Invariant Latent Refinement (TILR), a training-free intervention framework for identifying and manipulating stable reasoning directions in latent space. TILR first learns a low-rank invariant subspace from contrastive trajectory differences across inputs, then constrains latent interventions to this subspace while suppressing poorly aligned updates through an adaptive alignment gate. Across six reasoning benchmarks, we find that a small number of latent directions explain most variation between strong and weak reasoning trajectories. Interventions on these directions causally improve reasoning consistency and reduce trajectory instability under paraphrases and perturbations. TILR improves answer consistency under paraphrase by ~10% and reduces latent trajectory variance by up to

50\%

while preserving reasoning accuracy. These results support a geometric view of latent reasoning in which transferable reasoning behavior emerges from stable low-dimensional structure within hidden-state trajectories.

Arun Vignesh Malarkkan, Manan Roy Choudhury, Utkarsh Byahut +3

Arizona State University, Tempe, Arizona.

Reasoning Consistency in Language Models Implicit Reasoning in Language Models Causal Interventions in Language Models Language Model Robustness Representation Geometry in Language Models Latent Reasoning

Ask PDF

Abstract

LLM Interpretability

Explore similar work

DeltaTrace: Efficient Signed Attribution for Reasoning Language Models

Answer-Distribution Trajectories: A Stochastic-Dynamics View of LLM Reasoning

Invariant Reasoning Directions in Latent Trajectories of Language Models