DeltaTrace: Efficient Signed Attribution for Reasoning Language Models

Chen Liang; Daoping Wu; Donghua Yang; Zhiyu Liang; Ying Zhang; Shuguang Xiong; Hongzhi Wang

cs.LGOct 19, 2026

DeltaTrace: Efficient Signed Attribution for Reasoning Language Models

Authors: Chen Liang, Daoping Wu, Donghua Yang, Zhiyu Liang, Ying Zhang, Shuguang Xiong, Hongzhi Wang

Organizations: Harbin Institute of Technology · Zhejiang Lab

Abstract

We study how words in the context support or oppose a language model's response and present DeltaTrace, a novel attribution framework that explains the effect of input tokens on a fixed model response. DeltaTrace measures how the response's log-probability changes when the context tokens being explained are masked. To distribute this difference among source tokens, it constructs operator-specific local rules for finite changes and composes them backward through the computational graph using effective secant slopes. Each token is assigned a signed attribution as the inner product of its embedding displacement and the corresponding backward-propagated slope. Local conservation makes these contributions sum to the measured score difference and preserves their meaning when aggregated over sentences or passages. DeltaTrace supports both attention and linear-attention models, with auxiliary memory growing linearly with sequence length at fixed model dimensions and chunk size. On the evaluated attention model, DeltaTrace achieves the lowest deletion-faithfulness scores (RISE and MAS) on most retrieval and reasoning tasks and the highest needle-in-a-haystack Recall, exceeding FlashTrace by 5.67-24.33 percentage points. On the evaluated hybrid attention and linear-attention model, DeltaTrace lowers RISE on 9 tasks and MAS on 12 relative to FlashTrace.

Figures & tables

Figure 1

	Ordinary backpropagation Tangent	DeltaTrace Secant
Local slope	$f^{\prime}(a_{1})$	$\dfrac{\Delta h}{\Delta a}=\dfrac{h_{1}-h_{0}}{a_{1}-a_{0}}$
Output change	$dh=f^{\prime}(a_{1})\,da$	$\Delta h=\dfrac{\Delta h}{\Delta a}\,\Delta a$
Backward calculation	$\dfrac{\partial F}{\partial a}=f^{\prime}(a_{1})\dfrac{\partial F}{\partial h}$	$m_{a}=\dfrac{\Delta h}{\Delta a}\,m_{h}$

Table 1: The same activation, two backward calculations. The activation has input

a

and output

h=f(a)

. Here

d

denotes an infinitesimal change and

\Delta

the observed difference between the two executions.

Task	$n$	Perturbation	REAGENT	CLP	IFR	AttnLRP	FT	DT
MQ-Q2	100	0.0945	0.1174	0.0977	0.0747	0.1955	0.0682	0.0723
MQ-Q4	100	0.2389	0.2599	0.2534	0.1147	0.2628	0.1134	0.1448
MQ-Q8	100	0.4986	0.4865	0.5096	0.3709	0.3766	0.3519	0.4286
MV-V2	100	0.1343	0.1876	0.1559	0.0691	0.1403	0.0686	0.0597
MV-V4	100	0.1862	0.2107	0.2166	0.0726	0.1934	0.0704	0.0587
MV-V8	100	0.3514	0.3685	0.3929	0.2051	0.2852	0.1828	0.1159

Table 2: RISE

\downarrow

on the complete released Qwen3-8B tasks. FT is FlashTrace ( Pan et al., 2026 ) and DT is DeltaTrace . DT uses signed ranking. FT uses the same evaluation examples as DT.

n

is the released task size. Bold marks the lowest observed mean across all seven methods.

Task	$n$	Perturbation	REAGENT	CLP	IFR	AttnLRP	FT	DT
MQ-Q2	100	0.1442	0.1966	0.1663	0.1397	0.3261	0.1628	0.1176
MQ-Q4	100	0.3270	0.3574	0.3203	0.1770	0.4509	0.1937	0.2011
MQ-Q8	100	0.7093	0.6942	0.6566	0.4600	0.6020	0.4271	0.6047
MV-V2	100	0.1870	0.2756	0.2159	0.1343	0.2290	0.1569	0.1046
MV-V4	100	0.2436	0.2905	0.2804	0.1416	0.3248	0.1605	0.1058
MV-V8	100	0.4580	0.4938	0.5114	0.2746	0.4752	0.2477	0.1717

Table 3: MAS

\downarrow

on the same released tasks and seven methods as Table 2 . DT uses its positive source contributions. Bold marks the lowest observed mean.

Task	$n$	Budget	Perturbation	REAGENT	CLP	IFR	AttnLRP	FT	DT
MQ-Q2	100	10%	39.05	24.35	39.85	47.10	21.49	46.32	67.24
MQ-Q4	100	10%	9.04	8.49	8.58	32.84	20.42	39.58	47.16
MQ-Q8	100	10%	0.97	0.47	0.84	1.19	7.63	7.72	13.59
MV-V2	100	10%	25.48	18.02	20.67	57.50	25.44	53.46	77.79
MV-V4	100	10%	16.05	15.60	14.61	45.24	24.25	49.37	56.79
MV-V8	100	10%	8.02	7.37	7.29	17.87	15.88	19.52	25.19

Table 4: Evidence recovery (

\%

, higher is better) at the listed token budgets. NIAH/VT report token Recall. HotpotQA reports Full-support Recall@30%, the fraction of examples for which every official supporting fact is selected. Sentence scores sum positive token contributions. All body tokens are included in the budget calculation. VT macro averages its four rows. FT uses K3 for recovery. Bold marks the highest observed mean.

Task	$n$	Budget	Recall (%) $\uparrow$		RISE $\downarrow$		MAS $\downarrow$
			FT	DT	FT	DT	FT	DT
MQ-Q2	100	10%	52.92	71.94	0.0982	0.0996	0.2549	0.1306
MQ-Q4	100	10%	38.56	54.18	0.1414	0.1579	0.2651	0.2242
MQ-Q8	100	10%	10.72	14.63	0.3704	0.4028	0.4390	0.5891
MV-V2	100	10%	60.51	83.53	0.0974	0.0804	0.2467	0.1080
MV-V4	100	10%	46.19	61.42	0.1284	0.1130	0.2538	0.1510

Table 5: Attribution quality on Qwen3.5-9B. All tasks use the full set of released examples. Recovery is token Recall for NIAH/VT and Full-support Recall@30% for HotpotQA, as defined in Table 4 . FT denotes FlashTrace, with K3 for Recall and K1 for RISE/MAS. DT uses signed RISE and positive-part MAS. Bold marks the better mean. Dashes indicate undefined recovery.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Operation	Change decomposition
Linear and residual	$\Delta(Wx+b)=W\Delta x$ ; $\Delta(a+b)=\Delta a+\Delta b$
Attention values	$\Delta(PV)=\bar{P}\Delta V+\Delta P\bar{V}$
Attention scores	$\Delta(QK^{\top})=\Delta Q\bar{K}^{\top}+\bar{Q}\Delta K^{\top}$
SwiGLU	$\Delta(us)=\bar{s}\odot\Delta u+\bar{u}\odot\Delta s$ , $s=\operatorname{SiLU}(g)$
GDN output gating	$\Delta(n\odot s)=\bar{s}\odot\Delta n+\bar{n}\odot\Delta s$
RMS normalization	Analytic finite rule for $wx/\sqrt{\operatorname{mean}(x^{2})+\epsilon}$

Appendix

Table 6: Finite propagation rules by operator. Bars denote endpoint averages. The gated-memory recurrence is given in Appendix A.4 .

Method	Principle	Strength	DT advantage
Perturbation ( Pan et al., 2026 )	Measure effects of separate masks	Direct intervention	Additive allocation of the joint effect
REAGENT ( Zhao and Shan, 2024 )	Iteratively replace tokens and update importance	Black-box attribution	Signed scores without iterative sampling
CLP ( Cífka and Liutkus, 2023 )	Measure prediction changes as context expands	Reveals newly added information	Allocates the joint effect of selected inputs
IFR ( Ferrando and Voita, 2024 )	Trace contributions to internal representations	Identifies information paths	Includes changes in attention weights
AttnLRP ( Achtibat et al., 2024 )	Propagate output relevance backward	Input and neuron attribution	Explains the measured finite score change
FlashTrace ( Pan et al., 2026 )	Aggregate spans and recursively trace sources	Efficient multi-token attribution	Signed, additive response-score contributions

Appendix

Table 7: Attribution methods.

Model	Task	Method	RISE $\downarrow$	MAS $\downarrow$	Recovery@10% $\uparrow$
Qwen3-8B	MQ-Q2	FlashTrace	0.0685	0.1547	56.45
		DeltaTrace	0.0574	0.0806	77.52
	MoreHopQA	FlashTrace	0.1111	0.1969	—
		DeltaTrace	0.0585	0.1173	—
Qwen3.5-9B	MQ-Q2	FlashTrace	0.0942	0.2485	63.92
		DeltaTrace	0.1115	0.1518	73.35

Appendix

Table 8: Paired development results on the first eight MQ-Q2 and eight MoreHopQA examples per model. This gives 32 model–example pairs. FT uses one recursive hop for RISE/MAS and three for recovery. RISE is signed, while MAS and recovery use positive scores.

Figure 2: Signed evidence for retrieving two numbers. (a,b) The same niah_mq_q2 input on Qwen3-8B and Qwen3.5-9B. Teal and coral show positive and negative DT contributions in nats. Marks locate excerpts. (c) Each model’s DT (solid) and one-hop FT (dashed) curves show the normalized log-probability of the full fixed response plus EOS under 20-step deletion. Color identifies the model. The released normalization uses that model’s full-input and fully-masked scores, clipping and a cumulative minimum. DT deletion uses positive ranking. Answer summarizes the full stored response.

Figure 3: Signed evidence across a multi-hop question. (a,b) The MoreHopQA input links Townsend Coleman to Sing , Garth Jennings, and his surname’s consonants. (c) Normalized log-probability of the full fixed response plus EOS, comparing each model’s DT with its own one-hop FT baseline. Model colors, method line styles, and per-model normalization follow Figure 2 . Both heatmaps share the signed scale within this example.

Feature Attribution

Feature attribution estimates which input features or internal components contribute to a model’s predictions.

See all 231 papers

Local Causal Attribution of Chain-of-Thought Reasoning
Jun 20, 2026Dennis Wei, Yannis Belkhiter, Erik Miehling +1LLM Interpretability Causal Attribution
1IBM Research.
Directional Reasoning Trajectory Change (DRTC): Identifying Critical Trace Segments in Reasoning Models
Feb 17, 2026Waldemar ChangLLM Interpretability Reasoning Trace Analysis
Johns Hopkins University
Matryoshka attribution: Learning to attribute language model outputs to representations and weights
Sep 22, 2026Aryaman Arora, Kirill Acharya, Nathan Hu +4LLM Interpretability Circuits in Language Models
Stanford University

Jun 20, 2026J/K move · Enter open · S save

Local Causal Attribution of Chain-of-Thought Reasoning

Dennis Wei, Yannis Belkhiter, Erik Miehling, Radu Marinescu

1IBM Research.

LLM Interpretability Causal Attribution Causal Reasoning in Language Models Causal Interpretability Reasoning Trace Analysis Structured Reasoning in Language Models

Open paper AskPDF arXiv

Understanding the causal structure of a language model's thought process is a problem of significant importance for both transparency and safety. In this work, we take a local approach toward this goal by analyzing the causal relationships among individual components, termed units, of a given, specific chain-of-thought trace. We construct a structural causal model on these units and relate each unit to the log probability of generating (subsequent) output units. Our algorithm, termed AttriCoT, is a black-box method that performs attribution by estimating importance parameters in the structural causal model using

O(U)

forward passes through the model, where

U

is the number of units. Evaluation of perturbation curves across 5 datasets and 4 reasoning models shows that AttriCoT produces attributions that are more faithful to the model's behavior than alternative methods. The attribution results also reveal notable differences in thought structure between models and domains.

Abstract

Feature Attribution

Explore similar work

Local Causal Attribution of Chain-of-Thought Reasoning

Directional Reasoning Trajectory Change (DRTC): Identifying Critical Trace Segments in Reasoning Models

Matryoshka attribution: Learning to attribute language model outputs to representations and weights