Which context tokens drive a language model's answer? Graph-based tools such as attention rollout and attention flow answer this from a single forward pass, yet their faithfulness in decoder-only models is rarely tested against interventions. We view the layers of a decoder as a causal network whose conductances are attention weights, and read a token's influence on the answer from the resistance between them. We propose RIFT (Resistance-based Information Flow in Transformers), which scores a token by the series resistance of its path of least resistance to the answer, found exactly by a classic shortest-path dynamic program. On synthetic in-context retrieval tasks and extractive question answering over Wikipedia paragraphs, RIFT predicts the effect of replacing a token with a Spearman correlation of 0.52-0.68; predicting a single replacement, it reaches 77-96% of the agreement between two different random replacements. It needs no gradients and no extra forward passes, scoring every token of a prompt in 0.9-3.4 ms on one CPU thread, whereas the replacement experiment needs a forward pass per token. On 576 held-out prompts from four models of two families (Qwen3 and Llama 3.2), it outperforms token distance, attention rollout and attention flow in every pre-specified comparison, by 0.23 to 0.88 in mean Spearman correlation, and it is the best or tied-best of all attention-based scores we test. Aggregates that add up all routes (attention flow, effective resistance and rollout) instead mostly encode token position: as a simple proposition shows, early tokens have many more parallel routes to the answer.
Figures & tables
Fig. 1: The resistance view. A decoder unrolled over layers is a causal network; node (ℓ,i) is token i after layer ℓ , and edges carry attention mixing weights (plus the residual) as conductances. An early token reaches the answer through many parallel routes (gray), which all-path aggregates add up. RIFT scores only the strongest single route, the path of least resistance (blue).
TABLE I: Predicting causal token influence: mean within-prompt Spearman correlation with the replacement label (the mean of two replacements), and the paired advantage of RIFT over each baseline with 95% bootstrap intervals. Acc.: share of the scored prompts answered correctly. Rel.: agreement between two single replacements; RIFT 1 : RIFT against a single replacement, the like-for-like counterpart of Rel. A–E and H use Qwen3, F, G and I Llama-3.2-3B-Instruct; H and I are correctly answered SQuAD questions, with the log-probability label. In the Qwen3 datasets, attention flow ranks tokens from earliest to latest (exactly in all but one prompt), so its correlation with the label is the negative of that of distance.
Task
N
Rel.
RIFT
Distance
Rollout
Attn. flow
Single-hop
283
0.62
0.55
− 0.11
0.23
0.08
Two-hop
34
0.84
0.68
0.50
− 0.03
− 0.50
First-value
131
0.65
0.56
0.04
0.21
− 0.04
SQuAD
128
0.65
0.68
− 0.15
0.32
0.14
TABLE II: Confirmation results by task family (prompt-weighted means over the nine datasets; N is the number of prompts). Rel.: agreement between two single replacements; the score columns use their mean.
Fig. 2: Confirmatory results for the nine held-out datasets (A–E and H: Qwen3; F, G and I: Llama-3.2-3B-Instruct; H and I: SQuAD). RIFT (blue) is far above token distance and rollout in every dataset; the widest path (violet), its single-route sibling and a diagnostic outside the decision rule, comes close. Attention flow is omitted (for Qwen3 it is the mirror image of distance); Table I gives all values with the label reliability.
Fig. 3: A representative SQuAD question for Qwen3-1.7B (dataset H; 207 tokens): “What type of animal did onlookers claim was piloting the boat?” (answer: monkey ). Columns are the 12 scored words in position order (subscript: position; the answer word and its question words in bold); darker cells mark tokens ranked as more influential within the prompt.
Fig. 4: Accuracy versus cost on the SQuAD prompts (datasets H and I). Accuracy: Spearman correlation with the effect of one replacement (for the replacement experiment, the agreement between two replacements). Cost: median time to score every token of a prompt. RIFT, rollout and distance are timed on one CPU thread, attention flow and effective resistance from their per-source solver times, and the replacement experiment from its per-pass GPU time (one forward pass per token).
− max. attention
− widest path
− multiplicative path
A
+0.05 [+0.01, +0.10]
+0.05 [+0.02, +0.08]
+0.03 [ − 0.01, +0.06]
B
+0.20 [+0.15, +0.24]
+0.00 [ − 0.03, +0.03]
+0.10 [+0.08, +0.13]
C
+0.15 [+0.11, +0.20]
− 0.01 [ − 0.03, +0.02]
+0.09 [+0.06, +0.12]
D
+0.03 [ − 0.01, +0.06]
+0.05 [+0.01, +0.08]
+0.01 [ − 0.02, +0.04]
E
+0.18 [+0.14, +0.23]
− 0.01 [ − 0.04, +0.03]
+0.09 [+0.05, +0.12]
F
+0.05 [+0.01, +0.09]
+0.04 [ − 0.01, +0.08]
+0.04 [+0.01, +0.08]
TABLE III: Advantage of RIFT over other attention-based scores (paired difference in mean Spearman with 95% intervals; computed after confirmation, diagnostic).
Score
Position ρ
Label ρ
Token distance
1.00 / 1.00
0.13 / 0.00
Attention rollout
− 0.73 / − 0.60
0.02 / 0.28
Attention flow
− 1.00 / − 1.00
− 0.13 / 0.00
Eff. resistance, attention
− 0.96 / − 0.91
− 0.14 / − 0.01
Eff. resistance, Jacobian
− 0.99 / − 0.98
− 0.14 / − 0.02
Structural null
− 1.00 / − 1.00
− 0.13 / 0.00
TABLE IV: Position dependence versus validity (exploration, 192 prompts per 1.7B model, base / instruct). Position: Spearman with inverse distance to the answer; label: Spearman with the replacement label. Structural null: effective resistance with every conductance set to the same constant.
Fig. 5: Position dependence as the attention graph deepens (mean of the two 1.7B models). All-path aggregates become functions of position within a few layers; attention flow (hollow diamonds) and effective resistance nearly coincide. RIFT does not. Negative values mean that earlier tokens score higher.