SinkTrack: Attention Sink based Context Anchoring for Large Language Models

Xu Liu; Guikun Chen; Wenguan Wang

cs.CVApr 11, 2026

SinkTrack: Attention Sink based Context Anchoring for Large Language Models

Authors: Xu Liu, Guikun Chen, Wenguan Wang

Organizations: The State Key Lab of Brain-Machine Intelligence, Zhejiang University

Abstract

Large language models (LLMs) suffer from hallucination and context forgetting. Prior studies suggest that attention drift is a primary cause of these problems, where LLMs' focus shifts towards newly generated tokens and away from the initial input context. To counteract this, we make use of a related, intrinsic characteristic of LLMs: attention sink -- the tendency to consistently allocate high attention to the very first token (i.e., <BOS>) of a sequence. Concretely, we propose an advanced context anchoring method, SinkTrack, which treats <BOS> as an information anchor and injects key contextual features (such as those derived from the input image or instruction) into its representation. As such, LLM remains anchored to the initial input context throughout the entire generation process. SinkTrack is training-free, plug-and-play, and introduces negligible inference overhead. Experiments demonstrate that SinkTrack mitigates hallucination and context forgetting across both textual (e.g., +21.6% on SQuAD2.0 with Llama3.1-8B-Instruct) and multi-modal (e.g., +22.8% on M3CoT with Qwen2.5-VL-7B-Instruct) tasks. Its consistent gains across different architectures and scales underscore the robustness and generalizability. We also analyze its underlying working mechanism from the perspective of information delivery. Our source code is available at https://github.com/67L1/SinkTrack.

Figures & tables

Figure 1: Hallucination and context forgetting are pervasive obstacles in LLMs.

Figure 2: Attention drift and sink (the darker the color, the greater the weight).

Panel of Figure 3 — Figure 3: An overview of our methodology exploration.

Figure 4: Soft injection achieves considerable performance improvements (§ 2.3 ).

Figure 5: SinkTrack further improves performance.

Figure 6: Effect of mean-pooling in information injection (Eq. 4 ). (a) Using the mean-pooled information vector; (b) using the original (unpooled) one.

Figure 7: Qualitative comparison between CoT and SinkTrack .

Method	Acc (%)
CoT (Baseline)	43.83
Inject Every Layer	51.02
Inject Every 5 Layers	60.04
Intermittent & Increasing	59.87
Intermittent & Decaying	60.69

Table 3: Performance of different soft injection variants (§ 2.3 ).

Figure 8: The illustration of information flow with and without SinkTrack (§ 2.4 ).

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure S1: Comparison between the

\langle

BOS

\rangle

’s attention weight vectors before and after applying SinkTrack .

Gen. Idx	Seq. Pos	Attn to $\langle$ BOS $\rangle$	Max Attn (Others)	Ratio
1	8083	0.561	0.057	9.87x
64	8146	0.570	0.048	11.97x
128	8210	0.612	0.054	11.25x
256	8338	0.560	0.050	11.24x
512	8594	0.558	0.037	15.17x
1024	9106	0.582	0.042	13.96x

Appendix

Table S1: SinkTrack Drift Test on Llama3.1-8B. The attention to

\langle

BOS

\rangle

consistently dominates other tokens even after generating 1024 new tokens.

Model	w/o Injection	w/ Injection	Overhead
Llama3.1-8B	35.90 ms	36.66 ms	+0.76 ms
Qwen2.5-VL-7B	107.13 ms	110.26 ms	+3.13 ms

Appendix

Table S2: Prefill stage latency measurement on NVIDIA GeForce RTX 4090 (averaged over 500 samples). SinkTrack introduces negligible computational overhead.

LLM Context Management

Large language model context management studies how to select, compress, retrieve, and organize information within bounded model inputs.

See all 248 papers

Attention Sinks as Internal Signals for Hallucination Detection in Large Language Models
Apr 12, 2026Jakub Binkowski, Kamil Adamczewski, Tomasz KajdanowiczLLM Hallucination Detection Attention Sinks
Department of Artificial Intelligence, Wroclaw University of Science and Technology
Resonant Context Anchoring: Decoupling Attention Routing and Signal Gain at Inference Time
Jun 1, 2026Mingkuan Zhao, Yide Gao, Wentao Hu +6Contextual Faithfulness Hallucination in Language Models
Xi’an Jiaotong University · University of Science and Technology of China · Tongji University +1
Hallucinations as Orthogonal Noise: Inference-Time Manifold Alignment via Dynamic Contextual Orthogonalization
Jun 2, 2026Mingkuan Zhao, Wentao Hu, Tianchen Huang +6LLM Hallucination Mitigation Hallucination in Language Models
Xi’an Jiaotong University · Xingchen AGI Lab, China Telecom AI Technology (Beijing) Co., Ltd. · University of Science and Technology of China +2

Apr 12, 2026J/K move · Enter open · S save

Attention Sinks as Internal Signals for Hallucination Detection in Large Language Models

Jakub Binkowski, Kamil Adamczewski, Tomasz Kajdanowicz

Department of Artificial Intelligence, Wroclaw University of Science and Technology

LLM Hallucination Detection Attention Sinks LLM Interpretability

Open paper AskPDF arXiv

Large language models frequently exhibit hallucinations: fluent and confident outputs that are factually incorrect or unsupported by the input context. While recent hallucination detection methods have explored various features derived from attention maps, the underlying mechanisms they exploit remain poorly understood. In this work, we propose SinkProbe, a hallucination detection method grounded in the observation that hallucinations are deeply entangled with attention sinks - tokens that accumulate disproportionate attention mass during generation - indicating a transition from distributed, input-grounded attention to compressed, prior-dominated computation. Importantly, although sink scores are computed solely from attention maps, we find that the classifier preferentially relies on sinks whose associated value vectors have large norms. Moreover, we show that previous methods implicitly depend on attention sinks by establishing their mathematical relationship to sink scores. Our findings yield a novel hallucination detection method grounded in theory that produces state-of-the-art results across popular datasets and LLMs.

Abstract

LLM Context Management

Explore similar work

Attention Sinks as Internal Signals for Hallucination Detection in Large Language Models

Resonant Context Anchoring: Decoupling Attention Routing and Signal Gain at Inference Time

Hallucinations as Orthogonal Noise: Inference-Time Manifold Alignment via Dynamic Contextual Orthogonalization