Lost-in-the-Middle Effect

Latest papers

9 papers0 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

An Adjoint-Sensitivity Framework for Lost-in-the-Middle Phenomena in Causal Residual Transformers
Jul 20, 2026Cheng Huan, Hongwei YuanLost-in-the-Middle Effect Transformer Attention
Department of Statistics and Data Science, The Chinese University of Hong Kong. · Department of Mathematics, University of Macau.
LongCrafter: Towards Diverse Long-Context Understanding via Evidence-Graph-Guided Instruction Synthesis
Jul 7, 2026Chenhao Yuan, Yinhao Xu, Shuwen Xu +8Lost-in-the-Middle Effect LLM Fine-Tuning
University of Chinese Academy of Sciences, Beijing, China · The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
Prefilling-dLLM: Predictive Prefilling for Long-Context Inference in Diffusion Language Models
Jun 9, 2026Jing Xiong, Qi Han, Shansan Gong +5Lost-in-the-Middle Effect Prefill-Decode Disaggregation
The University of Hong Kong · University of Michigan, Ann Arbor · LMSYS Org
Kinetic theory for Transformers and the lost-in-the-middle phenomenon
May 9, 2026Mitia Duerinckx, Borjan Geshkovski, Stefano RossiLost-in-the-Middle Effect Self-Attention
BERAG: Bayesian Ensemble Retrieval-Augmented Generation for Knowledge-based Visual Question Answering
Apr 24, 2026Jinghong Chen, Jingbiao Mei, Guangyu Yang +1Visual Question Answering Retrieval-Augmented Generation
Department of Engineering University of Cambridge
Escaping the Context Bottleneck: Active Context Curation for LLM Agents via Reinforcement Learning
Apr 13, 2026Xiaozhe Li, Tianyi Lyu, Yizhao Yang +6Lost-in-the-Middle Effect LLM Context Management
Tongji University · CurrentsAI Research · Stanford University +1
Self-Describing Structured Data with Dual-Layer Guidance: A Lightweight Alternative to RAG for Precision Retrieval in Large-Scale LLM Knowledge Navigation
Mar 28, 2026Hung Ming LiuLost-in-the-Middle Effect Information Retrieval
PARRAWA AI
Lost in the Middle at Birth: An Exact Theory of Transformer Position Bias
Mar 10, 2026Borun D ChowdhuryLost-in-the-Middle Effect Transformer
Meta, London, W1T 1FB, UK
A Structural Theory of Position Bias in Transformers
Feb 18, 2026Hanna Herasimchyk, Robin Labryga, Tomislav Prusina +1Lost-in-the-Middle Effect Position Bias
University of Hamburg Hamburg, Germany

Jul 20, 2026J/K move · Enter open · S save

An Adjoint-Sensitivity Framework for Lost-in-the-Middle Phenomena in Causal Residual Transformers

Cheng Huan, Hongwei Yuan

Department of Statistics and Data Science, The Chinese University of Hong Kong. · Department of Mathematics, University of Macau.

Lost-in-the-Middle Effect Transformer Attention Positional Encoding Sensitivity Analysis

Open paper AskPDF arXiv

We develop an adjoint-sensitivity framework for positional influence in causal residual Transformers and separate unconditional analytic results from conditional boundary-shape conclusions. The principal unconditional theorem is the residual-to-depth-flow estimate for layer controls converging in

L^1

, complemented by a finite-token-to-Volterra attention estimate that explicitly controls the first cells near the causal endpoint. We define a normalized adjoint-energy influence density and derive its exact evolution along full-batch gradient flow. The adjoint admits an exact generator-term decomposition into residual transmission, nonlocal Volterra, and local channels, including all covariance cross terms. Causal masking can amplify early-position sensitivity and residual identity paths can transmit a right-localized terminal bias, but neither mechanism alone forces a U-shaped profile. We therefore state boundary advantages under independently checkable energy, correlation, and local-channel bounds; these conditions are sufficient rather than necessary. Finite-token influence balancing, positional reweighting, and task-aligned observability are presented as diagnostics or regularizers with explicit differentiation requirements, computational costs, and limitations. Controlled simulations illustrate that each intervention controls its designated surrogate, while observability balance or outer-loop reweighting need not monotonically reduce the influence-based Lost-in-the-Middle diagnostic.