cs.AIFeb 14, 2026

Attention in Constant Time: Vashista Sparse Attention for Long-Context Decoding with Exponential Guarantees

Authors: Vashista Nobaub

Organizations: Datar Consulting

Abstract

Large language models spend most of their inference cost on attention over long contexts, yet empirical behavior suggests that only a small subset of tokens meaningfully contributes to each query. We formalize this phenomenon by modeling attention as a projection onto the convex hull of key vectors and analyzing its entropic (softmax-like) relaxation. Our main theoretical contribution is a face-stability theorem showing that, under a strict complementarity margin (a support gap (Δ) certified by KKT multipliers), entropic attention concentrates on a constant-size active face: the total mass assigned to inactive tokens decays exponentially as (\exp(-Ω(Δ/\varepsilon))), while the error on the active face scales linearly in the temperature/regularization parameter (\varepsilon). This yields a practical criterion for when sparse long-context decoding is safe and provides a principled knob to trade accuracy for compute. Building on these guarantees, we introduce Vashista Sparse Attention, a drop-in mechanism that maintains a small candidate set per query through a paging-style context selection strategy compatible with modern inference stacks. Across long-context evaluations, we observe stable constant-size effective support, strong wall-clock speedups, and minimal quality degradation in the regimes predicted by the support-gap diagnostics. Finally, we discuss deployment implications for privacy-sensitive and air-gapped settings, where interchangeable attention modules enable predictable latency and cost without external retrieval dependencies.

Figures & tables

Explore similar work

CardsList
  1. Stratified Sparse Attention: Reducing the Sparse-to-Dense Gap at Extreme Sparsity

    Oct 20, 2026Sahil Joshi, Prithvi Dixit, Aryan Katiyar +5Sparse AttentionLong-Context Language Model Inference

  2. Elastic Threshold Attention: Learned Contextual Sparsity for Long-Context Decoding

    Sep 16, 2026Themistoklis Haris, Henry Li, Maryam KarimzadehganLong-Context Language Model InferenceSparse Attention