Transformer Inference

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 39

All topics

EntroPrefill: Renyi-Guided Context Pruning with Conditional Stability Guarantees for Retrieval-Augmented Generation
Oct 7, 2026Inbasekaran SToken Pruning Transformer Inference
SRM Institute of Science and Technology, India
Stochastic Rounding in Low-Precision Transformer Inference: A Variable-Precision Emulation Study of a Small GPT-2
Oct 1, 2026Yohan Chatelain, Pablo de Oliveira CastroTransformer Inference Low-Bit Quantization
Krembil Centre for Neuroinformatics, CAMH, Toronto, Canada · Université Paris-Saclay, UVSQ, LI-PaRAD Versailles, France · Universite Paris-Saclay, UVSQ, LI-PaRAD, Versailles, France
The Geometry of Inference in Transformer Residual Streams
Sep 29, 2026Timur Mudarisov, Mikhail Burtsev, Radu StateTransformer Interpretability Transformer Inference
University of Luxembourg, Luxembourg · London Institute for Mathematical Sciences, London, UK
A Smaller Transformer in Your Transformer
Sep 17, 2026Dhananjay Tomar, Marius Aasan, Andreas Kleppe +1Transformer Inference Vision Transformer
Institute for Cancer Genetics and Informatics Oslo University Hospital Oslo, Norway · Department of Informatics University of Oslo Oslo, Norway · SFI Visual Intelligence UiT The Arctic University of Norway Tromsø, Norway
MeshKV: A Network-on-Chip KV Cache Fabric for Scalable Transformer Decoding Accelerators
Sep 16, 2026Dong Liu, Yanxuan YuTransformer Inference KV Caching
Columbia University New York, NY, USA
Recent Developments in Transformer Inference Deployment on FPGA Platforms: A Survey
Sep 1, 2026Arjan Blankestijn, Uraz Odyurt, Amirreza YousefzadehEfficient Transformer Inference Transformer Inference
Computer Architecture for Embedded Systems, University of Twente Enschede, The Netherlands · Faculty of Engineering Technology, University of Twente Enschede, The Netherlands
CacheBridge: Efficient Cross-Model KV Cache Transfer
Sep 1, 2026Xingyu Qu, Siyuan Lu, Zhiyu Chen +2Transformer Inference KV Caching
Westlake University. · †Work conducted during an internship at Westlake University. · Amazon. +1
The Transformer Revolution, Part 1: Dynamic Processing through Output-Weight Interconnections
Aug 4, 2026Marco Giunti, Fabrizia Giulia GaravagliaTransformer Inference Prompt Sensitivity
ALOPHIS – Applied Logic Philosophy and History of Science Università di Cagliari · Dip. di Pedagogia, Psicologia, Filosoﬁa Università di Cagliari
At-the-Roofline Sparse Tensor Contractions on Vector Processors for Transformer Inference
Jul 28, 2026Bowen Wang, Chi Zhang, Diyou Shen +3Efficient Neural Network Inference Transformer Inference
*IIS, ETH Zürich, Switzerland · ‡Computing Systems Lab, Huawei, Switzerland · †DEI, Universit`a di Bologna, Italy
Optimizing Transformer Neural Network for Real-Time Outlier Detection on FPGAs
Jul 24, 2026Ilia Sobakinskikh, Paul Alexander BilokonTransformer Inference Financial Time Series
Agree on the Model, Verify the Inference: GKR Protocols for HND-Based Transformer Inference
Jul 23, 2026Xiaolong Liang, Juanjuan Li, Rui Qin +1Neural Network Verification Transformer Inference
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
MoA-Structured Decode Attention DNF Derivation, KV-Cache Accumulation, GQA/MQA, and OpenACC Kernel
Jul 21, 2026Lenore Mulin, Gaetan HainsEfficient Transformer Inference Grouped-Query Attention
University at Albany, SUNY · LACL, Universit´e Paris-Est Cr´eteil
On the Diverse Dynamical Behaviors Arising in Deep Linear Transformers
Jul 20, 2026Sixu Li, Thomas Jacob Maranzatto, Jan Peszek +5Dynamical Systems Transformer Inference
University of Wisconsin–Madison, Department of Statistics · University of Maryland, Department of Electrical and Computer Engineering · University of Warsaw, Institute of Applied Mathematics and Mechanics +2
Per-Token Fixed-Point Convergence in Depth-Recurrent Transformers
Jul 15, 2026Joe LoganTransformer Inference Recurrent Transformers
Independent researcher
Layer-Parallel Inference Reduces Encrypted Nonlinear Depth in Transformers
Jul 6, 2026Ligong Han, Kai Xu, Hao Wang +3Efficient Transformer Inference Transformer Inference
MBZUAI IFM · Red Hat AI Innovation · MIT-IBM Watson AI Lab +3
ELiTeFormer: An Efficient Transformer for FPGAs
Jul 4, 2026Victor Agostinelli, Nicolas Bohm Agostini, Antonino TumeoLLM Quantization Ternary Quantization
Oregon State University Corvallis, Oregon, USA · Pacific Northwest National Laboratory Richland, Washington, USA
The risk of KV cache compression
Jul 1, 2026Lukas Haverbeck, Carmen Amo Alonso, Andres Felipe Posada-Moreno +2Transformer Inference KV Caching
RWTH Aachen University · Stanford University
The Context-Ready Transformer
Jun 25, 2026Mahesh GodavartiEfficient Transformer Inference Transformer Inference
A Carrot, Inc
Keyless Attention: Value-Space Routing and Value-Only Caching for Efficient Transformers
Jun 20, 2026Xin GaoTransformer Inference KV Caching
Department of Mathematics and Statistics York University Toronto, M3J 1P3
MemNovo: Look Back at the Spectrum for Balanced De Novo Peptide Sequencing from Mass Spectrometry
Jun 10, 2026Dongxin Lyu, Jingbo Zhou, Hongxin Xiang +2Memory-Augmented Neural Networks Transformer Inference
Westlake University Hangzhou, Zhejiang, China · Hunan University Changsha, Hunan, China · Shanghai Artificial Intelligence Laboratory Shanghai, China +1
Latent Recurrent Transformer: Architecture Exploration, Training Strategies, and Scaling Behavior
May 26, 2026Zeyi Huang, Xuehai He, LiLiang Ren +8Language Model Pretraining Transformer Inference
Microsoft · University of Washington · University of Wisconsin-Madison
Profiling-Driven Adaptive Distributed Transformer Inference on Embedded Edge Deployment
May 25, 2026Muhammad Azlan Qazi, Alexandros Iosifidis, Qi ZhangEfficient Transformer Inference Transformer Inference
Aarhus University · Aarhus, Denmark · Tampere University +1
Training-Free Looped Transformers
May 22, 2026Lizhang Chen, Jonathan Li, Chen Liang +2Transformer Inference Recurrent Transformers
University of Texas at Austin
SNLP: Layer-Parallel Inference via Structured Newton Corrections
May 18, 2026Ligong Han, Kai Xu, Hao Wang +1Transformer Inference Parallel Decoding
1Red Hat AI Innovation · 2MIT-IBM Watson AI Lab · 3Core AI, IBM
Going Beyond the Edge: Distributed Inference of Transformer Models on Ultra-Low-Power Wireless Devices
May 15, 2026Alexander Gräfe, Ding Huo, Vincent de Bakker +3Efficient Transformer Inference Transformer Inference
1RWTH Aachen University · 2TU Darmstadt
Transformer-like Inference from Optimal Control
May 15, 2026Aditya Kudre, Heng-Sheng Chang, Prashant G. MehtaTransformer Inference Decoder-Only Language Models
Coordinated Science Laboratory · Electrical and Computer Engineering · University of Illinois Urbana-Champaign +1
Transformer Scalability Crisis: The First Comprehensive Empirical Analysis of Performance Walls in Modern Language Models
May 14, 2026Mahdi Naser Moghadasi, Faezeh GhaderiTransformer Inference Transformer Attention
Research Division, BrightMind AI, Seattle, WA · Texas Tech University, Lubbock, TX · University of Texas at Arlington, Arlington, TX
Attention Once Is All You Need: Efficient Streaming Inference with Stateful Transformers
May 13, 2026Victor NorgrenTransformer Inference LLM Inference
LayerScale, Inc.
ASAP: Amortized Doubly-Stochastic Attention via Sliced Dual Projection
May 13, 2026Huy Tran, Max Milkert, David HydeTransformer Inference Self-Attention
Department of Computer Science Vanderbilt University
KV-Fold: One-Step KV-Cache Recurrence for Long-Context Inference
May 12, 2026Alireza Nadali, Patrick Cooper, Ashutosh Trivedi +1Transformer Inference KV Caching
Department of Computer Science University of Colorado Boulder Boulder, Colorado, 80301

Oct 7, 2026, cs.LGJ/K move · Enter open · S save

EntroPrefill: Renyi-Guided Context Pruning with Conditional Stability Guarantees for Retrieval-Augmented Generation

Inbasekaran S

SRM Institute of Science and Technology, India

Token Pruning Transformer Inference Retrieval-Augmented Generation

Open paper AskPDF arXiv

Mid-prefill pruning can reduce the sequence processed by deeper transformer layers, but attention concentration alone does not certify that discarded context is dispensable. We formulate EntroPrefill as a Renyi-guided proposal mechanism coupled to explicit constraints on discarded attention mass. Sink-isolated, regularized head pooling respects grouped-query attention while exposing a quantitative trade-off between specialization and worst-head coverage. We derive a mixture-to-head deletion envelope, a computable upper bound on feasible token removal, and a finite-sample observer guarantee that remains valid when the pruning layer is selected adaptively. We then establish a conditional transformer perturbation bound with explicit sufficient Lipschitz constants and a first-token decision-margin corollary. A counterexample shows why shallow observations alone cannot imply an unconditional future-output guarantee. The systems analysis distinguishes query-head unions, physical page allocation, and KV-transfer payload, and gives an arithmetic break-even condition for pruning. This manuscript is theoretical in scope: it defines the procedure, its assumptions, and its formal limits, but does not report measured acceleration or task-accuracy preservation. Experiments are reserved for subsequent validation of the assumptions, approximation tightness, and end-to-end resource trade-offs.

Momentum

Related topics

Latest papers 39