Efficient Transformer Inference

Follow this topic Ask about this topic

Momentum

9 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 79

All topics

Beyond Autoregressive RTG: Conditioning via Injection Outside Sequential Modeling in Decision Transformer
May 7, 2026Yongyi Wang, Hanyu Liu, Lingfeng Li +6Efficient Transformer Inference Reinforcement Learning
School of Computer Science Peking University Beijing, China 100871
Budgeted Attention Allocation: Cost-Conditioned Compute Control for Efficient Transformers
May 7, 2026Amrit NidhiEfficient Transformer Inference Self-Attention
Independent Researcher
Linearizing Vision Transformer with Test-Time Training
May 4, 2026Yining Li, Dongchen Han, Zeyu Liu +3Efficient Transformer Inference Self-Attention
Tsinghua University, Beijing, China
LEAP: Layer-wise Exit-Aware Pretraining for Efficient Transformer Inference
May 1, 2026Shashank Kapadia, Deep Naryan Mishra, Sujal Reddy Alugubelli +4Efficient Transformer Inference Dynamic Neural Networks
Walmart Inc. · Sam’s Club
ELSA: Exact Linear-Scan Attention for Fast and Memory-Light Vision Transformers
Apr 26, 2026Chih-Chung Hsu, Xin-Di Ma, Wo-Ting Liao +1Efficient Transformer Inference Softmax Attention
Advanced Computer Vision Laboratory, National Yang Ming Chiao Tung University
Evaluating CUDA Tile for AI Workloads on Hopper and Blackwell GPUs
Apr 25, 2026Divakar Kumar Yadav, Tian Zhao, Deepak KumarEfficient Transformer Inference GPU Kernel Optimization
University of Wisconsin–Milwaukee Milwaukee, WI, USA · Illinois Institute of Technology Chicago, IL, USA
The Recurrent Transformer: Greater Effective Depth and Efficient Decoding
Apr 23, 2026Costin-Andrei Oncescu, Depen Morwani, Samy Jelassi +3Efficient Transformer Inference Self-Attention
Harvard University
Decoding Text Spans for Efficient and Accurate Named-Entity Recognition
Apr 22, 2026Andrea Maracani, Savas Ozkan, Junyi Zhu +2Efficient Transformer Inference Named Entity Recognition
Samsung Research UK, Staines-upon-Thames, United Kingdom
EX-FIQA: Leveraging Intermediate Early eXit Representations from Vision Transformers for Face Image Quality Assessment
Apr 21, 2026Guray Ozgur, Tahar Chettaoui, Eduarda Caldeira +4Efficient Transformer Inference Efficient Inference
Fraunhofer Institute for Computer Graphics Research IGD, Darmstadt, Germany · Department of Computer Science, TU Darmstadt, Germany
Depth Adaptive Efficient Visual Autoregressive Modeling
Apr 19, 2026Chunliang Li, Tianze Cao, Sanyuan ZhaoEfficient Transformer Inference Visual Autoregressive Models
Beijing Institute of Technology · Beijing Institute of Technology, Zhuhai
Adaptive Computation Depth via Learned Token Routing in Transformers
Apr 18, 2026Ahmed Abdelmuniem Abdalla MohammedEfficient Transformer Inference Transformer
Independent Researcher
SecureRouter: Encrypted Routing for Efficient Secure Inference
Apr 16, 2026Yukuan Zhang, Mengxin Zheng, Qian LouEfficient Transformer Inference Secure Multi-Party Computation
University of Central Florida · Orlando, Florida, USA
Ragged Paged Attention: A High-Performance and Flexible LLM Inference Kernel for TPU
Apr 16, 2026Jevin Jiang, Ying Chen, Blake A. Hechtman +2Efficient Transformer Inference AI Accelerator Inference
Google
Dispatch-Aware Ragged Attention for Pruned Vision Transformers
Apr 16, 2026Seifeldin Abdellatif, Ahmad AlmasriEfficient Transformer Inference Visual Attention
College of Engineering Al Ain University Abu Dhabi, United Arab Emirates
Thinking Deeper, Not Longer: Memory-Efficient Test-Time Reasoning with Depth-Recurrent Transformers for Compositional Generalization
Mar 23, 2026Hung-Hsuan ChenEfficient Transformer Inference Recurrent Transformers
Computer Science and Information Engineering National Central University Taoyuan, Taiwan
Revisiting the Shape Convention of Transformer Language Models
Feb 6, 2026Feng-Ting Liao, Guan-Ting Yi, Tzu-Quan Lin +2Efficient Transformer Inference Transformer FFNs
MediaTek Research · National Taiwan University
Transform Trained Transformer for Accelerating Native 4K Video Generation
Dec 15, 2025Jiangning Zhang, Junwei Zhu, Teng Hu +9Efficient Transformer Inference Efficient Attention
Youtu Lab, Tencent · Zhejiang University
Pianist Transformer: Towards Expressive Piano Performance Rendering via Scalable Self-Supervised Pre-Training
Dec 2, 2025Hong-Jie You, Jie-Jing Shao, Xiao-Wen Yang +3Efficient Transformer Inference Self-Supervised Pre-Training
State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing 210023, China · School of Artificial Intelligence, Nanjing University, Nanjing 210023, China · School of Intelligence Science and Technology, Nanjing University, Suzhou 215163, China
Controllably Efficient Language Models
Nov 7, 2025Jatin Prakash, Aahlad Puli, Rajesh RanganathEfficient Transformer Inference Long-Context Language Modeling
New York University

May 7, 2026, cs.LGJ/K move · Enter open · S save

Beyond Autoregressive RTG: Conditioning via Injection Outside Sequential Modeling in Decision Transformer

Yongyi Wang, Hanyu Liu, Lingfeng Li, Bozhou Chen+5

School of Computer Science Peking University Beijing, China 100871

Efficient Transformer Inference Reinforcement Learning Transformer Transformer-Based RL Decision Transformer Offline RL

Open paper AskPDF arXiv

Decision Transformer (DT) formulates offline reinforcement learning as autoregressive sequence modeling, achieving promising results by predicting actions from a sequence of Return-to-Go (RTG), state, and action tokens. However, RTG is a scalar that summarizes future rewards, containing far less information than typical state or action vectors, yet it consumes the same computational budget per token. Worse, the self-attention cost of Transformers grows quadratically with sequence length, so including RTG as a separate token adds unnecessary overhead. We propose SlimDT, which removes RTG from the autoregressive sequence. Instead, we inject RTG information into the state representations before the sequential modeling step, allowing the Transformer to process only a compact (state, action) sequence. This reduces the sequence length by one-third, directly improving inference efficiency. On the D4RL benchmark, SlimDT surpasses standard DT across various tasks and achieves performance comparable to existing state-of-the-art methods. Decoupling a sparse conditioning signal from an information-rich sequence thus yields both computational gains and higher task performance.

Momentum

Related topics

Latest papers 79