Autoregressive Transformers

Follow this topic Ask about this topic

Recent momentum

-50%

2 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

2 new papers

A weekly snapshot of new work published in Autoregressive Transformers.

33 papers

Latest in Autoregressive Transformers

Browse all topics →

MeshKV: A Network-on-Chip KV Cache Fabric for Scalable Transformer Decoding Accelerators
Sep 16, 2026Dong Liu, Yanxuan YuKey-Value Cache Accelerator
DA-DLM: Explicitly Modeling Token Dependencies in Diffusion Language Models
Sep 14, 2026Pengyu Ji, Zichen Zhang, Xiang Hu +1Diffusion Language Models Autoregressive Transformers
Neural Introspection Gating for Adaptive KV-Cache Reuse in Vision-Language-Action Models
Aug 11, 2026Zhijie Wu, Kento Kawaharazuka, Kei OkadaCache Visual Tokens
Full-bandwidth transformer
Aug 9, 2026Xi Wang, Ziyang Cai, Zheng Zhan +5Autoregressive Transformers Transformer Architectures
Scaling an Autoregressive Transformer for Single-Cell Generation
Aug 3, 2026Aleksandr Sharipov, Yusif Mukhtarov, Igor MolybogAutoregressive Transformers Single-Cell Rna Sequencing
Interpretable GOHR Agents via Sparse Autoencoders
Jul 27, 2026Shiwei Tan, Yusong Zhao, Weiyi Qin +6Agent Decisions Autoregressive Transformers
CAPT: A Multi-task Continuous Autoregressive Transformer enabling Cross-dataset and Cross-species Transfer for Calcium Population Dynamics
Jul 25, 2026Xinhong Xu, Yimeng Zhang, Yuanlong ZhangAutoregressive Transformers Causal Attention
Autoregressive Modeling of Film with Applications in Video Montage
Jul 16, 2026Marcelo Sandoval-Castañeda, Fabian Caba Heilbron, Shiry Ginosar +4Autoregressive Video Generation Films
COBART: Controlled, Optimized, Bidirectional and Auto-Regressive Transformer for Ad Headline Generation
Jul 9, 2026Yashal Shakti Kanungo, Gyanendra Das, Pooja A +1Ads Click-Through Rate Prediction
EVOTS: Evolutionary Transformer Search for Time Series Forecasting
Jun 30, 2026AbdElRahman ElSaid, Damir PulatovMultivariate Time Series Forecasting Evolutionary Search
Generative Lane Topology Reasoning via Autoregressive Model with Geometry Prior
Jun 30, 2026Jiahui Fu, Zehao Huang, Han Li +2Lane Topology Geometry-Aware Foundation Model
How Token Influence Decays with Distance: A Green-Function View of Trained Language Models
Jun 28, 2026Matthias Brändel, Stephan Köhler, Oliver RheinbachAutoregressive Transformers Next-Token Prediction
Long-Term Prediction of Local and Global Human Motion with Occlusion Recovery
Jun 26, 2026Qiaoyue Yang, Sven Heutger, Christopher Niemann +3Human Motion Prediction Human Motion Generation
SamatNext v0.2-B: An Exploratory Study of RMS-Normalized Hybrid Decoders for Curriculum Retention in Small Code Models
Jun 20, 2026Samat ZharassovAutoregressive Transformers Catastrophic Forgetting
Discrete Autoregressive Transformer for Generative Mechanism Synthesis
Jun 16, 2026Anar Nurizada, Anurag PurwarAutomated Synthesis Techniques Kinematic Chain
GPT-Based Fast Simulation of CLAS12 Detector Hits via Conditional Autoregressive Generation
Jun 14, 2026Cole Granger, James Giroux, Richard Tyson +2High Energy Physics Autoregressive Generation
SPADE: Split-and-Delay Embeddings for Autoregressive High-Granularity Calorimeter Simulation
Jun 9, 2026Joschka Birk, Frank Gaede, Anna Hallin +3Autoregressive Transformers Autoregressive Model
In-Context Graphical Inference
Jun 3, 2026Zehua Cheng, Wei Dai, Jiahao SunProbabilistic Inference Bayesian
Data-Driven Forecasting of three-Component Seismograms Using Transformer Architectures
Jun 1, 2026Waleed Esmail, Stuart Russell, Jana Klinge +2Full Waveform Inversion Autoregressive Transformers
Cognitive Fatigue in Autoregressive Transformers: Formalization and Measurement
May 29, 2026Riju Marwah, Ritvik Garimella, Vishal Pallagani +3Autoregressive Language Models Large Language Model Performance
Sequential Neural Probabilistic Amplitude Shaping: Learning the Channel's Language
May 27, 2026Mohammad Taha Askari, Lutz Lampe, Amirhossein GhazisaeidiSource-Channel Coding Neural Audio Codecs
PRISM: Position-encoded Regressive Inverse Spectral Model for Multilayer Thin-Film Design
May 26, 2026Runtian Wang, Renhao Xue, Baige Chen +1Autoregressive Transformers Generative Inverse Design
Transformer Field Theory: A Response-Theoretic Approach to Mechanistic Interpretability
May 24, 2026David N. Olivieri, Antonio F. Pérez RodríguezAutoregressive Transformers Mechanistic Interpretability
UniT: Unified Geometry Learning with Group Autoregressive Transformer
May 20, 2026Haotian Wang, Yusong Huang, Zhaonian Kuang +4Cross-Modal Attention Autoregressive Transformers
Position: The Turing-Completeness of Autoregressive Transformers Relies Heavily on Context Management
May 19, 2026Guanyu Cui, Zhewei Wei, Kun HeAutoregressive Transformers Transformer Architectures
Where Should Diffusion Enter a Language Model? Geometry-Guided Hidden-State Replacement
May 14, 2026Injin Kong, Hyoungjoon Lee, Yohan JoDiffusion Language Models Autoregressive Transformers
N-vium: Mixture-of-Exits Transformer for Accelerated Exact Generation
May 13, 2026Aleksander Lorenc, Frédéric Berdoz, Joël Mathys +1Autoregressive Transformers Transformer Architectures
Exploring the Potential of Probabilistic Transformer for Time Series Modeling: A Report on the ST-PT Framework
Apr 29, 2026Zhangzhi Xiong, Haoyi Wu, You Wu +3Time Series Spatio-Temporal Transformers
Architecture Determines Observability of Transformers
Apr 27, 2026Thomas CarmichaelAutoregressive Transformers Transformer Architectures
A Hybridizable Neural Time Integrator for Stable Autoregressive Forecasting
Apr 22, 2026Brooks Kinch, Xiaozhe Hu, Yilong Huang +6Autoregressive Transformers Long-Horizon Forecasting
The Illusion of Equivalence: Systematic FP16 Divergence in KV-Cached Autoregressive Inference
Apr 16, 2026Ranjith Chodavarapu, Lei XuSystematic Fp16 Divergence Key-Value Cache Quantization
GaussianGPT: Towards Autoregressive 3D Gaussian Scene Generation
Mar 27, 2026Nicolas von Lützow, Barbara Rössle, Katharina Schmid +13D Scenes 3D Object Understanding
Normalizing Flows are Capable Models for Continuous Control
May 29, 2025Raj Ghugare, Benjamin EysenbachNormalizing Flows Flow-Based Policy

Sep 16, 2026 · cs.ARJ/K move · Enter open · S save

MeshKV: A Network-on-Chip KV Cache Fabric for Scalable Transformer Decoding Accelerators

Dong Liu, Yanxuan Yu

Columbia University New York, NY, USA

Autoregressive transformer decoding is constrained by irregular key-value (KV) cache movement on tiled accelerators. Prior compression and DRAM-placement systems still concentrate traffic on centralized memory paths that bottleneck long-context serving. We present MeshKV, a KV cache fabric that moves blocks as packetized flows over a lightweight NoC. It co-designs (i) TaKV affine striping to spread homes and cut hotspot load, (ii) Mare multicast with verified duplicate suppression, and (iii) Pad, which overlaps prefetch, tile multiply, and streaming softmax behind credit-aligned FIFOs. Together they convert bisection back-pressure into useful KV transfer. On our 8x8 FPGA implementation with LLaMA-2-7B and Mistral-7B at 8K-32K, MeshKV reduces interconnect traffic by up to 58%, improves KV bandwidth utilization by 2.1x, and delivers up to 1.9x multi-stream throughput.

Key-Value Cache Accelerator Throughput Autoregressive Transformers Transformer Encoder

Open paper Ask about it PDF

Recent momentum

Neighbouring directions

Recent digests

2 new papers

Latest in Autoregressive Transformers