Large Language Model Decoding

Follow this topic Ask about this topic

Recent momentum

-55%

5 papers in the last 28 days · 0.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-14

1 new paper

A weekly snapshot of new work published in Large Language Model Decoding.

Period ending 2026-09-07

8 new papers

A weekly snapshot of new work published in Large Language Model Decoding.

89 papers

Latest in Large Language Model Decoding

Open your feed →

Fixed State, Long Reach: What a Constant-Size Cache Buys Block Diffusion at Scale
Sep 14, 2026Vaibhav Singh, Pierre-André Noël, Torsten Scholak +2Diffusion Language Models Large Language Model Decoding
K/V-Cache Interventions Dissociate Representation Alignment from Persona Expression in Decoder-Only Language Models
Sep 11, 2026Yu Sun, Mengyin Lu, Cong Feng +2Kv-Cache Steering Personas
Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM
Sep 3, 2026Sergii Kozyrev, Davyd MaiborodaLarge Language Model Quantization Fp8
Faster Than Flash: Exploiting Attention Sparsity for Efficient Long-Context Decoding
Aug 31, 2026Zhigeng Liu, Zhiyuan Ning, Ruixiao Li +5Efficient Long-Context Inference Dynamic Sparse Attention
DASC: Decay-Aware State Compression for Hybrid Linear-Attention Serving
Aug 31, 2026Yanqi Yu, Pingwei Sun, Jianchao Tan +4Dynamic Sparse Attention Recurrent State
CateKV: On Sequential Consistency for Long-Context LLM Inference Acceleration
Aug 31, 2026Haoyun Jiang, Haolin Li, Jianwei Zhang +7Efficient Long-Context Inference LLM Inference Efficiency
Depth-Wise Probing and Pruning of the Planning Token in a Driving Vision-Language-Action Model
Aug 7, 2026Harisankar Babu, Benjamin Coors, Christopher Lang +3Vision-Language-Action Dead Reckoning
Can MLLMs Decode the Creative Leap? Introducing C4 for Cross-Concept Understanding
Aug 6, 2026Ming Wang, Yuqing Zhang, Tingna Xie +5Creativity Multi-Dimensional Evaluation
NeuroPB: Scaling Neural Decoding with Pretrained Behavioral Representations
Aug 5, 2026Luyao Jin, Yonghao Song, Huan Zhao +2Brain-Computer Interface Behavioral Foundation Models
Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIs
Aug 2, 2026Neil F. Johnson, Frank Yingjie Huo, Bella Xinrui LiNonlinear Dynamics Emergence
Leveraging Transfer Learning with Class-Specific Decoders for Laparoscopic Segmentation
Jul 31, 2026Priya Tomar, Aditya Parikh, Christian Bauckhage +1Medical Image Segmentation Transfer Learning
Stage-Replay Divergence Follows the KV Cache: Fixed-Prefix Precision Controls and Bidirectional Cache Transplantation
Jul 30, 2026Alexander Boesgaard LorupCache Prefill
Memory Decoder at Scale: A Pretrained, Parametric Long-Term Memory
Jul 30, 2026Rubin Wei, Jiaqi Cao, Jiarui Wang +4Large Language Model Memory Large Language Model Scale
Phoneme- vs. Character-Level Targets and Selective State-Space Models for Intracortical Brain-to-Text
Jul 29, 2026Lucas Zamora Vera, Jose A. Gonzalez-LopezGrapheme-To-Phoneme Large Language Model Decoding
Decoupled Visual Processing: Efficient Multimodal Adaptation via Modality-Specific Transformer Substitution
Jul 29, 2026Mingkuan Feng, Zhengqi Wen, Jianhua TaoMultimodal Transformers Large Language Model Decoding
SyRuP: Enhancing System-Prompt Following via Reward-Guided Prediction in LLM Decoding
Jul 27, 2026Seoyeon Kim, Minjae Kang, Jaehyung KimInstruction-Tuned Large Language Models Large Language Model Decoding
HiKV: Hierarchical Importance-Aware KV Cache with Hardware Acceleration for LLM Decoding
Jul 24, 2026Chao Fang, Jun Yin, Man Shi +1Key-Value Cache Large Language Model Decoding
A Modern Multimodal Assistant on a 6 GB 2011 GPU: Stage-Validated, All-GPU CUDA Inference for Fermi
Jul 16, 2026A. C. Opus, J. Q. LuRtx Cuda
Leveraging unlabelled data for generalizable neural population decoding
Jul 15, 2026Ximeng Mao, Nanda H. Krishna, Avery Hee-Woon Ryoo +2Neural Recordings Self-Supervised Learning
DepthWeave-KV: Token-Adaptive Cross-Layer Residual Factorization for Long-Context KV Cache Compression
Jul 7, 2026Anna Cordoba, Adam Puente Tercero, Nerea Angulo Hijo +4Key-Value Cache Compression Efficient Long-Context Inference
FreqDepthKV: Frequency-Guided Depth Sharing for Robust KV Cache Compression in Long-Context LLM Inference
Jul 7, 2026Anna Córdoba, Adam Puente Tercero, Nerea Angulo Hijo +4Key-Value Cache Compression LLM Inference Efficiency
Latency-Constrained Hardware-Aware Quantum Error Correction Co-Design with Adaptive Confidence-Gated Neural Decoding for the Rotated Surface Code
Jul 7, 2026Sumit ChongderQuantum Error Correction Error Correction
TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decoding
Jul 3, 2026Chengcheng Wang, Tingzhang Luo, Wenhao Li +2Diffusion Language Models Large Language Model Decoding
Expander Sparse Autoencoders: Parameter-Efficient Dictionaries for Mechanistic Interpretability
Jul 2, 2026Rodrigo Mendoza-SmithImproving Sparse Autoencoders Sparse Autoencoders
PARTREP: Learning What to Repeat for Decoder-only LLMs
Jul 2, 2026Andikawati P Widjaja, Yongjun Kim, Hyounghun Kim +1Large Language Model Decoding Causal Attention
Predict, Reuse, and Repair: Accelerating Dynamic Sparse Attention for Long-Context LLM Decoding
Jun 29, 2026Tianyu Wang, Gourav Rattihalli, Aditya Dhakal +4Dynamic Sparse Attention Efficient Long-Context Inference
One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Grounding
Jun 29, 2026Chen Liu, Ling Chen, Hanzhang Zhou +5Graphical User Interface Target Localization
Depth Exploration for LLM Decoding
Jun 28, 2026Weisi Yang, Zipeng Sun, Stephen XiaLarge Language Model Decoding Constrained Decoding Strategies
BTI-Net: Bidirectional Decoder-Level Task Interaction via Uncertainty-Aware Gating for Multi-Task Medical Image Analysis
Jun 27, 2026Abdullah Al Shafi, Md Kawsar Mahmud Khan Zunayed, Safin Ahmmed +2Medical Image Segmentation Medical Imaging
Masked Diffusion Decoding as $x$ -Prediction Flow
Jun 27, 2026Weitian Wang, Lianlei Shan, Shubham Rai +2Masked Diffusion Language Models Large Language Model Decoding
Locker-based Truck-Drone Routing with Integrated Considerations of Pickups, Deliveries, and No-Fly Zones
Jun 26, 2026Xuanyu Liu, Hui Hu, Jiao Zhao +2Multi-Drone Coordination Last Mile
Textual Belief States for World Models: Identifiable Representation Learning Under Strict Mediation
Jun 26, 2026Xiang Gao, Kaiwen Dong, Yuguang Yao +2Intermediate Latent States Latent Reasoning
Neural Texture Compression using Hypernetworks
Jun 25, 2026Belcour LaurentConvolutional Neural Network Compression Large Language Model Decoding
Breaking the Likelihood Trap: Variance-Calibrated Modulation for Large Language Model Decoding
Jun 21, 2026Yuanhao Ding, Meimingwei Li, Esteban Garces Arias +3Large Language Model Decoding
HERALD: High-Throughput Block Diffusion LLM Serving via CPU-GPU Cooperative KV Cache Retrieval
Jun 19, 2026Omin Kwon, Doyeon Kim, Jongseok Park +3LLM Inference Efficiency Graphics Processing Unit Memory
Mixtures of Subspaces for Bandwidth Efficient Context Parallel Training
Jun 15, 2026Sameera Ramasinghe, Ajanthan Thalaiyasingam, Hadi Mohaghegh Dolatabadi +5Large Language Model Compression Efficient Long-Context Inference
QK-Normed MLA: QK normalization without full key caching
Jun 15, 2026Yizhou Han, Yao Zhao, Jun Zhou +2Multi-Head Attention Key-Value Cache Quantization
Prefill/Decode-Aware Evaluation of LLM Inference on Emerging AI Accelerators
Jun 14, 2026Shun Usami, Venkatram Vishwanath, E. Wes BethelLLM Inference Efficiency Accelerator
Re-feeding Is Not Replaying: Measuring Replay Noise in Counterfactual Token-Credit Estimation
Jun 14, 2026Nils MattesonToken-Level Uncertainty Flips
From Rigid to Dynamic: Entropy-Guided Adaptive Inference for Long-Context LLMs
Jun 8, 2026Zhanchao Xu, Haoyang Li, Qingfa Xiao +4LLM Inference Efficiency Dynamic Sparse Attention
Pretrained, Frozen, Still Leaking: Auditing Cross-Encoder Attribute Transfer in EEG Foundation Models
Jun 8, 2026Jianwei TaiElectroencephalography Foundation Models Electroencephalogram
TRADE: Transducer-Augmented Decoder for Speech LLM
Jun 7, 2026Yun Tang, Shanil Puri, Shinji Watanabe +1Large Audio Language Models Large Language Model Decoding
You Only Index Once: Cross-Layer Sparse Attention with Shared Routing
Jun 4, 2026Yutao Sun, Yanqi Zhang, Li Dong +2Efficient Long-Context Inference Dynamic Sparse Attention
Subspace-Aware Sparse Autoencoders for Effective Mechanistic Interpretability
Jun 4, 2026Seyed Arshan Dalili, Mehrdad MahdaviSparse Autoencoders Mechanistic Interpretability
SparDA: Sparse Decoupled Attention for Efficient Long-Context LLM Inference
Jun 3, 2026Yaosheng Fu, Guangxuan Xiao, Xin Dong +2Dynamic Sparse Attention LLM Inference Efficiency
Observation, Not Prediction: Conversation-Level Disaggregated Scheduling for Agentic Serving
Jun 1, 2026Jianru Ding, Ryien Hosseini, Pouya Mahdi Gholami +2Per-Token Latency Prefill
Spatially Distributed Task-Oriented Compression for Multi-Emitter Localization and Characterization with Spectral Overlap
May 31, 2026H. Nazim Bicer, J. Nicholas LanemanSpectrum Sparse Observations
Mitigating Hallucinations in Large Language Models Via Decoder Layer Skipping
May 30, 2026Hanze Li, Jinhao You, Yichen Guo +3Large Language Model Hallucination Mitigating Hallucinations
FlowOVD: Learning Generative Latent Flows for Zero-shot Open-vocabulary Detection
May 30, 2026Yao Wei, Andrea Cavallaro, Changjae OhOpen-Vocabulary Object Detection Open-Vocabulary
Score $\times$ Decoder: A Unified View of Unsupervised Inference-Time Scaling for Hallucination MitigationScore \times Decoder: A Unified View of Unsupervised Inference-Time Scaling for Hallucination Mitigation
May 30, 2026Yun-Chen Cheng, Che-Yu Lin, Cheng-Lin YangLarge Language Model Hallucination Inference-Time
Probing the Prompt KV Cache: Where It Becomes Dispensable
May 28, 2026Vinayshekhar Bannihatti Kumar, Manoj Ghuhan Arivazhagan, Disha Makhija +1Key-Value Cache Compression Key-Value Cache
Memory-Bound but Not Bandwidth-Limited: The Physical AI Inference Gap in Batch-1 LLM Decode
May 28, 2026Josef ChenLLM Inference Efficiency Single-Inference Workloads
The Missing Piece in Pre-trained Model Evaluation: Reward-Guided Decoding Unlocks Task-Oriented Behavior Without Parameter Updates
May 27, 2026Shaobo Wang, Guo Chen, Ziyue Wang +5Large Language Model Post-Training Large Language Model Decoding
QAM-W: Joint 2D Codebook Quantization for LLM Weights via Hadamard Rotation and Activation-Aware Scaling
May 25, 2026Preetam Sharma, Kacper DobekPost-Training Quantization Model Weights
DeCoDrift: Stabilizing Decoder Coupling in Closed-Loop Foundation Segmentation
May 25, 2026H. M. Shadman Tabib, Md. Shamsuzzoha Bayzid, M Sohel RahmanLarge Language Model Decoding Attention-Guided Transunet
Decoding Stimulus Reconstruction-Based Auditory Attention Robustly in Unbalanced EEG Datasets
May 25, 2026Yuanming Zhang, Yayun Liang, Zhibin Lin +1Auditory Attention Electroencephalogram
End-to-End Intracortical Speech Decoding from Neural Activity
May 23, 2026Owais Mujtaba Khanday, Jose A. Gonzalez-Lopez, Marc Ouellet +2Large Language Model Decoding Word Error Rate
MDS-DETR: DETR with Masked Duplicate Suppressor
May 22, 2026Chanho Lee, Seunghee Koh, Yunho Jeon +1Rt-Detr Large Language Model Decoding
Learned Relay Representations for Forward-Thinking Discrete Diffusion Models
May 21, 2026Benjamin Rozonoyer, Jacopo Minniti, Dhruvesh Patel +4Masked Diffusion Language Models Diffusion Language Models
Gated DeltaNet-2: Decoupling Erase and Write in Linear Attention
May 21, 2026Ali Hatamizadeh, Yejin Choi, Jan KautzEfficient Long-Context Inference Large Language Model Decoding

Sep 14, 2026 · cs.LGJ/K move · Enter open · S save

Fixed State, Long Reach: What a Constant-Size Cache Buys Block Diffusion at Scale

Vaibhav Singh, Pierre-André Noël, Torsten Scholak, Eugene Belilovsky+1

Diffusion language models decode tokens in parallel, but their bidirectional denoiser rules out the naive key--value (KV) cache behind fast autoregressive inference. Block diffusion restores caching by decoding block-by-block, and the block caches deployed on it so far are tied to attention: O(L)in memory and, if used as training-free retrofits, only an approximation of the model's computation. Both constraints can be overcome: sequence mixers that summarize finalized blocks into a reusable state support block caching, and the corresponding block-causal training objective makes the cache exact. We study this recipe at scale, pretraining three 3B block-diffusion denoisers (attention, mamba, and hybrid) on 300B tokens under one single-frontier objective and decoding all three through a single cached interface. Only the state-space cache is O(1) in sequence length: its memory and per-step latency stay constant at any context length, while an attention cache remains O(L). At 256k tokens (where attention has grown to 82GB and 29 ms/step), the Mamba cache delivers 4.3x lower latency, 11x less memory, and 2.6x higher single-stream throughput; and because that footprint is constant it scales with batch as well, reaching 14x the aggregate throughput, where attention cannot run beyond a single stream. The same linear-state bias lets the Mamba and hybrid backbones keep retrieving out to 8-16x their training length, whereas attention's retrieval collapses at 2x, at no measured quality cost.

Diffusion Language Models Large Language Model Decoding

Open paper Ask about it PDF