Tensor Programs

Ask about this topic

Recent momentum

-75%

2 papers in the last 28 days · 0.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-07

2 new papers

A weekly snapshot of new work published in Tensor Programs.

54 papers

Latest in Tensor Programs

Open your feed →
CardsList

  1. Hardware-Aware FP4 FlashAttention-4

    Sep 3, 2026Robert HuFp8Tensor Programs

  2. Superposed Latent Autoencoder

    Sep 1, 2026Quanling Zhao, Jiaying Yang, Tianqi Zhang +4AutoencoderLatent Representation

  3. Hand-Written PTX Tensor-Core GEMM Kernels: A Multi-Precision Study on NVIDIA L4

    Aug 10, 2026Matt J. Borowski, Blazej OsinskiTensor ProgramsMulti-Gpu Systems

  4. SwiftQK: Fast and Communication-Efficient Tensor Parallelism for Query-Key Normalization

    Aug 10, 2026Gyudong Kim, Wonjun Han, Young Geun KimTensor ProgramsBatch Normalization

  5. Tensor Probabilistic Model Checking of Finite-Horizon Markov Chains (Extended Version)

    Aug 1, 2026Jianlin Li, Nick Guo, Peter Ye +1Probabilistic Model CheckingMarkov

  6. It's All Just Vectorization: einx, a Universal Notation for Tensor Operations

    Jul 30, 2026Florian Fervers, Sebastian Bullinger, Christoph Bodensteiner +1Tensor ProgramsVectorization

  7. At-the-Roofline Sparse Tensor Contractions on Vector Processors for Transformer Inference

    Jul 28, 2026Bowen Wang, Chi Zhang, Diyou Shen +3Tensor ProgramsVectorization

  8. X-Stage: Modeling Post-Issue Backpressure in GPU Communication--Computation Fusion

    Jul 25, 2026Jianwen Xian, Zhiyuan Xu, Yuchen Li +9Tensor ProgramsGraphics Processing Unit Kernels

  9. BaseRT: Advancing Best-in-Class LLM Inference with Apple M5 Neural Accelerators

    Jul 21, 2026Fabian Waschkowski, Prabod Rathnayaka, Lukas WesemannLLM Inference EfficiencyAccelerator

  10. Sobek: Streaming Equivariant Tensor Product Convolutions

    Jul 20, 2026Vladimir Chorošajev, Cédric BényTensor ProgramsConvolutional

  11. Quantitative Gaussian-Process limits of Tensor Programs

    Jul 7, 2026Andrea Agazzi, Eloy Mosig García, Dario TrevisanGaussian ProcessWasserstein Distance

  12. From Tensor Buffer to Distributed Memory Hierarchy: A Survey of KV Cache Management for LLM Serving

    Jun 30, 2026Jie Li, Tongyang Wang, Yong ChenKey-Value CacheCache

  13. Online TT-ALS for Streaming Tensor Decomposition with Incremental Orthogonalization

    Jun 30, 2026Hiroki Takeda, Yuto Miyatake, Daisuke FurihataLow-Rank Tensor DecompositionTensor Programs

  14. Axon: A Synthesizing Superoptimizer for Tensor Programs

    Jun 24, 2026Akash Kothari, Shaowei Zhu, Daniel Kroening +1Tensor ProgramsAccelerator

  15. Test-Input Generation for Tensor Programs: What Actually Finds Kernel Bugs

    Jun 23, 2026Dipankar SarkarFuzzingTensor Programs

  16. FORGE: Fused On-Register Gradient Elimination for Memory-Efficient LLM Training

    Jun 22, 2026Dikshant Kukreja, Kritarth Prasad, Avinash Anand +6Tensor ProgramsLow-Precision Training

  17. Demystifying Numerical Instability in LLM Inference: Achieving Reproducible Inference for Mission-Critical Tasks with HEAL

    Jun 19, 2026Zhenting Zhu, Lucas Thai, Shan Yu +5Low-Precision FormatsLLM Inference Efficiency

  18. Realizing Native INT8 Compute for Diffusion Transformers on Consumer GPUs: A Fused INT8 GEMM Kernel for Ideogram 4.0

    Jun 12, 2026Ali Asaria, Tony Salomone, Deep GandhiDiffusion TransformersSystematic Fp16 Divergence

  19. Performance Analysis and Optimization of 3D Generative Diffusion Models across GPU Architectures

    Jun 11, 2026Jeeho Ryoo, Yongchan Jung, Muhammad Ali Khaliq +33D Latent Diffusion ModelTensor Programs

  20. Recursive Binding on a Budget: Subspace Carving in Order-p Tensor Memories

    Jun 9, 2026Travis Pence, Daisuke Yamada, Vikas SinghTensorTensor Programs

  21. Toward Compiler World Models: Learning Latent Dynamics for Efficient Tensor Program Search

    Jun 8, 2026Haolin Pan, Lianghong Huang, Xvlin Zhou +2Tensor ProgramsAdaptive Scheduling

  22. APEX4: Efficient Pure W4A4 LLM Inference via Intra-SM Compute Rebalancing

    Jun 7, 2026Hong Guo, Nianhui Guo, Weixing Wang +3Tensor ProgramsLLM Inference Efficiency

  23. MusaCoder: Native GPU Kernel Generation with Full-Stack Training on Moore Threads GPU

    Jun 3, 2026Kun Cheng, Songshuo Lu, Sicong Liao +7Graphics Processing Unit KernelsLarge Language Model Training

  24. Operator Fusion for LLM Inference on the Tensix Architecture

    Jun 3, 2026Qingbo Wu, Ke Li, Wenzhu Wang +3Tensor ProgramsLightweight Convolutional Neural Network Hardware Accelerator

  25. On Efficient Scaling of GNNs via IO-Aware Layers Implementations

    May 29, 2026Daria Fomina, Daniil Krasylnikov, Alexey Boykov +3Graph Neural NetworksGraphics Processing Unit Kernels

  26. Graphical einops: bridging tensor networks and computation graphs

    May 29, 2026Vincent Wang-Maścianica, Nikhil KhatriTensor NetworksTensor Programs

  27. Revisiting Padded Transformer Expressivity: Which Architectural Choices Matter and Which Don't

    May 28, 2026Anej Svete, William Merrill, Ryan Cotterell +1Transformer VariantTransformer Attention

  28. SANA-Streaming: Real-time Streaming Video Editing with Hybrid Diffusion Transformer

    May 28, 2026Yuyang Zhao, Yicheng Pan, Qiyuan He +6Video EditingVideo Diffusion Models

  29. RW-TTT: Batched Serving for Request-Owned Test-Time Training State

    May 27, 2026Jian Yang, Zhizhuo Kou, Yao Tian +4Large Language Model ServingBatch Size

  30. Tensor Memory: Fixed-Size Recurrent State for Long-Horizon Transformers

    May 26, 2026Kabir Swain, Sijie Han, Daniel Karl I. Weidele +2Recurrent StateRecurrent Model

  31. Step-TP: A Grounded, Step-Level Dataset with Chain-of-Thought Reasoning for LLM-Guided Tensor Program Optimization

    May 25, 2026Mengfan Liu, Da Zheng, Junwei Su +1Tensor ProgramsCode Optimization

  32. Tensor Cache: Eviction-conditioned Associative Memory for Transformers

    May 21, 2026Kabir Swain, Sijie Han, Daniel Karl I. Weidele +2Transformer AttentionTensor Programs

  33. A Typed Tensor Language for Federated Learning

    May 20, 2026Theofilos Mailis, Kalliopi-Christina Despotidou, Konstantinos Filippopolitis +6TensorTensor Programs

  34. Sutra: Tensor-Op RNNs as a Compilation Target for Vector Symbolic Architectures

    May 20, 2026Emma LeonhartPytorchTensor Programs

  35. Instant GPU Efficiency Visibility at Fleet Scale

    May 20, 2026Connor Pedersen, Dong H. Ahn, Michel Migdal +2High-Performance ComputingTensor Programs

  36. Multi-Scale Dequant: Eliminating Dequantization Bottleneck via Activation Decomposition for Efficient LLM Inference

    May 13, 2026Lingchao Zheng, Yuwei Fan, Jun Li +5Low-Bit QuantizationSystematic Fp16 Divergence

  37. SceneFactory: GPU-Accelerated Multi-Agent Driving Simulation with Physics-Based Vehicle Dynamics

    May 8, 2026Yicheng Zhu, Yang Chen, Tao Li +1Autonomous Driving SimulationWaymo Open Motion Dataset

  38. StreamIndex: Memory-Bounded Compressed Sparse Attention via Streaming Top-k

    May 4, 2026Jaber Jaber, Osama JaberDynamic Sparse AttentionIndex

  39. Folding Tensor and Sequence Parallelism for Memory-Efficient Transformer Training & Inference

    Apr 29, 2026Vasu Shyam, Anna Golubeva, Quentin AnthonyTensor ProgramsTensor

  40. Block-Wise Differentiable Sinkhorn Attention: Tail-Refinement Gradients with a Gap-Aware Dustbin Bridge

    Apr 28, 2026Dylan FordeSinkhornTensor Programs

  41. QFlash: Bridging Quantization and Memory Efficiency in Vision Transformer Attention

    Apr 28, 2026Sehyeon Oh, Yongin Kwon, Jemin LeeQuantizationVision Transformer

  42. TACO: Efficient Communication Compression of Intermediate Tensors for Scalable Tensor-Parallel LLM Training

    Apr 27, 2026Man Liu, Xingchen Liu, Xingjian Tian +8Tensor ProgramsTensor

  43. Evaluating CUDA Tile for AI Workloads on Hopper and Blackwell GPUs

    Apr 25, 2026Divakar Kumar Yadav, Tian Zhao, Deepak KumarCudaTensor Programs

  44. HGQ-LUT: Fast LUT-Aware Training and Efficient Architectures for DNN Inference

    Apr 24, 2026Chang Sun, Zhiqiang Que, Bakhtiar Zadeh +4Graph Neural Network InferenceField-Programmable Gate Arrays

  45. How Much Cache Does Reasoning Need? Depth-Cache Tradeoffs in KV-Compressed Transformers

    Apr 20, 2026Xiao WangKey-Value Cache CompressionKey-Value Cache

  46. Ragged Paged Attention: A High-Performance and Flexible LLM Inference Kernel for TPU

    Apr 16, 2026Jevin Jiang, Ying Chen, Blake A. Hechtman +2LLM Inference EfficiencyTensor Programs

  47. Prism: Symbolic Superoptimization of Tensor Programs

    Apr 16, 2026Mengdi Wu, Xiaoyu Jiang, Oded Padon +1Tensor ProgramsCode Optimization

  48. Nautilus: An Auto-Scheduling Tensor Compiler for Efficient Tiled GPU Kernels

    Apr 16, 2026Yifan Zhao, Yuchen Yang, Matei Budiu +1Tensor ProgramsGraphics Processing Unit Kernels

  49. VTC: DNN Compilation with Virtual Tensors for Data Movement Elimination

    Feb 11, 2026Muyan Hu, Ahan Gupta, Jiachen Yuan +7Tensor ProgramsSingle-Inference Workloads

  50. DynamiQ: Accelerating Gradient Synchronization using Compressed Multi-hop All-reduce

    Feb 9, 2026Wenchen Han, Shay Vargaftik, Michael Mitzenmacher +1Low-Bit QuantizationTensor Programs

  51. Scalable Synthesis of distributed LLM workloads through Symbolic Tensor Graphs

    Nov 13, 2025Changhai Man, Joongun Park, Hanjiang Wu +3Single-Inference WorkloadsTensor Programs

  52. Correctness Forensics for Batch Speculative Decoding: Diagnosing the Ragged Tensor Problem

    Oct 26, 2025Ranran Haoran Zhang, Soumik Dey, Ashirbad Mishra +3Speculative DecodingBatch Size

  53. FP8 is All You Need (Part 1): Debunking Hardware FP64 as the HPC Holy Grail (Sep 3rd version)

    Date pendingSatoshi MatsuokaHigh-Performance ComputingFp8

  54. FP8 is All You Need (Part 2): Full-FP64 3-D FFT on FP8-Generation Tensor CoresThe Integer-Epilogue Wall and the Minimal Hardware That Would Remove It

    Date pendingSatoshi MatsuokaFp8Tensor Programs