VTC: DNN Compilation with Virtual Tensors for Data Movement Elimination

We present Nautilus, a novel tensor compiler that moves toward fully automated math-to-kernel optimization. Nautilus compiles a high-level algebraic specification of tensor operators into efficient tiled GPU kernels. Nautilus's successive lowering design allows high-level optimizations, expression rewrites, and tile optimizations to be jointly applied in a single end-to-end system. Nautilus presents a novel auto-scheduler that discovers sequences of high-level optimizations, while preserving the regular program structure needed by tile optimizers. Nautilus's auto-scheduler captures complex interactions and trade-offs in the high-level optimizations, including aggressive global transformations like advanced reduction fusion. Nautilus is the first end-to-end tensor compiler capable of starting from a math-like description of attention and automatically discovering FlashAttention-3-like kernels, offloading the entire burden of optimization from the programmer to the compiler. Across five transformer-based models and 150 evaluation configurations on NVIDIA GH200 and RTX 5090 GPUs, Nautilus achieves up to 23% higher throughput than state-of-the-art compilers on GH200 and up to 42% on RTX 5090, while matching or exceeding manually written cuDNN kernels on many long-sequence configurations.

High-performance Host processors can integrate Processing-In-Memory (PIM) devices, which can accelerate memory-intensive kernels of Machine Learning (ML) models, including Large Language Models (LLMs), by leveraging the large memory bandwidth available at PIM cores. However, Host processor needs consecutive elements distributed across DRAM banks, while PIM cores need consecutive elements within their local banks. This necessitates data rearrangements in ML kernel execution that pose significant performance and programmability challenges, further exacerbated by the need to support diverse PIM devices. Current compilation approaches lack systematic optimization for diverse ML kernels and multiple PIM devices, and may largely ignore data rearrangement costs during the compute code optimization step. We show that data rearrangements and compute code optimization are interdependent, and need to be jointly optimized during the tuning process. Therefore, we design DCC, the first data-centric ML compiler for PIM systems that jointly co-optimizes data rearrangements and compute code in a unified tuning process. DCC integrates a multi-layer PIM abstraction to support multiple PIM backends. DCC enables effective co-optimization of data partitioning strategies with compute loop partitioning schemes. DCC applies PIM-specific code optimizations, and leverages a fast and accurate performance prediction model to select the bestperforming code schedule for a given kernel on a target PIM architecture. Our evaluations in various individual ML kernels show that DCC achieves up to 7.68x speedup (2.21x average) on HBM-PIM, and up to 13.17x speedup (3.92x average) on AttAcc PIM, over GPU-only execution. In end-to-end LLM inference, DCC on AttAcc accelerates GPT-3 and LLaMA-2 by 4.52x average (up to 7.71x in LLaMA-2) over GPU. DCC is open-sourced at https://github.com/SPIN-Research-Group/DCC.

Large deep neural networks are costly to store and deploy because inference must move and evaluate many parameters. This paper studies \emph{Automatically Differentiable Nonlinear Tensor Networks} (ADNTNs), compact differentiable weight generators for replacing selected dense, convolutional, and attention layers. An ADNTN maps a small set of trainable tensor cores to a full weight tensor through hierarchical contractions and learnable nonlinearities; the generated layer is then used as an ordinary linear or convolutional operator. We investigate three multilayered topologies: Tree Tensor Networks, augmented Tree Tensor Networks with boundary disentanglers, and MERA-style multi-scale decoders. Compared with flat brick-wall automatically differentiable tensor networks, these hierarchies provide logarithmic-depth communication between tensorised modes and optional lateral mixing, which can improve long-range structure without large increases in stored parameters. We give a unified forward--adjoint formulation showing how reverse-mode automatic differentiation computes pre-activation adjoints and contracted-environment gradients for all trainable cores. The formulation supports task losses, reconstruction losses, distillation, quantisation-aware terms, batching, and modern optimisers. Proof-of-concept experiments on selected AlexNet and VGG-16 layers on CIFAR-10 datasets achieve per-layer parameter-compression ratios from about $2{,}000\times$ to $430{,}000\times$. Several VGG-16 compressed models match or slightly exceed the dense baseline, whereas AlexNet shows moderate degradation under more restrictive redundancy. These results indicate that nonlinear tensor-network generators are a promising structured route to compact pattern-recognition models, while also showing that contraction schedules and hardware-aware implementations remain essential for practical speedups.