In Scientific Computing and modern Machine Learning (ML) workloads, sequences of dependent General Matrix Multiplications (GEMMs) often dominate execution time. While state-of-the-art BLAS libraries aggressively optimize individual GEMM calls, they remain constrained by the BLAS API, which requires each call to independently pack input matrices and restore outputs to a canonical memory layout. In sequential GEMMs, these constraints cause redundant packing and unpacking, wasting valuable computational resources. This paper introduces LP-GEMM, a decomposition of the GEMM kernel that enables packing-layout propagation across sequential GEMM operations. This approach eliminates unnecessary data repacking while preserving full BLAS semantic correctness at the boundaries. We evaluate LP-GEMM on x86 (AVX-512) and RISC-V (RVV 1.0) architectures across MLP-like and Attention-like workloads. Our results show average speedups of 2.25x over OpenBLAS on Intel x86 for sequential GEMMs and competitive gains relative to vendor-optimized libraries such as Intel MKL. We demonstrate the practicality of the approach beyond microbenchmarks by implementing a standalone C++ version of the Llama-3.2 inference path using exclusively BLAS-level GEMM calls. These results confirm that leveraging data layout propagation between operations can significantly boost performance.
Figures & tables
Fig. 1 : Comparison of OpenBLAS and LP-GEMM kernel execution of consecutive GEMM operations. The relative block sizes do not represent execution time. Weight packing is omitted for clarity.
Fig. 2 : Visual representation of GotoBLAS approach [ 6 ] . (a) shows how matrices are tiled for the architecture’s memory hierarchy, while (b) depicts how each block At and Bt is further tiled and packed for better usage of the processor’s registers during CT computing of µKernel. Finally, (c) shows the final GotoBLAS GEMM algorithm.
Fig. 3 : Overview of sequential GEMM operations using different data layouts. The results are the same as using OpenBLAS with column-major or row-major, respectively.
Fig. 4 : Micro-kernel Layout.
CPU info
Intel Xeon Gold 6252
Spacemit X60
CPU Family
Skylakex (x86-64)
RV64GCVB
Vector Extension
AVX512
RVV1.0
mc,nc and kc
448, 16384, 448
128, 16385, 128
mr and nr
16, 4
16, 8
L1 cache size (KiB)
32
32
L2 cache size (KiB)
1024
512 shared
TABLE I : Summary of evaluated systems.
Fig. 5 : Speedup of a single GEMM extracted from gemmbench [ 16 ] using different state-of-art kernels compared to the three separate LP-GEMM kernels.
Fig. 6 : Speedup of Attention layer from LLaMA 3.2 model, with embedded dimension of 2048, and MLP weights with dimension of 8129. Comparing LP-GEMM and layout propagation to OpenBLAS with no propagation.
Fig. 7 : Usage comparison of LP-GEMM with FlashGEMM and OpenBLAS, using benchmarks extracted from DNN [ 27 ] .