cs.LGJan 29, 2026

LAMP: Look-Ahead Mixed-Precision Inference of Large Language Models

Authors: Stanislav Budzinskiy, Marian Gloser, Tolunay Yilmaz, Ying Hong Tham, Yuanyi Lin, Wenyi Fang, Fan Wu, Philipp Petersen

Organizations: Faculty of Mathematics University of Vienna, Austria · Huawei Technologies

Abstract

Mixed-precision computations are a hallmark of the current stage of AI, driving the progress in large language models towards efficient, locally deployable solutions. This article addresses the floating-point computation of compositionally-rich functions, concentrating on transformer inference. Based on the rounding error analysis of a composition f(g(x))f(g(\mathrm{x})), we provide an adaptive strategy that selects a small subset of components of g(x)g(\mathrm{x}) to be computed more accurately while all other computations can be carried out with lower accuracy. We then explain how this strategy can be applied to different compositions within a transformer and illustrate its overall effect on transformer inference. We study the effectiveness of this algorithm numerically on GPT-2 models and demonstrate that already very low recomputation rates allow for improvements of up to two orders of magnitude in accuracy.

Figures & tables

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Diagonal-Tiled Mixed-Precision Attention for Efficient Low-Bit MXFP Inference

    Apr 5, 2026Yifu Ding, Xinhao Zhang, Jinyang GuoLLM Inference AccelerationGPU Kernel Optimization

  2. Numerical stability analysis of large language models

    Mar 13, 2025Stanislav Budzinskiy, Wenyi Fang, Longbin Zeng +1Transformer InferenceTransformer Attention