Paper ID: 2406.19657

LLMEasyQuant -- An Easy to Use Toolkit for LLM Quantization

Dong Liu, Meng Jiang, Kaiser Pister

Currently, there are many quantization methods appeared for LLM quantization, yet few are user-friendly and easy to be deployed locally. Packages like TensorRT and Quantohave many underlying structures and self-invoking internal functions, which are not conducive to developers' personalized development and learning for deployment. Therefore, we develop LLMEasyQuant, it is a package aiming to for easy quantization deployment which is user-friendly and suitable for beginners' learning.

Submitted: Jun 28, 2024