4-Bit Quantization

Four-bit quantization studies how to run or train neural networks with compact INT4 or FP4 values, reducing memory use and data movement for deployment. Post-training methods use calibration, learned scales or codebooks, rotations, and outlier or residual compensation; quantization-aware training instead adapts model parameters to low precision during training. Work spans large language models, vision and diffusion models, and inference components such as attention and KV caches, with evaluations measuring accuracy alongside latency, throughput, and memory. Many systems now approach higher-precision quality in specific settings, but results remain sensitive to activation outliers, calibration data, model architecture, and hardware format, so robustness and practical speedups are still key challenges.