LLM Compression

LLM: Large Language Model

Overview

LLM compression studies how to make language models smaller or cheaper to run while retaining useful abilities. Common approaches include low-bit quantization, pruning weights, layers, or experts, low-rank factorization, distillation into smaller models, and compressing prompts or KV caches. Methods increasingly use calibration data, sensitivity estimates, or adaptive budget allocation, often paired with specialized inference kernels to turn compact representations into practical speedups. A central challenge is that quality, reasoning, reliability, and actual runtime can change in different ways: fewer parameters or tokens do not always mean faster execution or preserved behavior.

Latest papers

CardsList

301 papers25 papers in the last four weeks, down 42% on the four weeks before. 0.2% of all new papers.

  1. Beyond Size: How Gradients Shape Pruning Decisions in Large Language Models

    Nov 8, 2023Rocktim Jyoti Das, Mukul Ranjan, Mingjie Sun +2LVLM PruningLLM Pruning