Neural Network Compression

Neural network compression studies how to represent, train, or run models with fewer parameters and less memory or computation, making them practical on devices with limited resources. Common approaches include pruning weights or neurons, quantization, low-rank and tensor factorization, parameter sharing, and knowledge distillation; some methods also compress activations exchanged during distributed training. Work spans small edge models and microcontrollers as well as large language, vision, and diffusion models, evaluated on tasks such as image classification, language modeling, and video coding. A central challenge is that smaller parameter counts or fewer theoretical operations do not always mean faster inference, and aggressive compression can harm accuracy, robustness, or task-specific behavior.