Model Pruning

Model pruning aims to reduce the computational cost and memory footprint of large neural networks by removing less important parameters, while preserving or even improving performance. Current research focuses on developing efficient one-shot pruning methods, particularly for large language models (LLMs) and vision transformers (ViTs), often incorporating techniques like gradient-based importance scoring, block-aware optimization, and prompt-based approaches. These advancements are crucial for deploying sophisticated AI models on resource-constrained devices and improving the efficiency of training and inference, impacting both scientific understanding of model architectures and practical applications across various domains.