Sparse Sub Model

Sparse sub-model research aims to drastically reduce the computational cost and memory footprint of deep neural networks without significant performance loss. Current efforts focus on identifying and training highly sparse sub-networks within existing architectures like diffusion models and various convolutional and graph neural networks, often employing techniques inspired by the Lottery Ticket Hypothesis. This work is driven by the need to deploy complex models on resource-constrained devices (e.g., edge devices, IoT) and improve training efficiency for large models, leading to significant advancements in both model efficiency and federated learning.