Weight Sharing Supernet

Weight-sharing supernets are a powerful technique in neural architecture search (NAS), aiming to efficiently train a single large network encompassing many smaller architectures (subnets). Current research focuses on improving the training stability and accuracy of these supernets, often employing techniques like information-based measurements to select optimal subnets and novel loss functions to address weight sharing issues. This approach significantly accelerates the NAS process, enabling the rapid discovery of efficient and accurate models tailored to various resource constraints (e.g., latency, memory, energy), with applications ranging from mobile devices to large-scale datacenter deployments.