Self Ensemble

Self-ensembling techniques aim to improve model performance and robustness by creating multiple diverse predictions from a single model, rather than training multiple separate models. Current research focuses on efficient self-ensemble methods, often leveraging multi-branch architectures, weight pruning, or stochastic sampling of intermediate model states to generate diverse predictions, and employing techniques like knowledge distillation to improve overall performance. These approaches offer significant advantages in reducing computational costs and improving generalization, particularly in resource-intensive tasks like semantic segmentation and time series classification, while also enhancing robustness to noisy labels and adversarial attacks.