Last Layer Retraining

Last-layer retraining (LLR) is a technique for improving the robustness and fairness of machine learning models by selectively retraining only the final classification layer. Current research focuses on mitigating the reliance on spurious correlations in models like CLIP, enhancing robustness to noisy labels, and reducing the need for extensive annotated datasets through methods such as text-based retraining or leveraging object detection scores. This efficient approach offers significant potential for improving model generalization and fairness across various applications, particularly in scenarios with limited data or computational resources.