Logit Vector

Logit vectors, representing the pre-softmax outputs of neural networks, are increasingly central to improving model performance and interpretability across various machine learning tasks. Current research focuses on leveraging logits for addressing challenges like long-tailed data distributions (where some classes are under-represented), improving test-time adaptation in the face of distribution shifts, and enhancing the robustness of federated learning. These efforts involve modifying loss functions, adjusting logits directly (e.g., through group-wise or class-level perturbations), and employing techniques like logit switching and similarity distillation to improve model accuracy and generalization. The insights gained from studying logits are impacting diverse applications, from action segmentation in videos to more robust and reliable classification in general.