Kernel Alignment

Kernel alignment (KA) quantifies the similarity between the feature representations learned by different models or layers within a model, primarily using metrics like Centered Kernel Alignment (CKA). Current research focuses on improving KA's accuracy and robustness, addressing biases, and applying it to diverse applications such as knowledge distillation, model architecture refinement, and evaluating the representational similarity of vision and language models. KA's ability to diagnose model behavior, guide model training, and facilitate cross-model comparisons makes it a valuable tool for understanding and improving deep learning models across various domains.