Post Hoc Orthogonalization

Post-hoc orthogonalization is a technique used to improve the properties of learned representations in machine learning models, primarily aiming to enhance disentanglement, reduce bias, and improve generalization. Current research focuses on applying this technique to various model architectures, including neural networks and knowledge graph embeddings, often incorporating contrastive learning or other regularization methods to achieve orthogonal feature spaces. This approach holds significance for mitigating biases in sensitive applications like healthcare and improving the efficiency and performance of various machine learning tasks, particularly in scenarios with limited data or complex relationships between features.