Orthogonal prOjection

Orthogonal projection, a mathematical technique for decomposing vectors or data into orthogonal subspaces, is finding increasing application across diverse fields. Current research focuses on improving its efficiency and robustness in various contexts, including neural network training (e.g., using Approximated Orthogonal Projection Units or scaled gradient projection methods), continual learning (via null-space projections or key-query orthogonal projections), and signal processing (e.g., enhanced Orthogonal Matching Pursuit). These advancements are leading to improved model stability, reduced computational complexity, and enhanced performance in applications ranging from soft sensors and image recognition to spiking neural networks and knowledge distillation.