Paper ID: 2501.07930 • Published Jan 14, 2025
An Adaptive Orthogonal Convolution Scheme for Efficient and Flexible CNN Architectures
Thibaut Boissin (IRIT, ANITI), Franck Mamalet, Thomas Fel, Agustin Martin Picard, Thomas Massena (IRIT), Mathieu Serrurier (IRIT...
TL;DR
Get AI-generated summaries with premium
Get AI-generated summaries with premium
Orthogonal convolutional layers are the workhorse of multiple areas in
machine learning, such as adversarial robustness, normalizing flows, GANs, and
Lipschitzconstrained models. Their ability to preserve norms and ensure stable
gradient propagation makes them valuable for a large range of problems. Despite
their promise, the deployment of orthogonal convolution in large-scale
applications is a significant challenge due to computational overhead and
limited support for modern features like strides, dilations, group
convolutions, and transposed convolutions.In this paper, we introduce AOC
(Adaptative Orthogonal Convolution), a scalable method for constructing
orthogonal convolutions, effectively overcoming these limitations. This
advancement unlocks the construction of architectures that were previously
considered impractical. We demonstrate through our experiments that our method
produces expressive models that become increasingly efficient as they scale. To
foster further advancement, we provide an open-source library implementing this
method, available at this https URL