Invariant Layer

Invariant layers in neural networks aim to create models insensitive to certain input transformations, such as rotations, permutations, or translations, improving efficiency and robustness. Current research focuses on designing and analyzing these layers within various architectures, including convolutional neural networks for image and mesh processing, and graph neural networks for structured data, exploring optimal pooling strategies and the interplay between invariant and equivariant components. This work is significant because it enhances the performance and generalizability of machine learning models across diverse applications, particularly in areas like biomedical image analysis and point cloud processing, by reducing the need for extensive data augmentation and simplifying model design.

Papers

September 29, 2023

Benefits of mirror weight symmetry for 3D mesh segmentation in biomedical applications
Vladislav Dordiuk, Maksim Dzhigil, Konstantin Ushenin
Convolutional Neural Network Convolutional Layer Complementary Benefit Biomedical Image Segmentation Biomedical Application Mesh Segmentation Invariant Layer

June 8, 2023

Equivariant vs. Invariant Layers: A Comparison of Backbone and Pooling for Point Cloud Classification
Ashkan Shahbazi, Abihith Kothapalli, Xinran Liu, Robert Sheng, Soheil Kolouri
Geometric Deep Learning Point Cloud Classification Equivariant Map Numerous Cutting Edge Backbone Global Pooling Invariant Layer Equivariant Framework

February 27, 2023

Invariant Layers for Graphs with Nodes of Different Types
Dmitry Rybin, Ruoyu Sun, Zhi-Quan Luo
Graph Drawing Random Node Machine Permutation Different Type Invariant Network Structure Tensor Invariant Layer

January 30, 2023

Equivariant Architectures for Learning in Deep Weight Spaces
Aviv Navon, Aviv Shamsian, Idan Achituve, Ethan Fetaya, Gal Chechik, Haggai Maron
Neural Network LeArning Abstract Pre Trained Network Equivariant Architecture Weight Space Invariant Layer

May 18, 2022

A Classification of $G$-invariant Shallow Neural Networks
Devanshu Agrawal, James Ostrowski
Classification Code Gene Level GNN Shallow Neural Network Invariant Layer

December 23, 2021

Revisiting Transformation Invariant Geometric Deep Learning: Are Initial Representations All You Need?
Ziwei Zhang, Xin Wang, Zeyang Zhang, Peng Cui, Wenwu Zhu
Meaningful Representation Geometric Deep Learning Invariant Network Transformation Invariance Invariant Layer

November 29, 2021

SPIN: Simplifying Polar Invariance for Neural networks Application to vision-based irradiance forecasting
Quentin Paletta, Anthony Hu, Guillaume Arbod, Philippe Blanc, Joan Lasenby
Rotation Invariance Solar Irradiance Spin Direction Neural Network Application Invariant Layer

Invariant Layer

Papers

Benefits of mirror weight symmetry for 3D mesh segmentation in biomedical applications

Equivariant vs. Invariant Layers: A Comparison of Backbone and Pooling for Point Cloud Classification

Invariant Layers for Graphs with Nodes of Different Types

Equivariant Architectures for Learning in Deep Weight Spaces

A Classification of $G$-invariant Shallow Neural Networks

Revisiting Transformation Invariant Geometric Deep Learning: Are Initial Representations All You Need?

SPIN: Simplifying Polar Invariance for Neural networks Application to vision-based irradiance forecasting