Kernel Banach Space

Reproducing Kernel Banach Spaces (RKBSs) offer a powerful mathematical framework for analyzing neural networks, moving beyond the limitations of traditional Reproducing Kernel Hilbert Spaces (RKHSs) by accommodating the sparsity and non-Hilbert space properties of practical network architectures. Current research focuses on applying RKBS theory to deep networks, including graph convolutional networks, and establishing representer theorems that justify the use of finite-width architectures. This theoretical advancement provides a deeper understanding of neural network function spaces, leading to insights into generalization, optimization, and network compression techniques.

Papers

October 14, 2024

Which Spaces can be Embedded in $\mathcal{L}_p$-type Reproducing Kernel Banach Space? A Characterization via Metric Entropy
Yiping Lu, Daozhe Lin, Qiang Du
New Characterization Kernel Hilbert Space Mixed Space Banach Space Kernel Banach Space Information Complexity Metric Entropy

August 9, 2024

Decomposition of one-layer neural networks via the infinite sum of reproducing kernel Banach spaces
Seungcheol Shin, Myungjoo Kang
Microbial Decomposition Feature Space Partial Sum Layer Neural Network Kernel Banach Space

March 13, 2024

Neural reproducing kernel Banach spaces and representer theorems for deep networks
Francesca Bartolucci, Ernesto De Vito, Lorenzo Rosasco, Stefano Vigogna
Deep Network Neural Topic Kernel Hilbert Space Function Space Kernel Banach Space Representer Theorem

March 5, 2024

Hypothesis Spaces for Deep Learning
Rui Wang, Yuesheng Xu, Mingsong Yan
Deep Learning Deep Neural Network Banach Space Hypothesis Space Kernel Banach Space

November 6, 2023

Barron Space for Graph Convolution Neural Networks
Seok-Young Chung, Qiyu Sun
Kernel Hilbert Space Graph Convolutional Neural Network Graph Domain Neural Kernel Kernel Banach Space Barron Space

October 5, 2023

Function-Space Optimality of Neural Architectures With Multivariate Nonlinearities
Rahul Parhi, Michael Unser
Neural Architecture Shallow Neural Network Kernel Banach Space Multivariate System Space Optimality

May 25, 2023

Variation Spaces for Multi-Output Neural Networks: Insights on Multi-Task Learning and Network Compression
Joseph Shenouda, Rahul Parhi, Kangwook Lee, Robert D. Nowak
Multi Task Learning DCU Insight AQ Vector Valued Network Compression Decay Regularization Kernel Banach Space Variation Space

May 21, 2023

Sparse Representer Theorems for Learning in Reproducing Kernel Banach Spaces
Rui Wang, Yuesheng Xu, Mingsong Yan
LeArning Abstract Sparsity Increase Kernel Banach Space Sparse Kernel Representer Theorem Sparse Learning Method

February 1, 2023

Gradient Descent in Neural Networks as Sequential Learning in RKBS
Alistair Shilton, Sunil Gupta, Santu Rana, Svetha Venkatesh
Neural Network Gradient Descent Neural Tangent Kernel Kernel Hilbert Space Sequential Learning Generalization Property Taylor Expansion Weak Convergence Kernel Banach Space

November 9, 2022

Duality for Neural Networks through Reproducing Kernel Banach Spaces
Len Spek, Tjeerd Jan Heeringa, Felix Schwenninger, Christoph Brune
Neural Network Primal Dual Reproducing Kernel Hilbert Space Strong Duality Kernel Banach Space

Kernel Banach Space

Papers

Which Spaces can be Embedded in $\mathcal{L}_p$-type Reproducing Kernel Banach Space? A Characterization via Metric Entropy

Decomposition of one-layer neural networks via the infinite sum of reproducing kernel Banach spaces

Neural reproducing kernel Banach spaces and representer theorems for deep networks

Hypothesis Spaces for Deep Learning

Barron Space for Graph Convolution Neural Networks

Function-Space Optimality of Neural Architectures With Multivariate Nonlinearities

Variation Spaces for Multi-Output Neural Networks: Insights on Multi-Task Learning and Network Compression

Sparse Representer Theorems for Learning in Reproducing Kernel Banach Spaces

Gradient Descent in Neural Networks as Sequential Learning in RKBS

Duality for Neural Networks through Reproducing Kernel Banach Spaces