Function Space

Function space research explores the properties of spaces where the objects of study are functions, rather than vectors or scalars, aiming to understand how functions are represented and manipulated within machine learning and scientific computing. Current research focuses on characterizing the geometry and properties of function spaces generated by neural networks, particularly deep ReLU networks and attention mechanisms, and developing algorithms for learning and approximating functions within these spaces, including Bayesian methods and neural operators. This work has significant implications for improving the efficiency and accuracy of machine learning models, particularly in areas like scientific data analysis, generative modeling, and reinforcement learning, by enabling more sophisticated representations and algorithms.

Papers

October 28, 2024

Detection of moving objects through turbulent media. Decomposition of Oscillatory vs Non-Oscillatory spatio-temporal vector fields
Jerome Gilles, Francis Alvarez, Nicholas B. Ferrante, Margaret Fortman, Lena Tahir, Alex Tarter, Anneke von Seeger
Data Detection Vector Field Function Space Non Converging Artificial Oscillation Moving Object Cartoon Texture Medium STORM

September 23, 2024

Embedding Knowledge Graph in Function Space
Louis Mozart Kamdem Teyou, Caglar Demir, Axel-Cyrille Ngonga Ngomo
Knowledge Graph Jina Embeddings Multi Layer Entity Representation Function Space Better Representation

September 2, 2024

On the optimal approximation of Sobolev and Besov functions using deep ReLU neural networks
Yunfei Yang
Neural Network Average Approximation Function Space Approximation Rate Sparse Vector Sobolev Space

August 30, 2024

Geometry of Lightning Self-Attention: Identifiability and Dimension
Nathan W. Henry, Giovanni Luca Marchetti, Kathlén Kohn
Geometric Analysis Box Dimension Novel Identifiability Test Function Space Self Attention Network

August 16, 2024

Error Bounds for Learning Fourier Linear Operators
Unique Subedi, Ambuj Tewari
Fourier Neural Operator Low Rank Approximation Error Bound Function Space Discretization Error

August 2, 2024

Autoencoders in Function Space
Justin Bunker, Mark Girolami, Hefin Lambley, Andrew M. Stuart, T. J. Sullivan
Generative Model Supervised Autoencoder Variational Autoencoder Function Space

June 3, 2024

Learning the Target Network in Function Space
Kavosh Asadi, Yao Liu, Shoham Sabach, Ming Yin, Rasool Fakoor
Reinforcement Learning Deep Reinforcement Learning Value Function Function Space Adaptive Network Model Equivalence Goal Node

June 1, 2024

Cross-Table Pretraining towards a Universal Function Space for Heterogeneous Tabular Data
Jintai Chen, Zhen Lin, Qiyuan Chen, Jimeng Sun
Cross Dataset Function Space Tabular Learning Heterogeneous Tabular Data Tabular Prediction Task Cross Table

May 31, 2024

Stochastic Optimal Control for Diffusion Bridges in Function Spaces
Byoungwoo Park, Jungwon Choi, Sungbin Lim, Juho Lee
Diffusion Model Diffusion Based Generative Model Function Space Diffusion Bridge Stochastic Optimal Control Time Series Diffusion

May 10, 2024

Approximation Error and Complexity Bounds for ReLU Networks on Low-Regular Function Spaces
Owen Davis, Gianluca Geraci, Mohammad Motamed
Neural Network Activation Function ReLU Network Function Space Approximation Error Complexity Bound

April 28, 2024

IMEX-Reg: Implicit-Explicit Regularization in the Function Space for Continual Learning
Prashant Bhat, Bharath Renjith, Elahe Arani, Bahram Zonooz
Continual LEArning Generalization Performance Consistency Regularization Function Space Explicit Regularization Recency Bias

April 3, 2024

Composite Bayesian Optimization In Function Spaces Using NEON -- Neural Epistemic Operator Networks
Leonardo Ferreira Guilhoto, Paris Perdikaris
Machine Learning Model Deep Ensemble Operator Learning Function Space Epistemic Neural Network

March 16, 2024

Function-space Parameterization of Neural Networks for Sequential Learning
Aidan Scannell, Riccardo Mereu, Paul Chang, Ella Tamir, Joni Pajarinen, Arno Solin
Neural Network Continual LEArning Gradient Based Sequential Learning Function Space

March 13, 2024

Neural reproducing kernel Banach spaces and representer theorems for deep networks
Francesca Bartolucci, Ernesto De Vito, Lorenzo Rosasco, Stefano Vigogna
Deep Network Neural Topic Kernel Hilbert Space Function Space Kernel Banach Space Representer Theorem

November 15, 2023

Value FULCRA: Mapping Large Language Models to the Multidimensional Spectrum of Basic Human Values
Jing Yao, Xiaoyuan Yi, Xiting Wang, Yifan Gong, Xing Xie
Net Present Value Value Alignment Human Value Vector Valued Function Space Spectral Energy Distribution

November 12, 2023

How do Minimum-Norm Shallow Denoisers Look in Function Space?
Chen Zeno, Greg Ongie, Yaniv Blumenfeld, Nir Weinberger, Daniel Soudry
Neural Network Function Space Linear Interpolation

October 27, 2023

Lipschitz and H\"older Continuity in Reproducing Kernel Hilbert Spaces
Christian Fiedler
Reproducing Kernel Hilbert Space Small Kernel Continuous Speech Function Space Lipschitz Operator H\"older Function Regularity Condition

September 18, 2023

Neural Feature Learning in Function Space
Xiangxiang Xu, Lizhong Zheng
Deep Feature Feature Representation Geometric Feature Function Space Optimal Feature

August 17, 2023

Tipping Point Forecasting in Non-Stationary Dynamics on Function Spaces
Miguel Liu-Schiaffini, Clare E. Singer, Nikola Kovachki, Tapio Schneider, Kamyar Azizzadenesheli, Anima Anandkumar
Non Stationary Function Space Tipping Point Point Forecasting Climate Tipping Point

July 25, 2023

Modify Training Directions in Function Space to Reduce Generalization Error
Yi Yu, Wenlian Lu, Boyu Chen
Neural Tangent Kernel Generalization Error Function Space Natural Gradient Descent