Learned Function

Learned function research focuses on developing methods for neural networks and other machine learning models to effectively approximate and generalize from complex, often high-dimensional functions. Current efforts concentrate on improving model architectures (like Transformers and Kolmogorov-Arnold Networks), incorporating prior knowledge about function properties (e.g., invertibility, Lipschitz continuity), and developing efficient algorithms for learning functions on various data structures (matrices, point clouds, graphs). This field is crucial for advancing numerous applications, from improving the accuracy and efficiency of existing machine learning models to enabling new capabilities in areas like visual question answering, drug discovery, and policy analysis.

Papers

October 25, 2024

LArctan-SKAN: Simple and Efficient Single-Parameterized Kolmogorov-Arnold Networks using Learnable Trigonometric Function
Zhijie Chen, Xinglin Zhang
High Efficiency Kolmogorov Arnold Network MNIST Dataset Learned Function

October 3, 2024

MLP-KAN: Unifying Deep Representation and Function Learning
Yunhong He, Yifeng Xie, Zhengqing Yuan, Lichao Sun
Representation Learning Kolmogorov Arnold Network Deep Representation Learned Function

August 23, 2024

JacNet: Learning Functions with Structured Jacobians
Jonathan Lorraine, Safwan Hossain
Neural Network Learned Function Jacobian Matrix Fast Mapping Input Output

June 19, 2024

LIVE: Learnable In-Context Vector for Visual Question Answering
Yingzhe Peng, Chenduo Hao, Xu Yang, Jiawei Peng, Xinting Hu, Xin Geng
Language Model Context Learning Visual Question Answering Large Multimodal Model Learned Function Context Vector

May 27, 2024

Learning Social Welfare Functions
Kanad Shrikar Pardeshi, Itai Shapira, Ariel D. Procaccia, Aarti Singh
Vector Valued Learned Function Learning Task Social Welfare

May 26, 2024

Chain of Tools: Large Language Model is an Automatic Multi-tool Learner
Zhengliang Shi, Shen Gao, Xiuyi Chen, Yue Feng, Lingyong Yan, Haibo Shi, Dawei Yin, Zhumin Chen, Suzan Verberne, Zhaochun Ren
Large Language Model Side Chain Easy Tool Tool Learning Learned Function

May 13, 2024

Learning functions on symmetric matrices and point clouds via lightweight invariant features
Ben Blum-Smith, Ningyuan Huang, Marco Cuturi, Soledad Villar
Machine Learning Point Cloud Molecular Property Invariant Feature Learned Function Symmetric Positive Definite Matrix Invariant Function

May 12, 2024

Chebyshev Polynomial-Based Kolmogorov-Arnold Networks: An Efficient Architecture for Nonlinear Function Approximation
Sidharth SS, Keerthana AR, Gokul R, Anas KP
High Dimensional Kolmogorov Arnold Network Function Approximation Efficient Architecture Learned Function Nonlinear Function Approximation Chebyshev Davidson Algorithm

March 18, 2024

Compositional learning of functions in humans and machines
Yanli Zhou, Brenden M. Lake, Adina Williams
New Machine Real Human Simple Function Compositional Learning Learned Function Human Like Generalization Function Composition

February 19, 2024

Function Class Learning with Genetic Programming: Towards Explainable Meta Learning for Tumor Growth Functionals
E. M. C. Sijben, J. C. Jansen, P. A. N. Bosman, T. Alderliesten
Meta Learning Genetic Programming Cancer Diagnosis Learned Function Tumor Growth

February 2, 2024

Why do Random Forests Work? Understanding Tree Ensembles as Self-Regularizing Adaptive Smoothers
Alicia Curth, Alan Jeffares, Mihaela van der Schaar
Random Forest Variance Reduction Tree Ensemble Hypothesis Class Learned Function

October 8, 2023

How Graph Neural Networks Learn: Lessons from Training Dynamics
Chenxiao Yang, Qitian Wu, David Wipf, Ruoyu Sun, Junchi Yan
Graph Neural Network Gradient Descent Critical Lesson Training Dynamic Learned Function Kernel Target Alignment

July 30, 2023

On Neural Network approximation of ideal adversarial attack and convergence of adversarial training
Rajdeep Haldar, Qifan Song
Adversarial Attack Adversarial Training Early Stage Convergence Adversarial Loss Learned Function Neural Network Approximation Optimal Attack

July 26, 2023

Fixed Integral Neural Networks
Ryan Kortvelesy
Neural Network First Integral Learned Function Neural Integral Equation

June 23, 2023

A new approach to generalisation error of machine learning algorithms: Estimates and convergence
Michail Loulakis, Charalambos G. Makridakis
Neural Network Deep Learning Estimation Task Early Stage Convergence Novel Approach Machine Learning Algorithm Generalization Error Learned Function Neural Interpolation

May 23, 2023

Reviewing Evolution of Learning Functions and Semantic Information Measures for Understanding Deep Learning
Chenguang Lu
Deep Learning Mutual Information Information Theory Learned Function

April 19, 2023

Beyond Transformers for Function Learning
Simon Segert, Jonathan Cohen
Transformer Megatron Decepticons Inductive Bias Cognitive Science Large Deep Learning Model Learned Function Extrapolation Capability

March 20, 2023

How (Implicit) Regularization of ReLU Neural Networks Characterizes the Learned Function -- Part II: the Multi-D Case of Two Layers with Random First Layer
Jakob Heiss, Josef Teichmann, Hanna Wutte
Neural Network Model Additive Model Generalized Additive Model Learned Function ReLU Neural Network Functional Regularization

February 15, 2023

InfoNCE Loss Provably Learns Cluster-Preserving Representations
Advait Parulekar, Liam Collins, Karthikeyan Shanmugam, Aryan Mokhtari, Sanjay Shakkottai
LeArning Abstract Meaningful Representation Unsupervised Visual Representation Learning Class Loss Learned Function Weak Augmentation

October 5, 2022

Spectral Regularization Allows Data-frugal Learning over Combinatorial Spaces
Amirali Aghazadeh, Nived Rajaraman, Tony Tu, Kannan Ramchandran
Generalization Performance Improved Generalization Learned Function Spectral Regularization Optimal Generalization Frugal Learning High Dimensional Combinatorial