Nonlinear Function Approximation

Nonlinear function approximation aims to efficiently represent complex relationships using models beyond simple linear functions, a crucial task in diverse fields like reinforcement learning and machine learning. Current research focuses on developing and analyzing algorithms and architectures, including deep neural networks with various activation functions (e.g., ReLU, Chebyshev polynomials), kernel methods, and spiking neural networks, to improve approximation accuracy and efficiency while addressing challenges like the curse of dimensionality and sample complexity. These advancements are vital for improving the performance and theoretical understanding of machine learning algorithms, particularly in high-dimensional settings where linear models are insufficient, leading to more robust and efficient solutions for real-world problems. The development of provably efficient algorithms with strong theoretical guarantees is a major focus.

Papers

June 21, 2024

Open Problem: Order Optimal Regret Bounds for Kernel-Based Reinforcement Learning
Sattar Vakili
Reinforcement Learning Deep Learning Open Problem Nonlinear Function Approximation Order Optimal Regret Kernel Based Reinforcement Learning

June 8, 2024

Deep Neural Networks are Adaptive to Function Regularity and Data Distribution in Approximation and Estimation
Hao Liu, Jiahui Cheng, Wenjing Liao
Deep Learning Deep Neural Network Estimation Task Average Approximation Data Distribution Deep ReLU Network Nonlinear Function Approximation Geometric Regularity

May 27, 2024

Provably Efficient Reinforcement Learning with Multinomial Logit Function Approximation
Long-Fei Li, Yu-Jie Zhang, Peng Zhao, Zhi-Hua Zhou
Reinforcement Learning Linear Function Approximation O$ Regret Nonlinear Function Approximation Multinomial Logit

May 12, 2024

Chebyshev Polynomial-Based Kolmogorov-Arnold Networks: An Efficient Architecture for Nonlinear Function Approximation
Sidharth SS, Keerthana AR, Gokul R, Anas KP
High Dimensional Kolmogorov Arnold Network Function Approximation Efficient Architecture Learned Function Chebyshev Davidson Algorithm Nonlinear Function Approximation

May 3, 2024

Stochastic Multivariate Universal-Radix Finite-State Machine: a Theoretically and Practically Elegant Nonlinear Function Approximator
Xincheng Feng, Guodong Shen, Jianhao Hu, Meng Li, Ngai Wong
Practical Application Finite State Machine Nonlinear Function Approximation Multivariate Probability Distribution Stochastic Computing Function Generation

October 2, 2023

Pessimistic Nonlinear Least-Squares Value Iteration for Offline Reinforcement Learning
Qiwei Di, Heyang Zhao, Jiafan He, Quanquan Gu
Offline Reinforcement Learning Value Iteration Instance Dependent Regret Nonlinear Function Approximation

July 18, 2023

Approximating nonlinear functions with latent boundaries in low-rank excitatory-inhibitory spiking networks
William F. Podlaski, Christian K. Machens
Network Programming Single Neuron Level Spiking Neuron Spike Based Nonlinear Function Approximation Real Neuron

March 9, 2023

Quantum Splines for Non-Linear Approximations
Antonio Macaluso, Luca Clissa, Stefano Lodi, Claudio Sartori
Quantum Computing Nonlinear Function Approximation Lloyd Algorithm

February 25, 2023

Gauss-Newton Temporal Difference Learning with Nonlinear Function Approximation
Zhifa Ke, Junyu Zhang, Zaiwen Wen
Q Learning Temporal Difference Learning Temporal Difference Nonlinear Function Approximation

February 20, 2023

Reinforcement Learning with Function Approximation: From Linear to Nonlinear
Jihao Long, Jiequn Han
Reinforcement Learning Nonlinear Model Function Approximation Linear Ordered Data Modern Reinforcement Learning Nonlinear Function Approximation

January 30, 2023

Optimal Approximation Complexity of High-Dimensional Functions with Neural Networks
Vincent P. H. Goverse, Jad Hamdan, Jared Tanner
Neural Network High Dimensional Deep ReLU Network Approximation Rate Nonlinear Function Approximation Dimensional Subspace

December 12, 2022

VO$Q$L: Towards Optimal Regret in Model-free RL with Nonlinear Function Approximation
Alekh Agarwal, Yujia Jin, Tong Zhang
Q Learning Sparse Reward Model Free Reinforcement Learning Regret Analysis Q$ Learning Linear MDPs Nonlinear Function Approximation

October 30, 2022

Representation Learning for General-sum Low-rank Markov Games
Chengzhuo Ni, Yuda Song, Xuezhou Zhang, Chi Jin, Mengdi Wang
Representation Learning Low Rank Markov Game Nonlinear Function Approximation General Sum Markov Game Zero Sum

October 10, 2022

Actor-Critic or Critic-Actor? A Tale of Two Time Scales
Shalabh Bhatnagar, Vivek S. Borkar, Soumyajit Guin
Stochastic Approximation Actor Critic Algorithm Policy Iteration Cautionary TALE Time Scale Nonlinear Function Approximation

August 10, 2022

Learning Two-Player Mixture Markov Games: Kernel Function Approximation and Correlated Equilibrium
Chris Junchi Li, Dongruo Zhou, Quanquan Gu, Michael I. Jordan
Nash Equilibrium Markov Game Nonlinear Function Approximation Correlated Equilibrium Kernel Approximation Linear Mixture Markov Decision Process

July 19, 2022

Approximation Power of Deep Neural Networks: an explanatory mathematical survey
Mohammad Motamed
Deep Neural Network Deep Network Back Propagation Deep ReLU Network Nonlinear Function Approximation Approximation Power

March 15, 2022

Non-Linear Reinforcement Learning in Large Action Spaces: Structural Conditions and Sample-efficiency of Posterior Sampling
Alekh Agarwal, Tong Zhang
Action Space Posterior Sampling Sample Efficiency Sample Efficient Reinforcement Learning Nonlinear Function Approximation Structural Damage

February 28, 2022

Provably Efficient Convergence of Primal-Dual Actor-Critic with Nonlinear Function Approximation
Jing Dong, Li Shen, Yinggan Xu, Baoxiang Wang
Reinforcement Learning Primal Dual Actor Critic Algorithm Nonlinear Function Approximation Stochastic Gradient Descent Ascent Primal Dual Reinforcement Learning

Nonlinear Function Approximation

Papers

Open Problem: Order Optimal Regret Bounds for Kernel-Based Reinforcement Learning

Deep Neural Networks are Adaptive to Function Regularity and Data Distribution in Approximation and Estimation

Provably Efficient Reinforcement Learning with Multinomial Logit Function Approximation

Chebyshev Polynomial-Based Kolmogorov-Arnold Networks: An Efficient Architecture for Nonlinear Function Approximation

Stochastic Multivariate Universal-Radix Finite-State Machine: a Theoretically and Practically Elegant Nonlinear Function Approximator

Pessimistic Nonlinear Least-Squares Value Iteration for Offline Reinforcement Learning

Approximating nonlinear functions with latent boundaries in low-rank excitatory-inhibitory spiking networks

Quantum Splines for Non-Linear Approximations

Gauss-Newton Temporal Difference Learning with Nonlinear Function Approximation

Reinforcement Learning with Function Approximation: From Linear to Nonlinear

Optimal Approximation Complexity of High-Dimensional Functions with Neural Networks

VO$Q$L: Towards Optimal Regret in Model-free RL with Nonlinear Function Approximation

Representation Learning for General-sum Low-rank Markov Games

Actor-Critic or Critic-Actor? A Tale of Two Time Scales

Learning Two-Player Mixture Markov Games: Kernel Function Approximation and Correlated Equilibrium

Approximation Power of Deep Neural Networks: an explanatory mathematical survey

Non-Linear Reinforcement Learning in Large Action Spaces: Structural Conditions and Sample-efficiency of Posterior Sampling

Provably Efficient Convergence of Primal-Dual Actor-Critic with Nonlinear Function Approximation