Linear Function Approximation

Linear function approximation aims to efficiently represent value functions in reinforcement learning and other sequential decision-making problems by using linear combinations of features. Current research emphasizes developing provably efficient algorithms for various settings, including those with off-policy learning, delayed feedback, heavy-tailed rewards, and multiple agents, often employing techniques like temporal difference learning, fitted Q-iteration, and natural policy gradients. These advancements address the computational challenges of large state spaces and improve the theoretical understanding and practical performance of reinforcement learning algorithms, impacting fields like robotics, control systems, and AI safety.

Papers

January 30, 2023

January 26, 2023

Multi-Agent Congestion Cost Minimization With Linear Function Approximations
Prashant Trivedi, Nandyala Hemachandra
Multi Agent Reinforcement Learning Multiple Agent Linear Function Approximation Value Iteration Stochastic Shortest Path

November 28, 2022

Provably Efficient Model-free RL in Leader-Follower MDP with Linear Function Approximation
Arnob Ghosh
High Efficiency Bandit Feedback Model Free Markov Game Single Agent Linear Function Approximation Linear MDPs

November 23, 2022

On Instance-Dependent Bounds for Offline Reinforcement Learning with Linear Function Approximation
Thanh Nguyen-Tang, Ming Yin, Sunil Gupta, Svetha Venkatesh, Raman Arora
Offline Reinforcement Learning Information Theoretic Linear Function Approximation Function Approximation Instance Dependent

November 14, 2022

Linear Reinforcement Learning with Ball Structure Action Space
Zeyu Jia, Randy Jia, Dhruv Madeka, Dean P. Foster
Reinforcement Learning Optimal Policy Action Space Linear Function Approximation Sample Efficient Action Value Function

October 20, 2022

Monotonic Risk Relationships under Distribution Shifts for Regularized Risk Minimization
Daniel LeJeune, Jiayu Liu, Reinhard Heckel
Distribution Shift Covariate Shift Linear Function Approximation Linear Inverse Problem Distribution Performance Monotonic Representation Ridge Regularized

October 13, 2022

Reinforcement Learning with Unbiased Policy Evaluation and Linear Function Approximation
Anna Winnicki, R. Srikant
Reinforcement Learning Markov Decision Process Stochastic Approximation Linear Function Approximation Policy Iteration

October 12, 2022

Finite time analysis of temporal difference learning with linear function approximation: Tail averaging and regularisation
Gandharv Patil, Prashanth L.A., Dheeraj Nagaraj, Doina Precup
Regularization Model Linear Function Approximation Finite Time Temporal Difference Polyak Ruppert

October 3, 2022

September 22, 2022

Proximal Point Imitation Learning
Luca Viano, Angeliki Kamoutsi, Gergely Neu, Igor Krawczuk, Volkan Cevher
Proximal Policy Optimization Linear Function Approximation Expert Trajectory Proximal Point Imitation Game

September 14, 2022

Distributionally Robust Offline Reinforcement Learning with Linear Function Approximation
Xiaoteng Ma, Zhipeng Liang, Jose Blanchet, Mingwen Liu, Li Xia, Jiheng Zhang, Qianchuan Zhao, Zhengyuan Zhou
Reinforcement Learning Optimal Policy Robust Policy Linear Function Approximation Robust Offline Reinforcement Learning

July 28, 2022

Exploiting and Defending Against the Approximate Linearity of Apple's NeuralHash
Jagdeep Singh Bhatia, Kevin Meng
Linear Function Approximation Black Box Attack Hash Function Image Hashing Perceptual Hashing

July 18, 2022

A Few Expert Queries Suffices for Sample-Efficient RL with Resets and Linear Value Approximation
Philip Amortila, Nan Jiang, Dhruv Madeka, Dean P. Foster
Sample Complexity Optimal Policy Feature Imitation Linear Function Approximation Sample Efficient Sample Efficient Reinforcement Learning Global Reset Feature

June 23, 2022

June 1, 2022

May 31, 2022

Nearly Minimax Optimal Offline Reinforcement Learning with Linear Function Approximation: Single-Agent MDP and Markov Game
Wei Xiong, Han Zhong, Chengshuai Shi, Cong Shen, Liwei Wang, Tong Zhang
Offline Reinforcement Learning Markov Game Single Agent Linear Function Approximation Offline Multi Agent Reinforcement Learning Tabular Markov Decision Process Minimax Optimal Algorithm

Linear Function Approximation

Papers

Improved Regret for Efficient Online Reinforcement Learning with Linear Function Approximation

Refined Regret for Adversarial MDPs with Linear Function Approximation

Multi-Agent Congestion Cost Minimization With Linear Function Approximations

Provably Efficient Model-free RL in Leader-Follower MDP with Linear Function Approximation

On Instance-Dependent Bounds for Offline Reinforcement Learning with Linear Function Approximation

Linear Reinforcement Learning with Ball Structure Action Space

Monotonic Risk Relationships under Distribution Shifts for Regularized Risk Minimization

Reinforcement Learning with Unbiased Policy Evaluation and Linear Function Approximation

Finite time analysis of temporal difference learning with linear function approximation: Tail averaging and regularisation

Bias and Extrapolation in Markovian Linear Stochastic Approximation with Constant Stepsizes

Near-Optimal Deployment Efficiency in Reward-Free Reinforcement Learning with Linear Function Approximation

Proximal Point Imitation Learning

Distributionally Robust Offline Reinforcement Learning with Linear Function Approximation

Exploiting and Defending Against the Approximate Linearity of Apple's NeuralHash

A Few Expert Queries Suffices for Sample-Efficient RL with Resets and Linear Value Approximation

Provably Efficient Model-Free Constrained RL with Linear Function Approximation

Nearly Minimax Optimal Reinforcement Learning with Linear Function Approximation

Stabilizing Q-learning with Linear Architectures for Provably Efficient Learning

Provably Efficient Lifelong Reinforcement Learning with Linear Function Approximation

Nearly Minimax Optimal Offline Reinforcement Learning with Linear Function Approximation: Single-Agent MDP and Markov Game