Linear Markov decision processes (MDPs) represent rewards or transition dynamics with unknown parameters that are linear in known features, making learning possible without enumerating every state and action. Researchers develop methods such as optimistic least-squares value iteration, model-based planning, actor-critic, and primal-dual algorithms for episodic, constrained, and changing environments. Guarantees are commonly measured by regret or the samples needed to learn a near-optimal policy, including settings with large or continuous state and action spaces. A central tension is that strong guarantees often rely on realizability or access to computational oracles, while lower bounds reveal unavoidable costs in feature dimension, horizon, and reward gaps.
Independent Researcher · Department of Mathematics, University of Southern California · Department of Mathematics, University of California Los Angeles +1
School of Civil and Environmental Engineering, Cornell University, Ithaca, NY, USA · Dept. of Electrical and Electronic Engineering, Imperial College London, London SW7 2AZ, UK · Dept. of Electronic Systems, Aalborg University, Denmark +2
Department of Statistics, The Pennsylvania State University · Department of Applied and Computational Mathematics and Statistics, University of Notre Dame
Department of Statistics and Data Science, National University of Singapore · Department of Neurology, University of Pittsburgh · Department of Biostatistics, Harvard T.H. Chan School of Public Health +2