Model-Free RL

RL: Reinforcement Learning

Overview

Model-free reinforcement learning studies how agents can choose actions and improve behavior directly from rewards and experience, which is useful when system dynamics are unknown or costly to model. Common approaches include Q-learning and temporal-difference methods, policy gradients and actor-critic algorithms such as SAC and TD3, plus offline learning from fixed datasets. Research spans continuous control, safety and exploration, multi-agent systems, and average-reward problems, with theory increasingly addressing convergence, regret, and sample complexity. A central tension is improving sample efficiency, robustness, and safety while retaining the simplicity of learning without an explicit dynamics model; predictive representations also blur the boundary between model-free and model-based methods.

Latest papers

CardsList

67 papers6 papers in the last four weeks, down 50% on the four weeks before. 0.0% of all new papers.

  1. Sample-Efficient Ensemble Reinforcement Learning at Scale

    Nov 15, 2026Yoav Feigenbaum, Dmitrii Krylov, Roy FoxReinforcement LearningSample-Efficient RL

  2. Provably Efficient Representation Learning for Low-Rank CMDPs

    Nov 5, 2026Kaixuan Liu, Guojun Xiong, Shengpu Tang +2Primal-Dual RLMarkov Decision Processes

  3. Statistical Foundations of Reinforcement Learning: A Non-Asymptotic Perspective

    Oct 30, 2026Yuxin Chen, Yuejie Chi, Jianqing Fan +3Sample-Efficient RLModel-Based RL

  4. A Model-Free Continuous-Time RL Algorithm from Discrete-Time Off-Policy Data with End-to-End Guarantees

    Oct 20, 2026Chaorui Yao, Yuming Paul Zhang, Lin Yang +1Continuous-Time RLModel-Free RL

  5. Continuous-Time Policy Gradient from Discrete Data: Convergence Analysis and Model-Free Algorithms

    Oct 20, 2026Martin Hernandez, Xinyan Qing, Mo Zhou +1Continuous-Time RLModel-Free RL

  6. Reinforcement Learning with Hawkes-Structured Networks for Market Making

    Oct 12, 2026Fernando Acero, Philipp Plank, Yufei ZhangMarket MakingHawkes Processes

  7. Entropy-Regularized Mean-Field Games with Average-Reward Criterion: Span-Seminorm Contractions and Model-Free Learning

    Oct 11, 2026Gunes Akbas, Naci Saldi, Berkay Anahtarci +1Mean Field GamesModel-Free RL

  8. Visualizing the World Model in an Agent's Value Function

    Oct 9, 2026Shawheen Naderi, Fatemeh Akherati, Yubei ChenWorld ModelsInterpretability in RL

  9. Randomized Transport Maps for Model-Free Policy-Gradient Mean-Field Control

    Oct 8, 2026Adonis Jamal, Samy Mekkaoui, Yadh Hafsi +1Policy Gradient MethodsModel-Free RL

  10. RLDISCOVER: LLM-driven co-evolution of reinforcement learning algorithms

    Oct 6, 2026Haoran Li, Zengle Ge, Xiaomin Yuan +14Automated Algorithm DiscoveryEvolutionary RL

  11. Reachability-Aware Diffusion Policy Optimization

    Oct 5, 2026Hikmet Simsir, Kutay Demiray, Ozgur S. OguzConstrained RLDiffusion Policy

  12. Taylor Representations for Model-Free RL in Networked MDPs

    Oct 4, 2026Salah Chikhi, Abdelhaq Chaoui, Asuman Ozdaglar +1Markov Decision ProcessesRepresentation Learning for RL

  13. Variance-Aware Fine-Grained Gap-Dependent Bounds for Online Reinforcement Learning

    Oct 3, 2026Haochen Zhang, Lingzhou Xue, Zhong ZhengReinforcement LearningModel-Free RL

  14. Q-Learning for Reachability in MEC-Free MDPs

    Oct 1, 2026Lu-Chin Chang, Suguman BansalMarkov Decision ProcessesReinforcement Learning

  15. Deep Epistemic Value Functions for Optimistic Exploration

    Sep 28, 2026Leander Diaz-Bone, Marco Bagatella, Jonas Hübotter +1Value Function EstimationRL Exploration

  16. GraphHCA: Closed-Form Hindsight Credit Assignment for Long-Horizon LLM Agents

    Sep 28, 2026Haodong Zhu, Yangyang Ren, Changbai Li +4Credit Assignment in RLStep-Level Credit Assignment in RL

  17. Lifted Bellman Linear Programming for Offline Reinforcement Learning

    Sep 21, 2026Hyukjun Yang, Jongchan Park, Narim Jeong +1Reinforcement LearningOffline RL

  18. Deep Reinforcement Learning with Buffered Quantile Objectives

    Sep 18, 2026Mohammad Alipour-vaezi, Sajad KhodadadianRisk-Sensitive RLDistributional RL

  19. ProDVI: Programmatic Dynamics Priors for Value Network Initialization

    Aug 6, 2026Xinwei Liu, Junyuan Liang, Jianting Zhang +1Reinforcement LearningLLM-Guided RL

  20. Observation-Grounded Self-Predictive Reinforcement Learning for Visual Continuous Control

    Aug 6, 2026Xinwei Liu, Junyuan Liang, Jianting Zhang +1Predictive Representation LearningContinuous Control

  21. Exact Model-Free Policy Iteration for Co-safe LTL Planning

    Aug 5, 2026Zetong Xuan, Yu WangReinforcement LearningPolicy Iteration

  22. Finite-Time Analysis of Discounted Exponential-Utility Reinforcement Learning

    Aug 3, 2026Ankur Naskar, Vivek T A, Aditya Kumar +2Reinforcement LearningRisk-Sensitive RL

  23. Evaluating Rollout-Verified Reinforcement Fine-Tuning of a Reasoning Model for Multi-Zone Variable-Air-Volume Control

    Jul 30, 2026Takumi Shioda, Kohei Terashima, Tatsuo NagaiLanguage Model-Based ControlModel-Free RL

  24. Physics-Aware End-to-End Deep Reinforcement Learning for Quadcopter Control with Actuator Dynamics

    Jul 28, 2026Ya-Chia Shen, Woei-Leong ChanRL ControlQuadrotor Control

  25. Principled Direction-Free Intrinsic Motivation through Model-Free Epistemic Free-Energy Estimators

    Jul 18, 2026Alireza Furutanpey, Schahram DustdarFree Energy CalculationIntrinsic Reward Methods

  26. Actor-Critic Learning for Extended Mean Field Control with Deterministic Policies

    Jul 13, 2026Ziheng Cheng, Xin Guo, Huyên Pham +1Policy GradientContinuous-Time RL

  27. Can Reinforcement Learning Efficiently Discover Price Manipulation?

    Jul 7, 2026Ioanna-Yvonni Tsaknaki, Andrea Macrì, Fabrizio LilloModel-Free RLAlgorithmic Trading