Model-Based RL

RL: Reinforcement Learning

Ask about this topic

Overview

Model-based reinforcement learning studies how agents can learn or use a model of how actions change the environment, then plan with that model to improve decisions and reduce costly interaction. Common approaches include latent world models that predict future states, Dyna-style learning from imagined rollouts, and model-predictive control or tree search; Bayesian models and ensembles help represent uncertainty, while control barriers and other safety methods constrain actions. Methods are tested on suites such as DeepMind Control, MuJoCo, Atari, and D4RL, as well as robotic manipulation, autonomous driving, and other physical systems, alongside theoretical work on regret and sample complexity. The central tension is that imagined experience can make learning more efficient, but model errors may accumulate over long horizons or be exploited by the policy, so reliable uncertainty estimates, safe planning, and transfer to real systems remain open challenges.

Latest papers

CardsList

245 papers32 papers in the last four weeks, up 10% on the four weeks before. 0.2% of all new papers.

  1. Vision-Language Agents Can Learn in Their Own Latent Worlds

    Oct 27, 2026Haoxi Li, Sikai Bai, Jianfei Ma +5RL for VLMsWorld Models for VLA Models

    Department of Computer Science and Engineering, HKUST, Hong Kong, China · School of Computing, National University of Singapore, Singapore · Department of Computer Science, University of Southern California, Los Angeles, USA

  2. Frequency-Uniform Log-Regret Reinforcement Learning for Diffusions

    Oct 27, 2026Jose Blanchet, Lutong Hao, Renyuan XuRegret Minimization in RLModel-Based RL

    Department of Management Science and Engineering, Stanford University.

  3. Dream to Assist: Intent-Conditioned World Models for Shared Vehicle Control

    Oct 26, 2026Jonathan DeCastro, Thomas Balch, Emily Sumner +9Shared AutonomyHuman-in-the-Loop AI

    Toyota Research Institute, Cambridge, MA 02139, USA · Toyota Research Institute, Los Altos, CA 94022, USA · Massachusetts Institute of Technology, Cambridge, MA 02139, USA

  4. HumanoidNDM: Closing the Training-Deployment Gap of Motion Tracking via Humanoid Neural Dynamics Model

    Oct 25, 2026Chao Chen, Zhenrong Wang, Jilong Wang +4Robot Policy AdaptationHumanoid Motion Tracking

    CFCS, School of Computer Science, Peking University · Galbot · Institute of Automation, Chinese Academy of Sciences +2

  5. Reinforcement Learning with Multi-Step Transition Predictions: A Bellman Theory

    Oct 23, 2026Chenbei Lu, Zaiwei Chen, Tongxin Li +2Reinforcement LearningModel-Based RL

    Duffield College of Engineering, Cornell University · Edwardson School of Industrial Engineering, Purdue University · School of Data Science, The Chinese University of Hong Kong (Shenzhen) +2

  6. DiffDreamer: Coupling World Models with Differentiable Simulation for Agile Quadrotor Control

    Oct 22, 2026Fanxing Li, Yuxiang Huang, Shengyang Wang +7Quadrotor ControlReinforcement Learning

  7. Primal-Dual Representation Learning for Low-Rank Constrained MDPs

    Oct 22, 2026Chenhao Zhou, Chao Zhang, Wensong Bai +2Markov Decision ProcessesConstrained RL

    College of Computer Science and Technology, Zhejiang University

  8. Second Thoughts: Exploring and Planning through Comparative Imagination in World Models

    Oct 20, 2026Hieu Tran, Hong YuWorld Model-Based PlanningModel-Based RL

    University of Massachusetts Amherst · University of Massachusetts Lowell

  9. SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model

    Oct 20, 2026Qiayuan Liao, Yiyang Shao, Junfeng Long +4Humanoid Robot ControlHumanoid Motion Tracking

    UC Berkeley

  10. From POMDP Theory to Deep RL with Particle Filters

    Oct 20, 2026Kevin Tan, Ziping XuPartially Observable RLModel-Based RL

    Wharton Statistics and Data Science, University of Pennsylvania · School of Data and Information Sciences, University of North Carolina at Chapel Hill

  11. Simulation Priors for Rapid Adaptation in Model-Based RL for Autonomous Racing

    Oct 19, 2026Adrian Remonda, Jiajun Xi, Nicklas Hansen +5RL for Autonomous DrivingModel-Based RL

    Toyota Research Institute, Los Altos, CA, USA. · University of California San Diego (UCSD), La Jolla, CA, USA.

  12. Dynamics-Aligned Representation Learning under Induced Partial Observability

    Oct 18, 2026SM Mazharul Islam, Manfred HuberModel-Based RLRepresentation Learning for RL

    Department of Computer Science and Engineering, University of Texas at Arlington, TX-76019, USA

  13. Constrained Multi-Agent Reinforcement Learning in Imagination

    Oct 15, 2026Raz Shmueli, Shahaf S. ShperbergConstrained MARLMulti-Agent Reinforcement Learning

    Ben-Gurion University of the Negev, Beer-Sheva, Israel

  14. Manifold Generative Annealing for Contact Constraint-aware Impedance Control with Reinforcement Learning Prior

    Oct 15, 2026Zhilin He, Guoquan Qi, Changliu LiuContact-Rich Robotic ManipulationImpedance Control

    Carnegie Mellon University · Institute of Automation, Chinese Academy of Sciences

  15. Model-Based Online Decision Making via Generative Trajectory Planning

    Oct 13, 2026Haldun Balim, Yilun Du, Na LiModel-Based PlanningModel-Based RL

    Harvard University

  16. WoVR+: World Model as Reliable Simulator for Versatile VLA Policy Optimization with RL

    Oct 13, 2026Shangqing Zhou, Zhennan Jiang, Ning Ma +9Robotic RLRL for VLMs

    Institute of Automation, Chinese Academy of Sciences. · School of Artificial Intelligence, University of Chinese Academy of Sciences. · Zhongguancun Academy. +3

  17. Trust-Calibrated VLA Policy Post-Training with Evolving Imagination

    Oct 12, 2026Wenzhan Li, Yiran Qin, Heng Zhou +3RL Post-TrainingVLMs for Robotics

    Sun Yat-sen University · Shenzhen Loop Area Institute · The Chinese University of Hong Kong, Shenzhen +3

  18. Chunk-wise Model-based Policy Optimization

    Oct 10, 2026Weixi Song, Donglin Wang, Yong-Lu LiModel-Based RL

    Zhejiang University · Shanghai Innovation Institute · Westlake University +1

  19. Dream to Remember: Continual Steering of Generalist Robot Policies with World Models

    Oct 9, 2026Bahey Tharwat, Iman Nematollahi, Marvin Alles +3Continual Robot LearningRobotic RL

    University of Freiburg · Technical University of Munich

  20. Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks

    Oct 8, 2026Haoyu Zhao, Zhengxu Yu, Zhiyuan He +5Continual Learning for LLM AgentsRecursive Self-Improvement

    University College London · Huawei Noah’s Ark Lab, UK · University of Liverpool

  21. RIWANav: Recursive World-Action Models with Self-Improvement for Urban Navigation

    Oct 6, 2026Jing Xie, Shouwei Ruan, Yubin Wang +4Robot Policy LearningWorld Model Learning

    MoE Key Lab of Artificial Intelligence, Institute of AI, School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · Beihang University, Beijing, China · The Hong Kong University of Science and Technology, Hong Kong, China +3

  22. Learning in Dreams, Winning in Reality: A Continuous Dyna Loop for a Ten-Hero MOBA

    Oct 6, 2026Jordy KietoGame-Playing AgentsModel-Based RL

  23. Considering Context: When World Models Need Context Encoders

    Oct 5, 2026Oleg Smirnov, Sofiane Ennadir, John Pertoft +2Reinforcement LearningWorld Model Learning

    King AI Labs, Microsoft Gaming

  24. Mind the Execution Gap: Action-Semantic Mismatch in World-Model Control

    Oct 5, 2026Shengtao Wen, Xiang Chen, Yu Tian +3World Model-Based PlanningAction-Conditioned World Models

    Nanjing University of Aeronautics and Astronautics, Nanjing, China · Tsinghua University, Beijing, China · Nanjing University, Nanjing, China

  25. Optimal Control with Learned Critics under Unmodeled State Dependencies

    Oct 4, 2026Philipp Schoch, Markus RyllResidual Dynamics LearningRobot Dynamics Learning

    Autonomous Aerial Systems Lab Technical University of Munich

  26. R2R^2-WAM: Repair-and-Reject Post-Training for World Action Models

    Oct 4, 2026Ruiyan Xu, Haisheng Su, Sixu Lin +4World Action ModelsAction-Conditioned World Models

    The Chinese University of Hong Kong, Shenzhen · Manifold AI · Southeast University +1

  27. Near-Optimal Sample Complexity for Recursive Entropic Risk Reinforcement Learning with a Generative Model

    Oct 3, 2026Amirparsa Bahrami, Oliver Mortensen, Mohammad Sadegh TalebiRisk-Sensitive RLSample-Efficient RL

    School of Electrical Engineering, Sharif University of Technology · Department of Computer Science, University of Copenhagen

  28. Learning Transferable Policies from Action-free Time Series Through Dynamical Embeddings

    Oct 2, 2026Niklas Emonds, Georgia KoppeTransfer Learning in RLReinforcement Learning

    Hector Institute for AI in Psychiatry (HITKIP) & Department of Psychiatry and Psychotherapy, Central Institute of Mental Health (CIMH), Medical Faculty Mannheim, Heidelberg University · Interdisciplinary Center for Scientific Computing (IWR), Faculty of Mathematics and Computer Science, Heidelberg University · Hertie Institute for AI in Brain Health, University of Tübingen, Germany

  29. VIGOR: Zero-Shot Visual Generalization via Latent-Space Consistency in Model-Based Reinforcement Learning

    Oct 2, 2026Mingyu Park, Samyeul Noh, Hyun Myung +1Latent-Space RegularizationGeneralization in RL

    Robotics Program KAIST, ETRI · Electrical Engineering KAIST

  30. Calibration-risk routing for controlled world-model adaptation

    Oct 1, 2026Yifan Zhang, Liang Zheng, Qifan ZhangWorld Models for RoboticsWorld Models

    Central South University