Model-Based RL

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

24 papers in the last four weeks, up 243% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 142

All topics

Scaling Automatic Research Agents via World Models
Aug 12, 2026Xiyuan Yang, Sheikh Sarwar, Jingru Cheng +8World Model Learning RL Post-Training
University of Illinois Urbana-Champaign · Amazon
Dynamics Models for Offline Hyperparameter Selection in Real-World RL
Aug 11, 2026Jordan Coblin, Han Wang, Martha White +1RL Control Offline RL
University of Alberta · Alberta Machine Intelligence Institute (Amii) · Canada CIFAR AI Chair at Amii
IADD-TR: Intervention-Aware Dynamics Decoupling with Targeted Regularization for Model-Based Reinforcement Learning
Aug 11, 2026Zefeng Liang, Jie Qiao, Ruichu Cai +2Reinforcement Learning Robust RL
School of Computer Science, Guangdong University of Technology, Guangzhou, China · College of Science, Shantou University, Shantou, China
Dreamer-SAC: Off-Policy Learning in Latent World Models for Sample-Efficient Autonomous Driving
Aug 11, 2026Jiazhuo Li, Linjiang Cao, Qi Liu +1World Models for Autonomous Driving RL for Autonomous Driving
Tongji University
LUCID: Latent-Skill Unified Control via Imagined Dynamics for Long-Horizon Humanoid Loco-Manipulation
Aug 7, 2026Cheng Guo, Mingzhe Ni, Angelo Cangelosi +1Humanoid Loco-Manipulation Latent Action Learning
Department of Computer Science, University of Manchester, Manchester, UK · Human-Robot Interfaces and Interaction Laboratory, Italian Institute of Technology, Genoa, Italy
Dueling World Models: Advantage-Style Action Channels for Common-Mode Distractor Rejection
Aug 7, 2026Jiazhuo Li, Yiming Fei, Zhiruo Zhou +1Action-Conditioned World Models Latent World Models
University of Michigan, Ann Arbor, MI, USA · Zhejiang University, Hangzhou, China · Wuhan University of Technology, Wuhan, China +1
Stochastic Multiple Shooting Trajectory Optimization via Sequential Local Policy Evaluation
Aug 4, 2026Ashwin Gupta, Joseph MooreTrajectory Optimization Stochastic Optimal Control
Johns Hopkins University Whiting School of Engineering
Analytic Planning under Uncertainty with Moment Closure
Aug 3, 2026Shishir Sharma, Doina PrecupDecision-Making under Uncertainty Model-Based RL
1McGill University, Montreal, Canada · 2Mila – Quebec Artificial Intelligence Institute, Montreal, Canada
Climate-Dyna Deep Hedging for XVAs: Model-Based Reinforcement Learning, Residual Climate HVA, and Hedge-Instrument Discovery
Aug 2, 2026Xiaozhen Wang, Francois Buet-GolfouseQuantitative Finance Residual Policy Learning
CEREMADE Université Paris Dauphine–PSL, France · AIML Global Markets Barclays, United Kingdom
DreamQAS: Learning a Decision-Useful World Model for VQE-Efficient Quantum Architecture Search
Jul 31, 2026Jiayang Niu, Yan Wang, Jie Li +4World Models Quantum RL
School of Computing Technologies, RMIT University · Quantum Systems, Data61, CSIRO · RMIT University
Sample Efficient Hierarchical Reinforcement Learning via Best Policy Identification
Jul 31, 2026Anders Jonsson, Emilie Kaufmann, Gianmarco Tedeschi +1Hierarchical RL Policy Learning
Department of Engineering Universitat Pompeu Fabra · Univ. Lille, CNRS, Inria Centrale Lille, UMR 9189-CRIStAL · Dept. Electronics, Information, and Bioengineering Politecnico di Milano
QQWorld: Quantile-Quantile Matching for World Model Regularization
Jul 30, 2026Zhoushun Yu, Xiaoyu Hu, Xiangyu XuWorld Model Learning World Model-Based Planning
Xi’an Jiaotong University
Learning Implicit Causal World Models from Multi-Agent Demonstrations
Jul 28, 2026Jasorsi GhoshCausal Representation Learning Causal World Models
Department of Computer Science Purdue University West Lafayette, Indiana, USA
Reinformed Dreamer: An Asymmetric World Model Efficiently Trained through Latent Guidance
Jul 28, 2026Gaspard Lambrechts, Adrien Bolland, Daniel Ebi +1Reinforcement Learning World Models
1McGill University, Canada · 3Université de Liège, Belgium · 4Karlsruhe Institute of Technology, Germany
Dreamer-CPC: Message Learning with World Models for Decentralized Multi-agent Reinforcement Learning
Jul 22, 2026Taisuke Takayama, Naoto Yoshida, Tadahiro TaniguchiPredictive Coding Partially Observable RL
Graduate School of Informatics, Kyoto University, Kyoto, Japan · Research Organization of Science and Technology, Ritsumeikan University, Shiga, Japan
The World Model Remembers, the Actor Forgets: Dream Rehearsal for Continual Model-Based RL
Jul 22, 2026Gurp NijjerGenerative Replay Reinforcement Learning
Quantegra Research, Canada
Koopman Dreamer: Spectrally Constrained Latent Dynamics for Stable World-Model Imagination
Jul 22, 2026Jiaqi Li, Xinglong Zhang, Haibin Xie +3Koopman Operator Learning Latent Dynamics Modeling
College of Intelligence Science and Technology, National University of Defense Technology, Changsha, China
Reinforcement Learning: From Algorithms To Foundation Models
Jul 20, 2026Zihan DingReinforcement Learning RL for Video Generation
Department of Electrical Computer Engineering, Princeton University
Enhancing Personalized Bladder Cancer Treatment Through Reinforcement Learning: A Recurrent Patient State Transition Decision Support Framework
Jul 18, 2026Divyansh Chawla, Anshu Garg, Isshaan SinghReinforcement Learning Healthcare
School of Computer Science and Engineering, Vellore Institute of Technology, Chennai-600 0127, Tamil Nadu, INDIA
Learning from World Feedback: Why Model Uncertainty Fails as a Risk Signal in Model-Based RL
Jul 18, 2026Zhaohui WangRisk-Sensitive RL Model Predictive Control
Viterbi School of Engineering, University of Southern California, Los Angeles, CA, USA.
Certifiable Safe Model-Based Reinforcement Learning with Control-Affine Dynamics Approximation
Jul 17, 2026Hao Zhou, Yanze Zhang, Cameron Reid +1Safe RL Model-Based RL
Department of Computer Science, University of Illinois Chicago, Chicago, IL 60607, USA · Department of Computer Science and Engineering, Texas A&M University, College Station, TX 77843, USA
Concept-Guided Spatial Regularization for World Models in Atari Pong
Jul 16, 2026Yukuan Lu, Zaishuo Xia, Weyl Lu +1Game-Playing Agents World Model Learning
UC Davis
RENEW: Towards Learning World Models and Repairing Model Exploitation from Preferences
Jul 15, 2026Logan Mondal Bhamidipaty, Mykel Kochenderfer, Subramanian RamamoorthyWorld Model Learning Offline RL
University of Edinburgh · 2Stanford University
Learning Safe Agent Behaviour from Human Preferences and Justifications via World Models
Jul 14, 2026Ilias Kazantzidis, Timothy J. Norman, Yali Du +1Safe RL Reward Design for RL
University of Southampton, Southampton SO17 1BJ, United Kingdom · King’s College London, London WC2R 2LS, United Kingdom
Shortcut Trajectory Planning for Efficient Offline Reinforcement Learning
Jul 10, 2026Guanquan Wang, Yoshimasa TsuruokaTrajectory Generation Offline RL
Department of Information and Communication Engineering · The University of Tokyo
Operator-on-F complements value-equivalence: a planning-time diagnostic for latent world models
Jul 5, 2026Donna VakalisWorld Model-Based Planning Latent World Models
1Mila – Quebec AI Institute · 2Université de Montréal
Learning Task-Sufficient World Models by Synergizing Agentic Exploration and Structured Modeling
Jul 5, 2026Fan Feng, Yujia Zheng, Minghao Fu +5World Model Learning World Models
1UCSD · 2MBZUAI · 3CMU +1
Reward Observability and the Limits of Offline Checkpoint Selection in RSSM World Models
Jul 2, 2026Nikolai Smolyanskiy, Jonathan ShockReinforcement Learning Latent World Models
Solving Markov Decision Processes with Future Information via MPC
Jun 23, 2026Shambhuraj Sawant, Akhil S Anand, Dirk Reinhardt +1RL Control Markov Decision Processes
Norwegian University of Science and Technology (NTNU), Trondheim, Norway.
Inverting the Bellman Equation: From $Q$ -Values to World Models
Jun 19, 2026Alistair Letcher, Mattie Fellows, Alexander D. Goldie +3Reinforcement Learning Model-Free RL
FLAIR, University of Oxford · Google DeepMind · Mila, University of Montreal

Aug 12, 2026, cs.LGJ/K move · Enter open · S save

Scaling Automatic Research Agents via World Models

Xiyuan Yang, Sheikh Sarwar, Jingru Cheng, Zhan Shi+7

University of Illinois Urbana-Champaign · Amazon

World Model Learning RL Post-Training RL for LLM Agents Autonomous Research Agents Model-Based RL

Open paper AskPDF arXiv

Automating empirical research is a long-standing direction of AI. Recent automatic research (AutoResearch) agents bring this goal within reach, as modern LLMs show the capability to independently implement solutions and learn from the execution outcomes. Behind these gains, post-training (especially RL) plays a central role. In this paper, we identify a fundamental tension when scaling RL for these agents: the two components of every AutoResearch trajectory (agent generation and environment execution) scale in very different manners, since all generation shares compute through batching, while each execution occupies its exclusive sandbox and real machine time. As a result, the environment execution dominates the training cost and becomes the bottleneck as trajectories grow. To resolve this tension, we propose World Model RL (WMRL), which replaces environment execution with a world model to remove this bottleneck. Additionally, the world model can be imperfect, as its rewards are corrupted by bias and noise. Therefore, we further equip WMRL with two mitigations, Online Debiasing and Inverse-Variance Denoising, which offset the bias and suppress the noise respectively. Theoretically, we prove that both mitigations of WMRL strictly improve the convergence guarantee. Empirically, WMRL accelerates training by 3-4x on various tasks at different agent scales, while exceeding the performance of standard RL baselines. Moreover, our post-trained 4B and 9B agents outperform much larger open-weight agents of 48B and 120B on held-out benchmarks. Beyond AutoResearch, WMRL also transfers to post-training embodied VLA policies, which demonstrates the generalizability of our method.

Momentum

Related topics

Latest papers 142