Generalization in RL

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

9 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 47

All topics

Understanding Goal Generalisation in Sequential Reinforcement Learning
May 22, 2026Jason Ross Brown, Edward James YoungReinforcement Learning OOD Generalization
University of Cambridge Cambridge, United Kingdom · University of Cambridge & Geodesic Research Cambridge, United Kingdom
Smaller Abstract State Spaces Enable Cross-Scale Generalization in Reinforcement Learning
May 19, 2026Nasehatul Mustakim, Lucas LehnertPartially Observable RL OOD Generalization
Department of Computer Science University of Saskatchewan Saskatoon, Saskatchewan, Canada
GRLO: Towards Generalizable Reinforcement Learning in Open-Ended Environments from Zero
May 14, 2026Shangjian Yin, Yu Fu, Yue Dong +1Reinforcement Learning RL for Language Model Reasoning
University of California, Riverside
On the Importance of Multistability for Horizon Generalization in Reinforcement Learning
May 12, 2026Asad Bakija, Florent De Geeter, Julien Brandoit +2Dynamical Systems Reinforcement Learning
Dept. of Electrical Engineering and Computer Science · University of Liège · Liège, Belgium
One for All: A Non-Linear Transformer can Enable Cross-Domain Generalization for In-Context Reinforcement Learning
May 10, 2026Bowen He, Juncheng Dong, Lin Lin +1Reinforcement Learning Transformer
Duke University, Durham, U.S.A.
Improving Zero-Shot Offline RL via Behavioral Task Sampling
Apr 28, 2026Nazim Bendib, Nicolas Perrin-Gilbert, Olivier SigaudReinforcement Learning Offline RL
Institut des Systèmes Intelligents et de Robotique, ISIR, Sorbonne Université, Paris, France.
Why Does Reinforcement Learning Generalize? A Feature-Level Mechanistic Study of Post-Training in Large Language Models
Apr 27, 2026Dan Shi, Zhuowen Han, Simon Ostermann +3Reinforcement Learning RL for Language Model Reasoning
TJUNLP Lab, School of Computer Science and Technology, Tianjin University, China · German Research Center for Artificial Intelligence (DFKI), Saarbrücken, Germany · Saarland University, Saarbrücken, Germany
Rethinking Reinforcement Fine-Tuning in LVLM: Convergence, Reward Decomposition, and Generalization
Apr 21, 2026Carter Adams, Rafael Oliveira, Gabriel Almeida +1Tool Use in VLMs Reinforcement Learning
Federal University of Bahia
Temporal Consistency Improves Generalization in Contextual Offline Meta Reinforcement Learning
Mar 3, 2026Mohammadreza Nakheai, Aidan Scannell, Kevin Luck +1Representation Learning Offline RL
Aalto University · Toyota Research Institute · Vrije Universiteit Amsterdam
Multi-Agent Reinforcement Learning for V2X Resource Allocation: Disentangling MARL Challenges Through Benchmarking
Feb 18, 2026Siyuan Wang, Lei Lei, Pranav Maheshwari +2RL Benchmarks Reinforcement Learning
College of Engineering, University of Guelph, Guelph, Ontario, Canada · College of Electrical Engineering and Computer Sciences, Ningbo University, Ningbo, 315211, China
PlatoLTL: Scaling LTL-Guided Multi-Task RL
Jan 30, 2026Jacques Cloete, Mathias Jackermeier, Ioannis Havoutis +1RL for Robotics Constrained RL
Oxford Robotics Institute University of Oxford Oxford, United Kingdom · Department of Computer Science University of Oxford Oxford, United Kingdom
KAGE-Bench: Fast Known-Axis Visual Generalization Evaluation for Reinforcement Learning
Jan 20, 2026Egor Cherepanov, Daniil Zelezetsky, Alexey K. Kovalev +1RL Benchmarks Reinforcement Learning
1AXXX, Moscow, Russia · 2MIRAI, Moscow, Russia
Meta-RL with Bayesian Linear Task Models
Dec 24, 2025Jingyang You, Hanna KurniawatiRepresentation Learning Reinforcement Learning
School of Computing Australian National University
RAD: Retrieval High-quality Demonstrations to Enhance Decision-making
Jul 21, 2025Lu Guo, Yixiang Shan, Zhengbang Zhu +5Offline RL Generalization in RL
School of Artificial Intelligence, Jilin University, Changchun, China · School of Computer Science, Shanghai Jiao Tong University, Shanghai, China
Towards More Efficient, Robust, Instance-adaptive, and Generalizable Sequential Decision making
Apr 12, 2025Zhiyong WangMulti-Armed Bandits Reinforcement Learning
Department of Computer Science and Engineering, The Chinese University of Hong Kong
Training on Irrelevant States Implies Data Augmentation: Generalization in Contextual MDPs
Oct 4, 2024Max Weltevrede, Caroline Horsch, Matthijs T. J. Spaan +1Reinforcement Learning RL Exploration
Delft University of Technology, Delft, The Netherlands
Policy Learning with a Language Bottleneck
May 7, 2024Megha Srivastava, Cedric Colas, Dorsa Sadigh +1Policy Learning LLM-Guided RL
Stanford University · Massachusetts Institute of Technology Inria

May 22, 2026, cs.LGJ/K move · Enter open · S save

Understanding Goal Generalisation in Sequential Reinforcement Learning

Jason Ross Brown, Edward James Young

University of Cambridge Cambridge, United Kingdom · University of Cambridge & Geodesic Research Cambridge, United Kingdom

Reinforcement Learning OOD Generalization Generalization in RL

Open paper AskPDF arXiv

Reinforcement learning agents often exhibit unintended goal-directed behaviour outside their training distribution, but we currently lack a principled understanding of how such agents will generalise to novel environments based on their training history. We address this gap for agents trained sequentially on one or more tasks. We study over 100 sequential training pipelines, evaluating behaviour across over 250 out-of-distribution environments. We find that salient features drive generalisation, and that goals learnt early in training can persist and influence those acquired later. To explain these phenomena, we introduce latent policy gradients, a method that predicts what out-of-distribution behaviour a training pipeline will likely induce. Our method simulates the evolution of low-dimensional latent variables during training according to what would achieve high reward on the training objective with respect to a simple model of how the latent variables map to behaviour. It achieves strong predictive accuracy, generalises to unseen types of training pipeline, and is interpretable. Our findings demonstrate that while out-of-distribution RL agent behaviour is dependent on the whole training pipeline, this dependence has an underlying structure we can capture, laying groundwork for understanding goal generalisation from a developmental perspective.

Momentum

Related topics

Latest papers 47