Contextual RL

RL: Reinforcement Learning

Latest papers

19 papers3 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Provable Transformer Context Compression for In-Context Reinforcement Learning
Oct 14, 2026Yajie Zhao, Yingjun Mo, Zhiyong Wang +3In-Context RL In-Context Learning
Southern University of Science and Technology · University of Electronic Science and Technology of China · Harbin Institute of Technology
Do LLMs Learn from Rewards in Context? : Rethinking the role of reward in In-Context RL
Oct 8, 2026Minchan Kwon, Seunghee Koh, Sunghyun Baek +2In-Context RL Contextual RL
Korea Advanced Institute of Science and Technology (KAIST)
Considering Context: When World Models Need Context Encoders
Oct 5, 2026Oleg Smirnov, Sofiane Ennadir, John Pertoft +2Reinforcement Learning World Model Learning
King AI Labs, Microsoft Gaming
Downstream-Aware Context Selection for Online In-Context Reinforcement Learning
Sep 27, 2026Ruihan A. Li, Shangtong Zhang, Rohan ChandraIn-Context RL In-Context Learning
University of Illinois Urbana-Champaign · University of Virginia
Dynamic Context Scheduling: Learning Beyond the Static Universe
Aug 21, 2026Martin Mráz, André BiedenkappContextual RL Reinforcement Learning
University of Freiburg
Task Specialization Fine-Tuning for Contextual Reinforcement Learning
Aug 17, 2026Jianan Zhou, Jung-Hoon Cho, Tianyue Zhou +5Contextual RL Reinforcement Learning
Nanyang Technological University · MIT · UIUC
Contextual Quality-Diversity Evolutionary Reinforcement Learning for HVAC Control in Tropical Commercial Buildings
Aug 11, 2026Tran Le VuReinforcement Learning Building Energy Management
In-Context Reinforcement Learning under Non-Stationarity: A Survey
Jul 1, 2026A Run, Ziluo DingNon-Stationary RL In-Context RL
Independent Researcher · Beijing Innovation Center of Humanoid Robotics
MATE: Solving Contextual Markov Decision Processes with Memory of Accumulated Transition Embeddings
May 17, 2026Himchan Hwang, Hyeokju Jeong, Gene Chung +3Markov Decision Processes Meta-Reinforcement Learning
Seoul National University · Saige · Ulsan National Institute of Science and Technology (UNIST)
Tighter Regret Bounds for Contextual Action-Set Reinforcement Learning
May 15, 2026Zijun Chen, Zihan ZhangRegret Minimization in RL Reinforcement Learning
Department of Computer Science and Engineering Hong Kong University of Science and Technology
Beyond Linear Attention: Softmax Transformers Implement In-Context Reinforcement Learning
May 8, 2026Zixuan Xie, Xinyu Liu, Claire Chen +3Reinforcement Learning Transformer-Based RL
University of Virginia · California Institute of Technology · Purdue University
Data Deletion Can Help in Adaptive RL
Apr 30, 2026Param Budhraja, Aditya Gangrade, Alex Olshevsky +1Non-Stationary RL Reinforcement Learning
Department of Electrical and Computer Engineering, Boston University, Boston, MA 02215, USA · Faculty of Electrical and Computer Engineering, Systems Engineering, Boston University, Boston, MA 02215, USA
PiCSRL: Physics-Informed Contextual Spectral Reinforcement Learning
Mar 26, 2026Mitra Nasr Azadani, Syed Usama Imtiaz, Nasrin AlamdariContextual RL Adaptive Sensing
Department of Civil and Environmental Engineering Florida State University, Tallahassee, FL, USA
Self Paced Gaussian Contextual Reinforcement Learning
Mar 24, 2026Mohsen Sahraei Ardakani, Rui SongContextual RL Reinforcement Learning
Department of Electrical and Computer Engineering North Carolina State University Raleigh, NC · Department of Statistics North Carolina State University Raleigh, NC
Temporal Consistency Improves Generalization in Contextual Offline Meta Reinforcement Learning
Mar 3, 2026Mohammadreza Nakheai, Aidan Scannell, Kevin Luck +1Representation Learning Offline RL
Aalto University · Toyota Research Institute · Vrije Universiteit Amsterdam
In-Context Planning with Latent Temporal Abstractions
Feb 21, 2026Baiting Luo, Yunuo Zhang, Nathaniel S. Keplinger +3Model-Based Planning Offline RL
Vanderbilt University · William & Mary
Contextual Intelligence The Next Leap for Reinforcement Learning
Feb 17, 2026André BiedenkappContextual RL Zero-Shot RL
Albert-Ludwigs-Universität Freiburg, Germany
Shared Modular Recurrence in Contextual MDPs for Universal Morphology Control
Jun 10, 2025Laurens Engwegen, Max Weltevrede, Caroline Horsch +2Robotic Control Reinforcement Learning
Delft University of Technology Delft, The Netherlands
Training on Irrelevant States Implies Data Augmentation: Generalization in Contextual MDPs
Oct 4, 2024Max Weltevrede, Caroline Horsch, Matthijs T. J. Spaan +1Reinforcement Learning RL Exploration
Delft University of Technology, Delft, The Netherlands

Oct 14, 2026J/K move · Enter open · S save

Provable Transformer Context Compression for In-Context Reinforcement Learning

Yajie Zhao, Yingjun Mo, Zhiyong Wang, Bo Tang+2

Southern University of Science and Technology · University of Electronic Science and Technology of China · Harbin Institute of Technology

In-Context RL In-Context Learning Transformer-Based RL Contextual RL Policy Evaluation Temporal-Difference Learning

Open paper AskPDF arXiv

Transformers can perform in-context reinforcement learning without parameter updates. However, existing theoretical constructions often require context sizes that grow with the number of sampled transitions, leading to increasing memory costs in large-scale problems. This raises a fundamental scalability question: how much context is intrinsically necessary to carry out such learning? We answer this question for fixed-policy TD evaluation. We show that, with a shared linear decoder, the minimum context dimension required to exactly reproduce the first

L

TD iterates is the Krylov dimension induced by the sampled transitions. When the available context dimension

m

falls below this threshold, exact reproduction is no longer possible. In this regime, we characterize the optimal approximation error and show that it decays as

Θ(γ^m\max\{1,1/(m(1-γ))\})

where

γ

is the discounting factor. Moving from representation complexity to actual memory cost, we characterize the optimal retained memory for exact continuous representations as

Θ(nk_L)

and show experimentally that on a sampled task whose exact rank is full but whose numerical rank is small, a context accurate to

10^{-2}

88.7\%

smaller than the sparse sufficient statistics of TD. Finally, we establish the learnability of the shared executor through pretraining by proving global convergence of an auxiliary objective over effective attention parameters under suitable task diversity. Experiments across diverse TD evaluation problems validate parameter recovery and the predicted context--accuracy tradeoff.