In-Context RL

RL: Reinforcement Learning

In-context reinforcement learning (ICRL) studies agents that use examples, rewards, and past interactions in their context to adapt to new tasks without changing model weights. Approaches include training sequence models across many environments, using Transformers to implement updates such as temporal-difference or Q-learning, and guiding language-model agents with demonstrations, reflections, or retrieved experience. Research spans control and robotics, including MuJoCo and deployment-time adaptation, as well as tool use and multi-agent settings such as Overcooked. Open questions include whether agents truly learn from reward signals, how to retain useful history efficiently, and whether adaptation transfers reliably to unfamiliar tasks and teammates.

Latest papers

45 papers11 papers in the last four weeks, up 175% on the four weeks before. 0.1% of all new papers.

Asymmetric On-Policy Distillation Learns In-Context Exploration
Oct 21, 2026Sriyash Poddar, Yanda Bao, Jacob Krantz +5RL Exploration Partially Observable RL
Paul G. Allen School of Computer Science and Engineering, University of Washington · Work partially done while at Meta
Is In-Context Reinforcement Learning A Posterior Sampler? An Empirical Study
Oct 20, 2026Weiqin Chen, Xinlong Du, Gesualdo Scutari +1In-Context RL Transformer-Based RL
Edwardson School of Industrial Engineering, Purdue University West Lafayette, IN 47907, USA
Remember Less, Adapt Longer: Bounded-State In-Context Reinforcement Learning with Recurrent Summary Memory
Oct 19, 2026Jiawen Wang, Cheng Xue, Peng Zhang +1Reinforcement Learning In-Context RL
The Australian National University
Agentic In-Context Learning: What Algorithms Can Transformers Learn in Closed-Loop Interaction?
Oct 16, 2026Shiguang Wu, Yaqing Wang, Quanming YaoIn-Context Learning In-Context RL
Tsinghua University · Beijing Institute of Mathematical Sciences and Applications
Provable Transformer Context Compression for In-Context Reinforcement Learning
Oct 14, 2026Yajie Zhao, Yingjun Mo, Zhiyong Wang +3In-Context RL In-Context Learning
Southern University of Science and Technology · University of Electronic Science and Technology of China · Harbin Institute of Technology
VLRLA: Deployment-Time Adaptation of Vision-Language-Action Models through In-Context Reinforcement Learning
Oct 14, 2026Diyuan Shi, Yiqi Tang, Hongyin Zhang +2Test-Time Adaptation of VLA Models In-Context RL
Zhejiang University · Westlake University
From Policy Evaluation to Off-Policy Control: Provable Emergence of In-Context Q-Learning
Oct 13, 2026Yingjun Mo, Yajie Zhao, Fang KongIn-Context RL Transformer-Based RL
Southern University of Science and Technology (SUSTech) · University of Electronic Science and Technology of China (UESTC)
Explore with Memory, Learn without It: In-Context Exploration and Self-Distillation for LLM Reasoning
Oct 13, 2026Yutong Chen, Zhichao Xu, Rui Liu +1RL for Language Model Reasoning RL Exploration
Department of Computer Science, Clemson University · Kahlert School of Computing, University of Utah
Do LLMs Learn from Rewards in Context? : Rethinking the role of reward in In-Context RL
Oct 8, 2026Minchan Kwon, Seunghee Koh, Sunghyun Baek +2In-Context RL Contextual RL
Korea Advanced Institute of Science and Technology (KAIST)
Independent Multi-Agent Reinforcement Learning with Counterfactual Semantic-Social World Models
Oct 6, 2026Fernando Martinez, Tao Li, Yingdong Lu +1Sequential MARL Action-Conditioned World Models
Department of Computer and Information Sciences, Fordham University, New York, NY, 10023, USA · Department of Systems Engineering, City University of Hong Kong, Hong Kong SAR, 999077, China · IBM Research, Yorktown Heights, NY, 10598, USA
Scalable In-Context Reinforcement Learning with Recurrent Algorithm Distillation
Sep 28, 2026Yuanqing Ma, Zhenrui Zheng, Chenjun XiaoTransformer In-Context RL
The Chinese University of Hong Kong, Shenzhen
Downstream-Aware Context Selection for Online In-Context Reinforcement Learning
Sep 27, 2026Ruihan A. Li, Shangtong Zhang, Rohan ChandraIn-Context RL In-Context Learning
University of Illinois Urbana-Champaign · University of Virginia
Privacy-Preserving Prompted Policy Search for Robotic Control
Sep 24, 2026Ali Irshayyid, Feng Lin, Chong Li +1LLM-Guided RL Robot Policy Learning
Department of Electrical and Computer Engineering, Oakland University, Rochester, MI 48309 USA · Department of Electrical and Computer Engineering, Wayne State University, Detroit, MI 48202, USA · OORT and the Department of Electrical Engineering, Columbia University, NYC, 10027, USA
From Weak Data to Strong Policy: Q-Targets Enable Provable In-Context Reinforcement Learning
Sep 24, 2026Yichen Lin, Xuyuan Xiong, Xue Wang +3In-Context RL Transformer-Based RL
Shanghai Jiao Tong University, Shanghai, China · Alibaba Group · School of Management, University at Buffalo, Buffalo, NY, USA
A neural-astrocyte architecture implements a hybrid automaton for evidence accumulation
Sep 14, 2026Giacomo Vedovati, Ilya E. Monosov, Thomas J. Papouin +1Dynamical Systems Reinforcement Learning
Department of Electrical and Systems Engineering, Washington University in St. Louis, St. Louis, MO 63130, USA · Department of Neuroscience, Johns Hopkins University, Baltimore, MD 21218 USA · Department of Neuroscience, Washington University in St. Louis, St. Louis, MO 63130, USA
LLMs are Few-Shot Decision-Makers: Generalized Context-Aware Microgrid Frequency Control through Prompt Decision Transformer
Aug 22, 2026Xu Yang, Chenhui Lin, Haotian Liu +3Power Systems Transformer-Based RL
State Key Laboratory of Power Systems, Department of Electrical Engineering, Tsinghua University, Beijing 100084, China
From Answers to Policies: Efficient In-Context Learning System through Emulating Expert Investigation
Aug 17, 2026Minh-Ha Nguyen, Ngoc-Ngo Quang Tran, Thuy Dung Nguyen +1In-Context Learning In-Context RL
Department of Epidemiology, Vanderbilt University, Nashville, TN, USA · Department of Pediatrics, Vanderbilt University Medical Center, Nashville, TN, USA · Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN, USA +1
Semantic Bandits: In-Context Exploration-Exploitation is Biased by Semantic Priors
Aug 17, 2026David Eric Austin, Kaheer Suleman, Jackie Chi Kit CheungExploration-Exploitation Tradeoff Linguistic Bias in Language Models
McGill University · Mila – Quebec AI Institute · Skyfall AI +1
In-Context Learning as Implicit Policy Gradient
Jul 25, 2026Masahiro Kaneko, Timothy BaldwinRL for Language Models Policy Gradient
MBZUAI
Sample-Efficient Learning from Agent Experience
Jul 23, 2026Chenhui Gou, Haoqin Tu, Yunhao Fang +2In-Context RL Language Model Distillation
Monash University · ByteDance Seed
In-Context Reinforcement Learning under Non-Stationarity: A Survey
Jul 1, 2026A Run, Ziluo DingNon-Stationary RL In-Context RL
Independent Researcher · Beijing Innovation Center of Humanoid Robotics
Reinforcement Learning Foundation Models Should Already Be A Thing
Jun 17, 2026Abdelrahman Zighem, Jill-Jênn VieSynthetic Data Pretraining Reinforcement Learning
Ecole normale supérieure de Paris, PSL University, Paris, France · Soda team, Inria Saclay, Palaiseau, France.
Latent Q-Barrier Shielding for Safe In-Context Reinforcement Learning
May 24, 2026Minjae Kwon, Amir Moeini, Shangtong Zhang +1Reinforcement Learning Control Barrier Functions
University of Virginia
Benchmarking the Limits of In-Context Reinforcement Learning for Ad-Hoc Teamwork
May 23, 2026Yuheng Jing, Kai Li, Ziwen Zhang +8RL Benchmarks Multi-Agent Coordination
C2DL, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · University of Science and Technology of China +5
TabQL: In-Context Q-Learning with Tabular Foundation Models
May 18, 2026Qisai Liu, Zhanhong Jiang, Timilehin Ayanlade +4Deep Q-Learning Q-Learning
Department of Mechanical Engineering Iowa State University Ames, IA 50011 · Translational AI Center Iowa State University Ames, IA 50011 · Department of Computer Science Iowa State University Ames, IA 50011 +1
One for All: A Non-Linear Transformer can Enable Cross-Domain Generalization for In-Context Reinforcement Learning
May 10, 2026Bowen He, Juncheng Dong, Lin Lin +1Reinforcement Learning Transformer
Duke University, Durham, U.S.A.
Beyond Linear Attention: Softmax Transformers Implement In-Context Reinforcement Learning
May 8, 2026Zixuan Xie, Xinyu Liu, Claire Chen +3Reinforcement Learning Transformer-Based RL
University of Virginia · California Institute of Technology · Purdue University
Convergence and Emergence of In-Context Reinforcement Learning with Chain of Thought
May 8, 2026Zixuan Xie, Xinyu Liu, Rohan Chandra +1Reinforcement Learning Transformer
University of Virginia
Transformers Provably Implement In-Context Reinforcement Learning with Policy Improvement
May 7, 2026Haodong Liang, Lifeng LaiReinforcement Learning Transformer-Based RL
Department of Electrical and Computer Engineering, University of California, Davis
Vintix II: Decision Pre-Trained Transformer is a Scalable In-Context Reinforcement Learner
Apr 6, 2026Andrei Polubarov, Lyubaykin Nikita, Alexander Derevyagin +11In-Context RL Transformer-Based RL
AXXX, Applied AI Institute · AXXX, Innopolis University · AXXX, HSE +7

Oct 21, 2026J/K move · Enter open · S save

Asymmetric On-Policy Distillation Learns In-Context Exploration

Sriyash Poddar, Yanda Bao, Jacob Krantz, Matthew Chang+4

Paul G. Allen School of Computer Science and Engineering, University of Washington · Work partially done while at Meta

RL Exploration Partially Observable RL On-Policy Distillation Robot Policy Learning In-Context RL

Open paper AskPDF arXiv

Sequential decision-making under partial observability is difficult because agents must explore to infer hidden, task-relevant information before acting effectively. Learning such information-gathering behaviors tabula rasa via online reinforcement learning is sample-inefficient, whereas offline imitation averages expert actions across hidden states and fails to learn exploration. In this work, we propose ASTEROID, a simple and effective student--teacher supervised learning framework that can learn test-time exploration. Although a privileged expert never needs to explore, we demonstrate that a history-conditioned student can learn to do so when trained with on-policy contexts: the student first collects on-policy context under partial observability, and the privileged expert provides action labels conditioned on that context. By distilling expert actions directly conditioned on the student's on-policy history, the student learns to systematically resolve partial observability, i.e., search while uncertain and exploit once the hidden state is revealed. This enables efficient test-time exploration (similar to Bayesian posterior sampling) without incurring the impractical training-data costs of reinforcement learning. Across a diverse range of eleven different environments, including discrete exploration benchmarks, robotic manipulation, navigation, and Procgen environments, ASTEROID achieves ~2x higher returns under fixed exploration budgets, uses ~100x fewer interactions than RL baselines, and outperforms frontier models at exploration. Finally, our method transfers policies trained in simulation to vision-denied real-world robot manipulation, exhibiting efficient exploration and increasing success by +40% over standard distillation. Overall, it provides a simple yet surprisingly effective framework to learn efficient test-time exploration.