ICL for Robotics

ICL: In-Context Learning

In-context learning for robotics studies how robots can acquire or adapt behaviors from examples, observations, and feedback supplied during use rather than by retraining their models. Common approaches condition vision-language-action policies on demonstrations, retrieve relevant episodes from memory, or use action effects and world models to infer how the robot and environment behave. These methods are tested on simulated and real manipulation, including long-horizon tasks, novel objects, contact-rich actions, and changing viewpoints, with some also addressing bimanual control and deformable objects. A central challenge is making context useful and compact enough for reliable adaptation while ensuring gains transfer across tasks, robots, and real-world conditions.

Latest papers

40 papers14 papers in the last four weeks, up 100% on the four weeks before. 0.1% of all new papers.

Decompose and Recompose: Reasoning New Skills from Existing Abilities for Cross-Task Robotic Manipulation
May 2, 2026Xitie Zhang, Aming Wu, Yahong HanICL for Robotics Zero-Shot Robotic Manipulation
School of Artificial Intelligence, College of Intelligence and Computing, Tianjin University, China · School of Computer Science and Information Engineering, Hefei University of Technology, China
Bimanual Robot Manipulation via Multi-Agent In-Context Learning
Apr 22, 2026Alessio Palma, Indro Spinelli, Vignesh Prasad +4ICL for Robotics Language Model-Based Control
Sapienza University of Rome, Italy · TU Darmstadt, Germany · Hessian.AI, Germany
A Hierarchical Spatiotemporal Action Tokenizer for In-Context Imitation Learning in Robotics
Apr 16, 2026Fawad Javed Fateh, Ali Shah Ali, Murad Popattia +4ICL for Robotics Vector Quantization
Retrocausal, Inc., Redmond, WA 98052, USA
Diffusion Sequence Models for Generative In-Context Meta-Learning of Robot Dynamics
Apr 15, 2026Angelo Moroncelli, Matteo Rufolo, Gunes Cagin Aydin +2Robot Dynamics Identification Robot System Identification
University of Applied Science and Arts of Southern Switzerland, Department of Innovative Technologies, IDSIA-SUPSI, Lugano, Switzerland. · Università della Svizzera Italiana, Faculty of Informatics, Lugano, Switzerland. · Politecnico di Milano, Mechanical Department, Milano, Italy.
DiffusionAnything: End-to-End In-context Diffusion Learning for Unified Navigation and Pre-Grasp Motion
Mar 27, 2026Iana Zhura, Yara Mahmoud, Jeffrin Sam +4Visual Robot Navigation Diffusion-Based Planning
The authors are with the Intelligent Space Robotics Laboratory, Center for Digital Engineering, Skolkovo Institute of Science and Technology.
RoboHarness: A Memory-Augmented Policy Harness for Vision-Language-Action Model Robustness via In-Context Adaptation
Mar 25, 2026Zhuoran Li, Zhiyang Li, Kaijun Zhou +1Robustness of VLA Models Test-Time Adaptation of VLA Models
School of Computer Science, Shanghai Jiao Tong University, Shanghai, China. · Shanghai Syslong Information Technology Co., Ltd.
Confusion-Aware In-Context-Learning for Vision-Language Models in Robotic Manipulation
Mar 16, 2026Yayun He, Zuheng Kang, Botao Zhao +3VLM Robustness Vision-Language Models
Ping An Technology (Shenzhen) Co., Ltd. · Shenzhen Bao’an Middle School
ICLR: In-Context Imitation Learning with Visual Reasoning
Mar 8, 2026Toan Nguyen, Weiduo Yuan, Songlin Wei +3Robot Imitation Learning ICL for Robotics
University of Southern California · Autodesk Research
MEM: Multi-Scale Embodied Memory for Vision Language Action Models
Mar 4, 2026Marcel Torne, Karl Pertsch, Homer Walke +14Memory-Augmented Agents Long-Term Memory for Embodied Agents
Physical Intelligence · Stanford University · UC Berkeley +1
Act-Observe-Rewrite: Multimodal Coding Agents as In-Context Policy Learners for Robot Manipulation
Mar 3, 2026Vaishak KumarCode Generation for Robotics Robot Policy Learning
General Bionix, Inc.

May 2, 2026J/K move · Enter open · S save

Decompose and Recompose: Reasoning New Skills from Existing Abilities for Cross-Task Robotic Manipulation

Xitie Zhang, Aming Wu, Yahong Han

School of Artificial Intelligence, College of Intelligence and Computing, Tianjin University, China · School of Computer Science and Information Engineering, Hefei University of Technology, China

ICL for Robotics Zero-Shot Robotic Manipulation Robot Skill Learning Generalization in Robotic Manipulation Multi-Task Robotic Manipulation Robotic Manipulation+2

Open paper AskPDF arXiv

Cross-task generalization is a core challenge in open-world robotic manipulation, and the key lies in extracting transferable manipulation knowledge from seen tasks. Recent in-context learning approaches leverage seen task demonstrations to generate actions for unseen tasks without parameter updates. However, existing methods provide only low-level continuous action sequences as context, failing to capture composable skill knowledge and causing models to degenerate into superficial trajectory imitation. We propose Decompose and Recompose, a skill reasoning framework using atomic skill-action pairs as intermediate representations. Our approach decomposes seen demonstrations into interpretable skill--action alignments, enabling the model to recompose these skills for unseen tasks through compositional reasoning. Specifically, we construct a task-adaptive dynamic demonstration library via visual-semantic retrieval combined with skill sequences from a planning agent, complemented by a coverage-aware static library to fill missing skill patterns. Together, these yield skill-comprehensive demonstrations that explicitly elicit compositional reasoning for skill composition and execution ordering. Experiments on the AGNOSTOS benchmark and real-world environments validate our method's zero-shot cross-task generalization capability.