Language Conditioned
Language-conditioned robotics focuses on enabling robots to understand and execute tasks described in natural language, bridging the gap between human instruction and robotic action. Current research emphasizes developing robust models, often leveraging vision-language models (VLMs) and large language models (LLMs), to translate instructions into executable control sequences, often within reinforcement learning or imitation learning frameworks. This field is significant because it promises to greatly simplify human-robot interaction and enable robots to perform more complex and diverse tasks in unstructured environments, impacting fields like manufacturing, healthcare, and domestic assistance. A key challenge remains improving generalization to unseen tasks and environments.
Papers
AnyBimanual: Transferring Unimanual Policy for General Bimanual Manipulation
Guanxing Lu, Tengbo Yu, Haoyuan Deng, Season Si Chen, Yansong Tang, Ziwei Wang
LLMs for Generalizable Language-Conditioned Policy Learning under Minimal Data Requirements
Thomas Pouplin, Katarzyna Kobalczyk, Hao Sun, Mihaela van der Schaar
Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulation
Homanga Bharadhwaj, Debidatta Dwibedi, Abhinav Gupta, Shubham Tulsiani, Carl Doersch, Ted Xiao, Dhruv Shah, Fei Xia, Dorsa Sadigh, Sean Kirmani
From Goal-Conditioned to Language-Conditioned Agents via Vision-Language Models
Theo Cachet, Christopher R. Dance, Olivier Sigaud