Language-Conditioned Robot Control

Ask about this topic

Overview

The field studies how robots can turn spoken or written instructions into actions that fit what they perceive and what their bodies can do, from object manipulation and navigation to humanoid movement. Approaches include end-to-end vision-language-action policies, language-model planners paired with low-level controllers, and reusable skills or robot programs; training draws on demonstrations, simulation, reinforcement learning, and motion data. Work is expanding from simulated benchmarks such as LIBERO to real robots, including mobile platforms, drones, arms, and humanoids, with added tools for user feedback, safety checks, and adaptation. A central challenge is making language understanding reliable under ambiguity and distribution shift while ensuring that generated actions remain physically feasible, safe, and responsive in closed-loop execution.

Latest papers

CardsList

74 papers12 papers in the last four weeks, up 71% on the four weeks before. 0.1% of all new papers.

  1. Low-Burden LLM-Based Preference Learning: Personalizing Assistive Robots from Natural Language Feedback for Users with Paralysis

    Apr 1, 2026Keshav Shankar, Dan Ding, Wei GaoAssistive RoboticsRobot Learning

    Electrical and Computer Engineering, University of Pittsburgh, Pittsburgh, PA. · Rehabilitation Science and Technology, University of Pittsburgh, Pittsburgh, PA.

  2. ThermoAct:Thermal-Aware Vision-Language-Action Models for Robotic Perception and Decision-Making

    Mar 26, 2026Young-Chae Son, Dae-Kwan Ko, Yoon-Ji Choi +1Robotic PerceptionLanguage-Conditioned Robot Control

    Dongguk University

  3. QuadFM: Foundational Text-Driven Quadruped Motion Dataset for Generation and Control

    Mar 25, 2026Li Gao, Fuzhi Yang, Jianhui Chen +4Robot Motion GenerationLegged Robotics

    AMAP, Alibaba Group

  4. SafeFlow: Real-Time Text-Driven Humanoid Whole-Body Control via Physics-Guided Rectified Flow and Selective Safety Gating

    Mar 25, 2026Hanbyel Cho, Sang-Hun Kim, Jeonguk Kang +1Robotic ControlHumanoid Robot Control

    Physical AI Lab, RX, Samsung Electronics

  5. RoboForge: Physically Optimized Text-guided Whole-Body Locomotion for Humanoids

    Mar 18, 2026Xichen Yuan, Zhe Li, Bofan Lyu +4Humanoid Robot LocomotionHuman Motion Generation

    MARS Lab, Nanyang Technological University

  6. Interpreting Context-Aware Human Preferences for Multi-Objective Robot Navigation

    Mar 18, 2026Tharun Sethuraman, Subham Agrawal, Nils Dengler +3Robot NavigationRobot Policy Adaptation

    Hochschule Bonn-Rhein-Sieg, Germany · University of Bonn, Germany · Lamarr Institute for Machine Learning and Artificial Intelligence, Germany

  7. ECHO: Edge-Cloud Humanoid Orchestration for Language-to-Motion Control

    Mar 17, 2026Haozhe Jia, Jianfei Song, Yuan Zhang +5Language-Conditioned Robot ControlRobot Motion Generation

    The Hong Kong University of Science and Technology (Guangzhou). · LimX Dynamics Technology Co., Ltd. · Shandong University. +1

  8. IROSA: Interactive Robot Skill Adaptation using Natural Language

    Mar 4, 2026Markus Knauer, Samuel Bustamante, Thomas Eiband +3Robot Policy AdaptationRobot Skill Learning

    All authors are with the German Aerospace Center (DLR), Institute of Robotics and Mechatronics (RMC), Münchener Str. 20, 82234 Weßling, Germany. · Markus Knauer, Alin Albu-Schäffer and Samuel Bustamante are also with the School of Computation, Information and Technology (CIT), Technical University of Munich (TUM), Arcisstr. 21, 80333 Munich, Germany.

  9. LAD-Drive: Bridging Language and Trajectory with Action-Aware Diffusion Transformers

    Mar 2, 2026Fabian Schmidt, Karol Fedurko, Markus Enzweiler +1Language-Conditioned Robot PlanningAutonomous Driving Planning

    Institute for Intelligent Systems, Esslingen University of Applied Sciences, Germany. · Department of Computer Science, University of Freiburg, Germany.

  10. TMR-VLA:Vision-Language-Action Model for Magnetic Motion Control of Tri-leg Silicone-based Soft Robot

    Feb 28, 2026Ruijie Tang, Chi Kit Ng, Kaixuan Wu +5Language-Conditioned Robot ControlSoft Robotics

  11. SignVLA: A Gloss-Free Vision-Language-Action Framework for Real-Time Sign Language-Guided Robotic Manipulation

    Feb 26, 2026Xinyu Tan, Ningwei Bai, Harry Gardener +6Vision-Language-Action ModelsHuman-Robot Interaction

    Department of Computer Science, University College London (UCL), London, WC1E 6BT, U.K.

  12. An Approach to Combining Video and Speech with Large Language Models in Human-Robot Interaction

    Feb 23, 2026Guanting Shen, Zi TianHuman-Robot InteractionLanguage-Conditioned Robot Manipulation

    The authors are with the Dalian University of Technology. Dalian, China.

  13. When Vision Overrides Language: Evaluating and Mitigating Counterfactual Failures in VLAs

    Feb 19, 2026Yu Fang, Yuchun Feng, Dong Jing +5VLM RobustnessCounterfactual Evaluation of VLMs

    University of North Carolina at Chapel Hill

  14. Prompting Robot Teams with Natural Language

    Sep 29, 2025Eduardo Sebastián, Nicolas Pfitzer, Ajay Shankar +1Large Language Model-Based Robot PlanningMulti-Robot Coordination

    Department of Computer Science and Technology, University of Cambridge, UK · Department of Mechanical and Process Engineering, ETH Zurich, Switzerland