Language-Conditioned Robot Manipulation

Follow this topic Ask about this topic

Momentum

43 papers in the last four weeks, up 760% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 173

All topics

A Conversational Framework for Human-Robot Collaborative Manipulation with Distributed Generative AI models
Jun 4, 2026Arash Ghasemzadeh Kakroudi, Roel PietersLanguage-Conditioned Robot Manipulation Large Language Model-Based Robot Planning
Automation Technology and Mechanical Engineering, Tampere University, 33720, Tampere, Finland
Affordance2Action: Task-Conditioned Scene-level Affordance Grounding for Real-Time Manipulation
Jun 2, 2026Litao Liu, Yifan Han, Pengfei Yi +9Language-Conditioned Robot Manipulation Robotic Manipulation
Department of Computer Science, Rutgers University-New Brunswick · The Hong Kong University of Science and Technology (GZ) · Shanghai AI Laboratory
AFUN: Towards an Affordance Foundation Model for Functionality Understanding
Jun 1, 2026Zhaoning Wang, Yi Zhong, Jiawei Fu +2Language-Conditioned Robot Manipulation Robot Foundation Models
University of Michigan · University of California, San Diego · NVIDIA
RoboSemanticBench: Diagnosing Semantic Grounding in Action Prediction for VLA Models
Jun 1, 2026Bin Yu, Yao Zhang, Haishan Liu +9Language-Conditioned Robot Manipulation Vision-Language Grounding
1HIT · 2ZGCA · 3ZGCI +6
DeMaVLA: A Vision-Language-Action Foundation Model for Generalizable Deformable Manipulation
May 29, 2026Taiyi Su, Jian Zhu, Tianjian Wang +9Language-Conditioned Robot Manipulation Deformable Object Manipulation
1AIRC, Midea Group · 2Tongji University
ELAN4D: Embodiment-Centric 4D Supervision for Vision-Language-Action Models via Plug-and-Play Adaptation
May 28, 2026Zeyuan He, Bowen Yang, Zhirui Fang +9Language-Conditioned Robot Manipulation Visuomotor Policy Learning
Torr Vision Group, University of Oxford · The Chinese University of Hong Kong, Shenzhen · Shanghai Jiao Tong University +3
Gaze2Act: Gaze-Conditioned Vision-Language-Action Policies for Interactive Robot Manipulation
May 28, 2026Kuangji Zuo, Gen Li, Bofan Lyu +9Language-Conditioned Robot Manipulation Robotic Manipulation
1MARS Lab, Nanyang Technological University
Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments
May 28, 2026Qiuyue Wang, Mingsheng Li, Jian Guan +37Language-Conditioned Robot Manipulation Robot Foundation Models
Qwen Team
Mag-VLA: Vision-Language-Action Model for Bimanual Magnetically Actuated Microrobot Manipulation
May 27, 2026Yongchen Wang, Kangyi Lu, Lan Wei +1Language-Conditioned Robot Manipulation Bimanual Robotic Manipulation
Department of Bioengineering, Imperial-X, Imperial College London
Tabero: Learning Gentle Manipulation with Closed-Loop Force Feedback from Vision, Touch, and Language
May 27, 2026Qiwei Wu, Rui Zhang, Xin Xiang +4Language-Conditioned Robot Manipulation Robotic Manipulation
Nvidia, Beijing, China · Hong Kong University of Sci
Agentic Language-to-Objective Synthesis for Optofluidic Assembly
May 26, 2026Ivan Saraev, Elena Erben, Weida Liao +4Language-Conditioned Robot Manipulation Agentic Workflows
1) Institute of Biological and Chemical Systems, Karlsruhe Institute of Technology, Germany · 2) Department of Applied Mathematics and Theoretical Physics, University of Cambridge, UK · 3) Department of Mathematics, Imperial College London, UK +1
FineVLA: Fine-Grained Instruction Alignment for Steerable Vision-Language-Action Policies
May 26, 2026Xintong Hu, Xuhong Huang, Jinyu Zhang +11Language-Conditioned Robot Manipulation Vision-Language-Action Models
XLANG Lab, The University of Hong Kong · Qwen Team, Alibaba Inc.
Spatial Memory for Out-of-Vision Manipulation in Vision-Language-Action
May 21, 2026Pengteng Li, Weiyu Guo, He Zhang +4Language-Conditioned Robot Manipulation Robotics
Thrust of Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou), China · AI2Robotics · Department of Computer Science and Engineering, The Hong Kong University of Science and Technology Hong Kong SAR, China.
PointACT: Vision-Language-Action Models with Multi-Scale Point-Action Interaction
May 20, 2026Shizhe Chen, Paul Pacaud, Cordelia SchmidLanguage-Conditioned Robot Manipulation Point Cloud Learning
Inria, ´Ecole normale sup´erieure, CNRS, PSL Research University
DISC: Decoupling Instruction from State-Conditioned Control via Policy Generation
May 20, 2026Hanxiang Ren, Pei Zhou, Xunzhe Zhou +1Language-Conditioned Robot Manipulation Visuomotor Policy Learning
‡Zhejiang University §The University of Hong Kong ¶TranscEngram
RoVLA: Multi-Consistency Constraints for Robust Vision-Language-Action Models
May 19, 2026Jingzhou Luo, Yifan Wen, Yongjie Bai +3VLM Robustness Language-Conditioned Robot Manipulation
Sun Yat-sen University, China · Peng Cheng Laboratory · Guangdong Key Laboratory of Big Data Analysis and Processing +1
ManiSoft: Towards Vision-Language Manipulation for Soft Continuum Robotics
May 18, 2026Ziyu Wei, Luting Wang, Chen Gao +2Language-Conditioned Robot Manipulation Robotic RL
Beihang University · Hangzhou Innovation Institute, Beihang University · National University of Singapore
Key-Gram: Extensible World Knowledge for Embodied Manipulation
May 18, 2026Jingjing Fan, Siyuan Li, Botao Ren +1Memory-Augmented VLMs Language-Conditioned Robot Manipulation
Department of Computer Science and Technology Tsinghua University · Department of Automation Tsinghua University
From Human Videos to Robot Manipulation: A Survey on Scalable Vision-Language-Action Learning with Human-Centric Data
May 18, 2026Zhiyuan Feng, Qixiu Li, Huizhi Liang +12Language-Conditioned Robot Manipulation Cross-Embodiment Robot Learning
1Tsinghua University · 5Microsoft Research Asia · 2HKUST +4
RoboFlow4D: A Lightweight Flow World Model Toward Real-Time Flow-Guided Robotic Manipulation
May 17, 2026Sixu Lin, Junliang Chen, Huaiyuan Xu +8World Models for Robotics Language-Conditioned Robot Manipulation
School of Data Science, The Chinese University of Hong Kong (Shenzhen) · The Hong Kong Polytechnic University · Shenzhen Loop Area Institute +1
AffordVLA: Injecting Affordance Representations into Vision-Language-Action Models via Implicit Feature Alignment
May 17, 2026Weijie Kong, Zhian Su, Wei Yu +1Language-Conditioned Robot Manipulation Vision-Language-Action Models
Grasp Lab, School of Mechanical Engineering of Zhejiang University, Hangzhou 310027, China · Torch Kernel Co., Ltd., Hangzhou, 310000, P. R. China
How to Instruct Your Robot: Dense Language Annotations Power Robot Policy Learning
May 16, 2026Bosung Kim, Ruiyi Wang, David Acuna +5Robot Policy Learning Robot Skill Learning
University of California, San Diego · 2NVIDIA
Health-Conditioned Vision-Language-Action Models for Malfunction-Aware Robot Control
May 15, 2026Hüseyin Arslan, Özgür ErkentRobotic Control Language-Conditioned Robot Manipulation
*Computer Engineering, Hacettepe University, Ankara, Turkey
Evo-Depth: A Lightweight Depth-Enhanced Vision-Language-Action Model
May 14, 2026Tao Lin, Yuxin Du, Jiting Liu +14Language-Conditioned Robot Manipulation Efficient VLA Models
School of AI, Shanghai Jiao Tong University · King Abdullah University of Science and Technology · SJTU-Quic Robot Joint Lab +1
IntentVLA: Short-Horizon Intent Modeling for Aliased Robot Manipulation
May 14, 2026Shijie Lian, Bin Yu, Xiaopeng Lin +8Robot Policy Learning Language-Conditioned Robot Manipulation
1HUST · 2ZGCA · 4HIT +7
SECOND-Grasp: Semantic Contact-guided Dexterous Grasping
May 13, 2026Han Yi Shin, Heeju Ko, Jaewon Mun +6Robotic Grasping Language-Conditioned Robot Manipulation
1Korea University · University of Texas at Austin · University of Michigan +1
Beyond World-Frame Action Heads: Motion-Centric Action Frames for Vision-Language-Action Models
May 12, 2026Huoren Yang, Jianchao Zhao, Hu Yusong +7Language-Conditioned Robot Manipulation Visuomotor Policy Learning
Xi’an Jiaotong University, Xi’an, Shaanxi, CN · One Robotics, Shenzhen, Guangdong, China · Shenzhen University of Advanced Technology, Shenzhen, Guangdong, CN
Forecast-aware Gaussian Splatting for Predictive 3D Representation in Language-Guided Pick-and-Place Manipulation
May 11, 2026Kaixin Jia, Jiacheng Xu3D Gaussian Splatting Language-Conditioned Robot Manipulation
KTH Royal Institute of Technology, Stockholm, Sweden.
ElasticFlow: One-Step Physics-Consistent Policy with Elastic Time Horizons for Language-Guided Manipulation
May 9, 2026Kewei Chen, Yayu Long, Shuai Li +1Flow Matching Long-Horizon Robotic Manipulation
Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences · Chongqing School, University of Chinese Academy of Sciences · Faculty of Information Technology and Electrical Engineering, University of Oulu, Finland
TriRelVLA: Triadic Relational Structure for Generalizable Embodied Manipulation
May 7, 2026Hanyu Zhou, Chuanhao Ma, Gim Hee LeeLanguage-Conditioned Robot Manipulation Robotic Manipulation
School of Computing, National University of Singapore · School of Artificial Intelligence and Automation, Huazhong University of Science and Technology

Jun 4, 2026, cs.ROJ/K move · Enter open · S save

A Conversational Framework for Human-Robot Collaborative Manipulation with Distributed Generative AI models

Arash Ghasemzadeh Kakroudi, Roel Pieters

Automation Technology and Mechanical Engineering, Tampere University, 33720, Tampere, Finland

Language-Conditioned Robot Manipulation Large Language Model-Based Robot Planning Human-Robot Collaboration VLMs for Robotics

Open paper AskPDF arXiv

This paper presents a distributed conversational framework for human-robot collaborative manipulation that integrates local language and vision-language models (VLMs) with a Robot Operating System 2 (ROS 2)-based execution stack. Language understanding, visual grounding, orchestration, and motion execution run as separate ROS 2 nodes, enabling flexible deployment across distributed hardware while maintaining a responsive control loop. From free-form user commands, the system generates structured action requests for pick, place, and handover. It uses a VLM to return image-space targets, which are converted into metric robot-frame goals using depth and calibration. A web dashboard exposes intermediate intent and grounding overlays (pixel, depth, and robot-frame) and requires explicit operator confirmation before any motion is executed. Experiments on a Franka FR3 platform evaluate end-to-end task reliability and latency under increasing working table scene ambiguity and compare alternative LLM/VLM configurations in the same pipeline. Code and full documentation are available at github.com/cogrob-tuni/franka-llm.

Momentum

Related topics

Latest papers 173