Vision-Based Robot Control

Follow this topic Ask about this topic

Momentum

31 papers in the last four weeks, up 288% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 134

All topics

Flow Control: Steering Vision-Language-Action Models with Simple Real-Time Inputs
Jun 8, 2026Jonathan C. Kao, Jason Chan, Andy WangRobotic Control Vision-Language-Action Models
Departments of Electrical and Computer Engineering, Computer Science, and Neurobiology, University of California, Los Angeles · Department of Computer Science, University of California, Los Angeles
CT-VAM: A Cerebello-Thalamic-Inspired Vision-Action Model for Efficient Visuomotor Control
Jun 8, 2026Jiacheng Li, Yize Guo, Jiabin Guo +2Robotic Control Visuomotor Policy Learning
AIRLab, Department of Automation University of Science and Technology of China Hefei, China
VAIC: Vision-Guided Humanoid Agile Object Interaction Control via Decoupled Commands
Jun 8, 2026Dongting Li, Qianyang Wu, Xingyu Chen +9Humanoid Robot Control Robotic Manipulation
Tsinghua University · Xiaomi Robotics Lab · HKUST(Guangzhou)
Autonomous FPV Flight with Translational Optical Flow and Uncertainty Mask
Jun 8, 2026Yang Deng, Yu Hu, Feng Yu +2Optical Flow UAV Navigation
Shanghai Jiao Tong University, Shanghai 200240, China
Vision-Guided Dual-Arm Humanoid Robotic Disassembly of End-of-Life 18650 Lithium-ion Battery Packs
Jun 6, 2026Yile Chen, Zhihao Liu, Xi Vincent Wang +1Bimanual Robotic Manipulation Robotic Manipulation
KTH Royal Institute of Technology, Brinellvägen 8, 114 28, Stockholm, Sweden
RhinoVLA Technical Report
Jun 5, 2026Huixi Technology, :, Chen Zhang +13Efficient VLM Inference Efficient VLA Model Inference
Huixi Technology
QuadVerse: An Integrated Framework Aligning Visual-Physical Reality for Quadruped Simulation
Jun 5, 2026Yuxiang Chen, Yuanhao Wang, Ziheng Zhang +6Robotics Simulation Sim-to-Real Transfer
1Nanjing University · 2BUPT · 3DEXMAL +1
Multi-Robot Planning and Control from CCTV Camera Networks in a Real Warehouse
Jun 4, 2026Luke Robinson, Benjamin Ramtoula, Anas Izaaryene +2Robotic Control Multi-Robot Motion Planning
Oxford Robotics Institute, University of Oxford, UK · Robot Systems Group, Technical University of Munich, Germany
MAD: Mapping-Aware World Models for Agile Quadrotor Flight
Jun 3, 2026Xinhong Zhang, Runqing Wang, Yunfan Ren +6Robot Policy Learning World Models for Robotics
State Key Lab of Autonomous Intelligent Unmanned Systems, Beijing Institute of Technology, Beijing 100081, China · Zhongguancun Academy, Beijing 100094, China · School of Computer Science and Technology, Tongji University +2
PointAction: 3D Points as Universal Action Representations for Robot Control
Jun 2, 2026Mutian Tong, Han Jiang, Qiao Feng +2Generalization in Robotic Manipulation Cross-Embodiment Robot Learning
University of Pennsylvania · Work done during internship at University of Pennsylvania.
PerchRL: Vision-Based Agile Perching on Inclined Platforms under Rapid and Irregular Motion
Jun 2, 2026Zihong Lu, Zongzhuo Liu, Huaxu Li +5RL for Robotics Aerial Robotics
1SUSTech · 2HITSZ · 3PCL +1
SSR: Scaling Surefooted and Symmetric Humanoid Traversal to the Open World
May 29, 2026Ruiqi Yu, Yiwen Wang, Yuan Hao +2Humanoid Robot Locomotion Terrain-Aware Robot Locomotion
Zhejiang University
MASt3R-Nav: WayPixel Navigation in Relative 3D Maps
May 22, 2026Vansh Garg, Rohit Jayanti, Krish Pandya +5Robot Navigation Robot Motion Planning
Robotics Research Center, IIIT-Hyderabad, India · University of Heidelberg · MBZUAI
Vision-Based Agile Landing on Turbulent Waters
May 22, 2026Dimosthenis Angelis, Leonard Bauersfeld, Davide Scaramuzza +1RL for Robotics Aerial Robotics
Department of Electrical and Photonics Engineering, Technical University of Denmark, Kgs. Lyngby, Denmark · Robotics and Perception Group, Department of Informatics, University of Zurich, Zurich, Switzerland
Terminal Constraint Model Predictive Control for Image-Based Visual Servoing of UAVs with Kalman Filter-Based Moment Loss Compensation
May 21, 2026X. Wang, Y. Cao, W. L. W. Leong +4Model Predictive Control Vision-Based Robot Control
College of Design and Engineering, National University of Singapore, Singapore
Action with Visual Primitives
May 21, 2026Weilong Guo, Yuchen Wang, Renping Zhou +5Generalization in Robotic Manipulation Visuomotor Policy Learning
Anyverse Dynamics · Tsinghua University
EvoScene-VLA: Evolving Scene Beliefs Inside the Action Decoder for Chunked Robot Control
May 21, 2026Chushan Zhang, Ruihan Lu, Jinguang Tong +3World Models for Robotics Vision-Language-Action Models
Australian National University · The University of Queensland · Beijing Normal University
Anomaly-Informed Confidence Calibration for Vision-Based Safety Prediction
May 20, 2026Zhenjiang Mao, Jiawen Wu, Gabriel Wagner +2Post-Hoc Calibration Autonomous Racing
Trustworthy Engineered Autonomy (TEA) Lab, Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL 32611, USA
Learning-Based Adaptive Control for Surgical Robotic Exposure Task on Deformable Tissues
May 18, 2026Jiayi Liu, Kaiqi Wei, Yiwei Wang +2Deformable Object Manipulation Vision-Based Robot Control
Huazhong University of Science and Technology, Wuhan, China
Behavior Cloning for Active Perception with Low-Resolution Egocentric Vision
May 13, 2026Anthony Bilic, Chen Chen, Ladislau BölöniActive Vision for Robot Manipulation Active Sensing
Department of Computer Science, University of Central Florida, Orlando, FL, USA
RotVLA: Rotational Latent Action for Vision-Language-Action Model
May 13, 2026Qiwei Li, Xicheng Gong, Xinghang Li +5Latent Action Learning Vision-Language-Action Models
Wangxuan Institute of Computer Technology, Peking University · Xiaomi Robotics · CASIA
Autonomous Laparoscope Control through Unified Mechanics-Based Representation of Multimodal Intraoperative Information
May 6, 2026Xiaojian Li, Jin Fang, Yudong Shi +6Vision-Based Robot Control Surgical Robotics
School of Management and the Key Laboratory of Process Optimization and Intelligent Decision-Making (Ministry of Education), Hefei University of Technology, Hefei 230009, China · Department of Thoracic Surgery, Shanghai Pulmonary Hospital, School of Medicine, Tongi University, Shanghai 200433, China · Department of Thoracic Surgery, Changhai Hospital, Shanghai 200433, China
SigLoMa: Learning Open-World Quadrupedal Loco-Manipulation from Ego-Centric Vision
May 5, 2026Shiyi Chen, Haiyi Liu, Mingye Yang +2Legged Robotics Sim-to-Real Transfer
Tsinghua University · Imperial College London
Temporally Consistent Object 6D Pose Estimation for Robot Control
May 4, 2026Kateryna Zorina, Vojtech Priban, Mederic Fourmy +2Factor Graph Optimization Object Pose Estimation
Czech Institute of Informatics, Robotics and Cybernetics, Czech Technical University in Prague
VOFA: Visual Object Goal Pushing with Force-Adaptive Control for Humanoids
May 2, 2026Zichao Hu, Zifan Xu, Dongsik Chang +6Humanoid Loco-Manipulation Whole-Body Control
Department of Computer Science, The University of Texas at Austin · Amazon Inc. · Sony AI
Evidence-Based Landing Site Selection and Vison-Based Landing for UAVs in Unstructured Environments
May 2, 2026Sina Sajjadi, Jacopo Panerati, Sina Soleymanpour +3Aerial Robotics Vision-Based Robot Control
Aerospace Research Centre, National Research Council Canada, Montreal, QC, Canada · Department of Mechanical and Industrial Engineering, Toronto Metropolitan University, Toronto, ON, Canada
Stereo Multistage Spatial Attention for Real-Time Mobile Manipulation Under Visual Scale Variation and Disturbances
May 1, 2026Xianbo Cai, Hideyuki Ichiwara, Hyogo Hiruma +3Mobile Manipulation Visuomotor Policy Learning
Department of Intermedia Art and Science, Waseda University, Tokyo, Japan · SB Intuitions Corp., Tokyo, Japan · Research and Development Group, Hitachi, Ltd., Ibaraki, Japan +1
ExoActor: Exocentric Video Generation as Generalizable Interactive Humanoid Control
Apr 30, 2026Yanghao Zhou, Jingyu Ma, Yibo Peng +3Humanoid Robot Control Language-Conditioned Robot Control
Beijing Academy of Artificial Intelligence (BAAI)
HiPAN: Hierarchical Posture-Adaptive Navigation for Quadruped Robots in Unstructured 3D Environments
Apr 29, 2026Jeil Jeong, Minsung Yoon, Seokryun Choi +3Robot Navigation Vision-Based Robot Control
School of Computing, KAIST, South Korea.
VISION-SLS: Safe Perception-Based Control from Learned Visual Representations via System Level Synthesis
Apr 27, 2026Antoine P. Leeman, Shuyu Zhan, Melanie N. Zeilinger +1Safety-Critical Control Robust Control
Institute for Dynamic Systems and Control, ETH Zürich · Georgia Institute of Technology, Schools of 2Interactive Computing, 3Aerospace Engineering

Jun 8, 2026, cs.ROJ/K move · Enter open · S save

Flow Control: Steering Vision-Language-Action Models with Simple Real-Time Inputs

Jonathan C. Kao, Jason Chan, Andy Wang

Departments of Electrical and Computer Engineering, Computer Science, and Neurobiology, University of California, Los Angeles · Department of Computer Science, University of California, Los Angeles

Robotic Control Vision-Language-Action Models Vision-Based Robot Control VLMs for Robotics

Open paper AskPDF arXiv

We introduce flow control of vision-language-action (VLA) models, a simple and effective way to steer VLA actions in real-time through generic inputs, such as a keyboard. This method can be used out-of-the-box and does not require retraining or fine-tuning VLAs. It enables relatively crude user inputs to steer a VLA to align with user intent. The VLA transforms these inputs into action samples drawn from the VLA expert action distribution learned during training, so that the generated actions are high quality (conformity to the action expert distribution) and high fidelity (reflecting the user's intent). We demonstrate that flow control has many desirable properties: (1) flow control accurately and responsively steers robot actions with user inputs, (2) it is robust to suboptimal user inputs, (3) it enables users to steer VLAs to achieve significantly higher success rates and faster task completion, and (4) fine-tuning a VLA on flow control trajectories improves the autonomous policy. Together, these results provide a simple and intuitive way for users to help steer VLA actions, increasing task performance.

Momentum

Related topics

Latest papers 134