Vision-Based Robot Control

Overview

The field studies how robots can use camera images to choose and revise actions, allowing them to operate when precise maps or external tracking are unavailable. Approaches range from image-based visual servoing and model-predictive control to imitation learning, reinforcement learning, and vision-language-action or world-model policies. These methods are tested in tasks such as object manipulation, legged locomotion, aerial navigation, and surgical camera control, using simulation benchmarks and real-robot trials. A central challenge is making policies transfer across cameras, robots, and environments while remaining responsive and safe under occlusion, latency, and changing conditions.

Latest papers

241 papers25 papers in the last four weeks, down 49% on the four weeks before. 0.2% of all new papers.

Mar 25, 2025

Design of an Underwater Vehicle With Orientation-Adjustable Thrusters and Adaptive Tracking Control

Unmanned underwater vehicles (UUVs) are essential for marine exploration and research. However, conventional designs often struggle with limited maneuverability in complex and dynamic underwater environments. This paper presents an orientation-adjustable thruster UUV (OAT-UUV) with vector thrusters, enabling full six-degree-of-freedom (6-DOF) motion. and composite maneuvers. To overcome challenges associated with uncertain model parameters and environmental disturbances, a vision-based tracking control system based on a novel feedforward adaptive model predictive controller (FF-AMPC) is proposed to achieve robust object tracking, which integrates realtime state feedback with adaptive parameter updating. Extensive experiments, including closed-loop tracking and composite motion tests in real-world environment, have been conducted. The obtained results demonstrate that the orientation-adjustable propulsion design significantly improves the underwater maneuverability, while the FF-AMPC controller achieves a 49.1% hovering improvement compared to conventional controller.