cs.ROSep 23, 2025

Look as You Leap: Planning Simultaneous Motion and Perception for High-DOF Robots

Authors: Qingxi Meng, Emiliano Flores, Carlos Quintero-Peña, Peizhu Qian, Zachary Kingston, Shannan K. Hamlin, Vaibhav Unhelkar, Lydia E. Kavraki

Organizations: Department of Computer Science, Rice University, Houston, TX 77005 USA · Houston Methodist Academic Institute, Houston, TX 77030 USA · Department of Computer Science, Rice University, Houston, TX 77005 USA, and also with the Ken Kennedy Institute, Rice University, Houston, TX 77005 USA

Abstract

Most common tasks for robots in dynamic spaces require that the environment is regularly and actively perceived. The perception task considered in this work can represent a broad range of robot perception objectives, including object detection, human activity recognition, and human face detection. For example, a service robot may need to continuously localize an object during manipulation, while an assistive robot may need to reliably perceive a human face or activity for interaction and safety. These tasks impose perception constraints on robot motion. However, solving motion and perception tasks simultaneously is challenging, as their requirements often conflict. Furthermore, robots must react quickly to environmental changes, while directly evaluating perception quality (e.g., object detection confidence) is often expensive or infeasible at runtime. This problem is especially important in human-centered environments, such as homes and hospitals, where effective perception is essential for safe operation. In this work, we address motion planning for high-degree-of-freedom (DoF) robots from a start to a goal configuration with continuous perception constraints in static and dynamic environments. Our solution is a GPU-parallelized perception-score-guided probabilistic roadmap planner with a neural surrogate model (PS-PRM). Unlike existing active perception-, visibility-aware, or learning-based planners, our work jointly considers perception tasks and constraints when searching for a motion-planning solution. Our method uses a neural surrogate model to approximate perception scores, incorporates them into roadmap planning, and leverages GPU parallelism for efficient online replanning. We demonstrate that our planner outperforms RL- and trajectory-optimization-based baselines in static and dynamic environments in simulation and real-robot experiments.

Figures & tables

Explore similar work

CardsList
  1. Sampling-Based Motion Planning with Scene Graphs Under Perception Constraints

    Mar 3, 2026Qingxi Meng, Emiliano Flores, Thai Duong +2perception-aware motion planningScene Graphs

  2. G-MAPP: GPU-accelerated Multi-Agent Planning and Perception for Reactive Motion Generation

    Jun 10, 2026Tanmay Bishnoi, Riddhiman Laha, Tobias Löw +3World Models for RoboticsRobot Motion Generation