A fundamental assumption in robotic perception is that the sensor's field of view (FoV) is fixed relative to the robot body. Motion-decoupled sensors, such as gimbal-mounted cameras and MEMS-based LiDARs, instead allow sensing direction to be controlled independently at runtime. This freedom creates a computational challenge: efficiently selecting useful viewing directions online in feature-dense environments. We propose PIVOT, a lightweight iterative method that optimizes sensor viewing direction along a fixed translation trajectory to maximize feature visibility. Under a conical FoV model, visibility depends only on the optical axis, yielding a two-degree-of-freedom optimization on the viewing sphere S2. Coordinate-free SO(3) exponential-map updates enable efficient continuous optimization without explicit angular parameterizations or exhaustive viewing-sphere search. Monte Carlo evaluations retain 98.1--99.6% of brute-force visibility with a 76--85x speedup. Photorealistic simulation and real-world experiments further demonstrate improved visual localization robustness and practical viewpoint control on a quadruped robot.
Existing robotic perception is constrained by sensors that are either robot-mounted or permanently fixed in the environment, locking perception to a limited set of viewpoints. Yet as robots perform increasingly diverse tasks, the most informative viewpoint shifts from one task to the next-often somewhere onboard sensor and static infrastructure can not readily satisfy. To address this gap, we propose SensorPerch, a novel realization of active perception that decouples sensing from both the robot embodiment and the environment by treating sensors as independent physical entities that the robot can autonomously detach and re-attach within the environment. SensorPerch presents one realization of this paradigm: a lightweight, wireless, reconfigurable sensor platform that can perch on diverse surfaces, paired with a viewpoint-selection framework that determines task-optimal sensor placements. Together, these enable robots to construct task-relevant viewpoints on demand, independent of the robot's current position and available fixed infrastructure. We demonstrate the paradigm on two task classes: (i) object-coupled perception, where SensorPerch enables persistent object-state detection beyond the robot's current position, achieving successful event detection even when the robot is not nearby; and (ii) policy-coupled perception, where SensorPerch allows robots to construct diverse, policy-specific viewpoints for various policies, achieving success rates comparable to those obtained using oracle viewpoints.
Human vision is a highly active process driven by gaze, which directs attention to task-relevant regions through foveation, dramatically reducing visual processing. In contrast, robot learning systems typically rely on passive, uniform processing of raw camera images. In this work, we explore how incorporating human-like active gaze into robotic policies can enhance efficiency and robustness. We develop GIAVA (Gaze Integrated Active-Vision ALOHA), a robot vision system that emulates human head and neck movement, and gaze adjustment for foveated processing. Extending the AV-ALOHA robot platform, we introduce a framework for simultaneously collecting eye-tracking, perspective control, and robot manipulation demonstration data from a human operator. We also open-source a simulation benchmark and dataset for training robot policies that incorporate human gaze. Inspired by recent work in foveated image segmentation and given the widespread use of Vision Transformers (ViTs) in robot learning, we integrate gaze information into ViTs using a foveated patch tokenization scheme. Compared to uniform patch tokenization, this significantly reduces the number of tokens, and thus computation. Our results show that our method for foveated robot vision drastically reduces computational overhead, and enhances robustness to background distractors. Notably, on certain high-precision tasks, foveated vision also improves performance, as reflected in higher success rates. Together, these findings suggest that human-inspired foveated visual processing offers untapped potential and should be further considered as a useful inductive bias in robotic vision systems. https://soltanilara.github.io/giava/
Visual navigation in unknown environments remains a core challenge in mobile robotics, especially for resource-constrained platforms. Most existing approaches rely on loosely coupled modular pipelines and strong assumptions on perception quality or environmental structure, often resorting to multi-modal sensor suites that increase system complexity and deployment cost. Vision-only navigation offers a lightweight alternative, but its performance degrades severely under motion blur, low texture, and illumination changes, largely because they neglect the tight coupling between commanded motion and perception. While perception-aware methods partially address this issue, they typically optimize individual modules and fail to propagate uncertainty consistently across the navigation stack. In this paper, we present UNSEEN, a unified uncertainty- and perception-aware navigation framework that explicitly couples localization, mapping, and planning using only a front-mounted camera. UNSEEN estimates sparse maps and robot poses with associated uncertainties at 6Hz, and leverages them to plan trajectories that jointly optimize task progress and estimation accuracy in receding-horizon. Simulations and extensive real-world experiments in unknown environments demonstrate the robustness of the proposed approach, with UNSEEN-SLAM reducing absolute translational error by 9.8% and UNSEEN-Plan improving estimation accuracy by up to 45% compared to state-of-the-art methods, while achieving a 100% task success rate.
Tommaso Faraci, Marco Camurri, Daniele Fontanelli +1