ROI-Driven Foveated Attention for Unified Egocentric Representations in Vision-Language-Action Systems
Organizations: Synthoid.ai, Shanghai, China · Politecnico di Milano, Milan, Italy
Abstract
The development of embodied AI systems is increasingly constrained by the availability and structure of physical interaction data. Despite recent advances in vision-language-action (VLA) models, current pipelines suffer from high data collection cost, limited cross-embodiment alignment, and poor transfer from internet-scale visual data to robot control. We propose a region-of-interest (ROI) driven engineering workflow that introduces an egocentric, geometry-grounded data representation. By projecting end-effector poses via forward kinematics (FK) into a single external camera, we derive movement-aligned hand-centric ROIs without requiring wrist-mounted cameras or multi-view systems. Unlike directly downsampling the full frame, ROI is cropped from the original image before resizing, preserving high local information density for contact-critical regions while retaining global context. We present a reproducible pipeline covering calibration, synchronization, ROI generation, deterministic boundary handling, and metadata governance. The resulting representation is embodiment-aligned and viewpoint-normalized, enabling data reuse across heterogeneous robots. We argue that egocentric ROI serves as a practical data abstraction for scalable collection and cross-embodiment learning, bridging internet-scale perception and robot-specific control.
Figures & tables
| Strategy | Cameras | Calibration Burden | Sync Cost | Hand-Centric Signal | Notes |
|---|---|---|---|---|---|
| Single external camera only | 1 fixed | Low–Medium (eye-to-hand) | Low | Low–Medium | Simplest setup; weak local contact detail. |
| Wrist camera (eye-in-hand) | 1+ wrist | High | Medium | High | Strong local view; recurring hand–eye maintenance required. |
| Multi-view exocentric setup | 2–4+ fixed | High | High | Medium–High | Good observability; expensive calibration/synchronization. |
| Single camera + FK-projected ROI (ours) | 1 fixed | Medium (one-time eye-to-hand + validation) | Medium | High | Derived hand-centric stream without extra sensors; requires robust QC. |