Lightweight and Resource-Efficient Perception for Robotic Guide Dogs

Jinse Kwon; Yoojin Lim; Choonghan Lee; Yongseung Yu; Yongin Kwon; Jemin Lee

cs.DCOct 2, 2026

Lightweight and Resource-Efficient Perception for Robotic Guide Dogs

Authors: Jinse Kwon, Yoojin Lim, Choonghan Lee, Yongseung Yu, Yongin Kwon, Jemin Lee

Organizations: Electronics and Telecommunications Research Institute (ETRI), Korea · School of Computer Science and Engineering, Pusan National University, Korea · Division of Electronics and Information Engineering, Jeonbuk National University, Korea

Abstract

Robotic guide dogs should understand their surroundings, objects, and potential risks. Prior research has focused on raw sensor data from cameras and 2D or 3D LiDAR, which precisely measure distance points rather than provide a semantic understanding of the scene. While these physical measurements are effective for robot-centric collision avoidance and robot safety, they are not suitable for human-centric guidance. The system should recognize the type and relevance of obstacles and explain them, clearly and actionably, in terms of their spatial relation to the user. We present complete on-device perception modules that fuse a 360 camera and a 2D LiDAR for reliable collision avoidance, with moving-object detection and tracking for human-centric guidance. Finally, in walking-impossible situations, a vision--language model delivers pathway explanations as a safety mechanism to reduce user anxiety. In experiments, verification of fused 360 camera--LiDAR depth shows reliable near-range perception but inherent mid-range bias, while the system as a whole sustained real-time performance under 55 W. On the real-world egocentric GuideDogQA benchmark, our system achieved 83.8% accuracy, compared with 67.1% for GPT-4o. These results demonstrate that practical human-centric guidance with real-time on-device inference is feasible even on quadrupeds.

Figures & tables

Panel of Figure 1 — Figure 1 : (a) A 360 camera and a 2D LiDAR fused with on-device AI and multi-NPU acceleration, moving from robot-centric toward human-centric guidance. (b) Overview of the proposed system, integrating perception, obstacle understanding, and VLM-based guidance.

Figure 2 : Omnidirectional depth pipeline: (1) gstreamer-based cube-map decomposition, (2) ZMQ streaming to parallel NPUs, (3) SC-DepthV3 depth estimation, (4) LiDAR-based affine correction, and (5) 3D point generation for collision avoidance.

Figure 3 : Moving-object description: four faces stream RGB–depth into NPU detection (YOLOv8m) and lightweight tracking for object kinematics; per-face tracks are aggregated with safety-based prioritization into one spoken description per cycle.

Figure 4 : Event-driven pathway explanation: on a blocked path, a VLM turns the front view into a space-aware description of obstacles and options, streamed sentence-by-sentence to TTS to explain why walking stopped.

Figure 5 : Hardware setup: Ricoh THETA Z1 360 camera, SLAMTEC C1 2D LiDAR, Jetson AGX Orin, and five Hailo-8 NPUs.

Module	AI Model (AI Task)	Resolution (W $\times$ H $\times$ B)	Thr./face (FPS)	Thr.	Power (Peak W)				Thermal ( ∘ C)
Module	AI Model (AI Task)	Resolution (W $\times$ H $\times$ B)	Thr./face (FPS)		CPU	GPU	SYS	NPU	CPU	GPU	SYS	NPU
IDLE	—	—	—	—	0.4 W	3.2 W	6.1 W	—	47.5	44.5	44.9	36.0
PCIe-m.2 ext-card x2	—	—	—	—	—	—	18.8 W	—	—	—	43.8	—
ERP-Dec	—	1920 $\times$ 960 $\times$ 1 $\rightarrow$ 480 $\times$ 480 $\times$ 4	30.0	120.0 FPS	0.8 W	—	0.4 W	—	48.5	—	46.5	—
3D-Depth	SC-DepthV3 (Depth Est)	480 $\times$ 480 $\times$ 8 $\rightarrow$ 320 $\times$ 256 $\times$ 8	25.3	* 101.3 FPS	2.1 W	—	—	8.2 W	53.1	—	—	38.7
MOD	YOLOv8m (Object Det)	480 $\times$ 480 $\times$ 4 $\rightarrow$ 640 $\times$ 640 $\times$ 4	11.4	45.7 FPS	0.3 W	—	—	1.2 W	53.5	—	—	40.5

Table 1 : Throughput and per-rail (CPU/GPU/SYS/NPU) peak power and temperature as modules are progressively enabled on the Jetson AGX Orin.

Figure 6 : Measured mean depth vs. true distance for a person at 1–10 m under different background-wall (BG) positions (error bars: std. dev.; dotted:

y=x

). Near-field accuracy is high (MAE 0.34 m at 1–3 m), but mid-range estimates drift toward the wall distance.

Figure 7 : Qualitative GuideDogQA examples from the proposed on-device system. (a) Relative-depth comparison. Estimated distances are shown above the two referenced objects (green: nearer, red: farther). Ground truth distances are shown in the labels. (b) Object recognition. The green box marks the queried object, the green text indicates the correct answer, and the check mark denotes the predicted result.

Model	Depth Comparison	Object Recognition
Random Chance	25.0	25.0
Cambrian-1	24.3	82.3
Molmo	28.5	34.0
LLaVA-1.6	30.0	80.9
LLaVA-OneVision	32.4	87.4
Qwen2.5-VL	22.2	85.7

Table 2 : GuideDogQA accuracy (%), with a random chance level of

25.0

. VLM baselines are reproduced from GuideDog [ 16 ] . Our method (SC-DepthV3 and YOLOv8m) is evaluated as a deployed NPU implementation. Bold and underlined values indicate the best and second-best results, respectively.

Figure 8 : Effect of input resolution on relative depth ordering accuracy. The proposed center-line correction improves ordering accuracy at most resolutions and achieves high accuracy at the operational resolution of

320\times 256

Method	Acc. (%)	MAE (m)
GPT-4o	59.8	n/a
SC-DepthV3 raw	74.3	35.76
Depth-Anything-V2	88.8	38.49
Ours (DA-V2 anchor)	72.8	37.29
Ours (LiDAR anchor)	74.7	31.93

Table 3: Relative-depth ordering accuracy and absolute depth error on 100 randomly sampled AI-Hub adverse-weather images using projected 3D LiDAR as ground truth. Bold/underlined values denote the best/second-best results.

Explore similar work

Milo, a Fully Autonomous Indoor/Outdoor Robotic Guide Dog
Jul 21, 2026Florian Golemo, Joanna Wolski, Joel Ruben Antony Moniz +1Autonomous Navigation Obstacle Avoidance
Mila - Quebec AI Institute Montreal, Canada · Mila - Quebec AI Institute, Polytechnique Montreal, Canada
CANINE: Coaching Visually Impaired Users for Interactive Navigation with a Robot Guide Dog
May 19, 2026Cunjun Yu, Zishuo Wang, Anxing Xiao +2Coaching Systems Navigation
School of Computing, 2Smart Systems Institute National University of Singapore
Learning Agile Perceptive Traversal of Sparse 3D Structures for Humanoids
Aug 30, 2026Efe Ongan, Chong Zhang, Boyang Sun +33D Perception Lidar
Robotic Systems Lab, ETH Zurich, Switzerland · ETH AI Center, ETH Zurich, Switzerland · Computer Vision and Geometry Group, ETH Zurich, Switzerland

Jul 21, 2026, cs.ROJ/K move · Enter open · S save

Milo, a Fully Autonomous Indoor/Outdoor Robotic Guide Dog

Florian Golemo, Joanna Wolski, Joel Ruben Antony Moniz, Christopher Pal

Mila - Quebec AI Institute Montreal, Canada · Mila - Quebec AI Institute, Polytechnique Montreal, Canada

Autonomous Navigation Obstacle Avoidance Navigation

Open paper AskPDF arXiv

Many Blind and Low-Vision (BLV) people rely on guide dogs for moment-to-moment navigation, such as staying on path and avoiding obstacles and pedestrians. However, guide dogs are expensive to acquire and maintain (approximately $50k USD plus ongoing costs), often involve long waiting lists, and have relatively short life expectancies. While robot guide dogs offer a promising alternative, existing approaches exploring this idea suffer from several drawbacks: They often lack the autonomy required for real-world deployment, relying on prior 3D scans of the environment, external computation, or limited awareness of the handler. In this work, we present Milo, the first open-source, low-cost (approximately $2k USD) robotic guide dog platform capable of fulfilling the basic collaborative navigation role expected of a guide dog. Milo is fully autonomous, requiring no a priori knowledge of the environment, completely self-contained with all computation performed onboard, and suitable for both indoor and outdoor navigation while avoiding obstacles and pedestrians. Our system consists of a modified Unitree Go2 robot (equipped with onboard compute, sensors, and a handle), a perception stack combining voxel mapping with floor, obstacle, and pedestrian detection, and a navigation stack based on an obstacle-avoidance policy trained in a custom bird's-eye-view simulator. We evaluate Milo in real indoor and outdoor obstacle courses and compare it against a costmap-based baseline, demonstrating smoother navigation and fewer handler collisions. To maximize accessibility for BLV users, we release both the robot hardware instructions and the complete software stack as open source.

Abstract

Explore similar work

Milo, a Fully Autonomous Indoor/Outdoor Robotic Guide Dog

CANINE: Coaching Visually Impaired Users for Interactive Navigation with a Robot Guide Dog

Learning Agile Perceptive Traversal of Sparse 3D Structures for Humanoids