NavPatch: Evidence-Guided Object-Level Costmap Correction with Vision-Language Models

Shiji Sun; Xingyu Tao; Hao Wang; Ling Wang; Zhengyi Chen

cs.ROSep 13, 2026

NavPatch: Evidence-Guided Object-Level Costmap Correction with Vision-Language Models

Authors: Shiji Sun, Xingyu Tao, Hao Wang, Ling Wang, Zhengyi Chen

Organizations: Department of Intelligent Construction and Operation, Southeast University, Nanjing 211189, China. · Department of Construction Management and Intelligence, The Hong Kong Polytechnic University, Hong Kong SAR, China. · Department of Automation, Tsinghua University, Beijing 100084, China.

Abstract

Mobile robots typically rely on geometric maps for obstacle avoidance and path planning, but the resulting obstacle representation does not always match how an object should affect navigation. A low lying cable may be missed, a flexible curtain may create spurious blockage, and a traffic cone may require an exclusion region larger than its observed footprint. We present NavPatch, an object level correction layer that assigns ADD, REMOVE, or EXTEND to navigation relevant object categories through periodic scene understanding with a vision-language model. Open vocabulary grounding localizes object instances, and LiDAR and RGB-D observations provide 3D support. Observation quality filtering and cross frame maintenance determine when each correction patch is committed, replaced, or revoked. In 50 real robot trials across five layouts, NavPatch achieves an overall success rate of 86.0%. An ablation study of four configurations with 200 runs in total shows that NavPatch improves the success rate from 70.0% to 86.0% and reduces the false commit rate from 68.4% to 40.7% compared with updates based only on the current observation.

Figures & tables

Fig. 1: Three representative geometry and navigation mismatches. (a) A traversable curtain creates spurious blockage, (b) a low profile obstacle is weakly represented or missed, and (c) traffic cones require an exclusion region beyond their physical footprint. NavPatch applies Remove , Add , and Extend , respectively. Dashed and solid paths show the plans before and after map correction.

Fig. 2: System overview of NavPatch . Periodic VLM scene understanding proposes object categories and map actions. GroundingDINO and SAM 2.1 localize instances, while LiDAR and RGB-D observations provide 3D support for local correction regions. Observation quality filtering and cross frame maintenance regulate patch updates before correction patches are fused with the base costmap for navigation.

Fig. 3: Cross frame maintenance of a REMOVE patch under changing viewpoints. A region accepted at $t_{1}$ is retained as a candidate before sufficient evidence is accumulated, and is committed as a patch at $t_{2}$ . The patch is retained at $t_{3}$ while a newly observed region is accumulated as a candidate, and is replaced at $t_{4}$ . Dashed outlines denote candidate regions under evidence accumulation, whereas filled regions denote committed patches.

Fig. 4: Real robot navigation in five layouts. Panels (a) and (b) show Mixed-A and Mixed-B, while (c), (d), and (e) focus on Add , Remove , and Extend , respectively. Each panel shows the corrected map and a representative navigation path, with an RGB inset displaying detected object instances.

Layout	Trials	SR	FCR	MCR	PUR
Mixed-A	10	$70.0\%$	$42.2\%$	$35.0\%$	$8.1\%$
Mixed-B	10	$70.0\%$	$37.3\%$	$34.7\%$	$8.5\%$
ADD-focused	10	$100.0\%$	$47.3\%$	$3.3\%$	$7.8\%$
REMOVE-focused	10	$100.0\%$	$27.3\%$	$20.0\%$	$1.7\%$
EXTEND-focused	10	$90.0\%$	$38.5\%$	$20.0\%$	$5.4\%$
Overall	50	$\mathbf{86.0\%}$	$\mathbf{40.7\%}$	$\mathbf{24.9\%}$	$\mathbf{6.9\%}$

TABLE I: Navigation and map correction results across five layouts

Configuration	SR	FCR	MCR	PUR
Frame-wise	$70.0\%$	$68.4\%$	$22.4\%$	$95.4\%$
Quality-only	$66.0\%$	$64.6\%$	$24.1\%$	$88.6\%$
Temporal-only	$54.0\%$	$54.1\%$	$27.6\%$	$5.9\%$
NavPatch	$\mathbf{86.0\%}$	$\mathbf{40.7\%}$	$24.9\%$	$6.9\%$

TABLE II: Ablation study of patch update strategies

Robot Navigation

Robot navigation studies how robots perceive, localize, plan, and move safely toward goals in static or changing environments.

See all 554 papers

SAFER-Nav: Enhancing Safety for Visual Robot Navigation via Segmentation-Aware Fine-Tuning
Jun 10, 2026Geonyeong Ko, Giung Lee, Changjoo NamRobot Policy Learning Trajectory Generation
Dept. of Electronic Engineering, Sogang University, Seoul, Korea · Dept. of Computer Science, Rice University, Houston, TX, USA · Vertical Labs, Co., Ltd., Seoul, Korea
CoSMO: Conformal Semantic Mapping for Open-World Safe Robot Navigation
Oct 13, 2026Shuqi Wang, Yiwei Zheng, Pingrui Zhang +2Safe Robot Navigation Conformal Prediction
Shanghai Jiao Tong University · Shanghai Innovation Institute · Fudan University
RECAST: Recasting Vision-Language Semantics into an Actionable Cost Map for Robot Navigation
Sep 26, 2026Incheol Cho, Jintae Park, Jinkyu Kim +3Robot Navigation Visual Robot Navigation
Korea University · Hanwha Aerospace · KAIST AI +1

Jun 10, 2026J/K move · Enter open · S save

SAFER-Nav: Enhancing Safety for Visual Robot Navigation via Segmentation-Aware Fine-Tuning

Geonyeong Ko, Giung Lee, Changjoo Nam

Dept. of Electronic Engineering, Sogang University, Seoul, Korea · Dept. of Computer Science, Rice University, Houston, TX, USA · Vertical Labs, Co., Ltd., Seoul, Korea

Robot Policy Learning Trajectory Generation Robot Navigation Safe Robot Navigation Robot Safety Visual Robot Navigation

Open paper AskPDF arXiv

Vision-based navigation models, particularly foundation models, generate viable trajectories from RGB observations alone. However, even state-of-the-art transformer- and diffusion-based policies struggle to generalize in unfamiliar deployment environments containing unseen obstacles or shifted conditions. The resulting trajectories often remain goal-directed but unsafe. Existing efforts improve safety through external trajectory correction or internal geometric priors, yet the resulting policies are not trained to explicitly represent obstacle boundaries or traversable free-space structure. To address this, we propose a navigation model that incorporates these structures directly into the policy via fine-tuning and is designed to be compatible with diverse RGB-based backbones. Across multiple robot platforms, indoor environments, and static and dynamic obstacle scenarios, our method reduces collision frequency relative to ViNT, NoMaD, and their CARE-augmented variants while maintaining goal-reaching performance.

Abstract

Robot Navigation

Explore similar work

SAFER-Nav: Enhancing Safety for Visual Robot Navigation via Segmentation-Aware Fine-Tuning

CoSMO: Conformal Semantic Mapping for Open-World Safe Robot Navigation

RECAST: Recasting Vision-Language Semantics into an Actionable Cost Map for Robot Navigation