OASIS: Online Anchor-centric Scene Inference System for Open-Vocabulary Retrieval and Navigation in Dynamic Environments

Kaiyao Liao; Qingfeng Li; Chen Chen; Xiaoze Wu; Qing Sun; Jianwei Niu

cs.ROOct 22, 2026

OASIS: Online Anchor-centric Scene Inference System for Open-Vocabulary Retrieval and Navigation in Dynamic Environments

Authors: Kaiyao Liao, Qingfeng Li, Chen Chen, Xiaoze Wu, Qing Sun, Jianwei Niu

Organizations: Beihang University, Beijing, China · Hangzhou Innovation Institute of Beihang University, Hangzhou, China · School of Computer Science and Engineering, Beihang University, Beijing, China

Abstract

Reliable robotic mapping in dynamic real-world environments is essential for embodied intelligence and long-term autonomy. However, existing methods often involve a trade-off: lightweight online mapping is efficient but captures limited semantic context, while dense mapping provides richer semantics at higher computational cost and is more vulnerable to distractors. They also struggle to relocalize objects after scene changes. To address these challenges, we propose the Online Anchor-centric Scene Inference System (OASIS), a unified system for online mapping, complex open-vocabulary retrieval, and navigation in dynamic environments. Motivated by the observation that indoor scenes are organized around stable semantic anchors, OASIS represents the environment using a sparse set of anchor nodes, each maintaining a unified multimodal representation that integrates semantic descriptions, visual features, and geometric states for reasoning, verification, and navigation. Its compact anchor-centric memory supports on-demand evidence extraction and hierarchical retrieval, reducing redundant storage while improving retrieval reliability. To handle scene dynamics, OASIS further reuses transient observations collected during navigation to relocalize moved objects. Experiments in both simulation and real-world environments, including standard retrieval benchmarks and navigation tasks under both static and dynamic settings, demonstrate that our system can perform complex open-vocabulary retrieval and navigation, while also enabling relocation of displaced objects after scene changes.

Figures & tables

Figure 1. Comparison of retrieval paradigms for complex open-vocabulary queries. Unlike traditional dense mapping (left) which struggles with redundant context and visual distractors, our OASIS (right) constructs a lightweight anchor-centric memory. By leveraging an LLM to filter irrelevant regions and performing on-demand VLM verification, OASIS efficiently isolates the target based on fine-grained attributes and spatial relationships, significantly improving retrieval reliability.

Figure 2. System architecture of OASIS. Given an RGB-D stream, the system concurrently performs episodic memory curation (top left) and online semantic mapping (bottom left) to extract 3D instances. To build the episodic memory, it utilizes scale-aware scoring and spatial-visual cluster assignment to continuously update and retain high-quality representative frames. By assigning these selected frames to their corresponding stable anchor instances, the system constructs the unified multimodal OASIS Graph , structured as shown on the right. Leveraging this graph alongside multimodal foundation models (LLM/VLM), the agent effectively executes hierarchical retrieval and robust view-driven navigation in dynamic environments.

Figure 3. Visual relocalization via Transient Visual Buffer (TVB). If the retrieved target is absent from the current view, OASIS uses CLIP to match transient crops to the query and a VLM to verify candidate frames and confirm the target’s new location.

Method	mAP $\uparrow$	AP 50 $\uparrow$	AP 25 $\uparrow$
ConceptGraphs ( Gu et al., 2024 )	2.74	6.18	13.82
HOV-SG ( Werby et al., 2024 )	3.72	7.59	14.94
Kassab2024 ( Kassab et al., 2024 )	0.66	2.15	5.34
OpenMask3D ( Takmaz et al., 2023 )	2.10	4.93	12.63
DualMap ( Jiang et al., 2025 )	6.19	11.72	20.06
Ours	6.58	12.52	20.63

Table 1. Fine-grained object retrieval. Comparison on the OpenLex3D (Replica) benchmark.

Method	Regions	Trials	SR (%) $\uparrow$
ConceptGraphs ( Gu et al., 2024 )	15.6	34.9	29.3
HOV-SG ( Werby et al., 2024 )			31.5
DualMap ( Jiang et al., 2025 )			53.8
Ours			57.7

Table 2. Functional region retrieval. Comparison on the HOV-SG (HM3D) benchmark.

Method	Overall (%)		Easy (%)		Hard (%)		View-Dependent (%)		View-Independent (%)
Method	[email protected] $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$
OpenFusion ( Yamazaki et al., 2024 )	12.6	2.4	14.0	2.4	1.3	1.1	3.8	2.5	13.7	2.4
BBQ-CLIP ( Linok et al., 2025 )	14.4	8.8	15.4	9.0	6.7	6.7	11.4	5.1	14.4	8.8
ConceptGraphs ( Gu et al., 2024 )	13.3	6.2	13.0	6.8	16.0	1.3	15.2	5.1	13.1	6.4
DualMap ( Jiang et al., 2025 )	28.4	21.6	29.9	22.4	17.3	16.0	16.5	11.4	30.1	23.0
BBQ ( Linok et al., 2025 )	34.2	22.7	34.3	22.7	33.3	22.7	32.9	20.3	34.4	23.0

Table 3. Spatial relation retrieval. Grounding accuracy on the Sr3D+ (ScanNet) benchmark across different difficulty splits.

Scene Type	Method	00829	00848	00880	Trials	Avg. SR $\uparrow$
Static	ConceptGraphs ( Gu et al., 2024 )	69.2	53.8	61.5	78	61.5
	HOV-SG ( Werby et al., 2024 )	53.8	46.2	57.7		52.6
	DualMap ( Jiang et al., 2025 )	73.1	69.2	69.2		70.5
	Ours	76.9	92.6	84.6		85.9
In-Anchor	DualMap ( Jiang et al., 2025 )	66.7	66.7	61.1	54	64.8
In-Anchor	Ours	83.3	83.3	77.8	54	81.5

Table 4. Object Navigation Success Rate (SR) on HM3D.

Clustering	Global	Local	Adaptive	Avg. Clusters $\downarrow$
Spatial-Only	87.18	83.08	83.99	5.3
Visual-Only	84.79	72.92	86.64	3.8
Hybrid (Ours)	90.82	89.30	91.13	6.5

Table 5. Memory curation ablation. Impact of different clustering and frame selection strategies on scene instance coverage.

LLM reasoning	VLM verification	[email protected] (%) $\uparrow$	[email protected] (%) $\uparrow$
✗	✗	28.4	21.6
✗	✓	36.5	30.2
✓	✗	32.8	26.9
✓	✓	44.0	37.4

Table 6. Hierarchical retrieval ablation. Impact of LLM proposal and VLM verification stages on the Sr3D+ benchmark.

Method	Attempts $\downarrow$	SR (%) $\uparrow$	SPL (%) $\uparrow$
Random Anchor	2.9	17.0	3.9
Ours w/o Transient Visual Buffer	2.5	56.6	17.3
Ours w/o Semantic Reasoning	1.8	73.6	24.1
Ours (Full)	1.6	77.4	25.8

Table 7. Dynamic navigation ablation. Impact of system components on relocalization performance in the Cross-Anchor setting.

Method	Static		Dynamic
Method	SR (%) $\uparrow$	SPL (%) $\uparrow$	SR (%) $\uparrow$	SPL (%) $\uparrow$	Attempts $\downarrow$
DualMap ( Jiang et al., 2025 )	63.3	58.2	56.7	18.1	2.1
Ours	86.7	76.3	76.7	27.6	1.4

Table 8. Real-world navigation. System performance under static and dynamic household settings.

Figure 4. Real-world household environment. The three-room scene contains 13 semantic anchors and visually similar objects with diverse colors, shapes, and states.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Value	Description
$\lambda$	0.7	Balance weight between coverage and entropy
$\alpha$	0.7	Weight for anchor count in the global score
$A_{\max},O_{\max}$	5.0, 10.0	Saturation thresholds for counts
$\beta$	0.5	Weight for completeness in the local score
$C_{\max}$	5.0	Saturation threshold for interactions
$\gamma$	0.7	Weight for max aggregation in local score fusion

Appendix

Table S1. Hyperparameters for frame scoring.

Figure S1. Sensitivity analysis of frame-scoring hyperparameters.

Dataset	Method	Avg. Mem (MB) $\downarrow$	Peak Mem (MB) $\downarrow$	TPF (s) $\downarrow$
Replica	ConceptGraphs	7148.9	23551.9	4.188
	HOV-SG	73368.0	158126.6	42.005
	DualMap	3095.2	4564.0	0.276
	Ours	3231.2	4971.1	0.312
ScanNet	ConceptGraphs	9780.3	26155.2	6.301
	HOV-SG	9223.0	25735.0	8.039

Appendix

Table S2. Memory and runtime overhead comparison.

Stage	API	Avg. Image Count	Mean Latency (s)
Stage 1	GPT-5.2	–	4.2
Stage 2	Gemini-3-Pro	6.4	14.9
Stage 3	Gemini-3-Pro	2.0	8.1
Stage 4	Gemini-3-Pro	6.0	12.7
Stage 5	Gemini-3-Pro	2.0	7.6

Appendix

Table S3. Latency of LLM/VLM interactions across stages.

Indoor Robot Navigation

See all 32 papers

Aug 24, 2026

SuperMap: A Spatio-Temporal SLAM System for Visual-Language Navigation

Robotic navigation in human environments requires a spatio-temporal semantic representation that can rec- oncile open-vocabulary perception with long-term environmental changes. While foundation models provide strong zero-shot recognition, their predictions are intermittent and view-dependent, and naively integrating them into mapping pipelines leads to identity drift and stale semantics over time. We present SuperMap, a 4D spatio-temporal mapping framework for language-guided navigation that integrates high-frequency geometric SLAM with asynchronous open-vocabulary perception. Our core contribution is a consistency-driven mapping engine that combines 3D-aware instance association/re-activation with a principled existence-and-label confidence update to maintain stable object identities and prune outdated map content under occlusions and scene changes. SuperMap produces a queryable 4D scene-graph representation that interfaces naturally with Vision-Language Models by supporting compositional queries over object semantics, relations, We demonstrate SuperMap on benchmarks and real robots, including dynamic scenes with appearance/disappearance and relocation, and provide ablations and runtime analysis. We release the full system as open-source to provide the community with a deployable baseline for open-vocabulary spatio-temporal mapping. Project website: superodometry.com/supermap.

Shibo Zhao, Guofei Chen, Honghao Zhu +7

The Robotics Institute, Carnegie Mellon University

Open-Vocabulary Semantic Mapping Semantic Mapping Simultaneous Localization and Mapping 3D Scene Representation Vision-Language Navigation

Ask PDF

Jul 16, 2026

OASIS-Map: Object-Level Change Detection in Multi-Session Mapping using Semantic Correspondence Matching

Map representations which are consistent across repeated visits to a real-world semi-static environment are very useful for long-term robotic inspection. In such settings, the scene may evolve while the robot is absent, with objects appearing, disappearing, moving, or being replaced, quickly making a static map outdated. Existing change-detection methods reason through geometry, category-level semantics, or object persistence. However, achieving reliable object association across revisits remains a key challenge, especially under partial views, occlusion, and imperfect segmentation. In this work, we propose OASIS-Map, a multi-session mapping system that maintains a spatio-temporally consistent object-level map by establishing dense patch-level semantic correspondences between temporal observations. These correspondences detect where the scene has changed and incrementally associate objects across revisits as the robot re-observes the environment. We demonstrate OASIS-Map on three challenging real-world scenarios: object rearrangements in 3RScan, visually similar car replacements in a car park, and large-scale scene changes in an outdoor market. We achieve 0.783 F1 on change detection in a car replacement scenario in a car park and 0.667 F1 on moved object association in 3RScan. https://dynamic.robots.ox.ac.uk/projects/oasis-map/

Haedam Oh, Yifu Tao, Nived Chebrolu +1

Oxford Robotics Institute, Department of Engineering Science, University of Oxford, UK · Department of Computer Science and Engineering, Indian Institute of Technology Bombay, India

Semantic Correspondence Semantic Change Detection Robotic Perception 3D Scene Understanding

Ask PDF

May 30, 2026

Dynamic Resilient Spatio-Semantic Memory with Hybrid Localization for Mobile Manipulation

Reliable mobile manipulation in dynamic indoor environments requires a scene representation that remains geometrically consistent, semantically queryable, and computationally bounded as the environment changes. Existing systems often rely on pre-built maps, static-scene assumptions, or highly accurate camera poses, which can lead to stale or misaligned scene information when target objects are relocated or pose estimates are corrected. This paper presents DREAM, a real-robot mobile manipulation framework that integrates perception, memory, localization, navigation, and manipulation in previously unseen indoor environments without a pre-built map. DREAM constructs an online spatio-semantic voxel memory from RGB-D observations registered by a LiDAR-inertial-visual SLAM backend. It further introduces pose-graph-aware Redundancy-Aware Memory Pruning (RMP) to update historical observations after pose corrections while keeping long-horizon observation history bounded. For target localization and reacquisition, DREAM combines language-conditioned 3D retrieval, open-vocabulary image detection, and multimodal large language model based semantic verification. Real-robot experiments in four dynamic indoor laboratory scenes show that DREAM improves long-horizon task success rates from 40%-60% with DynaMem to 55%-70%, while maintaining a memory footprint of 0.37-0.63 GB and an online memory-update time of 0.43-0.53 s across scenes.

Zhijie Yan, Shufei Li, Ze Zhang +3

School of Mechanical Engineering and Automation, Beihang University, Beijing 100191, China · Department of Systems Engineering, City University of Hong Kong, Hong Kong SAR 518057, China · School of Computing, National University of Singapore, 21 Lower Kent Ridge Rd 119077, Singapore

Long-Horizon Robotic Manipulation Mobile Robotics Semantic Mapping

Ask PDF

Abstract

Indoor Robot Navigation

Explore similar work

SuperMap: A Spatio-Temporal SLAM System for Visual-Language Navigation

OASIS-Map: Object-Level Change Detection in Multi-Session Mapping using Semantic Correspondence Matching

Dynamic Resilient Spatio-Semantic Memory with Hybrid Localization for Mobile Manipulation