F2A-ObjectNav: An Adaptive Agentic Architecture for Object-Goal Navigation
Organizations: Social and Cognitive Robotics Laboratory, Mechanical Engineering Department, Sharif University of Technology, Tehran, Iran
Abstract
Object-goal navigation requires an embodied agent to explore an unfamiliar environment, identify a target object, navigate toward candidate instances, and reliably determine whether the observed object satisfies the mission goal. This work investigates this problem through a two-stage development process. In Stage I, a fixed modular framework is used to examine combinations of exploration, object detection, and target validation methods. The results reveal clear trade-offs among navigation performance, validation reliability, execution time, and model-related cost, with no single fixed configuration remaining preferable across all evaluated conditions. These observations motivate F2A-ObjectNav (Fixed-to-Adaptive Object-Goal Navigation), an adaptive agentic architecture in which a rule-based Supervisor coordinates specialized components for navigation and exploration, object detection, target validation, semantic guidance, and Mission Memory. The architecture uses event-driven recovery, strategy revision, candidate management, and conditional semantic guidance while retaining lightweight execution during normal operation. F2A-ObjectNav is evaluated over 25 episodes in each of AI2-THOR, RoboTHOR, and ProcTHOR, achieving success rate (SR) / success-weighted path length (SPL) values of 0.84/0.52, 0.40/0.22, and 0.44/0.37, respectively. In addition to task-level metrics, architecture-specific behavioral metrics are used to characterize semantic-guidance usage, strategy switching, recovery activation, memory-supported behavior, and decision stability. The results show different patterns of adaptive intervention across the three environments, with higher intervention frequencies observed in RoboTHOR and ProcTHOR than in AI2-THOR.
Figures & tables
| Config. | Exploration | Detection | Validation |
|---|---|---|---|
| C1 | Random | YOLO11 | YOLO-World |
| C2 | Frontier-based | YOLO-World | GPT-4o |
| C3 | Frontier-based | BLIP-2 | YOLO-World |
| C4 | BLIP-2-guided | YOLO-World | GPT-4o |
| C5 | GPT-4o-guided | YOLO-World | BLIP-2 |
| C6 | Manual control (human-reference condition) | ||
| Metric | Symbol | Evaluation purpose |
|---|---|---|
| Success Rate | SR | Proportion of successfully completed missions |
| Success-weighted Path Length | SPL | Mission success weighted by path efficiency |
| Environmental Adaptability Score | EAS | Relative retention of SR and SPL from the simple to the complex layout |
| Task Time | TT | Average duration from mission initialization to termination |
| Validation Accuracy | VA | Correctness of candidate-validation decisions |
| API Cost | AC | Average model-related inference cost per episode |
| SR | SPL | ||||
|---|---|---|---|---|---|
| Config. | Simple | Complex | Simple | Complex | EAS |
| C1 | 0.73 | 0.27 | 0.51 | 0.09 | 0.273 |
| C2 | 0.87 | 0.67 | 0.60 | 0.25 | 0.593 |
| C3 | 0.87 | 0.60 | 0.58 | 0.29 | 0.595 |
| C4 | 1.00 | 0.87 | 0.76 | 0.54 | 0.790 |
| C5 | 1.00 | 0.80 | 0.71 | 0.51 | 0.759 |
| TT (s) | API Cost ($) | ||||
|---|---|---|---|---|---|
| Config. | Simple | Complex | Simple | Complex | VA |
| C1 | 211 | 304 | 0.0000 | 0.0000 | 0.73 |
| C2 | 105 | 281 | 0.0015 | 0.0015 | 0.91 |
| C3 | 93 | 311 | 0.0000 | 0.0000 | 0.73 |
| C4 | 74 | 367 | 0.0015 | 0.0015 | 0.91 |
| C5 | 79 | 383 | 0.0031 | 0.0079 | 0.84 |
| Stage I limitation | Stage II design response |
|---|---|
| One configuration remains active throughout the mission | Rule-based supervisory coordination and conditional strategy revision |
| Different failure events return to the same predefined execution process | Event-specific recovery and mission-level decision logic |
| Rejected candidates and unsuccessful search history are not systematically reused | Mission Memory and memory-informed candidate management |
| Semantic reasoning is determined by the selected configuration rather than current need | Conditional activation of the Semantic Advisor |
| Component | Main input | Main output | Primary function |
|---|---|---|---|
| Supervisor Agent | Mission state and execution events | High-level coordination decision | Selects the appropriate mission response using explicit rules and event conditions |
| Navigation and Exploration Agent | Pose, map, current goal, and mission state | Navigation actions and exploration subgoals | Explores the environment, approaches candidates, and reports navigation progress |
| Object Detection Agent | Visual observations and target representation | Candidate detections | Identifies possible target instances without declaring mission success |
| Target Validation Agent | Candidate observation and complete target description | Validation result | Determines whether an approached candidate satisfies the intended goal |
| Semantic Advisor Agent | Target, recent observations, and mission context | High-level semantic guidance | Suggests promising regions or search directions when conventional search is insufficient |
| Mission Memory | Mission events and intermediate outcomes | Stored episode-level records | Preserves information relevant to later candidate, exploration, recovery, and supervisory decisions |
| Adaptive mechanism | Typical activation condition | Role in the architecture |
|---|---|---|
| Strategy switching | Insufficient progress, repeated failure, or ineffective current exploration behavior | Revises the active exploration strategy using the alternatives available to the Supervisor |
| Semantic-guided exploration | Prolonged search, repeated exploration, unsuccessful recovery, or insufficient semantic evidence | Requests high-level contextual guidance about a promising region or direction without directly controlling robot motion |
| Candidate management | Candidate detection, rejection, loss, or re-observation | Maintains candidate status and uses Mission Memory to avoid repeated evaluation of rejected instances |
| Spatial candidate merging | Multiple detections correspond to approximately the same estimated location | Consolidates repeated observations into a single candidate record |
| Recovery | Navigation stagnation, repeated trajectory, or failure to reach the current navigation objective | Replans, changes viewpoint or subgoal, clears an ineffective candidate, or resumes exploration |
| Goal-aware validation | The agent reaches a candidate with sufficient visual evidence | Evaluates the candidate against the complete target description before mission termination |
| Component | Model | Role | Activation |
|---|---|---|---|
| Goal interpretation | GPT-4o mini | Extracts the primary target representation while preserving the complete description for validation | Mission initialization |
| Primary object detection | YOLO11x | Detects common object categories during routine perception | During normal search when applicable |
| Open-vocabulary detection | YOLO-World v8x | Detects target classes using flexible object names and accepted aliases | When open-vocabulary detection is required |
| Target validation | GPT-4.1 mini | Evaluates an approached candidate against the complete target description | After candidate approach |
| Semantic guidance | GPT-4o | Provides high-level contextual guidance about promising search regions or directions | On request from the Supervisor |
| Group | Metric | Purpose |
| Standard ObjectNav | Success Rate (SR) | Measures successful mission completion. |
| Standard ObjectNav | Success-weighted Path Length (SPL) | Measures mission success together with path efficiency. |
| System-level | Task Time (TT) | Measures average episode duration from initialization to termination. |
| System-level | Validation Accuracy (VA) | Measures the correctness of candidate-validation decisions. |
| System-level | API Cost (AC) | Measures the average inference cost of externally accessed models per episode. |
| Architecture-specific | Semantic Guidance Rate (SGR) | Measures the frequency of Semantic Advisor activation relative to episode length. |
| Environment | SR (%) | SPL | TT (s) | VA (%) | API Cost ($) |
|---|---|---|---|---|---|
| AI2-THOR | 84.00 | 0.52 | 48.63 | 89.5 | 0.0125 |
| RoboTHOR | 40.00 | 0.22 | 101.04 | 86.7 | 0.0273 |
| ProcTHOR | 44.00 | 0.37 | 109.73 | 84.2 | 0.0294 |
| Environment | SGR (%) | SSR (%) | Recovery (%) | Memory (%) | Stability (%) |
|---|---|---|---|---|---|
| AI2-THOR | 0.06 | 0.61 | 13.3 | 0.80 | 98.90 |
| RoboTHOR | 0.36 | 4.01 | 26.8 | 15.00 | 92.80 |
| ProcTHOR | 0.65 | 5.20 | 30.6 | 7.30 | 90.50 |
| Mechanism | Aggregate diagnostic evidence | Execution-trace evidence | Observed functional role |
|---|---|---|---|
| Mission Memory | Memory-Assisted Loop Response was 0.80%, 15.00%, and 7.30% in AI2-THOR, RoboTHOR, and ProcTHOR, respectively. | In the recorded laptop mission, an incorrect candidate was rejected, stored in Mission Memory, and excluded from subsequent consideration before a later candidate was successfully confirmed. | Preserves episode-level information so that previous candidate and navigation outcomes can influence later decisions. |
| Recovery | Recovery Activation increased from 13.3% in AI2-THOR to 26.8% in RoboTHOR and 30.6% in ProcTHOR. | In the difficult remote-control mission, recovery was activated repeatedly after repeated states and navigation difficulties. | Provides event-triggered responses to stagnation, failed progress, and ineffective navigation states. |
| Strategy switching | SSR increased from 0.61% in AI2-THOR to 4.01% in RoboTHOR and 5.20% in ProcTHOR. | The recorded remote-control mission included switching between frontier-based and random exploration when the current search behavior became ineffective. | Prevents the architecture from remaining indefinitely committed to one exploration strategy when progress deteriorates. |
| Semantic Advisor | SGR remained low at 0.06%, 0.36%, and 0.65% in AI2-THOR, RoboTHOR, and ProcTHOR, respectively. | Semantic guidance was not required in the direct microwave-success mission but was invoked repeatedly in the more difficult remote-control execution. | Acts as a conditional source of high-level contextual guidance rather than a continuously active controller. |
| Candidate management and validation | Validation Accuracy remained between 84.2% and 89.5% across the three evaluation environments. | In the laptop mission, an initial candidate was rejected and stored before exploration continued and a later candidate was successfully confirmed. | Separates candidate generation from final target acceptance and prevents detector output from being treated directly as mission completion. |
| Supervisory coordination | Decision Stability was 98.90%, 92.80%, and 90.50% in AI2-THOR, RoboTHOR, and ProcTHOR, respectively. | Straightforward episodes proceeded with little intervention, whereas difficult executions involved recovery, strategy revision, memory interaction, and semantic assistance. | Coordinates adaptive intervention while retaining the current execution strategy when additional action is unnecessary. |
| Environment | Method | Year | SR (%) | SPL |
| DAT [ 12 ] | 2023 | 82.39 | 0.489 | |
| IOM [ 36 ] | 2023 | 82.99 | 0.474 | |
| MT [ 13 ] | 2023 | 83.14 | 0.502 | |
| Memory-MambaNav [ 22 ] | 2025 | 83.19 | 0.498 | |
| AI2-THOR | F2A-ObjectNav (ours) | 2026 | 84.00 | 0.520 |
| ESC [ 5 ] | 2023 | 38.10 | 0.222 |
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
| Item | Setting |
|---|---|
| Evaluation type | Simulation-based ObjectNav evaluation in indoor environments |
| Evaluation environments | AI2-THOR, RoboTHOR, and ProcTHOR |
| Mission specification | Object category or natural-language target description |
| Goal representation | Primary object representation for detection together with the complete target description retained for validation |
| Supervisory controller | Rule-based, event-driven Supervisor; no learned policy or LLM is used as the primary mission-level controller |
| Agent observations | Visual observations together with available depth, pose, and simulator-specific state information required by the navigation and supervisory components |
| Environment | Role in the evaluation | Agent type | Main characteristic |
|---|---|---|---|
| AI2-THOR | Evaluation in structured residential scenes | Virtual first-person agent | Interactive indoor scenes including kitchens, living rooms, bedrooms, and bathrooms |
| RoboTHOR | Evaluation under robot-oriented navigation conditions | LoCoBot-based simulated agent | Indoor navigation with motion and perception constraints designed around a mobile-robot platform |
| ProcTHOR | Evaluation under greater structural and layout diversity | Virtual agent in procedurally generated environments | Procedurally generated houses with varied layouts, room connectivity, and object arrangements |
| Setting | Specification |
|---|---|
| Evaluation size | 25 episodes per environment; 75 in total |
| Scene selection | Random selection from the available scenes of each environment |
| Target specification | Object category or natural-language description; target present, location withheld |
| Initialization | Valid starting pose; mission state and Mission Memory reset per episode |
| Agent inputs | RGB observations, depth, and navigation-related pose/state information |
| Target interpretation | LLM-based category extraction and preparation of a detailed validation description |
| Symbol | Term | Description |
|---|---|---|
| Semantic Guidance Call Count | Number of Semantic Advisor activations in episode . | |
| Strategy Switch Count | Number of navigation or exploration strategy changes in episode . | |
| Recovery Event Count | Number of recovery actions executed in episode . | |
| Navigation Issue Count | Number of detected navigation problems in episode , including stagnation, repeated trajectories, candidate-approach failure, unproductive rotation, or prolonged lack of progress. | |
| Memory-Supported Escape Count | Number of corrective or escape actions supported by Mission Memory in episode . | |
| Navigation Loop Count | Number of detected trajectory-repetition or navigation-loop events in episode . |
| Component | Role | Model | Activation |
|---|---|---|---|
| Target Interpreter | Converts the mission goal into a detector-facing object class and a complete validation query | GPT-4o mini | Mission initialization |
| Semantic Advisor | Suggests a promising direction or region for continuing the search | GPT-4o | Conditional; used after insufficient progress, repeated navigation, or unsuccessful recovery |
| Attribute-Aware Vision Validator | Validates a candidate against the complete target description and explicit attributes | GPT-4.1 mini | Attribute-constrained target descriptions |
| Alias-Based Vision Validator | Validates a candidate using the target name and accepted equivalent terms | GPT-4.1 mini | Simple target categories without additional descriptive constraints |
| YOLO-World Class List | Defines the controlled target classes used for open-vocabulary detection | YOLO-World v8x | Open-vocabulary detection |
| YOLO11x | Performs primary detection for supported object categories | YOLO11x | Routine object detection |
| Output | Role in the analysis |
|---|---|
| run_meta.json | Stores episode-level metadata such as environment, target, final status, recorded step count, execution time, and summary information. |
| flow_compact.json | Provides a compact structured sequence of the principal execution events and state transitions. |
| flow_compact.txt | Provides a human-readable summary of the mission execution flow. |
| appendix_bundle.json | Contains execution information organized for qualitative analysis and appendix reporting. |
| appendix_handoffs.txt | Records selected coordination events and information exchanges among system components. |
| exploration_map.png | Visualizes explored regions together with the trajectory recorded during the episode. |
| Property | Value |
|---|---|
| Environment | AI2-THOR |
| Scene | FloorPlan1 |
| Target | Microwave |
| Final status | Successful |
| Recorded steps | Approximately 43 |
| Execution time | Approximately 42 s |
| Property | Value |
|---|---|
| Environment | AI2-THOR |
| Scene | FloorPlan201 |
| Target | Laptop |
| Final status | Successful |
| Recorded steps | Approximately 63 |
| Execution time | Approximately 67 s |
| Property | Value |
|---|---|
| Environment | AI2-THOR |
| Scene | FloorPlan205 |
| Target | Remote control |
| Final status | Unsuccessful |
| Recorded steps | Approximately 1,000 |
| Execution time | Approximately 253 s |
| Property | Value |
|---|---|
| Environment | ProcTHOR |
| Scene | ProcTHOR-train-00057 |
| Target | Soccer ball |
| Final status | Unsuccessful |
| Recorded steps | Approximately 1,000 |
| Execution time | Approximately 405 s |
Object-Goal Navigation
Object-goal navigation studies how robots find and approach requested objects in unfamiliar environments using visual, spatial, and language cues.
See all 86 papers