cs.RONov 12, 2026

Racing the Clock: Teaching VLNs Budget-Conditioned Object Search

Authors: Maitree Meher, Siming He, S. Shankar Sastry, Jitendra Malik, Claire Tomlin, Jonas Frey

Organizations: University of California, Berkeley · Stanford University

Abstract

How do action budgets shape vision-language navigation (VLN) exploration, and what expert supervision best teaches budget-aware search? We study budget-conditioned object search: finding as many distinct instances of a requested category as possible within a budget, with discovery defined geometrically by camera field of view and proximity. Distinct discoveries provide a denser evaluation signal than single-target success. We cast expert design as a choice of scene abstraction: a single budget-indexed Bellman recursion, applied to different abstractions of the full environment, yields experts that differ in what they know. An object-level Bellman expert is given every target location and reduces search to budgeted routing. A room-level Bellman expert retains observed room connectivity, doorway evidence, and cross-scene semantic priors but withholds the contents of unexplored rooms, so it must search for targets that its student also cannot see. We distill both experts into Uni-NaVid under matched training conditions and also evaluate greedy-frontier supervision on held-out HM3D scenes. Changing only the stated budget changes learned trajectories. Despite higher object-level expert return, room-level supervision achieves the highest mean collection among evaluated VLNs at every tested budget. At 2000 steps, it finds 72.7% of reachable targets, versus 68.5% under greedy-frontier supervision, 65.5% under object-level supervision, and 44.8% for the pretrained VLN. These results show that expert-return rankings need not predict supervision quality for VLN exploration, and motivate choosing an expert's abstraction with the student's information in mind rather than by expert return alone.

Figures & tables

Explore similar work

CardsList
  1. VLN-NF: Feasibility-Aware Vision-and-Language Navigation with False-Premise Instructions

    Apr 12, 2026Hung-Ting Su, Ting-Jun Wang, Jia-Fong Yeh +2VLN BenchmarksVision-Language Navigation

  2. History-Conditioned Spatio-Temporal Visual Token Pruning for Efficient Vision-Language Navigation

    Mar 6, 2026Qitong Wang, Yijun Liang, Ming Li +2Efficient VLM InferenceToken Pruning

  3. Seek Before You Move: Evidence Seeking for Progress Grounding in Vision-Language Navigation

    Sep 29, 2026Zhimin Wang, Meiyuan Zhu, Duo Wu +8Vision-Language NavigationRL for VLMs