cs.RONov 8, 2026

MOSU2: Long-range Navigation System with Multi-modal Scene Understanding

Authors: Jing Liang, Zhenyu Hou, Senthurbavan Kirubaharan, Md Safwan Mondal, Ashok Gudivada, Xuesu Xiao, Dinesh Manocha

Organizations: Stanford University, CA, USA · George Mason University, VA, USA · University of Maryland, College Park, MD, USA

Abstract

Robots that travel long distances through public outdoor spaces must do more than avoid obstacles: they must remain on terrain suited to their mobility and decide when to proceed, yield, or detour around pedestrians, cyclists, vehicles, traffic signals, and temporary restrictions. These decisions are difficult because their correctness often depends on how a scene evolves, such as whether an approaching car is slowing or a cyclist will cross the robot's path. Vision-language models (VLMs) can interpret such cues, but they reason over images that lack metric geometry and motion. Supplying this evidence at every step is too slow for real-time navigation, and hand-crafted triggers cannot anticipate the open-ended variety of outdoor situations. We present MOSU2, a long-range navigation system with two-stage trajectory selection, in which first-stage contextual reasoning determines when additional evidence is needed and what to retrieve. Only when an interaction remains unresolved does the first stage trigger the second stage to retrieve geometric and dynamic information and align it with scene context to make navigation decisions. Building on MOSU, MOSU2 follows routes from GPS-based route planning and filters candidate trajectories through multi-modal traversability analysis before selecting a trajectory to move the robot. In real-world experiments spanning stop gestures, construction sites, crosswalks, and roads shared with vehicles, cyclists, and pedestrians, MOSU2 raises mean navigation success from 61.8% to 94.2% and condition-detection accuracy from 72.2% to 91.4% over its single-image predecessor, at about 2.2 times its mean decision latency. Project website: https://jinggm.github.io/mosu2/

Figures & tables

Explore similar work

CardsList
  1. Walk With Me: Long-Horizon Social Navigation for Human-Centric Outdoor Assistance

    Apr 29, 2026Lingfeng Zhang, Xiaoshuai Hao, Xizhou Bu +11Social Robot NavigationVLMs for Robotics

  2. UniLM-Nav: A Unified Framework for Zero-Shot Last-Mile Navigation

    Jul 7, 2026Zhuofan Zhang, Tianxu Wang, Guoxi Zhang +6Mobile ManipulationRobot Navigation

  3. I2O2I-Nav: A Benchmark with a Modular Agent for Long-Horizon Indoor-to-Outdoor-to-Indoor Navigation

    Oct 13, 2026Zhengjie Shu, Wanyu Li, Di Xiao +11Indoor Robot NavigationRobot Navigation Benchmarks