cs.ROOct 10, 2026

Training-Free Multi-Agent Cooperative Embodied Search from Uncertain Vision-Language Priors

Authors: Fan Pu, Keshu Wu, Jiwan Jiang, Shiyu Shen, Zhiwen Fan, Zhengzhong Tu, Jiachen Li, Yang Zhou

Organizations: Department of Civil and Environmental Engineering, Texas A&M University, TX 77843. · Department of Civil and Environmental Engineering, University of Illinois at Urbana-Champaign, IL 61801. · Department of Electrical and Computer Engineering, Texas A&M University, TX 77843. · Department of Computer Science and Engineering, Texas A&M University, TX 77843. · 

Abstract

Real-world Vision-Language Navigation (VLN) requires agents to act under partial observability, where map structure, semantic cues, and target locations may be incomplete or incorrectly inferred by foundation models. This paper presents a lightweight, training-free, and uncertainty-aware framework for cooperative multi-agent multi-target search in mapless environments. Instead of learning an end-to-end navigation policy, the framework separates high-level semantic reasoning from low-level execution through a scalable graph-optimization backbone. Multimodal Large Language Models (MLLMs) serve only as hypothesis generators, proposing semantic zones, target-location cues, and possible navigable connections. These outputs are treated as uncertain priors and further refined using online observations and graph-context evidence. Cooperative navigation is then formulated as a rolling-horizon multi-agent mixed-integer linear program with a lexicographic objective that first maximizes target-directed reward and then reduces travel distance, uncertain-edge use, and repeated visits. Experiments derived from 100 Matterport3D test cases show that the proposed framework improves target coverage and search efficiency over representative baseline methods, while shared information and joint planning enable more balanced workloads across agents. By operating on an abstract graph without relying on embodiment-specific policy parameters, the framework scales across varying numbers of agents and targets, generalizes to unseen environments, and accommodates different perception modules and low-level controllers without task-specific retraining.

Figures & tables

Explore similar work

CardsList
  1. GoalVLM: VLM-driven Object Goal Navigation for Multi-Agent System

    Mar 18, 2026MoniJesu James, Amir Atef Habel, Aleksey Fedoseev +1Open-Vocabulary Object NavigationObject-Goal Navigation

  2. AgentVLN: Towards Agentic Vision-and-Language Navigation

    Mar 18, 2026Zihao Xin, Wentong Li, Yixuan Jiang +6Vision-Language NavigationRobot Navigation

  3. HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harness

    Sep 14, 2026Yang Chen, Lirong Che, Zhenyu Huang +7Embodied NavigationLarge Language Model-Based Robot Planning