cs.RONov 11, 2026

MOSAIC: Modular Obstacle and Spatial-Aware Framework for Manipulation In Cluttered Scenes

Authors: Richard Shi, Mohammad Mahdavian, Mehrdad Momeni Zadeh, Mohammed Elmahgiubi, Amir Rasouli

Organizations: Huawei Noah’s Ark Lab · University of Toronto · Simon Fraser University

Abstract

Vision-Language-Action (VLA) models have demonstrated strong performance in robotic manipulation, yet most rely primarily on 2D visual observations and language instructions, leaving scene geometry and obstacle relationships to be inferred implicitly. This limitation is particularly problematic in cluttered environments, where small spatial errors can lead to unintended contacts or collisions. We propose MOSAIC, a modular spatial- and obstacle-aware framework that can be integrated with pretrained VLA policies to improve object-level scene understanding and action generation. Our method augments the VLA backbone with an auxiliary semantic grounding task that predicts role-specific masks for the target object, destination region, and surrounding obstacles. We further combine the target and destination masks with depth estimated from 3D scene reconstruction to construct role-specific geometric tokens that are injected directly into the action expert. In addition, we propose obstacle-aware attention supervision that emphasizes obstacles near the robot body during task execution. The resulting attention features are subsequently injected into later action-expert layers to further guide action generation. Together, these components provide explicit semantic, geometric, and obstacle-aware cues while preserving the underlying pretrained VLA architecture. We evaluate our framework on pretrained VLA model using the RoboPRO benchmark, including visual perturbation tests, and further validate it on real-world manipulation tasks. MOSAIC improves the success rate by 5.1% in cluttered scenes, and by 9.7% under visual perturbations, with gains of up to 25.0% in the most challenging environment. We will release our code and dataset upon acceptance at the following Github page: https://mmahdavian.github.io/MOSAIC/

Figures & tables

Explore similar work

CardsList
  1. Overcoming Visual Clutter in Vision Language Action Models via Concept-Gated Visual Distillation

    Mar 11, 2026Sangmim Song, Sarath Kodagoda, Marc Carmichael +1Vision-Language-Action ModelsRobustness of VLA Models

  2. TAG: Target-Agnostic Guidance for Stable Object-Centric Inference in Vision-Language-Action Models

    Mar 25, 2026Jiaying Zhou, Zhihao Zhan, Ruifeng Zhai +5Vision-Language-Action ModelsRobustness of VLA Models

  3. AffordanceVLA: A Vision-Language-Action Model Empowering Action Generation through Affordance-Aware Understanding

    Jun 4, 2026Qize Yu, Jiadi You, Yuran Wang +12Language-Conditioned Robot ManipulationRobotic Manipulation