cs.LGNov 16, 2026

UNITE: Consolidating Embodied VLM Specialists via Shared-Anchor One-Bit Factorization

Authors: Tao Tao, Anke Tang, Enneng Yang, Zhi Hou, Xiaogang Wang, Dacheng Tao

Organizations: Sichuan University, Chengdu, China · ACE Robotics, Shanghai, China · Nanyang Technological University, Singapore · Shenzhen Campus of Sun Yat-sen University, Shenzhen 518107, China

Abstract

Embodied capabilities such as visual grounding, spatial reasoning, navigation, and driving understanding are often developed in separate vision-language specialists fine-tuned from a common pretrained model.Deploying the specialists separately multiplies storage, whereas merging them into one dense model forces conflicting task updates to share parameters and loses part of each specialist's expertise.We introduce UNITE, a framework that consolidates such specialists into a compact shared-backbone mixture of experts. UNITE keeps the feed-forward block of each specialist as a routable expert and represents the expert bank through a shared-anchor one-bit factorization: a shared floating-point anchor plus expert-specific, row-scaled binary deviations.The anchor is optimized jointly with the scales and signs through closed-form alternating updates, using only source weights and no calibration data.Router adaptation then adjusts token-level expert selection with the backbone and experts frozen, updating fewer than 0.60.6M parameters (<0.01%<0.01\% of the 8.88.8B active parameters). Built from four Qwen3-VL-8B specialists, the complete UNITE model occupies 20.320.3 GB, 1.16×1.16\times the storage of one dense specialist and 59.6%59.6\% below the uncompressed MoE. Across 2222 benchmarks and R2R Val-Unseen, UNITE matches six static merging methods in mean retention of best-specialist performance, leads all of them on Scan2Cap, ScanQA, and all four navigation metrics, and surpasses every source specialist on all five 3D scene understanding benchmarks (up to +19.1+19.1 on Scan2Cap) and on R2R navigation. Relative to the uncompressed MoE, factorization is nearly lossless on 2121 of 2222 benchmarks; the exception is VSI-Bench, where frame-count experiments suggest impaired use of cross-frame evidence.UNITE thus consolidates complementary embodied capabilities near single-model storage.

Figures & tables

Explore similar work

CardsList
  1. UniCompress: From Multi-LoRA Experts to One Compact Model for Edge Intelligence

    Nov 4, 2026Mingkang Liu, Peipei Jiao, Weiming Meng +3VLM CompressionVLM Quantization

  2. Pelican-Unify 1.0: A Unified Embodied Intelligence Model for Understanding, Reasoning, Imagination and Action

    May 14, 2026Yi Zhang, Yinda Chen, Che Liu +26Action-Conditioned Video GenerationUnified Multimodal Models

  3. Train Together or Merge Later? Unifying VLA Experts via a Shared Action Interface

    Sep 27, 2026Zhizhen Zhang, Yuxia Fu, Zijian Wang +2VLM MergingVision-Language-Action Models