cs.RONov 9, 2026

UniOMA: Unified Optimal-Transport Multi-Modal Structural Alignment for Robot Perception

Authors: Xinrui Zu, Mark Hoogendoorn, Kevin Sebastian Luck, Shujian Yu

Organizations: Vrije Universiteit Amsterdam, the Netherlands

Abstract

Achieving generalizable and well-aligned multimodal representations remains a core challenge in robot learning. While recent approaches have attempted to align modalities by modeling conditional or higher-order statistical dependencies, they do not explicitly consider the structural coherence across modalities. In this work, we propose a novel multimodal alignment method that augments existing contrastive losses with a geometry-aware Gromov--Wasserstein (GW) distance-based regularization. To this end, we encode intra-modal geometry for each modality and reduce cross-modal structural discrepancies through a dynamically learned GW barycenter, thereby encouraging structural consistency in addition to instance-level contrastive alignment. We apply this optimal-transport-based alignment strategy to robot perception tasks involving underexplored modalities such as force and tactile signals, where modality data often exhibit varying sample densities. Extensive experiments on five robotic benchmarks show that our method improves structural alignment and downstream performance over state-of-the-art multimodal contrastive baselines across a wide range of downstream tasks, including robot and environment state prediction. Moreover, our GW-based regularization can be seamlessly integrated into most InfoNCE-like objectives.

Figures & tables

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Multimodal Alignment Through Joint Kernel Entropic Gromov--Wasserstein Optimal Transport

    Aug 4, 2026Yixuan Florence Wu, Yilun Zhu, Naichen ShiCross-Modal AlignmentMultimodal Embedding

  2. GAAT: Geometry-Aware Alignment Transformer for Multimodal UAV Perception

    Aug 28, 2026Jingpu Yang, Debin Tang, Yilin Sun +4Cross-Modal AlignmentMultimodal Pretraining

  3. SomaAlign: Reliability-Aware Multimodal Representation Learning for Soft Robotic Grasping

    Nov 2, 2026Haoyuan Wang, Shuai Peng, Yongkai Liao +4Reliability-Aware Multimodal FusionRobotic Grasping