cs.AINov 18, 2026

NaviGATE: Noise-Aware Dual-View Generation for Multimodal Knowledge Graph Reasoning

Authors: Jie Peng, Yongxue Shan, Yongfu Zha, Cui Miao, Xiaodong Wang

Organizations: College of Computer Science and Technology, National University of Defense Technology, Changsha 410073, China

Abstract

Multimodal knowledge graph reasoning (MKGR) aims to infer missing facts by jointly exploiting relational structures, textual descriptions, and visual evidence. Existing embedding-based methods effectively capture graph topology, but their fixed multimodal representations are less effective for query-specific fine-grained semantic reasoning, while generation-based methods are vulnerable to noisy retrieval and may produce invalid entities. To address these challenges, we propose NaviGATE, a noise-aware generative framework for MKGR. NaviGATE first aligns graph-derived structural representations with textual semantics, then performs structure-aware internal knowledge activation by fine-tuning the multimodal language model with multimodal entity summaries and injected structural tokens. In the noise-aware dual-view instruction tuning stage, NaviGATE jointly optimizes retrieval-free and retrieval-aware views: reliable retrieved candidates are reinforced through contrastive logit sharpening, while misleading candidates are suppressed through parametric consistency distillation. In inference, structure-constrained decoding restricts generation to valid entities in the target knowledge graph. Experiments on standard benchmarks show that NaviGATE consistently outperforms strong unimodal, multimodal, and generation-based baselines, and remains robust under noisy retrieval scenarios.

Figures & tables

Explore similar work

CardsList
  1. GraphLoom: Reliability-Calibrated Graph Evidence Routing for Multimodal KG-RAG

    Aug 15, 2026Zafar Ali, Asad Khan, Aalia Malik +1Multimodal RAGGraph Retrieval-Augmented Generation

  2. MG2^2-RAG: Multi-Granularity Graph for Multimodal Retrieval-Augmented Generation

    Apr 4, 2026Sijun Dai, Qiang Huang, Xiaoxing You +1Cross-Modal LearningMultimodal RAG

  3. When Modalities Remember: Continual Learning for Multimodal Knowledge Graphs

    Apr 3, 2026Linyu Li, Zhi Jin, Yichi Zhang +5Multimodal KGsContinual Learning