cs.AIAug 29, 2026

More Perspectives, Stronger Signals: Multi-Perspective Enhancement and Progressive Fusion for Multimodal Entity Representation Learning

Authors: Chenyi Xiong, Yan Zhang, Jing Hu, Ziyue Qin, Kui Xiao, Xiaopan Lyu, Xiaoju Hou, Zhifei Li

Organizations: School of Computer Science, Hubei University Wuhan, China · School of Cyber Science and Technology, Hubei University Wuhan, China · Institute of Vocational Education, Guangdong Industry Polytechnic University Guangzhou, China · School of Computer Science, Hubei Key Laboratory of Big Data Intelligent Analysis and Application, Hubei University Wuhan, China

Abstract

Learning effective multimodal entity representations is fundamental for reasoning tasks such as multimodal knowledge graph completion (MMKGC). However, existing methods often suffer from semantic over-smoothing within modalities and ineffective noise filtration across modalities, particularly under sparse or ambiguous conditions. To overcome these limitations, we propose PrismF, a unified framework that synergizes multi-perspective enhancement with progressive fusion to extract stronger signals from diverse inputs. PrismF enhances fine-grained intra-modal semantics through a multi-perspective mechanism that decomposes each modality into complementary views and constrains them with a decoupling loss to reduce representation collapse. Furthermore, it improves cross-modal integration through a progressive fusion strategy that dynamically calibrates inter-modal interactions, enabling the model to emphasize informative signals while suppressing noisy or unreliable ones. Extensive experiments on three public benchmarks show that PrismF achieves the strongest overall performance, including relative improvements of 4.04% in MRR and 11.17% in Hits@1 on KVC16K. Our code can be found at https://github.com/HubuKG/PrismF.

Figures & tables

Explore similar work

Oct 12, 2026

Silence and Noise: Resolving Suppression and Overvaluation in Multimodal Entity Alignment

Multimodal entity alignment (MMEA) is fundamental to integrating heterogeneous knowledge graphs (KGs), yet existing paradigms remain critically vulnerable to incomplete modalities and cross-graph structural asymmetries. This paper exposes a fundamental oversight in current MMEA research: treating feature missingness and topological discrepancies as isolated preprocessing or denoising challenges, which overlooks their destructive coupling during multimodal fusion. We systematically formulate and evaluate two newly identified failure modes: 1) Detail Missing (DM)-induced overvaluation, where the absence of local modality details causes models to overemphasize uninformative, blurred representations; and 2) Context Imbalance (CI)-induced suppression, where topological discrepancies across KGs allow dense structural contexts to overwhelm weaker but highly discriminative modal signals, triggering high-level semantic drift. To break this cycle of degradation, we introduce ALMEA, a generative and calibration framework that establishes a bidirectional feedback loop between low-level feature reconstruction and high-level latent space alignment. Specifically, Cross-Modality Imputation (CMI) synthesizes missing modality details to mitigate overvaluation, while Latent Semantic Calibration (LSC) aligns cross-graph joint embeddings to suppress topology-driven semantic drift. Under sparse supervision, we develop Controlled Iterative Selection (CIS) to adaptively acquire representative and diverse pseudo-labeled pairs, optimizing latent calibration under constrained budgets. Extensive evaluations on standard benchmarks demonstrate that ALMEA consistently outperforms state-of-the-art baselines. At a 20% supervision rate, ALMEA yields substantial absolute improvements of up to 5.20% in MRR and 6.40% in Hits@1, maintaining robust gains across higher supervision thresholds.
Jul 8, 2026

Time Imprint: Learning Time-Aware Representations in Multi-Modal Knowledge Graphs

Multi-Modal Knowledge Graphs (MMKGs) enrich entities with multiple modalities such as text and images, yet entities with highly similar multi-modal features remain difficult to distinguish. Temporal information of an entity can serve as an additional modality to disambiguate such entities, but existing approaches rarely treat time as a separate modality alongside text and images due to two major challenges: (1) sparse temporal semantics, which hinder alignment with richer modalities, and (2) multiple timestamps, which introduce noise or reduce robustness in representation learning. To address these challenges, we propose Time Imprint, a framework that treats time as an entity-level modality and jointly aligns temporal, textual, and visual representations via a three-view contrastive objective. Additionally, to mitigate multi-timestamp ambiguity, Time Imprint studies a compact timestamp subset selection design space and aggregates the selected timestamps into a discriminative temporal embedding with attention pooling, balancing temporal specificity and robustness. Experiments on three MMKG benchmarks demonstrate that Time Imprint achieves state-of-the-art link prediction performance, improving Hits@1 by up to 6.07% overall and yielding up to 58% gains on the subset of the top-1% ambiguity samples. We further examine different fusion strategies and the sensitivity to timestamp availability and quality, clarifying when and why time-as-modality is most beneficial, while adding only modest training overhead. We release our code at https://anonymous.4open.science/r/Time-Imprint.
Mar 10, 2026

Progressive Representation Learning for Multimodal Sentiment Analysis with Incomplete Modalities

Multimodal Sentiment Analysis (MSA) seeks to infer human emotions by integrating textual, acoustic, and visual cues. However, existing approaches often rely on all modalities are completeness, whereas real-world applications frequently encounter noise, hardware failures, or privacy restrictions that result in missing modalities. There exists a significant feature misalignment between incomplete and complete modalities, and directly fusing them may even distort the well-learned representations of the intact modalities. To this end, we propose PRLF, a Progressive Representation Learning Framework designed for MSA under uncertain missing-modality conditions. PRLF introduces an Adaptive Modality Reliability Estimator (AMRE), which dynamically quantifies the reliability of each modality using recognition confidence and Fisher information to determine the dominant modality. In addition, the Progressive Interaction (ProgInteract) module iteratively aligns the other modalities with the dominant one, thereby enhancing cross-modal consistency while suppressing noise. Extensive experiments on CMU-MOSI, CMU-MOSEI, and SIMS verify that PRLF outperforms state-of-the-art methods across both inter- and intra-modality missing scenarios, demonstrating its robustness and generalization capability.