PSQE: A Theoretical-Practical Approach to Pseudo Seed Quality Enhancement for Unsupervised Multimodal Entity Alignment
Authors: Yunpeng Hong, Chenyang Bu, Jie Zhang, Yi He, Di Wu, Xindong Wu
Organizations: Key Laboratory of Knowledge Engineering with Big Data (the Ministry of Education of China), Hefei University of Technology, Hefei, Anhui, CN · Department of Data Science, College of William and Mary, Williamsburg, VA, USA · College of Computer and Information Science, Southwest University, Chongqing, CN
Multimodal Entity Alignment (MMEA) aims to identify equivalent entities across different data modalities, enabling structural data integration that in turn improves the performance of various large language model applications. To lift the requirement of labeled seed pairs that are difficult to obtain, recent methods shifted to an unsupervised paradigm using pseudo-alignment seeds. However, unsupervised entity alignment in multimodal settings remains underexplored, mainly because the incorporation of multimodal information often results in imbalanced coverage of pseudo-seeds within the knowledge graph. To overcome this, we propose PSQE (Pseudo-Seed Quality Enhancement) to improve the precision and graph coverage balance of pseudo seeds via multimodal information and clustering-resampling. Theoretical analysis reveals the impact of pseudo seeds on existing contrastive learning-based MMEA models. In particular, pseudo seeds can influence the attraction and the repulsion terms in contrastive learning at once, whereas imbalanced graph coverage causes models to prioritize high-density regions, thereby weakening their learning capability for entities in sparse regions. Experimental results validate our theoretical findings and show that PSQE as a plug-and-play module can improve the performance of baselines by considerable margins.
Figures & tables
Figure 1. Comparative analysis of three types of pseudo seed generation on the FR-EN dataset for Multimodal Entity Alignment (MMEA): (1) Type I: Unimodal, (2) Type II: Multimodal, and (3) Type III: Distribution-Aware Multimodal. An interesting phenomenon is observed: Type II achieves higher precision than Type I but performs worse in the downstream MMEA task. In contrast, Type III, due to its better distribution balance, yields the best performance.
Figure 2. Overall framework of PSQE. PSQE optimizes the precision and the graph coverage balance (distribution) of pseudo-alignment seeds in three stages to enhance their quality.
Figure 3. Example of an attraction term for Remark 1: x and y are correct entity pairs. Wrongly aligned seeds can push away vector representations from the correct entity, resulting in variability from the vector of the correct entity.
Figure 4. Examples of a repulsion term of Remark 2: x− is a neighboring entity, and red arrows indicate new negative samples. Negative samples were not added to x7− , x9− in the left figure. Unbalanced graph coverage leads to focusing on x1− , x2− , and x3− during optimization and under-optimization of entities in the region around x8− , which are difficult to distinguish, e.g., the bottom right corner of Fig. (a).
Model
Unsup
DBP15K ZH-EN
DBP15K JA-EN
DBP15K FR-EN
H@1
H@10
MRR
H@1
H@10
MRR
H@1
H@10
MRR
RDGCN (IJCAI, 2019)
×
.708
.846
.746
.767
.895
.812
.886
.957
.911
RPR-RHGT (IJCAI, 2022)
×
.693
-
.754
.886
-
.912
.889
-
.919
DESAlign (ICDE, 2024)
x
.826
.972
.885
.811
.963
.869
.810
.957
.865
EVA (AAAI, 2021)
✓
.783
.898
.825
.861
.946
.892
.914
.972
.936
EVA+PSQE
✓
.823
.930
.861
.873
.955
.904
.928
.977
.948
Table 1. Comparison of applying PSQE to four recent representative open-source unsupervised MMEA methods (EVA, MCLEA, MEAformer, PCMEA) on the DBP15K dataset. ‘MEAformer+PSQE’ denotes the application of PSQE to MEAformer. ‘Unsup’ indicates whether the method is unsupervised or not. It should be noted that the experimental results of EVA, MCLEA, MEAformer, and PCMEA deviate from the original paper in terms of specific values due to the differences in the setting of the initialization seeds.
Method
DW-V1
DW-V2
H@1
H@10
MRR
H@1
H@10
MRR
COTSAE (AAAI, 2020)
.709
.904
.77
.922
.983
.940
EVA
.880
.965
.914
.880
.963
.913
EVA+PSQE
.915
.971
.937
.900
.972
.929
MCLEA
.903
.975
.932
.896
.970
.926
MCLEA+PSQE
.912
.975
.937
.921
.977
.943
Table 2. Comparison results on the DWY15K dataset.
Figure 5. Case study of the results of MEAformer’s comparison on the unsupervised setting of PSQE vs. UVP on the JA-EN dataset, where correct entity pairs are marked with ‘✓’ and incorrect ones with ‘×’.
Figure 6. Ablation study of PSQE on the FR-EN dataset.
MEAformer
EVA
MCLEA
H@1
MRR
H@1
MRR
H@1
MRR
PSQE
.892
.921
.878
.907
.879
.911
PSQE (w/o rel)
.889
.919
.870
.901
.877
.908
PSQE (w/o attr)
.891
.921
.871
.902
.877
.909
PSQE (w/o visual)
.732
.795
.603
.667
.668
.735
Table 3. Component analysis for PSQE on MEAformer, EVA, and MCLEA models under the JA-EN dataset, where “w/o rel” denotes the removal of relational information from the PSQE, “w/o attr” denotes the removal of the attribute information and “w/o visual” denotes the removal of the visual information.
Figure 7. The impact of different clustering numbers in Stage I on the PSQE performance on the FR-EN dataset.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Model
ZH_EN
JA_EN
FR_EN
PSQE
82.5
81.0
81.0
PSQE - MIC
66.2
62.5
65.1
Appendix
Table 4. Effect of the MIC module on pseudo seed precision.
ZH-EN
JA-EN
FR-EN
stage I
75.3
64.3
66.2
stage II
502.4
457.5
458.6
stage III
5.2
4.7
5.4
Appendix
Table 5. Efficiency analysis of PSQE. PSQE runtime (seconds) statistics for generating pseudo-aligned seeds on the three DBP15K datasets.
Method
ZH_EN
JA_EN
FR_EN
MEAformer
0.838
0.887
0.929
MEAformer+PSQE
0.842
0.892
0.932
Appendix
Table 6. Comparison of H@1 accuracy with and without PSQE under fixed pseudo seed sizes.
Multimodal entity alignment (MMEA) is fundamental to integrating heterogeneous knowledge graphs (KGs), yet existing paradigms remain critically vulnerable to incomplete modalities and cross-graph structural asymmetries. This paper exposes a fundamental oversight in current MMEA research: treating feature missingness and topological discrepancies as isolated preprocessing or denoising challenges, which overlooks their destructive coupling during multimodal fusion. We systematically formulate and evaluate two newly identified failure modes: 1) Detail Missing (DM)-induced overvaluation, where the absence of local modality details causes models to overemphasize uninformative, blurred representations; and 2) Context Imbalance (CI)-induced suppression, where topological discrepancies across KGs allow dense structural contexts to overwhelm weaker but highly discriminative modal signals, triggering high-level semantic drift. To break this cycle of degradation, we introduce ALMEA, a generative and calibration framework that establishes a bidirectional feedback loop between low-level feature reconstruction and high-level latent space alignment. Specifically, Cross-Modality Imputation (CMI) synthesizes missing modality details to mitigate overvaluation, while Latent Semantic Calibration (LSC) aligns cross-graph joint embeddings to suppress topology-driven semantic drift. Under sparse supervision, we develop Controlled Iterative Selection (CIS) to adaptively acquire representative and diverse pseudo-labeled pairs, optimizing latent calibration under constrained budgets. Extensive evaluations on standard benchmarks demonstrate that ALMEA consistently outperforms state-of-the-art baselines. At a 20% supervision rate, ALMEA yields substantial absolute improvements of up to 5.20% in MRR and 6.40% in Hits@1, maintaining robust gains across higher supervision thresholds.
Xizhe Zhang, Jingfeng Zhang, Meng-Fen Chiang
School of Computer Science, The University of Auckland, New Zealand · College of Intelligent Robotics and Advanced Manufacturing/Institute of Trustworthy Embodied AI, Fudan University, China · College of Electrical and Computer Engineering, National Yang Ming Chiao Tung University, Taiwan
The visual modality, i.e., images, plays a key role in multi-modal entity alignment (MMEA). Existing approaches often directly fuse the image with other modalities to align different entities. Although simple, such strategies overlook the potential noise in the images and their semantic misalignment with corresponding entities, resulting in suboptimal fusion and degraded performance. Addressing this, we propose a novel Reliability-Aware framework for MMEA (RA-MMEA), which assesses visual reliability and adaptively improves unreliable visual representations for robust entity alignment. The core lies in two modules, including dependency-aware visual reliability prediction (DA-VRP) and stability-regularized visual embedding generation (SR-VEG). The former aims to estimate the reliability of an image by leveraging multi-modal dependency within the entity, while the latter focuses on producing alternative visual representation conditioned on semantics encoded in textual modalities for multi-modal fusion. Compared to current methods, RA-MMEA enables more reliable visual representations for modality fusion, thereby improving performance. In extensive experiments, RA-MMEA achieves state-of-the-art results, verifying the importance of reliable visual modality for entity alignment and the effectiveness of RA-MMEA. The code and results will be released.
Chenxiao Li, Yunhe Feng, Dongfang Liu +3
University of North Texas · Purdue University · Meta Inc
Multimodal Entity Linking (MEL) is a fundamental task in data management that maps ambiguous mentions with diverse modalities to the multimodal entities in a knowledge base. However, most existing MEL approaches primarily focus on optimizing instance-centric features and evidence, leaving broader forms of evidence and their intricate interdependencies insufficiently explored. Motivated by the observation that human expert decision-making process relies on multi-perspective judgment, in this work, we propose MSR-MEL, a Multi-perspective Evidence Synthesis and Reasoning framework with Large Language Models (LLMs) for unsupervised MEL. Specifically, we adopt a two-stage framework: (1) Offline Multi-Perspective Evidence Synthesis constructs a comprehensive set of evidence. This includes instance-centric evidence capturing the instance-centric multimodal information of mentions and entities, group-level evidence that aggregates neighborhood information, lexical evidence based on string overlap ratio, and statistical evidence based on simple summary statistics. A core contribution of our framework is the synthesis of group-level evidence, which effectively aggregates vital neighborhood information by graph. We first construct LLM-enhanced contextualized graphs. Subsequently, different modalities are jointly aligned through an asymmetric teacher-student graph neural network. (2) Online Multi-Perspective Evidence Reasoning leverages the power of LLM as a reasoning module to analyze the correlation and semantics of the multi-perspective evidence to induce an effective ranking strategy for accurate entity linking without supervision. Extensive experiments on widely used MEL benchmarks demonstrate that MSR-MEL consistently outperforms state-of-the-art unsupervised methods. The source code of this paper was available at: https://anonymous.4open.science/r/MSR-MEL-C21E/.
Mo Zhou, Jianwei Wang, Kai Wang +3
The University of New South Wales Sydney, Australia · Shanghai Jiao Tong University Shanghai, China · University of Technology Sydney Sydney, Australia