Silence and Noise: Resolving Suppression and Overvaluation in Multimodal Entity Alignment

Xizhe Zhang; Jingfeng Zhang; Meng-Fen Chiang

cs.LGOct 12, 2026

Silence and Noise: Resolving Suppression and Overvaluation in Multimodal Entity Alignment

Authors: Xizhe Zhang, Jingfeng Zhang, Meng-Fen Chiang

Organizations: School of Computer Science, The University of Auckland, New Zealand · College of Intelligent Robotics and Advanced Manufacturing/Institute of Trustworthy Embodied AI, Fudan University, China · College of Electrical and Computer Engineering, National Yang Ming Chiao Tung University, Taiwan

Abstract

Multimodal entity alignment (MMEA) is fundamental to integrating heterogeneous knowledge graphs (KGs), yet existing paradigms remain critically vulnerable to incomplete modalities and cross-graph structural asymmetries. This paper exposes a fundamental oversight in current MMEA research: treating feature missingness and topological discrepancies as isolated preprocessing or denoising challenges, which overlooks their destructive coupling during multimodal fusion. We systematically formulate and evaluate two newly identified failure modes: 1) Detail Missing (DM)-induced overvaluation, where the absence of local modality details causes models to overemphasize uninformative, blurred representations; and 2) Context Imbalance (CI)-induced suppression, where topological discrepancies across KGs allow dense structural contexts to overwhelm weaker but highly discriminative modal signals, triggering high-level semantic drift. To break this cycle of degradation, we introduce ALMEA, a generative and calibration framework that establishes a bidirectional feedback loop between low-level feature reconstruction and high-level latent space alignment. Specifically, Cross-Modality Imputation (CMI) synthesizes missing modality details to mitigate overvaluation, while Latent Semantic Calibration (LSC) aligns cross-graph joint embeddings to suppress topology-driven semantic drift. Under sparse supervision, we develop Controlled Iterative Selection (CIS) to adaptively acquire representative and diverse pseudo-labeled pairs, optimizing latent calibration under constrained budgets. Extensive evaluations on standard benchmarks demonstrate that ALMEA consistently outperforms state-of-the-art baselines. At a 20% supervision rate, ALMEA yields substantial absolute improvements of up to 5.20% in MRR and 6.40% in Hits@1, maintaining robust gains across higher supervision thresholds.

Figures & tables

Panel of Figure 1 — Figure 1: Overview of DM/CI challenges in MMEA benchmarks and the proposed ALMEA framework. Top: Two illustrative aligned entity pairs with Detail Missing (DM) and Context Imbalance (CI). “x” denotes a missing modality, and CI is measured using the modality-wise contextual discrepancies $\Delta(g,r,a)$ between ground-truth pairs across the source and target KGs. Middle: Dataset statistics on FB15K-DB15K and FB15K-YAGO15K: overall DM rates for visual/attribute modalities (left) and CI distributions over all ground-truth pairs (middle/right), grouped into Low/Mid/High regions by the averaged discrepancy score. Bottom: Our solutions to DM and CI. Entities are fused into joint embeddings $(h_{i},h_{j})$ with modality weights $\pi_{m}$ for nearest-neighbor retrieval. Cross-Modality Imputation (CMI) restores missing low-level details via a conditional VAE to mitigate DM-induced overvaluation, whereas Latent Semantic Calibration (LSC) aligns latent semantics across KGs to alleviate CI-induced suppression.

Figure 2: Representative failure cases of multimodal entity alignment under DM and CI. Each row shows modality weights $\pi_{m}$ ( $m\in\{v,g,r,a\}$ ) for the source entity ( S ), the top-1 incorrect target match ( T , wrong), and the ground-truth target at its retrieved rank ( T , GT@r). Red “x” denotes a missing modality. Cases 1 and 3 are CI-dominated failures, whereas Case 2 is dominated by DM. Abbreviations: MP = "Member of Parliament", LL = "Lucy Liu", ZQ = "Zachary Quinto", CCA = "Chelsea College of Arts", and ITFC = "Ipswich Town F.C".

	FB15K-DB15K (20%)			FB15K-DB15K (50%)			FB15K-DB15K (80%)
Methods	MRR(%)	Hits@1(%)	Hits@10(%)	MRR(%)	Hits@1(%)	Hits@10(%)	MRR(%)	Hits@1(%)	Hits@10(%)
TransE	13.40	0 7.80	24.00	30.60	23.00	44.60	50.70	42.60	65.90
GCN-align	0 8.70	0 5.30	17.40	29.30	22.60	43.50	47.20	41.40	63.50
SEA	25.50	17.00	42.50	47.00	37.30	65.70	50.50	51.20	78.40
MMEA	35.70	26.50	54.10	51.20	41.70	70.30	68.50	59.00	86.90
EVA	18.48	11.90	31.60	26.14	18.06	42.32	35.34	25.98	53.48

Table 1: Cross-MMKG alignment accuracy under 20%, 50%, and 80% supervision. Bold represents two variants and the improvement, while underlining indicates the strongest baseline;

*

denotes results reproduced from released code, and the shade of each block represents the result of a paired

t

-test with ALMEA, based on five matched runs: significant , highly significant , extremely significant , and not significant . Improv.: A Comparison of ALMEA and SimDiff

Setting	Methods	$\text{DBP15K}_{ZH-EN}$			$\text{DBP15K}_{JA-EN}$			$\text{DBP15K}_{FR-EN}$
		MRR(%)	Hits@1(%)	Hits@10(%)	MRR(%)	Hits@1(%)	Hits@10(%)	MRR(%)	Hits@1(%)	Hits@10(%)
w/o SF	BootEA	70.3	62.9	84.7	70.1	62.2	85.4	73.1	65.3	87.4
	EVA	81.4	76.1	90.7	81.7	76.2	91.3	84.7	79.3	94.2
	MCLEA	86.5	81.6	94.8	86.5	81.2	95.2	88.5	83.4	97.5
	SimDiff	87.7	82.9	96.3	88.3	83.3	96.6	90.5	86.1	98.0
	ALMEA ${}_{\text{w/o CIS}}$	81.8	74.7	94.4	81.3	74.5	93.3	82.7	75.9	94.7

Table 2: Multilingual MMKGs Alignment Accuracy Comparison. The results comparison on three bilingual datasets in different settings: without using Surface Forms (w/o SF), with Surface Forms (w/ SF). Under the unsupervised (unsup) setting, ALMEA-V and ALMEA-N use visual and name similarities, respectively, to initialize the alignment space. Bold values highlight the ALMEA variants rather than the column-wise best results.

	MCLEA	GEEA	MEAformer	OTMEA	SimDiff	A -	A +
Params (M)	9.06	47.46	10.58	11.02	10.31	14.34	14.34
Iter/s $\uparrow$	3.94	0.73	0.74	0.69	0.36	1.28	1.12

Table 3: Training efficiency. A - /A + denote ALMEA without/with CIS; higher Iter/s indicates faster training.

Panel of Figure 3 — Figure 3: Effectiveness of diversity in iterative learning. t-SNE plots show the sub-dataset (blue) and the CIS-selected subset (red), comparing selections without and with diversity score $D$ .

	Variants	FB15K-DB15K			FB15K-YAGO15K
		MRR	Hits@1	Hits@10	MRR	Hits@1	Hits@10
	ALMEA ${}_{\text{w/o CIS}}$	84.83	80.45	93.28	80.10	74.48	90.16
Mod.	w/o DMDS	82.80	77.13	90.94	78.43	72.45	88.83
Mod.	w/o LSC	81.73	76.41	89.89	77.06	71.21	87.48
Loss	w/o $\mathcal{L}_{DML}$	82.96	77.80	91.36	78.33	72.30	88.86
	w/o $\mathcal{L}_{URL}$	83.46	78.50	91.51	78.56	72.67	88.99

Table 4: Ablation study. All metrics are reported in %. Results are grouped into ALMEA

{}_{\text{w/o CIS}}

and ALMEA

{}_{\text{w/ CIS}}

. ALMEA

{}_{\text{w/o CIS}}

is tested under 80% supervision, where “Mod.”, “Loss”, and “Mdl.” denote module-, loss-, and modality-level ablations, respectively. ALMEA

{}_{\text{w/ CIS}}

is tested under 20% supervision. DMDS denotes DM Disturbance Simulation.

Source	Modal information
Neighborhood
Ground Truth	Qantas, Jakarta, Cairns, Darwin_Northern_Territory, Canberra, Brisbane
ALMEA	Qantas , Jakarta , Cairns , Darwin_Northern_Territory , Canberra , Brisbane , Nine_Network , Network_Ten , Keith_Urban , Australia_(2008_film) , Adelaide , University_of_Adelaide
GEEA	Brisbane , Qantas , Cairns , Nine_Network , Network_Ten , University_of_Adelaide
Attribute
Ground Truth	number_of_runways, longitude, mean_elevation, latitude

Table 5: Qualitative comparison of imputed modalities for Perth Airport on FB15K. For each imputed modality, the nearest neighbors are retrieved in the embedding space. Blue text denotes retrieved items that exactly match the ground truth, while underlined text denotes additional non-ground-truth neighbors returned by KNN.

Panel of Figure 4 — Figure 4: Sensitivity analysis of the sparsification factor $\alpha$ .

Panel of Figure 5 — Figure 5: Candidate Pairs Quality Analysis of CIS. True positive, false positive, and duplication rates are shown over $\tau\in[0,0.2]$ for $\alpha\in\{0.1,0.35,1\}$ , without $D$ ((a)–(c)) and with $D$ ((d)–(f)).

Figure 6: Downstream Gains of CIS in Low-Resource Settings.

Data	Variants	20%			50%			80%
Data	Variants	MRR	H@1	H@10	MRR	H@1	H@10	MRR	H@1	H@10
DB	M	55.4	46.5	72.2	73.3	66.4	85.4	82.4	77.5	90.5
	E	59.1	50.3	75.5	73.5	66.7	85.4	83.0	78.0	90.7
	C	59.4	50.8	75.6	74.2	67.4	86.2	83.8	79.1	92.4
YG	M	44.3	35.9	60.4	66.0	58.6	79.4	77.0	71.0	87.3
	E	50.2	41.6	66.6	67.1	59.7	80.9	78.0	71.7	88.5

Table 6: CIS selection strategies at 20%, 50%, and 80% supervision. DB and YG denote FB15K-DB15K and FB15K-YAGO15K; M, E, and C denote the margin, entropy, and CIS variants of ALMEA, respectively.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

$\mathcal{G}$	$\|\mathcal{E}\|$	$\|\mathcal{R}\|$	#Triples( $\mathcal{R}$ )	$\mathcal{A}$	#Triples( $\mathcal{A}$ )	$\|\mathcal{V}\|$	$\|GT\|$ (FB15K-DB15K)	$\|GT\|$ (FB15K-YAGO15K)
FB15K	14,951	1,345	592,213	116	29,385	13,444	12,846	11,199
DB15K	12,842	279	89,197	225	48,080	12,837	12,846	–
YAGO15K	15,404	32	122,886	7	23,532	11,194	–	11,199

Appendix

Table 7: Dataset Summary.

|GT|

represents the ground-truth entity alignments, indicating the number of aligned entity pairs between the source and target datasets. Specifically,

|GT|

(FB15K-DB15K) denotes the number of aligned entity pairs between FB15K and DB15K, while

|GT|

(FB15K-YAGO15K) denotes the aligned pairs between FB15K and YAGO15K. Some source entities may lack image modality or a corresponding target entity.

Datasets	$T_{\mathrm{total}}$	$T_{\mathrm{warm}}$	$T_{\mathrm{inter}}$	$T_{r}$	$T_{\mathrm{iter}}$	batch-size	masked dropping rate	learning rate	sparsification factor $\alpha$ (20%, 50%, 80%)
FB15K-DB15K	750	500	50	5	1000	3,500	0.45	0.001	[0.35, 0.45, 0.95]
FB15K-YAGO15K	550	300	50	5	1000	3,500	0.50	0.001	[0.25, 0.35, 0.45]

Appendix

Table 8: Hyperparameter settings in the main experiments. Training Configuration Settings

	FB15K-DB15K			FB15K-YAGO15K
Variant	MRR	Hits@1	Hits@10	MRR	Hits@1	Hits@10
ALMEA ${}_{\text{w/o CIS}}$	84.83	80.45	93.28	80.10	74.48	90.16
w/o $\mathcal{L}_{DML}$	82.96	77.80	91.36	78.33	72.30	88.86
w/o $\mathcal{L}_{URL}$	83.46	78.50	91.51	78.56	72.67	88.99
w/o $\mathcal{L}_{CMAL}$	83.43	78.25	91.38	78.33	72.45	88.72

Appendix

Table 9: Auxiliary-loss ablation under 80% supervision. All metrics are reported in %.

Dataset & Ratio	MCLEA	MSNEA	GEEA	MEAformer	OTMEA	SimDiff (SoTA)
FB15K-DB15K (20%)			**		**	**
FB15K-DB15K (50%)		**			**
FB15K-DB15K (80%)					—
FB15K-YAGO15K (20%)					**
FB15K-YAGO15K (50%)		**			—
FB15K-YAGO15K (80%)					—

Appendix

Table 10: Summary of statistical significance (paired t-test over 5 runs). Significance levels: * =

p<0.05

, ** =

p<0.01

, *** =

p<0.001

, — = not significant.

Ground Truth Nearest Neighbor	ALMEA Nearest Neighbor	GEEA Nearest Neighbor
# Entity Name : Bulgaria	# Entity Name : Bulgaria	# Entity Name : Bulgaria

# Neighborhood : Belgium, Hungary, Greece, Romania, Croatia	# Neighborhood : Hungary , Greece , Romania , Croatia , Slovenia , Chile , Colombia , Lithuania , Taiwan , Iceland	# Neighborhood : Belgium , Greece , Slovenia , Chile , Lithuania , Iceland
# Attribute : iso_numeric, longitude, latitude, area, date_founded, population_number, calling_code	# Attribute : iso_numeric , longitude , latitude , area , date_founded , population_number , calling_code , date_founded , date_dissolved , wasCreatedOnDate , populationDensity	# Attribute : iso_numeric , longitude , latitude , area , date_founded , population_number , calling_code , populationDensity

Appendix

Table 11: Qualitative comparison of imputed modalities for Bulgaria on FB15K. For each imputed modality, the nearest neighbors are retrieved in the embedding space. Blue text denotes retrieved items that exactly match the ground truth, while underlined text denotes additional non-ground-truth neighbors returned by KNN.

Ground Truth Nearest Neighbor	ALMEA Nearest Neighbor	GEEA Nearest Neighbor
# Entity Name : Mayor	# Entity Name : Mayor	# Entity Name : Mayor

# Neighborhood : Lafayette_Louisiana, Dnipropetrovsk, Laredo_Texas	# Neighborhood : Lafayette_Louisiana , Dnipropetrovsk , Laredo_Texas , Municipal Council	# Neighborhood : Lafayette_Louisiana , Laredo_Texas , California , Municipal_Councils , United_States
# Attribute : Null	# Attribute : date_established , date_founded , date_of_birth , height_meters , initial_release_date , interest	# Attribute : activeYearsEndYear , birthYear , activeYearsStartYear , birthDate

Appendix

Table 12: Qualitative comparison of imputed modalities for Mayor on FB15K. For each imputed modality, the nearest neighbors are retrieved in the embedding space. Blue text denotes retrieved items that exactly match the ground truth, while underlined text denotes additional non-ground-truth neighbors returned by KNN.

Ground Truth Nearest Neighbor	ALMEA Nearest Neighbor	GEEA Nearest Neighbor
# Entity Name : River Thames	# Entity Name : River Thames	# Entity Name : River Thames

# Neighborhood : London_Borough_of_Newham, London_Borough_of_Tower_Hamlets, City_of_London, Richmond_London	# Neighborhood : London_Borough_of_Newham , London_Borough_of_Tower_Hamlets , City_of_London , Richmond_London , Naveen_Andrews , United_Kingdom , Greater_London , Lincoln_England , St_Helens_Merseyside , London_Borough_of_Hackney	# Neighborhood : London_Borough_of_Newham , London_Borough_of_Tower_Hamlets , City_of_London , City_of_Westminster , Greater_London , London_Borough_of_Hackney
# Attribute : discharge, length, source_elevation, basin_area	# Attribute : discharge , length , source_elevation , basin_area , date_founded , latitude , longitude , elevation , areaCode	# Attribute : discharge , length , source_elevation , basin_area

Appendix

Table 13: Qualitative comparison of imputed modalities for River Thames on FB15K. For each imputed modality, the nearest neighbors are retrieved in the embedding space. Blue text denotes retrieved items that exactly match the ground truth, while underlined text denotes additional non-ground-truth neighbors returned by KNN.

Figure 7: Random recovery case under Limited Supervision. With only 5% of the supervision and just 2% of the budget, CIS elevated GT’s ranking to top 1.

	FB15K-DB15K (20%)			FB15K-DB15K (50%)			FB15K-DB15K (80%)
Methods	MRR(%)	Hits@1(%)	Hits@10(%)	MRR(%)	Hits@1(%)	Hits@10(%)	MRR(%)	Hits@1(%)	Hits@10(%)
ALMEA ${}_{\text{w/ margin}}$	55.42	46.52	72.24	73.30	66.40	85.39	82.40	77.53	90.54
ALMEA ${}_{\text{w/ entropy}}$	59.09	50.27	75.45	73.52	66.74	85.40	82.95	77.98	90.74
ALMEA ${}_{\text{w/ CIS}}$	59.43	50.79	75.64	74.16	67.40	86.16	83.80	79.08	92.41

Appendix

Table 14: CIS Selection Strategy Analysis under Limited Supervision. Results are under 20%, 50%, and 80% valid alignment pairs settings.

	FB15K-DB15K (20%)			FB15K-DB15K (50%)			FB15K-DB15K (80%)
Methods	MRR(%)	Hits@1(%)	Hits@10(%)	MRR(%)	Hits@1(%)	Hits@10(%)	MRR(%)	Hits@1(%)	Hits@10(%)
EVA	18.70	12.43	31.83	25.3	17.7	40.5	35.3	25.8	53.3
MCLEA	54.0	44.89	71.54	64.9	56.11	78.87	77.9	70.97	88.17
MSNEA	60.76	52.21	70.39	64.23	56.04	79.89	63.71	55.79	78.62
MEAformer	66.3	57.72	81.27	74.8	68.21	86.49	82.8	77.55	88.92
SimDiff	67.8	60.16	81.89	77.1	71.55	87.21	84.41	80.79	91.01

Appendix

Table 15: CIS under unlimited budget, also be described as iterative learning (IL). Here, ALMEA

{}_{\text{w/ CIS}}

stands for ALMEA with Controlled Iterative Selection (using a limited budget). ALMEA

{}_{\text{w/ IL}}

denotes ALMEA with Iterative Learning (use an unlimited budget).

	FB15K-DB15K (20%)
Methods	MRR(%)	Hits@1(%)	Hits@10(%)
ALMEA (Imbalance-0.05 $\times$ )	55.06	45.73	72.80
ALMEA (Imbalance-0.1 $\times$ )	55.51	46.21	73.36
ALMEA (Imbalance-0.5 $\times$ )	59.09	50.27	75.76
ALMEA	59.43	50.79	75.64

Appendix

Table 16: CIS robustness under imbalanced candidate spaces. We gradually reduce the number of target entities to simulate the scale difference between the source and target KGs, and report the resulting alignment performance.

Notation	Explanation
Multimodal Knowledge Graph (MMKG)
$\mathcal{G}$	Multimodal knowledge graph
$\mathcal{E}$	Set of entities
$\mathcal{R}$	Set of relations
$\mathcal{A}$	Set of attributes
$\mathcal{V}$	Visual information associated with the entities

Appendix

Table 17: Summary of Notations Used in ALMEA

Cross-Modal Alignment

Cross-modal alignment maps representations from different data types into compatible spaces so models can compare, transfer, and combine information.

See all 440 papers

Knowing When to Trust Images: Reliability-Aware Multi-modal Entity Alignment
Sep 20, 2026Chenxiao Li, Yunhe Feng, Dongfang Liu +3Cross-Modal Alignment Uncertainty-Aware Multimodal Fusion
University of North Texas · Purdue University · Meta Inc
PSQE: A Theoretical-Practical Approach to Pseudo Seed Quality Enhancement for Unsupervised Multimodal Entity Alignment
Feb 26, 2026Yunpeng Hong, Chenyang Bu, Jie Zhang +3Cross-Modal Alignment Pseudo-Labeling
Key Laboratory of Knowledge Engineering with Big Data (the Ministry of Education of China), Hefei University of Technology, Hefei, Anhui, CN · Department of Data Science, College of William and Mary, Williamsburg, VA, USA · College of Computer and Information Science, Southwest University, Chongqing, CN
More Perspectives, Stronger Signals: Multi-Perspective Enhancement and Progressive Fusion for Multimodal Entity Representation Learning
Aug 29, 2026Chenyi Xiong, Yan Zhang, Jing Hu +5Multimodal Learning Multimodal KGs
School of Computer Science, Hubei University Wuhan, China · School of Cyber Science and Technology, Hubei University Wuhan, China · Institute of Vocational Education, Guangdong Industry Polytechnic University Guangzhou, China +1

Sep 20, 2026J/K move · Enter open · S save

Knowing When to Trust Images: Reliability-Aware Multi-modal Entity Alignment

Chenxiao Li, Yunhe Feng, Dongfang Liu, Dong Nie+2

University of North Texas · Purdue University · Meta Inc

Cross-Modal Alignment Uncertainty-Aware Multimodal Fusion Multimodal Robustness

Open paper AskPDF arXiv

The visual modality, i.e., images, plays a key role in multi-modal entity alignment (MMEA). Existing approaches often directly fuse the image with other modalities to align different entities. Although simple, such strategies overlook the potential noise in the images and their semantic misalignment with corresponding entities, resulting in suboptimal fusion and degraded performance. Addressing this, we propose a novel Reliability-Aware framework for MMEA (RA-MMEA), which assesses visual reliability and adaptively improves unreliable visual representations for robust entity alignment. The core lies in two modules, including dependency-aware visual reliability prediction (DA-VRP) and stability-regularized visual embedding generation (SR-VEG). The former aims to estimate the reliability of an image by leveraging multi-modal dependency within the entity, while the latter focuses on producing alternative visual representation conditioned on semantics encoded in textual modalities for multi-modal fusion. Compared to current methods, RA-MMEA enables more reliable visual representations for modality fusion, thereby improving performance. In extensive experiments, RA-MMEA achieves state-of-the-art results, verifying the importance of reliable visual modality for entity alignment and the effectiveness of RA-MMEA. The code and results will be released.

Abstract

Cross-Modal Alignment

Explore similar work

Knowing When to Trust Images: Reliability-Aware Multi-modal Entity Alignment

PSQE: A Theoretical-Practical Approach to Pseudo Seed Quality Enhancement for Unsupervised Multimodal Entity Alignment

More Perspectives, Stronger Signals: Multi-Perspective Enhancement and Progressive Fusion for Multimodal Entity Representation Learning