CLEAR: Cross-Lingual Enhancement in Alignment via Reverse-training

Seungyoon Lee; Minhyuk Kim; Seongtae Hong; Youngjoon Jang; Dongsuk Oh; Heuiseok Lim

cs.CLApr 7, 2026

CLEAR: Cross-Lingual Enhancement in Alignment via Reverse-training

Authors: Seungyoon Lee, Minhyuk Kim, Seongtae Hong, Youngjoon Jang, Dongsuk Oh, Heuiseok Lim

Organizations: Department of Computer Science and Engineering, Korea University · Department of English Language and Literature, Kyungpook National University

Abstract

Existing multilingual embedding models often encounter challenges in cross-lingual scenarios due to imbalanced linguistic resources and less consideration of cross-lingual alignment during training. Although standardized contrastive learning approaches for cross-lingual adaptation are widely adopted, they may struggle to capture fundamental alignment between languages and degrade performance in well-aligned languages such as English. To address these challenges, we propose Cross-Lingual Enhancement in Retrieval via Reverse-training (CLEAR), a novel loss function utilizing a reverse training scheme to improve retrieval performance across diverse cross-lingual retrieval scenarios. CLEAR leverages an English passage as a bridge to strengthen alignments between the target language and English, ensuring robust performance in the cross-lingual retrieval task. Our extensive experiments demonstrate that CLEAR achieves notable improvements in cross-lingual scenarios, with gains up to 15%, particularly in low-resource languages, while minimizing performance degradation in English. Furthermore, our findings highlight that CLEAR offers promising effectiveness even in multilingual training, suggesting its potential for broad application and scalability. We release the code at https://github.com/dltmddbs100/CLEAR.

Figures & tables

Figure 1: Performance disparity of various embedding models across languages in a cross-lingual setup where English passage with other language queries in Belebele benchmark.

Figure 2: Comparison of the core idea of CLEAR with the standard InfoNCE loss. Solid arrows indicate the direct effects of training, while dashed arrows represent the indirect interactions, such as the resulting attraction or repulsion between representations.

Model	Language	English-English		English-Lang			Lang-English
Model	Language	InfoNCE	CLEAR	Base	InfoNCE	CLEAR	Base	InfoNCE	CLEAR
Belebele
bge-m3	zh	94.65 (-0.90)	95.47 (-0.08)	90.35	91.69	92.64	89.81	91.63	92.02
	es	95.51 (-0.04)	96.14 (+0.59)	91.99	92.81	93.39	92.05	93.50	93.82
	de	95.61 (+0.06)	96.11 (+0.56)	92.81	93.98	94.58	92.90	94.30	94.52
	hi	95.15 (-0.40)	95.75 (+0.20)	86.90	89.14	90.16	90.39	92.86	93.16

Table 1: Comprehensive cross-lingual evaluation results. In each cross-lingual setting, ‘Lang’ refers to the target language. The first word denotes the language of the passage, and the second one denotes the language of the query. The value in ‘()’ indicates the difference in performance from the original model.

Model	Passage-Query	Method	ar	de	zh	ru	es	hi	vi	te	bn
multilingual-e5	English-Lang	InfoNCE	82.58	89.99	86.84	89.16	89.46	83.10	86.69	76.48	81.85
	English-Lang	CLEAR	85.12	91.87	88.85	91.68	91.98	85.94	88.85	80.22	84.51
	Lang-English	InfoNCE	88.08	92.29	90.02	91.39	91.58	90.97	90.44	88.63	88.93
	Lang-English	CLEAR	88.89	93.47	90.78	92.25	93.14	92.39	91.47	89.50	90.68
gte-multilingual	English-Lang	InfoNCE	86.38	92.12	91.92	91.80	92.44	89.13	91.26	83.53	84.42
	English-Lang	CLEAR	87.77	92.92	92.70	92.56	93.29	90.13	91.82	85.19	85.70

Table 2: Cross-lingual evaluation results in multilingual training setup in Belebele benchmark.

Language	multilingual-e5		gte-multilingual
Language	InfoNCE	CLEAR	InfoNCE	CLEAR
en	94.94	95.61	94.66	95.46
ar	86.82	88.42	88.37	89.10
de	91.15	93.27	92.89	93.47
zh	91.28	93.05	92.85	93.65
ru	92.15	93.09	93.07	93.83

Table 3: Monolingual performances in multilingual training setup. Each row presents the result where the passage and query are in the same language.

Panel of Figure 3 — Figure 3: T-SNE visualization of the embeddings for multilingual-e5 with English passage and Arabic, Bengali queries after multilingual training. We randomly select 100 pairs from the Belebele and measure the distance between the embeddings of identical gold pairs.

Method	ar	de	zh	ru	es	hi	vi	te	bn	Average
CLEAR e5	83.78 / 88.23	91.94 / 92.86	88.89 / 90.75	90.59 / 92.16	91.61 / 93.13	86.10 / 92.13	88.63 / 91.95	80.97 / 88.99	85.44 / 89.92	87.55 / 91.12
[0.8pt / 1.2pt] w/o $\mathcal{L}_{\text{KL}}$	83.82 / 88.49	91.61 / 92.29	88.59 / 90.43	89.85 / 91.73	90.76 / 92.49	85.05 / 91.10	88.52 / 91.69	79.68 / 88.67	84.40 / 89.62	86.92 / 90.72
w/o Reverse	82.81 / 87.49	91.31 / 92.47	87.44 / 89.69	89.77 / 90.90	89.91 / 92.01	84.12 / 90.84	87.41 / 91.36	78.21 / 87.96	82.77 / 88.89	85.97 / 90.18
w/o $\mathcal{L}_{\text{NCE}_{\text{en}}}$	81.95 / 87.24	90.26 / 91.55	86.98 / 89.09	89.19 / 90.98	89.88 / 91.45	83.71 / 90.21	87.67 / 90.38	79.00 / 87.21	83.58 / 88.95	85.80 / 89.67
CLEAR gte	87.91 / 91.46	93.26 / 93.95	92.67 / 92.80	92.94 / 93.33	93.28 / 93.87	89.96 / 93.05	92.23 / 93.52	86.31 / 90.92	86.64 / 92.14	90.58 / 92.78
[0.8pt/1.2pt] w/o $\mathcal{L}_{\text{KL}}$	87.13 / 91.16	92.90 / 93.70	92.60 / 92.48	92.50 / 93.28	92.92 / 93.46	89.59 / 92.89	91.80 / 93.27	85.01 / 91.18	85.79 / 91.70	90.03 / 92.57

Table 4: Ablation results (English-Lang / Lang-English) for key components in CLEAR on multilingual-e5 and gte.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Details
bge-m3	BAAI/bge-m3
bge-m3	Chen et al. (2024)
multilingual-e5	intfloat/multilingual-e5-base
multilingual-e5	Wang et al. (2024b)
gte-multilingual	Alibaba-NLP/gte-multilingual-base
gte-multilingual	Zhang et al. (2024b)

Appendix

Table 5: Embedding models details.

Lang	bge-m3		multilingual-e5		gte-multilingual		jina-v3
Lang	InfoNCE	CLEAR	InfoNCE	CLEAR	InfoNCE	CLEAR	InfoNCE	CLEAR
Cross-lingual Scenario
ar	72.44	72.95	64.11	64.99	66.22	66.75	71.77	71.89
ru	76.93	76.77	70.84	71.83	74.36	74.76	76.99	77.59
te	68.78	69.66	55.56	59.42	68.36	69.26	67.63	67.83
bn	69.96	69.90	60.30	62.24	63.67	64.30	68.59	68.95

Appendix

Table 6: Cross-lingual evaluation results in XOR-TyDi. In the cross-lingual scenario, the results are from models fine-tuned individually for each language. In the multilingual expansion, the model is trained considering all languages equally, as in our main experiments.

$Q_{en}$	When was Winehouse quoted about the album release?
$P^{+}_{en}$	Winehouse and Ronson contributed a cover of Lesley Gore’s “It’s My Party” to the Quincy Jones tribute album Q Soul Bossa Nostra released 9 November 2010 … In July 2010, Winehouse was quoted as saying her next album would be released no later than January 2011, saying “It’s going to be very much the same as my second album, where there’s a lot of jukebox stuff and songs that are… just jukebox, really.”
$P^{-}_{en}$	On the other hand, Arabic is divided into over 27 dialects. Almost every Arab state has at least one local dialect of its own …
$Q_{\ell}$	zh: 懷恩豪斯何時提到這張專輯的發行?
	de: Wann hat Winehouse die Veröffentlichung dieses Albums erwähnt?
	ru: Когда Уайнхаус упомянула о выпуске этого альбома?
	es: ¿Cuándo mencionó Winehouse el lanzamiento de este álbum?

Appendix

Table 7: Example of inputs in the training.

Model	Lang	English-Lang		Lang-English
		InfoNCE	CLEAR	InfoNCE	CLEAR
bge-m3	ar	88.66	89.23	90.87	91.17
	zh	91.86	92.40	91.63	92.10
	es	92.66	93.17	93.07	93.44
	de	93.77	94.30	93.99	94.66
	ru	93.72	94.17	93.42	93.54

Appendix

Table 8: Cross-lingual evaluation result in Belebele trained with m2m100 translated dataset.

Panel of Figure 4 — Figure 4: Performance variation depending on the loss component weights ( $\lambda_{1}$ , $\lambda_{2}$ , $\lambda_{3}$ ) across multiple languages.

Model	Language	English-Lang						Lang-English
		nDCG@5			nDCG@10			nDCG@5			nDCG@10
		Base	InfoNCE	CLEAR	Base	InfoNCE	CLEAR	Base	InfoNCE	CLEAR	Base	InfoNCE	CLEAR
bge-m3	ar	86.62	88.22	88.29	87.77	89.06	89.02	86.42	90.66	90.93	87.63	91.61	91.91
	zh	89.70	90.93	92.14	90.35	91.69	92.64	89.08	90.97	91.51	89.81	91.63	92.02
	es	91.48	92.61	93.05	91.99	92.81	93.39	91.47	92.99	93.59	92.05	93.50	93.82
	de	92.20	93.62	94.31	92.81	93.98	94.58	92.47	93.87	94.30	92.90	94.30	94.52

Appendix

Table 9: Results on all languages under cross-lingual scenario in Belebele.

Model	Language	English-Lang						Lang-English
		nDCG@5			nDCG@10			nDCG@5			nDCG@10
		Base	InfoNCE	CLEAR	Base	InfoNCE	CLEAR	Base	InfoNCE	CLEAR	Base	InfoNCE	CLEAR
bge-m3	ar	91.84	92.69	93.24	92.24	93.08	93.50	91.91	94.03	94.44	92.38	94.41	94.71
	zh	93.85	93.66	94.34	94.04	94.03	94.68	92.74	93.80	94.50	93.18	94.21	94.82
	es	95.98	95.86	95.97	96.14	95.97	96.14	95.80	96.20	96.09	95.96	96.42	96.34
	de	95.29	95.64	95.91	95.52	95.75	96.04	95.89	95.53	95.91	96.17	95.67	95.93

Appendix

Table 10: Results on all languages under cross-lingual scenario in XQuAD.

Model	Language	English-Lang						Lang-English
		nDCG@5			nDCG@10			nDCG@5			nDCG@10
		Base	InfoNCE	CLEAR	Base	InfoNCE	CLEAR	Base	InfoNCE	CLEAR	Base	InfoNCE	CLEAR
bge-m3	ar	86.62	88.85	89.31	87.77	89.56	89.97	86.42	90.35	91.01	87.63	91.30	91.90
	zh	89.70	92.31	92.66	90.35	92.92	93.10	89.08	91.55	92.23	89.81	92.10	92.67
	es	91.48	92.03	93.22	91.99	92.66	93.44	91.47	92.60	93.29	92.05	93.26	93.81
	de	92.20	93.44	94.40	92.81	93.77	94.59	92.47	93.72	93.98	92.90	94.11	94.53

Appendix

Table 11: Results on all languages under the multilingual training setup in Belebele.

Model	Language	English-Lang						Lang-English
		nDCG@5			nDCG@10			nDCG@5			nDCG@10
		Base	InfoNCE	CLEAR	Base	InfoNCE	CLEAR	Base	InfoNCE	CLEAR	Base	InfoNCE	CLEAR
bge-m3	ar	91.84	92.29	93.19	92.24	92.53	93.46	91.91	94.12	94.26	92.38	94.52	94.62
	zh	93.85	94.22	94.21	94.04	94.50	94.58	92.74	93.53	93.98	93.18	93.83	94.26
	es	95.98	95.12	96.04	96.14	95.37	96.29	95.80	95.68	96.03	95.96	95.95	96.26
	de	95.29	95.06	95.65	95.52	95.22	95.81	95.89	95.29	95.81	96.17	95.46	95.91

Appendix

Table 12: Results on all languages under the multilingual training setup in XQuAD.

Multilingual IR

Multilingual information retrieval (IR) finds and ranks relevant information across languages, scripts, and language-mismatched queries and documents.

See all 53 papers

MIMO: Multilingual Information Retrieval via Monolingual Objectives
May 29, 2026Youngjoon Jang, Seongtae Hong, Heuiseok LimContrastive Learning Cross-Lingual Representation Alignment
Department of Computer Science and Engineering, Korea University
Enhancing Multilingual Embeddings via Multi-Way Parallel Text Alignment
Feb 25, 2026Barah Fazili, Koustava GoswamiCross-Lingual Representation Alignment Multilingual Language Models
IIT Bombay, Mumbai, India · Adobe Research, Bangalore, India
Bridging Linguistic Gaps: Cross-Lingual Mapping in Pre-Training and Dataset for Enhanced Multilingual LLM Performance
Apr 12, 2026Weihua Zheng, Chang Liu, Zhengyuan Liu +5Multilingual Language Models Cross-Lingual Representation Alignment
Singapore University of Technology and Design, Singapore · ByteDance, Singapore · Agency for Science, Technology and Research, Singapore

May 29, 2026J/K move · Enter open · S save

MIMO: Multilingual Information Retrieval via Monolingual Objectives

Youngjoon Jang, Seongtae Hong, Heuiseok Lim

Department of Computer Science and Engineering, Korea University

Contrastive Learning Cross-Lingual Representation Alignment Multilingual IR Representation Alignment

Open paper AskPDF arXiv

Multilingual Information Retrieval (MLIR) reflects real-world search environments in which queries and relevant documents may appear in different languages within a mixed-language corpus. However, existing embedding models are primarily optimized for Multi-Monolingual retrieval and their performance often degrades in MLIR settings. Moreover, directly applying conventional contrastive learning to MLIR can exacerbate language clustering and expose a trade-off between cross-lingual alignment and embedding uniformity. To address these limitations, we propose MIMO: Multilingual Information Retrieval via Monolingual Objectives, a two-stage framework that uses a stable English semantic space from a high-performing teacher model as an anchor. MIMO first initializes the student model's cross-lingual alignment through knowledge distillation, and then jointly optimizes distillation and cross-lingual contrastive learning to improve retrieval discrimination while preserving alignment. Extensive experiments show that MIMO consistently outperforms existing cross-lingual training baselines across various MLIR and Multi-Monolingual benchmarks. MIMO also remains competitive with off-the-shelf models of similar or larger parameter scales. Furthermore, our cross-lingual Alignment-Uniformity analysis clarifies the distinct roles of the two loss components and shows that their combination yields a favorable trade-off between alignment and uniformity.

Abstract

Multilingual IR

Explore similar work

MIMO: Multilingual Information Retrieval via Monolingual Objectives

Enhancing Multilingual Embeddings via Multi-Way Parallel Text Alignment

Bridging Linguistic Gaps: Cross-Lingual Mapping in Pre-Training and Dataset for Enhanced Multilingual LLM Performance