KhyontekAI at CheckThat! 2026: Multilingual Dense Retrieval and Cross-Encoder Reranking for Scientific Web Claims
Authors: Bhargab Jyoti Bhuyan, Pritam Deka
Organizations: Computer Science and Engineering Department, School of Engineering, Tezpur University, Assam, India · KhyontekAI, Guwahati, Assam, India
Scientific claims shared through social media platforms frequently reference research findings without explicitly linking to the corresponding scientific publications. Automatically retrieving the correct scientific source for such claims is a challenging multilingual retrieval problem involving semantic matching, paraphrasing, noisy social media text, and cross-domain language variation. In this work, we present a multilingual two-stage retrieval framework for CLEF 2026 CheckThat! Task 1 on Source Retrieval for Scientific Web Claims. Our system combines fine-tuned dense retrieval models with cross-encoder reranking for improving multilingual scientific document retrieval. We evaluate multiple transformer-based dense retrieval architectures including multilingual-E5, BGE, and Contriever under different metadata configurations. We further investigate multilingual joint training, retrieval depth analysis, metadata enrichment, and language balancing strategies for improving retrieval effectiveness. Experimental results demonstrate that metadata enrichment consistently improves retrieval effectiveness, while fine-tuned cross-encoder reranking provides additional gains in retrieval precision. Our final multilingual system achieves MRR@5 scores of 0.6222 for English, 0.5112 for German, and 0.6254 for French, with an overall average MRR@5 score of 0.5862.
Figures & tables
Figure 1: Overview of the proposed multilingual retrieval and reranking pipeline.
Parameter
Value
Loss Function
MultipleNegativesRankingLoss
Optimizer
AdamW
Learning Rate
2e-5
Batch Size
32
Epochs
3
Similarity Metric
Cosine Similarity
Table 1: Dense Retrieval Training Configuration
Model
MRR@5
multilingual-e5-large
0.4986
multilingual-e5-base
0.4854
bge-base-en-v1.5
0.4796
contriever-msmarco
0.4666
Table 2: Baseline Dense Retrieval Results on the English Development Set
Model
FULL
SIMPLE
multilingual-e5-large
0.5669
0.5596
multilingual-e5-base
0.5524
0.5451
bge-base-en-v1.5
0.5842
0.5789
contriever-msmarco
0.5500
0.5413
Table 3: Fine-Tuned Dense Retrieval Results on the English Development Set
System
MRR@5
Dense Retrieval Only
0.5842
Cross-Encoder Reranking
0.5948
Fine-Tuned Cross-Encoder
0.6222
Table 4: Impact of Cross-Encoder Reranking on the English Development Set
Top-k Retrieval Depth
MRR@5
100
0.5931
70
0.5934
50
0.5948
30
0.5941
20
0.5942
Table 5: Effect of Retrieval Depth During Reranking on the English Development Set
Language
Before Oversampling
After Oversampling
German
0.4758
0.5112
French
0.6154
0.6254
Table 6: Effect of German Oversampling During Multilingual Development Training
Language
MRR@5
English
0.6222
German
0.5112
French
0.6254
Average
0.5862
Table 7: Final Multilingual Development Phase Results
Language
MRR@5
English
0.5961
German
0.5014
French
0.5719
Average
0.5565
Table 8: Official CLEF 2026 Evaluation Phase Results on the Hidden Test Set
Team
Avg. MRR@5
claim2source
0.7628
team-outsider
0.7580
mattiaskvist
0.7464
CheckMate
0.7194
seupd2526-retrix
0.7108
KhyontekAI
0.5565
Table 9: Comparison with Top Systems on the Official CLEF 2026 Evaluation Phase