KhyontekAI at CheckThat! 2026: Multilingual Dense Retrieval and Cross-Encoder Reranking for Scientific Web Claims
Authors: Bhargab Jyoti Bhuyan, Pritam Deka
Organizations: Computer Science and Engineering Department, School of Engineering, Tezpur University, Assam, India · KhyontekAI, Guwahati, Assam, India
Scientific claims shared through social media platforms frequently reference research findings without explicitly linking to the corresponding scientific publications. Automatically retrieving the correct scientific source for such claims is a challenging multilingual retrieval problem involving semantic matching, paraphrasing, noisy social media text, and cross-domain language variation. In this work, we present a multilingual two-stage retrieval framework for CLEF 2026 CheckThat! Task 1 on Source Retrieval for Scientific Web Claims. Our system combines fine-tuned dense retrieval models with cross-encoder reranking for improving multilingual scientific document retrieval. We evaluate multiple transformer-based dense retrieval architectures including multilingual-E5, BGE, and Contriever under different metadata configurations. We further investigate multilingual joint training, retrieval depth analysis, metadata enrichment, and language balancing strategies for improving retrieval effectiveness. Experimental results demonstrate that metadata enrichment consistently improves retrieval effectiveness, while fine-tuned cross-encoder reranking provides additional gains in retrieval precision. Our final multilingual system achieves MRR@5 scores of 0.6222 for English, 0.5112 for German, and 0.6254 for French, with an overall average MRR@5 score of 0.5862.
Figures & tables
Figure 1: Overview of the proposed multilingual retrieval and reranking pipeline.
Parameter
Value
Loss Function
MultipleNegativesRankingLoss
Optimizer
AdamW
Learning Rate
2e-5
Batch Size
32
Epochs
3
Similarity Metric
Cosine Similarity
Table 1: Dense Retrieval Training Configuration
Model
MRR@5
multilingual-e5-large
0.4986
multilingual-e5-base
0.4854
bge-base-en-v1.5
0.4796
contriever-msmarco
0.4666
Table 2: Baseline Dense Retrieval Results on the English Development Set
Model
FULL
SIMPLE
multilingual-e5-large
0.5669
0.5596
multilingual-e5-base
0.5524
0.5451
bge-base-en-v1.5
0.5842
0.5789
contriever-msmarco
0.5500
0.5413
Table 3: Fine-Tuned Dense Retrieval Results on the English Development Set
System
MRR@5
Dense Retrieval Only
0.5842
Cross-Encoder Reranking
0.5948
Fine-Tuned Cross-Encoder
0.6222
Table 4: Impact of Cross-Encoder Reranking on the English Development Set
Top-k Retrieval Depth
MRR@5
100
0.5931
70
0.5934
50
0.5948
30
0.5941
20
0.5942
Table 5: Effect of Retrieval Depth During Reranking on the English Development Set
Language
Before Oversampling
After Oversampling
German
0.4758
0.5112
French
0.6154
0.6254
Table 6: Effect of German Oversampling During Multilingual Development Training
Language
MRR@5
English
0.6222
German
0.5112
French
0.6254
Average
0.5862
Table 7: Final Multilingual Development Phase Results
Language
MRR@5
English
0.5961
German
0.5014
French
0.5719
Average
0.5565
Table 8: Official CLEF 2026 Evaluation Phase Results on the Hidden Test Set
Team
Avg. MRR@5
claim2source
0.7628
team-outsider
0.7580
mattiaskvist
0.7464
CheckMate
0.7194
seupd2526-retrix
0.7108
KhyontekAI
0.5565
Table 9: Comparison with Top Systems on the Official CLEF 2026 Evaluation Phase
Misalignment between claims and their cited evidence is a common failure mode in reports generated by large language models, limiting their reliability in scientific and other high-stakes settings. We present DeepSciVerify, a two-stage pipeline for scientific claim-citation verification that combines abstract-level reasoning with selective escalation to passage-level evidence. The system first verifies claims using the abstract and defers uncertain cases, retrieving and analyzing full-text passages only when necessary. This design leverages complementary behaviors across LLMs, as some models are more conservative while others are more decisive under uncertainty. On the SCitance benchmark, DeepSciVerify achieves 86.7 Micro-F1, outperforming strong abstract-only baselines by +4.5 points while resolving 67% of instances without full-text retrieval. These results suggest that selective evidence escalation improves both accuracy and efficiency in claim-citation verification.
We present a multilingual fact-checking system deployed at Factiverse, designed for high-throughput and low-latency operation across diverse languages. The system follows a modular pipeline with three stages: claim detection, evidence retrieval and re-ranking, and veracity prediction. We fine-tune XLM-RoBERTa-Large for claim detection, mmBERT-base for three-label stance classification (Supports/Refutes/Mixed), and a SetFit-based multilingual re-ranker for claim--evidence matching. We compare these components against strong LLM baselines, including GPT-5.2, Claude Opus~4.6, and Qwen3-8b. Experiments on production data spanning 114 languages for claim detection and 28 languages for veracity prediction show that task-specific fine-tuning provides strong and stable multilingual performance, while the fine-tuned retrieval model remains competitive with modern proprietary embeddings. Same-hardware latency measurements further show large efficiency gains for encoder-based components, supporting their use in production deployments with tight cost and privacy constraints. Overall, compact fine-tuned, self-hosted models remain a practical and effective foundation for multilingual fact-checking at scale. Code and data used for this study are available at https://github.com/factiverse/factcheck-editor.
Pratuat Amatya, Vinay Setty
Factiverse AS and University of Stavanger Stavanger, Norway
Automatically verifying climate-related claims against scientific literature is a challenging task, complicated by the specialised nature of scholarly evidence and the diversity of rhetorical strategies underlying climate disinformation. ClimateCheck 2026 is the second iteration of a shared task addressing this challenge, expanding on the 2025 edition with tripled training data and a new disinformation narrative classification task. Running from January to February 2026 on the CodaBench platform, the competition attracted 20 registered participants and 8 leaderboard submissions, with systems combining dense retrieval pipelines, cross-encoder ensembles, and large language models with structured hierarchical reasoning. In addition to standard evaluation metrics (Recall@K and Binary Preference), we adapt an automated framework to assess retrieval quality under incomplete annotations, exposing systematic biases in how conventional metrics rank systems. A cross-task analysis further reveals that not all climate disinformation is equally verifiable, potentially implicating how future fact-checking systems should be designed.
Raia Abu Ahmad, Max Upravitelev, Aida Usmanova +2
Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI), Germany · Technische Universität Berlin, Germany · Leuphana Universität Lüneburg, Germany +1