Recurrent claims present a major challenge for automated fact-checking systems designed to combat misinformation, especially in multilingual settings. While tasks such as claim matching and fact-checked claim retrieval aim to address this problem by linking claim pairs, the broader challenge of effectively representing groups of similar claims that can be resolved with the same fact-check via claim clustering remains relatively underexplored. To address this gap, we introduce Claim2Vec, the first multilingual embedding model optimized to represent fact-check claims as vectors in an improved semantic embedding space. We fine-tune a multilingual encoder using contrastive learning with similar multilingual claim pairs. Experiments on the claim clustering task using three datasets, 14 multilingual embedding models, and 7 clustering algorithms demonstrate that Claim2Vec significantly improves clustering performance. Specifically, it enhances both cluster label alignment and the geometric structure of the embedding space across different cluster configurations. Our multilingual analysis shows that clusters containing multiple languages benefit from fine-tuning, demonstrating cross-lingual knowledge transfer.
Figures & tables
Figure 1: Sample claims from the same ground-truth cluster in MultiClaim Dataset, with BGE-M3 embeddings projected in 2D, predicted cluster memberships using Agglomerative clustering, and claim language.
Figure 2: Methodology of Claim2Vec learning and claim clustering
Figure 3: Positive and negative pairs’ cosine distance distribution - BGE-M3 vs Claim2Vec
Dataset
Clusters
Claims
Avg
Max
Languages
ClaimCheck
197
1187
6.03
28
22
ClaimMatch
192
1171
6.1
35
36
MultiClaim-Test
15,988
42.4K
2.65
31
64
Table 1: Statistics of cluster datasets
Model
Parameters
Embedding
Distiluse-base-multilingual-cased
135M
512
Paraphrase-multilingual-MiniLM-L12
118M
768
Paraphrase-multilingual-mpnet-base
278M
768
Gte-multilingual-base
305M
768
All-roberta-large-v1
355M
1024
LaBSE
471M
768
Table 2: Multilingual text embedding models
ClaimCheck
ClaimMatch
MultiClaim-Test
Approach
#Clusters
ARI
AMI
#Clusters
ARI
AMI
#Clusters
ARI
AMI
HDBSCAN
175
0.847
0.912
172
0.685
0.824
7765
0.007
0.416
Agglomerative
200
0.906
0.955
210
0.762
0.886
16344
0.625
0.757
Birch
332
0.789
0.867
333
0.684
0.822
24804
0.449
0.567
Optics
303
0.093
0.552
289
0.064
0.486
10323
0.001
0.227
MeanShift
354
0.836
0.879
329
0.589
0.815
–
–
–
Table 3: Clustering performance of different clustering algorithms.
ClaimCheck
ClaimMatch
MultiClaim-Test
Embedding Model
ARI
AMI
SS
ARI
AMI
SS
ARI
AMI
SS
Distiluse-base-multilingual-cased-v1
0.765
0.874
0.237
0.509
0.693
0.187
0.413
0.566
0.191
Paraphrase-multilingual-MiniLM-L12-v2
0.814
0.896
0.290
0.660
0.797
0.229
0.475
0.622
0.222
Paraphrase-multilingual-mpnet-base-v2
0.837
0.916
0.332
0.701
0.833
0.261
0.540
0.673
0.246
Gte-multilingual-base
0.882
0.929
0.285
0.613
0.797
0.211
0.569
0.698
0.237
All-roberta-large-v1
0.409
0.633
0.227
0.337
0.528
0.220
0.262
0.433
0.189
Table 4: Clustering Performance of Multilingual Embedding Models.
Dataset
Embedding Model
#Clusters
ARI
AMI
HOM
CMP
VM
SS
ClaimCheck (197)
BGE-M3
209
0.845
0.929
0.985
0.961
0.973
0.264
Claim2Vec
200
0.906
0.955
0.988
0.977
0.982
0.354
ClaimMatch (192)
BGE-M3
258
0.644
0.820
0.974
0.902
0.936
0.204
Claim2Vec
210
0.762
0.886
0.972
0.941
0.956
0.287
MultiClaim-Test (15988)
BGE-M3
17922
0.610
0.726
0.976
0.967
0.971
0.229
Claim2Vec
16338
0.626
0.758
0.972
0.975
0.973
0.284
Table 5: Clustering Comparison between BGE-M3 and Claim2Vec .
Figure 4: ARI, AMI, and Silhouette Score (SS) vs Number of Clusters Produced
Split
Mismerge
Dataset
BGE-M3
Claim2Vec
BGE-M3
Claim2Vec
ClaimCheck
57
37
26
18
ClaimMatch
146
83
48
37
MultiClaim-Test
8,474
6,383
3,495
3,258
Table 6: Number of split and mismerge errors produced by BGE-M3 and Claim2Vec across datasets.
Dataset
Model
AUC
Maximum
ARI
AMI
SS
ARI
AMI
ClaimCheck
BGE-M3
0.776
0.891
0.242
0.876
0.940
Claim2Vec
0.803
0.902
0.320
0.908
0.954
ClaimMatch
BGE-M3
0.668
0.826
0.186
0.746
0.863
Claim2Vec
0.693
0.846
0.259
0.762
0.885
MultiClaim-Test
BGE-M3
0.590
0.718
0.224
0.611
0.729
Table 7: AUC and Maximum of ARI, AMI and SS for Different Cluster Configurations.
Figure 5: Heatmap of Split and Merge Error Rates Across Languages in Claim2Vec
Figure 6: Multilingual vs Monolingual Gain (%)
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: 2D Projection of Two Topic Groups in MultiClaim
Figure 8: Top 30 Frequent Words of Two Topic Groups in MultiClaim
Approach
Hyperparameter
Search Options
ClaimCheck
ClaimMatch
MultiClaim-Test
HDBSAN
vector dimension
2-10
4
4
8
cluster selection epsilon
[0-0.1], step size of 0.01
0.1
0.1
0.055
min samples
Minimum value
1
1
1
min cluster size
Minimum value
2
2
2
Agglomerative
distance threshold
[0.5 - 1.5], step size of 0.05
1.38
1.258
0.95
linkage
Default
ward
ward
ward
Appendix
Table 8: Hyperparameters of Clustering Approaches
Figure 9: 2D projection of claims in ClaimCheck with the red color data points highlighting claims belonging to the split and mismerge error types
Figure 10: 2D projection of claims in ClaimMatch with the red color data points highlighting claims belonging to the split and mismerge error types
Claim (Language)
BGE-M3
Claim2Vec
COVID-19 vaccines contain graphene oxide (es)
53
2
Covid-19 vaccines contain graphene oxide (es)
53
2
COVID-19 vaccines contain graphene oxide. (fr)
53
2
Pfizer and Moderna COVID-19 vaccines contain toxic graphene oxide (es)
53
2
Covid19 vaccines have graphene oxide (es)
53
2
Covid vaccines contain graphene (es)
53
2
Appendix
Table 9: Split error example of claims belonging to the same ground-truth cluster in ClaimCheck .
Claim (Language)
BGE-M3
Claim2Vec
This video shows a massive full moon appearing in between Russia and Canada in the Arctic (en)
215
73
A video shows a huge Moon appearing in the Arctic Circle (it)
215
73
Video shows eclipse with giant Moon in the Arctic between Russia and Canada! (pt)
215
73
The video shows the moon rising in the Artice circle between Russia and Canada. (en)
215
73
A video shows a gigantic moon moving over the Arctic sky, eclipsing the sun, and disappearing, all in just 30 seconds. (en)
215
73
Video of a giant moon between Russia and Canada (en)
215
73
Appendix
Table 10: Split error example of claims belonging to the same ground-truth cluster in ClaimMatch .
Claim (Language)
BGE-M3
Claim2Vec
Dr. Tasuku Honjo, Professor of Medicine, Nobel Laureate of Japan, said that the Corona virus did not come naturally, but was created by China and released to the world (tel)
16039
14297
Japanese Nobel laureate Dr. Tasuku Honjo said the new coronavirus was engineered in a Chinese laboratory (eng)
16042
14297
Japanese professor, Nobel laureate Tasuku Honjo said that the coronavirus was created artificially in China (rus)
16043
14297
Nobel laureate Tasuku Honjo said that the new corona virus is not natural and that it was created in a Chinese laboratory (hbs)
16048
14297
Appendix
Table 11: Split error example of claims belonging to the same ground-truth cluster in MultiClaim-test .