cs.CLMar 17, 2026

Omnilingual SONAR: Cross-Lingual and Cross-Modal Sentence Embeddings Bridging Massively Multilingual Text and Speech

Authors: Omnilingual SONAR Team, João Maria Janeiro, Pere-Lluís Huguet Cabot, Ioannis Tsiamas, Yen Meng, Vivek Iyer, Guillem Ramírez, Loic Barrault, 

Organizations: FAIR at Meta

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Cross-lingual sentence encoders typically cover only a few hundred languages and often trade downstream quality for stronger alignment, limiting their adoption. We introduce OmniSONAR, a new family of omnilingual, cross-lingual and cross-modal sentence embedding models that natively embed text, speech, code, and mathematical expressions in a single semantic space, while delivering state-of-the-art downstream performance at the scale of thousands of languages, from high-resource to extremely low-resource varieties. To reach this scale without representation collapse, we use progressive training. We first learn a strong foundational space for 200 languages with an LLM-initialized encoder-decoder, combining token-level decoding with a novel split-softmax contrastive loss and synthetic hard negatives. Building on this foundation, we expand to several thousands language varieties via a two-stage teacher-student encoder distillation framework. Finally, we demonstrate the cross-modal extensibility of this space by seamlessly mapping 177 spoken languages into it. OmniSONAR halves cross-lingual similarity search error on the 200-language FLORES dataset and reduces error by a factor of 15 on the 1,560-language BIBLE benchmark. It also enables strong translation, outperforming NLLB-3B on multilingual benchmarks and exceeding prior models (including much larger LLMs) by 15 chrF++ points on 1,560 languages into English BIBLE translation. OmniSONAR also performs strongly on MTEB and XLCoST. For speech, OmniSONAR achieves a 43% lower similarity-search error and reaches 97% of SeamlessM4T speech-to-text quality, despite being zero-shot for translation (trained only on ASR data). Finally, by training an encoder-decoder LM, Spectrum, exclusively on English text processing OmniSONAR embedding sequences, we unlock high-performance transfer to thousands of languages and speech for complex downstream tasks.

Figures & tables

Figure 1: The training stages. In Stage 1 , we train our LLM-initialized encoder-decoder on translation data with a decoding loss. In Stage 2 , we introduce an encoder bottleneck via pooling and train with a combination of contrastive and decoding objectives. In Stage 3 , we introduce hard negatives and continue training with a split-softmax contrastive objective and the decoding loss. In Stage 4 , we extend the space to omnilingual-level language coverage by training with teacher-student distillation on 4,200 language varieties with a combination of MSE and Contrastive objectives, while first warming-up the omnilingual tokenization with MSE-based distillation. Lastly, in Stage 5 , we extend the omnilingual space to the speech modality with teacher-student distillation using ASR data.
Example 1: Python
Sourceif input_event["enable_points"] or event_info.get("bonus"):
TargetThe script determines if the "enable_points" in the input_event dictionary is set to True, or if the event_info dictionary includes a key called "bonus" with any associated value.
Example 2: JavaScript
Sourceconst DataHandler = require(’./lib/dataHandler.js’); let windowRef; const dataHandler = new DataHandler({});
TargetA constant named DataHandler is initialized by importing a module from a file called dataHandler.js located in a folder named lib , then a variable windowRef is declared, and a constant dataHandler is created as a new instance of DataHandler , passing an empty object to its constructor.
Example 3: Math
Table 1: Examples of paired data for code and mathematical expressions.
Original SentenceGenerated hard negatives
It was carrying a special mission unitIt was not carrying a special mission unit
It is carrying a normal mission unit
She was carrying a special mission unit
It will be carrying a special mission team
It was carrying 12 special mission units
Table 2: Examples of natural language hard negatives.
Figure 2: Example of OmniSONAR natural text prompting on both encoder and decoder sides. Source sentences, which are input to the encoder, are prefixed with language identifiers. The format used is “[language name]:”. Starting from the omnilingual extension, this prefix can be randomly replaced by ”Unspecified language:”. Target sentences, provided to the decoder, include task specifications, output language information, and information about data provenance (indicating whether the translation is human-labeled, automatically mined, or back-translated), following NLLB Team (2024) . Specifically, we use the prompts such as This is a possible translation in [language name]: for translation tasks and This is a possible natural language explanation in English: for code and math explanation tasks. This input formatting is applied consistently across all training stages. We provide the full list of prompts in Appendix Table 40 .
ParameterFoundationalNew
MSE weight ( λMSE\lambda^{\text{MSE}} )0.50.50.10.1
Student-Teacher Contrastive weight ( \lambda^{\text{s \rightarrow t}} )1.01.01.01.0
Teacher-Student Contrastive weight ( \lambda^{\text{t \rightarrow s}} )0.50.500
Logit scale ( τ\tau )10106060
Teacher Embedding ( zteacher\mathbf{z}^{\text{teacher}} )Interpolated source-target †Target
Language drop prob. ( punkp_{\text{unk}} )0.250.250.50.5
Table 3: Loss function and training parameter configurations depending on the source language type (foundational vs new) during the omnilingual extension. † When the source language is English, we use the source embedding of the teacher.
Table 4: X-Eng cross-lingual similarity search error rates xsim/xsim++ ( ↓\downarrow ) on different multilingual test sets.
Table 5: Results for GMMLU question mining (left) for all 42 languages and those covered by the baselines (common) and XLCOST (right). xsim ( ↓\downarrow ) reported for all models.
Table 6: Comparison of xsim and xsim++ results on FLEURS test set for the 36 languages covered by SONAR. Tables 43(a) and 43(b) .
Table 7: Classification and Pair Classification results from sentence-level MTEB tasks.
Table 8: X-Eng translation quality chrF++ ( ↑\uparrow ) / xCOMET( ↑\uparrow ) on different multilingual test sets. FLORES+ reports only the 11 languages not in FLORES200.
Table 9: Speech Translation (X-Eng) BLEU score on FLEURS, comparing OmniSONAR with SONAR and other Speech Translation models.
Table 10: Results of smaller distilled models on cross-lingual similarity search (FLORES200) and downstream tasks (MTEB). xsim/xsim++ report error rates ( ↓\downarrow ) and MTEB scores are higher-is-better ( ↑\uparrow ).
Table 11: Training objectives ablations : Ablations on training objectives to learn a massively multilingual sentence embedding space on the cross-lingual similarity search task of FLORES200 dev set, as measured by xsim and xsim++.
Table 12: Contrastive Learning ablations: Effect of hyper-parameters and modeling options in Contrastive Learning on cross-lingual similarity search on FLORES200 dev set.
Table 13: Model initialization ablations : Effect of model weight initialization for sequence-to-sequence stage as well as for contrastive learning stage on respectively decoding performance (spBLEU and chrF++) and cross-lingual similariy search (xsim and xsim++) on FLORES200 dev set.
Figure 3: Loss function parameters for new languages. Results on Bible dev . chrF++ obtained with the OmniSONAR-200 decoder. Vertical dashed lines indicate the final configuration used for OmniSONAR.
 BIBLEFLORES
ModelxsimchrF++xsim++chrF++
(a) Omnilingual Vocabulary Adaptation warm-up
without4.837.15.955.4
with4.837.75.656.0
(b) Loss function for foundational Languages
MSEContr. s →\rightarrow tContr. t →\rightarrow s    
Table 14: Ablations on the OmniSONAR omnilingual extension evaluated on BIBLE and FLORES dev sets. Cross-lingual similarity error rates xsim/xsim++ ( ↓\downarrow ) and translation quality chrF++ ( ↑\uparrow ). The final OmniSONAR configuration is highlighted in gray.
Table 15: Comparison of the foundational 200-language model (OmniSONAR-200) and the final omnilingual model (OmniSONAR) in cross-lingual similarity search (xsim/xsim++ ↓\downarrow ) and translation quality (chrF++/ xCOMET ↑\uparrow ) in Bible test and FLORES devtest . For translation with each model we use their dedicated decoders.
  BIBLEFLORESFLORES+
VocabularyParamsxsimchrF++Fert.Thr.xsim++chrF++Fert.Thr.xsim++chrF++Fert.Thr.
200-lang Vocabularies           
256K1.54.737.557.52645.655.943.252314.343.376.4130
Omnilingual Vocabularies           
8K1.04.637.573.22635.655.870.332114.543.778.2335
16K1.04.837.666.92975.655.962.736514.643.970.2378
Table 16: Vocabulary size ablations. Results in X-Eng dev sets. The 200-language vocabulary is the one used from OmniSONAR-200, which is extended from the Llama3 128K vocabulary. Model parameters in Billions. Metrics: cross-lingual similarity error rates with xsim/xsim++( ↓\downarrow ); translation quality with chrF++ ( ↑\uparrow ) obtained with the OmniSONAR-200 decoder; Tokenizer fertility( ↓\downarrow ); and Inference Throughput( ↑\uparrow ) in sentences per second.
 chrF++ (Translation Quality)xsim (Similarity Search Error)
FeatureImportance95% CIRankImportance95% CIRank
Tokenizer Fertility22.61[20.88, 24.75]15.12[4.43, 5.73]2
Family Examples14.19[13.23, 15.13]24.47[3.99, 5.19]3
Language Examples12.03[11.11, 12.99]36.86[6.07, 7.59]1
Latitude6.89[6.50, 7.42]43.63[3.18, 4.23]4
Longitude5.53[5.10, 6.09]53.46[3.06, 3.79]5
Table 17: Feature importance for predicting language performance on new languages (n=1,420) using Gradient Boosting with permutation-based importance and 95% confidence intervals. Importance values indicate increase in RMSE when a feature is randomly shuffled.
Table 18: Cross-lingual transfer (CLT) on SIB200Classification: Models trained on French, evaluated zero-shot on 199 languages (all) and 80 baseline-supported languages (common), reporting average relative performance to French.
Figure 4: Cross-lingual similarity (xsim) and translation quality (chrF++) for encoders trained on progressively more language groups, evaluated on BIBLE dev X-Eng. For translation we use the OmniSONAR-200 decoder. In the Top subfigures (a) and (b), we display Resource-based grouping: Groups A-G sorted by resourcefulness. In the Bottom subfigures (c) and (d), we display Family-based grouping: Group A = Indo-European (largest family) and Groups B-F = other families. In bold is the best performance per column (excluding OmniSONAR). The ”All” column is the average across all 1,560 languages. OmniSONAR is trained on all groups plus an additional group containing 1,864 extremely low resource languages.
Table 19: Speech Translation performance (X-Eng) on unseen languages during training.
Figure 5: Architecture of the Spectrum model. An encoder processes a sequence of sentence-level OmniSONAR embeddings, while the decoder is a token-level transformer that generates answers auto-regressively from a prefixed prompt. The decoder achieves this by performing cross-attention over the outputs of the encoder. We refer to the encoder as the “SONAR tower” and the decoder as the “token tower”.
Figure 6: Training phases of the Spectrum model, divided in Pre-training (Encoder Warmup, Teacher-Forced Sentence-Level LM and Prefix-LM) and Instruction Finetuning.
Figure 7: Example of a prompt for a Classification task from SIB200 using our diverse prompt generation approach. System prompt is in red , task prompt in blue , choices block in orange and the target format prompt in brown . The question block, used exclusively for MCQ and QA tasks, is absent here. This prompt is fed to the decoder, while the input document is passed through the encoder as usual.
Figure 8: FLOPs analysis of Spectrum architecture versus token-level LLM decoder-only architecture such as Llama3 models. Greener means that Spectrum is more compute efficient than token-level decoder-only models.
 AE BLEU1AE BLEU2AE BLEU3AE BLEU4AE BLEU
Spectrum auto-encode99.999.899.799.699.71
Table 20: Auto-encoding (AE) BLEU scores obtained by splitting and encoding long documents from WikiLingua with OmniSONAR and decoding them back into English with Spectrum auto-encode .
Figure 9: Spectrum pre-training performance across XHellaswag and Spoken StoryCloze benchmarks.
ModelXBelebeleSIB200Taxi1500
Llama 3.2 3B Instruct53.0365.6919.67
Llama 3.1 8B Instruct63.0666.0724.18
Spectrum 6B-3B61.8579.9536.9
Table 21: Zero-shot multilingual results of Spectrum on XBelebele, SIB, and Taxi1500, compared with Llama models
 T →\to TS →\to T
Speech-XBelebele63.6462.74
Speech-SIB10085.1182.59
Table 22: Zero-shot multilingual speech understanding results of Spectrum on Speech-XBelebele and Speech-SIB. We compare performance on Text →\to Text (T →\to T) and Speech →\to Text (S →\to T) splits of each benchmark to demonstrate cross-modal transfer.
Figure 10: Average AER across language pairs for different layer numbers. We see that OmniSONAR exhibits a parabolic shape, with inner layers showing better alignment than the final layers.
Table 23: Last layer AER for different encoders.
Dataset/task#Test languagesExample
Word alignment14The necessary correction will be made . Eine entsprechende Anderung wird vorgenommen werden .
Massive52What is today ’s forecast for Berlin DATE PLACE
PAN-X40REDIRECCIÓN Algarrobo ( Chile ) B-LOC I-LOC
WiC6\left.\begin{array}[]{@{}l@{}}\text{Bolivia holds a key \lx@text@underline{play} in this process of peace.}\\ \text{A musical \lx@text@underline{play} on the same subject was also staged.}\end{array}\right\}\ \begin{array}[]{@{}l@{}}\text{Different}\\ \text{meaning}\end{array}
Table 24: Summary of the tasks used for the evaluation of cross-lingual token representations.
Table 25: Performance for cross-lingual token representation tasks. The first column presents the mean Alignment Error Rate (AER) for the last layer. The remaining columns display the average F1 scores achieved by a classifier that was trained using English token embeddings and then evaluated on data from other languages.
Table 26: Performance for sentence-level tasks. We report xsim ( Artetxe and Schwenk, 2019 ) and xsim++ ( Chen et al., 2023c ) for cross-lingual mining and ChrF++ ( Popović, 2017a ) on the FLORES ( NLLB Team, 2024 ) dataset for translation performance.
Figure 11: Training dynamics of SONAR and OmniSONAR trained with the OmniSONAR-Token objective.
Table 27: Sentence-level performance comparison between OmniSONAR and OmniSONAR-LC on FLORES (error rate ↓\downarrow ) and MTEB (score ↑\uparrow ).
Table 28: Cross-lingual similarity search error rates on Belebele paragraphs.
Table 29: Cross-lingual similarity search error rates on IWSLT2017 documents (X →\rightarrow Eng).
Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 12: Distribution of sentence count differences from the mean across 9 languages from FLORES paragraphs. Each subplot shows the histogram of (language sentence count - average sentence count) across all paragraphs. Red dashed lines indicate zero (perfect consistency) and the mean deviation for each language. (a) The baseline SaT-12L-sm model exhibits substantial deviation from zero, with means ranging from -0.34 to 0.64 and an overall standard deviation of 1.48 across all 201 languages, indicating inconsistent segmentation. (b) Our custom model produces distributions tightly centered around zero, with means ranging from -0.09 to 0.13 and an overall standard deviation of 0.34.
PatternDescription
(.∗?)(.*?)Inline math (e.g., a2+b2=c2a^2 + b^2 = c^2 )
(.∗?)(.*?)Display math with double dollar signs
$(.*?)$Display math with ......
\begin{equation}(.*?)\end{equation}Equation environment
\begin{align}(.*?)\end{align}Align environment
\begin{align*}(.?)\end{align}Align* environment
Appendix
Table 30: Summary of regular expressions used for extracting LaTeX math expressions and splitting sentences.
Figure 13: Three prompt templates for translation, code snippet semantic description, and mathematical notation explanation.
Figure 14: Examples of hard negatives generation across Python code (blue), JavaScript code (orange), and mathematical expressions (green).
Original sentenceGenerated hard negatives
Tonight will be long.Tonight will not be short
 Yesterday may be long
 Tomorrow will be short
 Tonight may not be long
 Last night was short
You got the archaeologist and the psychologist wrong.You did not get the archaeologist and the psychologist right
Appendix
Table 31: Natural language hard negative examples
Figure 15: Prompt for generating xsim++ transformations with clear instructions and structure.
CodeLanguageScriptFamilySubgroupingRes. 
ace_ArabAcehneseArabicAustronesianMalayo-PolynesianLow 
ace_LatnAcehneseLatinAustronesianMalayo-PolynesianLow 
acm_ArabMesopotamian ArabicArabicAfro-AsiaticSemiticLow 
acq_ArabTa’izzi-Adeni ArabicArabicAfro-AsiaticSemiticLow 
aeb_ArabTunisian ArabicArabicAfro-AsiaticSemiticLow 
afr_LatnAfrikaansLatinIndo-EuropeanGermanicHigh 
Appendix
Table 47
CodeLanguageScriptFamilySubgroupingRes.
ita_LatnItalianLatinIndo-EuropeanItalicHigh
jav_LatnJavaneseLatinAustronesianMalayo-PolynesianLow
jpn_JpanJapaneseJapaneseJaponicJapanesicHigh
kab_LatnKabyleLatinAfro-AsiaticBerberLow
kac_LatnJingphoLatinSino-TibetanBrahmaputranLow
kam_LatnKambaLatinAtlantic-CongoBenue-CongoLow
Appendix
Table 48
CodeLanguageScriptFamilySubgroupingRes.
smo_LatnSamoanLatinAustronesianMalayo-PolynesianLow
sna_LatnShonaLatinAtlantic-CongoBenue-CongoLow
snd_ArabSindhiArabicIndo-EuropeanIndo-AryanLow
som_LatnSomaliLatinAfro-AsiaticCushiticLow
sot_LatnSouthern SothoLatinAtlantic-CongoBenue-CongoHigh
spa_LatnSpanishLatinIndo-EuropeanItalicHigh
Appendix
Table 32: Correspondence table from No Language Left Behind project for OmniSONAR-200 languages. This table was adapted from ( NLLB Team, 2024 ) .
Languages      
ace_Arabace_Latnacm_Arabacq_Arabaeb_Arabafr_Latnajp_Arab
aka_Latnals_Latnamh_Ethiapc_Arabarb_Arabars_Arabary_Arab
arz_Arabasm_Bengast_Latnawa_Devaayr_Latnazb_Arabazj_Latn
bak_Cyrlbam_Latnban_Latnbel_Cyrlbem_Latnben_Bengbho_Deva
bjn_Arabbjn_Latnbod_Tibtbos_Latnbug_Latnbul_Cyrlcat_Latn
ceb_Latnces_Latncjk_Latnckb_Arabcrh_Latncym_Latndan_Latn
Appendix
Table 33: Complete list of languages covered by OmniSONAR-200. Languages shown in bold are supported by all models in our comparison. Underlined languages are supported by OmniSONAR-speech 3B and 7B models. Our initial training covers 202 languages total, with 81 languages supported across all compared models.
 Seq2SeqContrastive
Datasetpairsdirssourcetargetpairsdirssourcetarget
BT Math445M5688––––
Dictionary153M3.3K18297––––
Code/Math →\to Eng1B9919.0M991
Eng →\to Code941M818––––
Eng →\to Math80M111––––
Appendix
Table 34: Training Data Statistics for Seq2Seq and Contrastive Learning Approaches. Each dataset shows: pairs (number of translation pairs), dirs (number of translation directions, i.e., language X to Y), source (number of source languages), and target (number of target languages).
 LangsExamples 
GroupTrainDevTotal (M)Median (K)Note
Resource-based grouping ( Figures 4(a) and 4(b) )
A2111402,764541High-resource
B30024693104 
C3002691755 
D3002751137 
Appendix
Table 35: Language group statistics for resource-based (top) and family-based (bottom) grouping experiments. The resource-based grouping includes all languages with sufficient parallel data, sorted by resourcefulness. The family-based grouping assigns Indo-European to Group A and distributes other families equally across Groups B-F.
ParameterLargeMediumSmallTinyxTiny
vocab_size256,232128,23264,23232,23216,232
hidden_size2,0481,7921,5361,152512
intermediate_size8,1927,1686,1444,6082,048
num_hidden_layers161412108
num_attention_heads3228241816
num_key_value_heads87664
Appendix
Table 36: Differences across the model sizes for OmniSONAR as explained in Section 4.8
Modelxsimxsim++
Mean0.689.25
CLS0.648.77
Appendix
Table 37: Ablation on different pooling strategies, evaluated on FLORES200 dev set.
modelstdmean
MEXMA0.0312-0.0011
mE5 large0.0312-0.0008
LaBSE0.03580.0049
SONAR0.00740.0000
OmniSONAR0.0356-0.0006
Appendix
Table 38: Standard deviation (std) and mean of embedding features for different models when encoding FLORES200 dev set on all common languages.
Figure 16: Singular values of embeddings from different models.
SystemSource SentenceDesired RetrievedActual Retrieved
OmniSONARO Corpo de Engenheiros dos EUA estimou que 15 cm de chuva podem romper os diques anteriormente danificados.The U.S. Corps of Engineers estimated that 6 inches of rainfall could breach the previously damaged levees.The U.S. Corps of Engineers estimated that 15 inches of rainfall could breach the previously damaged levees.
OmniSONAROs limites de velocidade anunciados sao visivelmente mais baixos do que nas secoes anteriores e subsequentes - comumente 55-65 km/h - e a estrita obediencia a eles e ainda mais importante do que o contrario.Posted speed limits are noticeably lower than in previous and subsequent sections commonly 35-40 mph (56-64 km/h) and strict obedience to them is even more important than otherwise.Posted speed limits are noticeably lower than in previous and subsequent sections commonly 35-90 mph (56-64 km/h) and strict obedience to them is even more important than otherwise.
MEXMAO Corpo de Engenheiros dos EUA estimou que 15 cm de chuva podem romper os diques anteriormente danificados.The U.S. Corps of Engineers estimated that 6 inches of rainfall could breach the previously damaged levees.The U.S. Corps of Engineers estimated that 15 inches of rainfall could breach the previously damaged levees.
MEXMAReportagens televisivas divulgam a fumaca esbranquicada saindo da planta.Television reports show white smoke coming from the plant.Television reports show black smoke coming from the plant.
SONARNo periodo de um ano, uma pessoa infectada pode infectar entre 10 e 15 contatos proximos.In one year’s time, an infected person may infect 10 to 15 close contacts.In one year’s time, an infected person will infect 10 to 15 close contacts.
SONARAconteceu novamente no mesmo mes em Mashhad, outro aviao comercial entrou em uma pista e atingiu uma parede, matando dezessete pessoas.The same month saw another airliner overrun a runway at Mashhad and strike a wall, killing seventeen.The same month did not saw another airliner overrun a runway at Mashhad and strike a wall, killing seventeen.
Appendix
Table 39: Comparison of three systems (OmniSONAR, MEXMA, SONAR) on two examples each. For each example, the table shows the original source sentence (in Portuguese) and the desired retrieved English sentence, the actual retrieved English sentence.
SourcePrompt Template
Encoder
Source/Anchor"<CLS><s> [LANGUAGE]:<SEP> [INPUT SENTENCE] </s>"
Decoder
Primary data"<s> This is a possible translation in [LANGUAGE]:<SEP> [INPUT SENTENCE] </s>"
Mined data"<s> This is a possible mined translation in [LANGUAGE]:<SEP> [INPUT SENTENCE] </s>"
Back-translations"<s> This is a possible back-translation in [LANGUAGE]:<SEP> [INPUT SENTENCE] </s>"
Appendix
Table 40: Prompt Templates for Encoder and Decoder Components. The encoder uses classification prompts to identify language and content, while the decoder uses descriptive prompts tailored to different data sources and translation types. Placeholders [LANGUAGE] and [INPUT SENTENCE] are replaced with actual values during training.
LangSONARLaBSEMEXMAOmniSONARmE5LangSONARLaBSEMEXMAOmniSONARmE5LangSONARLaBSEMEXMAOmniSONARmE5LangSONARLaBSEMEXMAOmniSONARmE5
ace_Arab6.2396.4457.918.6077.77ace_Latn0.3032.618.600.108.70acm_Arab0.100.300.000.000.10acq_Arab0.000.100.000.000.10
aeb_Arab0.404.940.200.200.99afr_Latn0.000.000.000.000.00ajp_Arab0.100.490.000.000.20aka_Latn0.3053.4644.860.107.21
als_Latn0.000.000.000.000.00amh_Ethi0.000.000.000.000.40apc_Arab0.101.090.000.000.10arb_Arab0.000.000.000.000.00
ars_Arab0.100.000.000.000.10ary_Arab0.9913.140.990.892.77arz_Arab0.200.690.100.000.40asm_Beng0.101.780.000.000.69
ast_Latn0.000.200.000.000.00awa_Deva0.991.090.890.890.99ayr_Latn3.8572.1354.251.6843.97azb_Arab2.9644.371.680.6911.36
azj_Latn0.300.300.200.200.20bak_Cyrl0.0041.9011.360.001.68bam_Latn4.0565.6152.372.1714.62ban_Latn0.408.401.090.302.57
Appendix
Table 41: xsim results for all models in all languages, x-eng in FLORES200 devtest set.
LangSONARLaBSEMEXMAOmniSONARmE5LangSONARLaBSEMEXMAOmniSONARmE5LangSONARLaBSEMEXMAOmniSONARmE5LangSONARLaBSEMEXMAOmniSONARmE5
ace_Arab55.34100.0092.0936.7698.91ace_Latn16.3082.2149.418.1048.22acm_Arab11.1752.679.395.8327.37acq_Arab8.3046.548.2010.8723.02
aeb_Arab11.7666.6013.646.7231.72afr_Latn5.149.494.641.1913.34ajp_Arab7.6154.159.195.2423.81aka_Latn19.4792.3985.1810.7742.79
als_Latn5.1410.979.391.7818.48amh_Ethi8.6028.856.233.3643.48apc_Arab9.5858.0012.655.1428.46arb_Arab6.8235.876.722.3719.27
ars_Arab13.9339.7212.658.5023.52ary_Arab14.0377.0822.2312.4539.03arz_Arab10.8758.9911.464.8424.51asm_Beng14.9262.1511.176.4241.50
ast_Latn9.6838.0414.136.8219.86awa_Deva11.0726.9812.063.7527.67ayr_Latn34.7897.4383.1019.2785.77azb_Arab42.5994.7632.6121.2565.32
azj_Latn14.0317.9810.776.1329.45bak_Cyrl11.4691.9051.093.5640.71bam_Latn30.1495.2685.6717.0961.56ban_Latn13.0460.3829.156.1336.26
Appendix
Table 42: xsim++ results for all models in all languages, x-eng in FLORES200 devtest set.
Table 43: OmniSONAR-speech modality results on 36 common languages SONAR and OmniSONAR-speech models cover, x-eng on FLEURS dev set. SONAR results are for 36 different language-specific encoders, while OmniSONAR-speech are one model across all languages.
FamilyLangsExamplesxsimchrF++ FamilyLangsExamplesxsimchrF++
Indo-European5356,329,5700.956.3 Huitotoan31076.132.3
Atlantic-Congo652464,7282.843.3 Border21056.131.4
Afroasiatic196419,3704.244.0 Eastern Daly31027.531.1
Turkic82410,6560.655.4 Kunimaipan3969.329.9
Dravidian39369,9571.949.7 Araucanian3904.538.9
Austronesian795315,3733.041.8 Lengua-Mascoy68412.231.4
Appendix
Table 44: Detailed statistics and results per language family. Results in BIBLE test . Number of examples is in thousands. Families sorted by number of examples. We removed 55 families that are absent from BIBLE test .
Figure 17: UMAP visualization of the sentence “ During his time with the team, he scored 403 goals in 468 appearances. ” from FLORES devset along closest hard negatives, shown as red crosses. Lines connect the translations to their English counterpart.
Figure 18: UMAP visualization of the whole space defined by the FLORES devset for 20 languages with different scripts.
More in this topic

Sentence Embeddings

Sentence embeddings map sentences to vectors for semantic similarity, retrieval, clustering, and downstream language tasks.

See all 35 papers

Explore similar work

CardsList

  1. Omnilingual-MsASR: Benchmarking Multimodal, Multilingual, Long-Form Multi-Speaker ASR of Omni Large Language Models

    Oct 21, 2026Yuan Ge, Pei Zhang, Xize Cheng +8Multi-Speaker ASRSpeech Recognition Benchmarks

    Northeastern University, China · Alibaba Token Hub, Alibaba · Kunming University of Science and Technology +1

  2. Cross-lingual Matryoshka Representation Learning across Speech and Text

    Feb 23, 2026Yaya Sy, Dioula Doucouré, Christophe Cerisara +1Representation LearningAudio-Text Retrieval

    LORIA, CNRS · Soynade Research

  3. Unified Vision-Language Modeling via Concept Space Alignment

    Mar 1, 2026Yifu Qiu, Paul-Ambroise Duquenne, Holger SchwenkVision-Language ModelsVision-Language Alignment

    University of Edinburgh · FAIR at Meta