MAEB: Massive Audio Embedding Benchmark

Adnan El Assadi; Isaac Chung; Chenghao Xiao; Roman Solomatin; Animesh Jha; Rahul Chand; Silky Singh; Kaitlyn Wang

cs.SDFeb 17, 2026

MAEB: Massive Audio Embedding Benchmark

Authors: Adnan El Assadi, Isaac Chung, Chenghao Xiao, Roman Solomatin, Animesh Jha, Rahul Chand, Silky Singh, Kaitlyn Wang,

Organizations: Carleton University · Zendesk · Durham University · MIRAI ·

Abstract

We introduce the Massive Audio Embedding Benchmark (MAEB), a large-scale benchmark covering 30 tasks across speech, music, environmental sounds, and cross-modal audio-text reasoning in 100+ languages. We evaluate 50+ models and find that no single model dominates across all tasks: contrastive audio-text models excel at environmental sound classification (e.g., ESC50) but score near random on multilingual speech tasks (e.g., SIB-FLEURS), while speech-pretrained models show the opposite pattern. Clustering remains challenging for all models, with even the best-performing model achieving only modest results. We observe that models excelling on acoustic understanding often perform poorly on linguistic tasks, and vice versa. We also show that the performance of audio encoders on MAEB correlates highly with their performance when used in audio large language models. MAEB is derived from MAEB+, a collection of 98 tasks. MAEB is designed to maintain task diversity while reducing evaluation cost, and it integrates into the MTEB ecosystem for unified evaluation across text, image, and audio modalities. We release MAEB and all 98 tasks along with code and a leaderboard at https://github.com/embeddings-benchmark/mteb.

Figures & tables

Figure 1: overview of task types and example subtypes in MAEB+. Values in parentheses denote numbers for MAEB.

Model	Params	MAEB	Extended	Speedup
YAMNet	3.7M	2.01	6.02	3.0 $\times$
wav2vec2-xls-r-2b	2B	26.93	45.62	1.7 $\times$
larger_clap_general	630M	11.52	32.23	2.8 $\times$
CLAP-htsat-fused	194M	13.03	35.35	2.7 $\times$

Table 1: Benchmark runtime comparison (GPU hours) between MAEB and MAEB(extended). Runtime measured on a single NVIDIA A100 GPU.

Figure 2: Domain-level performance on 94 tasks in MAEB+. Radial plot shows the top-performing model for each of the five acoustic domains: Speech (44 tasks), Music (13), Environmental (29), Bioacoustics (2), and Emotion (6). The dashed line represents an 80 target for universal performance, which remains unmet. Scores are averaged across all available task types (classification, clustering, retrieval, reranking). See Appendix D for methodology.

	Rank ( $\downarrow$ )		Average					Average per Category
Model	MAEB	Audio	All	Cat.	Eng.	Multi.	Aud.	Clf	M.Clf	PC	Rrnk	Clust	A. Rtrvl	X. Rtrvl	Zero Clf.
MAEB
Number of datasets			(30)	(30)	(15)	(23)	(19)	(10)	(2)	(3)	(1)	(3)	(1)	(8)	(2)
Large audio-language models
LCO-Embedding-Omni-7B	1	5	52.2	55.6	50.9	53.6	52.2	58.0	45.7	67.3	78.7	1.7	78.2	50.3	64.5
Qwen2-Audio-7B	2	1	33.7	34.0	30.1	27.6	50.8	62.7	10.7	56.9	80.8	12.7	33.9	1.6	12.4

Table 2: Top 30 models on the MAEB benchmark (30 tasks spanning audio-only and audio-text evaluation). Results are ranked using Borda count. The “Audio” column shows the model’s rank on MAEB(audio-only) for reference. We provide averages across all tasks, and per task category. “Eng.” shows the average for English-only tasks, “Multi.” shows the average excluding tasks with no linguistic content (zxx), and “Aud.” shows the average for audio-only tasks. Task categories are abbreviated as: Classification (Clf), Multilabel Classification (M.Clf), Pair Classification (PC), Reranking (Rrnk), Clustering (Clust), Audio Retrieval (A. Rtrvl), Cross-modal Retrieval (X. Rtrvl), Zero-shot Classification (Zero Clf.). We highlight the best score in bold and the best score with each model category using a grey cell.

Figure 3: MAEB+ embedding quality correlates with Audio LLM performance. MMAU evaluates Audio LLMs across Speech, Music, and Sound, the same domains covered by MAEB+. Each point plots an Audio LLM’s overall MMAU score (y-axis, averaged across domains) against its encoder’s MAEB+ score (x-axis, computed from 26 classification tasks aligned with MMAU domains). Preliminary correlation (R²=0.86, p=0.072, n=4) suggests a positive relationship between embedding quality and downstream reasoning, though the small sample size and statistical marginality warrant caution in interpreting this relationship.

Figure 4: Language distribution in the MAEB+ collection. English dominates with 70 tasks. We use zxx (No Linguistic Content) to tag datasets with no languages present.

Dataset	Citation	MAEB	N. Samples	Total Duration(s)	N. Langs	Domains	Main Metric
Any2AnyRetrieval
JamAltArtistA2ARetrieval	( Cífka et al., 2024 )	✓	6.7k	22992	4	Music	ndcg_at_10
Classification
AmbientAcousticContext	( Park et al., 2020 )		1k	1046	1	Spoken, Speech	accuracy
BeijingOpera	( Tian et al., 2014 )	✓	236	393	1	Music	accuracy
BirdCLEF	( Klinck et al., 2025 )	✓	1k	33602	1	Spoken, Speech, Bioacoustics	accuracy

Table 3: MAEB+ Audio-Only Tasks Overview. Tasks are grouped by type and show MAEB benchmark membership, dataset size, total audio duration, language coverage, domains, and main evaluation metric. * denotes values from huge datasets.

Dataset	Citation	MAEB	N. Samples	Total Secs	N. Langs	Modality	Domains	Main Metric
Audio-to-Text Retrieval
AudioCapsA2TRetrieval	( Kim et al., 2019 )		5.3k	8708	2	a2t	Encyclopaedic, Written	cv_recall_at_5
AudioSetStrongA2TRetrieval	( Hershey et al., 2021 )		1k	5065	1	a2t	AudioScene	cv_recall_at_5
CMUArcticA2TRetrieval	( Clark and Richmond, 2003 )		2.6k	4134	1	a2t	Spoken	cv_recall_at_5
ClothoA2TRetrieval	( Drossos et al., 2019 )		6.6k	23636	1	a2t	Encyclopaedic, Written	cv_recall_at_5
CommonVoiceMini17A2TRetrieval	( Ardila et al., 2020 )		46.8k	120220	50	a2t	Spoken	cv_recall_at_5

Table 4: MAEB+ Audio-Text Cross-Modal Tasks Overview. Tasks include zero-shot classification and bidirectional retrieval between audio and text modalities, with dataset size, total audio duration, and main evaluation metric. * denotes values from huge datasets.

Appendix figures & tables50 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Domain distributions in the MAEB+ collection, MAEB, and MAEB(audio-only).

Model Name	Model Size	Modalities
laion/clap-htsat-fused ( Wu et al., 2024 )	153	audio, text
laion/clap-htsat-unfused ( Wu et al., 2024 )	153	audio, text
laion/larger_clap_general ( Wu et al., 2024 )	193	audio, text
laion/larger_clap_music ( Wu et al., 2024 )	193	audio, text
laion/larger_clap_music_and_speech ( Wu et al., 2024 )	193	audio, text
MIT/ast-finetuned-audioset-10-10-0.4593 ( Gong et al., 2021 )	86	audio

Appendix

Table 5: List of all models evaluated in MAEB. Model sizes are in millions of parameters.

Model	AmbientAcoustic	BeijingOpera	BirdCLEF	CommonLangAge	CommonLangGender	CREMA-D	ESC50	FSDD
Qwen/Qwen2-Audio-7B	45.33	97.45	37.10	17.59	48.83	73.99	96.30	90.33
LCO-Embedding/LCO-Embedding-Omni-7B	39.67	92.79	34.10	16.16	30.70	36.05	94.15	99.00
LCO-Embedding/LCO-Embedding-Omni-3B	38.42	93.22	31.60	16.87	33.22	31.03	94.40	98.27
openai/whisper-medium	39.85	91.54	29.00	17.63	36.63	53.98	84.00	87.73
openai/whisper-small	36.89	88.13	26.40	15.85	45.87	49.19	77.35	91.47
MIT/ast-finetuned-audioset-10-10-0.4593	48.86	97.03	45.20	12.82	52.33	37.84	96.20	64.27

Appendix

Table 6: English classification results (datasets 1–8 of 23).

Model	GTZANGenre	GunshotTri	IEMOCAPEmo	IEMOCAPGender	LibriCount	MridinghamStroke	MridinghamTonic	NSynth
Qwen/Qwen2-Audio-7B	93.10	100.00	29.96	92.96	49.60	84.33	61.17	63.04
LCO-Embedding/LCO-Embedding-Omni-7B	82.30	96.67	24.35	70.75	33.22	61.89	42.07	58.09
LCO-Embedding/LCO-Embedding-Omni-3B	81.00	96.60	22.53	59.42	30.35	61.52	39.54	59.02
openai/whisper-medium	76.00	94.25	25.60	76.05	57.87	69.21	49.51	51.33
openai/whisper-small	71.50	94.44	24.21	69.93	53.37	69.06	45.49	49.88
MIT/ast-finetuned-audioset-10-10-0.4593	80.70	98.82	20.49	87.00	42.15	79.20	54.16	56.26

Appendix

Table 7: English classification results (datasets 9–16 of 23).

Model	SpeechCommands	SpokenQA	TUTAcoustic	VocalSound	VoxCelebSA	VoxPopuliAccent	MInDS14
Qwen/Qwen2-Audio-7B	75.60	21.35	34.30	91.82	29.54	39.35	25.51
LCO-Embedding/LCO-Embedding-Omni-7B	94.21	36.58	25.35	91.77	43.40	10.33	98.14
LCO-Embedding/LCO-Embedding-Omni-3B	94.40	37.00	25.00	91.31	48.97	8.97	98.48
openai/whisper-medium	72.43	21.74	26.55	80.13	33.92	54.04	48.30
openai/whisper-small	73.80	19.96	23.55	77.59	32.45	38.85	35.64
MIT/ast-finetuned-audioset-10-10-0.4593	24.27	15.86	30.45	82.11	28.33	23.61	7.94

Appendix

Table 8: English classification results (datasets 17–23 of 23).

Model	cs	de	en	es	fr	it	ko	nl	pl	pt	ru	zh
LCO-Embedding/LCO-Embedding-Omni-7B	72.3	91.5	98.1	97.9	97.4	84.2	92.9	94.2	68.7	82.5	95.6	97.8
LCO-Embedding/LCO-Embedding-Omni-3B	72.5	89.8	98.5	96.9	97.4	84.8	93.2	92.1	67.6	81.0	95.2	98.2
facebook/seamless-m4t-v2-large	92.3	89.4	89.2	92.0	90.4	78.9	90.5	90.2	67.8	63.4	90.4	93.0
facebook/mms-1b-fl102	77.0	78.4	77.0	76.8	75.3	71.8	70.3	78.7	69.8	66.2	66.6	71.3
facebook/mms-1b-all	64.3	54.7	58.5	46.1	62.0	51.1	69.9	64.2	47.3	53.5	54.0	76.9
facebook/mms-1b-l1107	55.6	57.4	64.0	58.2	63.1	51.9	44.8	56.7	43.8	46.5	52.9	37.6

Appendix

Table 9: MInDS-14 classification results across languages. Best result per language in bold.

Model	afr_Latn	amh_Ethi	arb_Arab	asm_Beng	ast_Latn	azj_Latn	bel_Cyrl	ben_Beng	bos_Latn	bul_Cyrl	cat_Latn	ceb_Latn	ces_Latn	ckb_Arab	cym_Latn
LCO-Embedding/LCO-Embedding-Omni-7B	47.3	39.3	71.5	32.1	70.6	50.9	70.5	41.1	51.7	55.4	74.3	53.4	45.5	28.5	32.0
LCO-Embedding/LCO-Embedding-Omni-3B	40.2	35.7	65.2	36.6	65.4	53.6	63.3	32.9	51.8	49.2	68.8	48.1	53.6	21.4	21.4
facebook/seamless-m4t-v2-large	42.0	41.9	54.6	51.7	51.8	41.0	43.8	46.5	48.1	53.6	45.7	24.2	46.6	32.9	34.8
openai/whisper-medium	26.8	18.8	28.6	15.9	44.5	36.5	39.1	24.0	29.5	29.6	32.1	31.1	30.4	11.6	21.4
facebook/mms-1b-fl102	17.9	23.1	30.4	34.9	27.7	20.5	34.0	27.7	22.3	16.1	26.8	19.6	27.6	23.2	25.8
facebook/mms-1b-all	25.0	27.7	21.5	26.0	27.7	17.8	29.4	25.9	26.9	21.5	18.7	21.4	18.7	20.4	24.9

Appendix

Table 10: SIB-FLEURS classification results (languages 1–15 of 102). Best per language in bold.

Model	dan_Latn	deu_Latn	ell_Grek	eng_Latn	est_Latn	fin_Latn	fra_Latn	fuv_Latn	gaz_Latn	gle_Latn	glg_Latn	guj_Gujr	hau_Latn	heb_Hebr	hin_Deva
LCO-Embedding/LCO-Embedding-Omni-7B	39.3	71.3	29.4	70.6	34.8	40.2	81.3	25.8	39.4	28.6	72.2	64.3	22.4	28.5	73.2
LCO-Embedding/LCO-Embedding-Omni-3B	43.8	68.7	42.0	68.9	36.6	32.1	74.1	18.7	31.2	27.7	67.6	49.1	19.6	31.2	65.2
facebook/seamless-m4t-v2-large	49.1	52.7	38.4	57.2	49.1	47.2	57.3	15.2	11.5	33.0	53.6	49.8	16.0	43.6	46.5
openai/whisper-medium	31.3	41.0	27.7	28.6	28.5	28.6	34.8	19.6	16.1	20.4	22.3	26.0	17.8	15.9	31.3
facebook/mms-1b-fl102	25.8	29.6	17.7	20.6	27.7	21.3	20.6	24.2	30.3	32.2	26.8	25.8	25.9	25.8	25.1
facebook/mms-1b-all	17.0	33.2	18.7	21.5	19.7	26.8	27.6	16.1	19.7	23.2	22.3	26.8	21.5	19.6	28.7

Appendix

Table 11: SIB-FLEURS classification results (languages 16–30 of 102). Best per language in bold.

Model	hrv_Latn	hun_Latn	hye_Armn	ibo_Latn	ind_Latn	isl_Latn	ita_Latn	jav_Latn	jpn_Jpan	kam_Latn	kan_Knda	kat_Geor	kaz_Cyrl	kea_Latn	khk_Cyrl
LCO-Embedding/LCO-Embedding-Omni-7B	51.8	44.6	25.9	25.0	82.9	33.0	72.3	50.2	74.2	26.0	51.0	34.9	44.7	72.4	20.5
LCO-Embedding/LCO-Embedding-Omni-3B	55.4	29.4	29.4	22.3	78.4	24.0	67.9	43.8	72.3	29.5	44.7	21.4	41.0	65.2	24.2
facebook/seamless-m4t-v2-large	56.2	40.1	48.4	20.4	54.4	61.6	49.1	44.7	49.4	14.3	49.1	57.2	48.1	36.6	34.7
openai/whisper-medium	31.2	37.4	20.5	19.6	33.8	15.1	40.9	19.6	26.8	24.1	17.8	16.1	24.0	28.5	23.1
facebook/mms-1b-fl102	28.5	27.7	22.3	27.6	30.3	29.5	25.9	19.6	19.8	24.2	20.5	23.1	26.8	32.1	31.3
facebook/mms-1b-all	27.7	26.8	23.1	22.3	27.6	25.9	31.1	22.4	19.8	18.8	23.3	24.9	24.1	25.9	18.8

Appendix

Table 12: SIB-FLEURS classification results (languages 31–45 of 102). Best per language in bold.

Model	khm_Khmr	kir_Cyrl	kor_Hang	lao_Laoo	lin_Latn	lit_Latn	ltz_Latn	lug_Latn	luo_Latn	lvs_Latn	mal_Mlym	mar_Deva	mkd_Cyrl	mlt_Latn	mri_Latn
LCO-Embedding/LCO-Embedding-Omni-7B	23.3	43.8	68.7	49.1	32.9	42.7	65.2	21.4	36.6	39.2	38.3	56.2	49.8	59.7	21.4
LCO-Embedding/LCO-Embedding-Omni-3B	22.3	48.1	70.5	42.7	33.0	39.1	59.9	24.2	29.4	41.9	33.0	52.8	58.9	56.2	23.3
facebook/seamless-m4t-v2-large	30.4	55.1	40.4	33.2	23.2	55.3	37.5	35.7	21.4	52.8	44.5	41.9	64.3	43.8	25.9
openai/whisper-medium	26.8	20.4	27.7	24.0	30.4	32.8	32.0	20.6	19.5	34.9	25.0	34.7	32.2	21.3	28.6
facebook/mms-1b-fl102	27.7	24.9	19.7	30.4	26.7	32.0	26.0	27.7	25.7	35.6	21.3	27.6	25.1	24.9	24.2
facebook/mms-1b-all	18.7	16.1	29.6	23.2	23.1	23.3	22.3	25.0	22.3	26.6	25.0	32.9	26.1	28.7	22.4

Appendix

Table 13: SIB-FLEURS classification results (languages 46–60 of 102). Best per language in bold.

Model	mya_Mymr	nld_Latn	nob_Latn	npi_Deva	nso_Latn	nya_Latn	oci_Latn	ory_Orya	pan_Guru	pbt_Arab	pes_Arab	pol_Latn	por_Latn	ron_Latn	rus_Cyrl
LCO-Embedding/LCO-Embedding-Omni-7B	15.3	71.5	42.0	55.3	23.3	23.1	56.3	49.0	53.7	29.3	38.5	66.2	71.4	58.0	69.7
LCO-Embedding/LCO-Embedding-Omni-3B	20.5	66.0	50.9	46.5	23.3	26.8	55.2	37.7	41.1	29.4	27.5	71.3	75.9	57.2	75.0
facebook/seamless-m4t-v2-large	34.0	52.9	46.5	42.0	20.5	45.6	30.4	40.2	50.0	25.9	55.3	43.8	50.9	44.9	52.7
openai/whisper-medium	8.1	31.3	38.3	26.7	17.7	17.8	26.7	24.9	33.0	21.4	24.8	36.6	42.8	33.2	33.0
facebook/mms-1b-fl102	27.7	32.2	30.4	24.2	30.4	33.0	26.8	26.8	25.0	25.0	21.3	22.4	33.8	18.0	29.5
facebook/mms-1b-all	25.0	30.4	27.6	20.6	26.8	25.0	29.4	15.2	27.6	20.6	24.8	30.3	30.3	18.8	21.5

Appendix

Table 14: SIB-FLEURS classification results (languages 61–75 of 102). Best per language in bold.

Model	slk_Latn	slv_Latn	sna_Latn	snd_Arab	som_Latn	spa_Latn	srp_Cyrl	swe_Latn	swh_Latn	tam_Taml	tel_Telu	tgk_Cyrl	tgl_Latn	tha_Thai	tur_Latn
LCO-Embedding/LCO-Embedding-Omni-7B	51.8	55.3	32.3	38.4	22.2	75.1	59.8	50.9	29.5	24.0	50.8	34.7	50.2	72.2	43.9
LCO-Embedding/LCO-Embedding-Omni-3B	49.0	44.5	40.2	37.5	25.8	69.6	51.8	52.7	29.5	31.3	50.0	28.5	43.0	65.8	49.1
facebook/seamless-m4t-v2-large	50.1	53.6	24.2	20.6	30.3	53.4	56.2	45.6	41.8	41.1	48.2	42.6	37.7	44.5	49.1
openai/whisper-medium	33.1	29.3	28.6	23.2	15.2	30.3	33.0	27.7	26.8	27.7	29.4	28.5	26.6	34.7	33.0
facebook/mms-1b-fl102	37.5	30.3	24.2	26.6	21.4	19.6	23.2	22.2	37.5	26.7	23.2	26.0	20.6	32.8	28.7
facebook/mms-1b-all	22.3	24.1	27.8	25.8	28.6	21.4	25.0	18.7	33.1	24.0	24.9	24.1	15.1	24.0	24.3

Appendix

Table 15: SIB-FLEURS classification results (languages 76–90 of 102). Best per language in bold.

Model	ukr_Cyrl	umb_Latn	urd_Arab	uzn_Latn	vie_Latn	wol_Latn	xho_Latn	yor_Latn	zho_Hans	zho_Hant	zsm_Latn	zul_Latn
LCO-Embedding/LCO-Embedding-Omni-7B	68.6	17.8	55.5	51.1	69.6	34.0	31.1	27.8	67.7	64.2	63.4	26.0
LCO-Embedding/LCO-Embedding-Omni-3B	71.3	20.6	57.2	40.2	66.0	33.0	25.0	32.1	65.9	62.5	55.5	26.7
facebook/seamless-m4t-v2-large	54.4	15.3	43.8	58.9	50.8	25.1	19.7	33.8	49.0	33.1	51.8	16.1
openai/whisper-medium	32.1	17.8	21.5	23.0	30.4	14.2	15.2	16.9	22.3	29.5	30.5	16.0
facebook/mms-1b-fl102	32.0	27.7	24.9	31.4	22.4	28.3	22.4	26.7	26.0	24.2	26.7	26.8
facebook/mms-1b-all	27.8	25.8	23.1	31.2	22.4	18.7	24.0	29.4	22.3	19.8	27.7	28.7

Appendix

Table 16: SIB-FLEURS classification results (languages 91–102 of 102). Best per language in bold.

Model	GenderID	LanguageID
laion/larger_clap_general	84.6	84.6
Qwen/Qwen2-Audio-7B	68.2	99.0
laion/larger_clap_music_and_speech	84.4	81.4
openai/whisper-tiny	66.6	96.6
openai/whisper-small	62.8	99.2
openai/whisper-base	63.6	98.0

Appendix

Table 17: VoxPopuli classification results. GenderID = gender classification, LanguageID = language identification. Both tasks are evaluated on multilingual audio samples containing English, French, Spanish, Polish, and German. Best result per task in bold.

Model	AmbientAcoustic	CREMA-D	ESC50	GTZANGenre	MusicGenre	VehicleSound	VoiceGender	VoxCeleb	VoxPopuliAccent
laion/larger_clap_music_and_speech	43.51	14.43	94.46	63.82	45.30	4.72	71.33	0.70	10.61
laion/clap-htsat-unfused	43.68	12.78	94.76	55.70	40.07	2.66	68.11	1.06	9.35
laion/larger_clap_general	43.78	13.17	94.10	65.09	43.38	3.37	26.63	0.68	11.20
Qwen/Qwen2-Audio-7B	39.92	32.37	88.69	73.94	41.84	5.52	13.43	0.18	4.25
microsoft/msclap-2023	44.12	10.73	95.33	61.91	36.57	2.86	31.43	0.78	9.58
MIT/ast-finetuned-audioset-10-10-0.4593	46.76	7.20	91.36	61.60	47.20	13.37	14.07	1.01	7.55

Appendix

Table 18: English clustering results.

Model	VoxPopuliGender
laion/clap-htsat-fused	52.68
microsoft/msclap-2022	46.37
microsoft/msclap-2023	32.09
laion/clap-htsat-unfused	22.20
google/vggish	8.20
speechbrain/cnn14-esc50	7.72

Appendix

Table 19: VoxPopuli gender clustering results. Task clusters audio samples by speaker gender across multilingual audio samples containing German, English, French, Spanish, and Polish. Best result in bold.

Model	CREMA-D	ESC50	NMSQA	VocalSound	VoxPopuliAccent
LCO-Embedding/LCO-Embedding-Omni-7B	53.56	99.44	97.60	94.47	50.73
LCO-Embedding/LCO-Embedding-Omni-3B	52.40	99.54	96.92	94.65	50.67
microsoft/msclap-2023	57.65	99.22	51.09	84.22	52.05
laion/clap-htsat-unfused	56.91	99.31	46.98	81.58	53.96
laion/larger_clap_music_and_speech	56.66	99.35	46.14	80.36	53.49
laion/larger_clap_general	54.71	99.14	47.42	81.09	53.55

Appendix

Table 20: Pair classification results. Models are evaluated on audio pair similarity tasks. Best result per task in bold.

Model	AudioSetMini	BirdSet	FSD2019Kaggle	FSD50K
MIT/ast-finetuned-audioset-10-10-0.4593	55.10	6.55	36.90	2.79
LCO-Embedding/LCO-Embedding-Omni-7B	43.60	3.03	44.19	4.90
LCO-Embedding/LCO-Embedding-Omni-3B	42.75	3.05	42.97	4.22
google/yamnet	46.46	2.51	18.95	1.34
Qwen/Qwen2-Audio-7B	46.28	0.81	7.05	13.78
openai/whisper-medium	38.15	2.77	17.84	3.25

Appendix

Table 21: Multilabel classification results. Models are evaluated on audio tagging tasks where each sample can have multiple labels. Best result per task in bold.

Model	ESC50	Ravdess	SpeechCmd v0.01	SpeechCmd v0.02	UrbanSound8k
LCO-Embedding/LCO-Embedding-Omni-7B	87.80	31.67	96.96	97.42	67.38
LCO-Embedding/LCO-Embedding-Omni-3B	87.65	26.94	96.61	97.40	68.80
laion/larger_clap_general	90.50	17.29	12.39	12.44	79.64
microsoft/msclap-2023	89.85	15.21	9.78	10.01	83.01
laion/larger_clap_music_and_speech	82.85	17.29	9.23	9.03	76.86
microsoft/msclap-2022	80.25	13.61	10.09	10.51	76.17

Appendix

Table 22: Zero-shot classification results. Models classify audio using text descriptions without task-specific training. Best result per task in bold.

Model	ESC50	FSDnoisy18k	GTZAN	UrbanSound8K	VocalSound
LCO-Embedding/LCO-Embedding-Omni-7B	97.58	84.61	78.71	79.27	89.94
LCO-Embedding/LCO-Embedding-Omni-3B	97.16	83.19	75.37	77.93	89.28
microsoft/msclap-2023	97.98	85.34	75.43	76.50	72.12
Qwen/Qwen2-Audio-7B	94.34	68.44	80.85	70.66	80.29
MIT/ast-finetuned-audioset-10-10-0.4593	96.89	77.35	77.65	75.91	65.56
microsoft/msclap-2022	96.28	79.11	62.91	77.96	59.69

Appendix

Table 23: Reranking results. Models are evaluated on audio-to-audio reranking tasks. Best result per task in bold.

Model	AudioCaps A2T	AudioCaps T2A	AudioSetStrong A2T	AudioSetStrong T2A	CMU Arctic A2T	CMU Arctic T2A	Clotho A2T	Clotho T2A	EmoVDB A2T
LCO-Embedding/LCO-Embedding-Omni-7B	56.74	53.82	44.14	42.19	99.92	99.54	37.32	33.89	97.37
LCO-Embedding/LCO-Embedding-Omni-3B	54.25	46.95	41.21	34.77	99.70	99.70	34.74	32.70	98.34
laion/larger_clap_general	76.22	57.06	87.30	85.55	21.23	34.12	44.69	33.72	1.80
laion/larger_clap_music_and_speech	74.18	63.14	46.29	47.07	22.21	44.20	38.95	32.39	3.94
microsoft/msclap-2023	53.11	49.94	53.71	53.71	1.82	1.44	54.64	41.93	2.63
laion/clap-htsat-unfused	75.20	62.30	46.29	46.29	1.67	0.61	44.69	34.18	1.93

Appendix

Table 24: English retrieval results (datasets 1–9 of 27).

Model	EmoVDB T2A	GigaSpeech A2T	GigaSpeech T2A	HiFiTTS A2T	HiFiTTS T2A	JLCorpus A2T	JLCorpus T2A	LibriTTS A2T	LibriTTS T2A
LCO-Embedding/LCO-Embedding-Omni-7B	97.24	83.11	83.29	99.67	100.00	69.83	65.33	99.96	99.91
LCO-Embedding/LCO-Embedding-Omni-3B	98.69	82.96	83.13	99.67	100.00	70.50	70.67	99.96	99.94
laion/larger_clap_general	2.42	0.13	0.16	4.33	5.33	4.46	6.67	0.21	0.64
laion/larger_clap_music_and_speech	4.98	0.15	0.21	3.00	3.67	3.96	9.33	0.13	0.70
microsoft/msclap-2023	2.42	0.19	0.10	3.00	3.33	6.42	6.67	0.36	0.36
laion/clap-htsat-unfused	2.35	0.09	0.36	4.00	5.33	4.00	6.00	0.45	0.43

Appendix

Table 25: English retrieval results (datasets 10–18 of 27).

Model	MACS A2T	MACS T2A	MusicCaps A2T	MusicCaps T2A	SoundDescs A2T	SoundDescs T2A	SpokenSQuAD T2A	UrbanSound8K A2T	UrbanSound8K T2A
LCO-Embedding/LCO-Embedding-Omni-7B	16.03	29.77	20.93	24.83	21.63	32.79	74.00	0.79	0.92
LCO-Embedding/LCO-Embedding-Omni-3B	18.32	22.65	19.69	18.83	18.82	29.09	72.00	0.67	0.88
laion/larger_clap_general	30.03	33.08	17.36	16.49	23.45	22.96	2.00	0.94	0.96
laion/larger_clap_music_and_speech	28.24	30.53	13.36	14.02	24.07	24.44	2.00	1.02	0.94
microsoft/msclap-2023	15.52	27.23	18.48	19.22	38.14	37.28	0.00	0.90	0.94
laion/clap-htsat-unfused	25.19	27.99	11.70	11.89	20.52	21.33	1.33	0.98	0.98

Appendix

Table 26: English retrieval results (datasets 19–27 of 27).

Model	af	am	ar	as	ast	az	be	bg	bn	bs	ca	ceb	ckb	cmn	cs	cy	da	de	el	en	es	et	fa	ff	fi	fil
LCO-Embedding/LCO-Embedding-Omni-3B	88.3	41.9	98.8	43.1	100.0	82.2	98.3	91.2	59.8	87.4	99.9	80.8	47.6	100.0	89.3	35.5	65.3	100.0	61.8	100.0	100.0	57.9	37.3	62.0	50.4	80.5
LCO-Embedding/LCO-Embedding-Omni-7B	90.5	37.0	98.8	40.5	100.0	75.9	97.9	92.4	60.8	84.2	99.6	79.1	39.3	100.0	78.1	33.3	68.4	99.9	39.5	100.0	100.0	54.1	40.5	65.2	36.5	77.6
Qwen/Qwen2-Audio-7B	1.9	1.7	2.8	1.2	1.9	1.1	0.8	0.9	1.2	1.3	1.7	1.3	1.1	1.7	1.4	0.5	1.2	0.9	2.0	3.4	3.6	1.6	0.9	1.4	1.4	0.8
microsoft/speecht5_multimodal	2.3	1.0	2.1	1.1	0.4	0.7	0.6	1.7	0.5	0.4	1.1	1.3	1.1	1.0	1.7	1.1	1.4	0.9	1.1	1.4	0.7	0.7	0.5	1.1	1.5	0.6
laion/larger_clap_music_and_speech	2.7	1.4	1.4	0.2	1.9	0.8	0.4	1.5	0.5	0.9	0.7	1.7	0.5	0.5	1.0	0.8	1.2	0.3	0.6	0.9	1.2	0.6	0.6	1.1	0.5	0.3
laion/clap-htsat-fused	2.7	0.4	0.9	0.6	0.8	0.3	0.4	0.5	0.4	0.4	0.4	1.1	0.8	1.1	1.1	0.6	0.8	0.6	0.2	0.9	0.4	0.8	0.6	0.6	0.8	0.3

Appendix

Table 27: FLEURS A2T retrieval results (languages 1–26 of 102). Best per language in bold.

Model	fr	ga	gl	gu	ha	he	hi	hr	hu	hy	id	ig	is	it	ja	jv	ka	kam	kea	kk	km	kn	ko	ky	lb	lg
LCO-Embedding/LCO-Embedding-Omni-3B	100.0	29.2	100.0	91.8	40.4	38.3	99.0	95.1	35.5	36.6	99.9	35.5	73.9	100.0	100.0	89.3	52.6	33.9	98.0	61.4	16.0	64.0	100.0	75.3	92.7	44.5
LCO-Embedding/LCO-Embedding-Omni-7B	100.0	27.6	100.0	92.2	31.1	37.2	98.8	87.7	31.7	27.5	99.9	32.7	54.3	100.0	100.0	89.3	47.4	38.0	98.6	59.3	16.5	64.9	100.0	69.7	93.3	42.3
Qwen/Qwen2-Audio-7B	3.3	1.2	2.9	1.2	0.8	1.0	1.9	1.5	1.2	1.3	2.5	0.8	21.7	3.1	2.9	0.8	1.3	1.0	1.5	1.6	0.4	0.6	2.6	1.4	1.2	1.0
microsoft/speecht5_multimodal	0.9	1.1	1.1	1.1	1.1	1.0	2.2	0.3	0.9	0.9	1.6	0.7	15.2	1.2	0.9	1.1	0.9	1.0	0.6	0.6	0.6	0.7	1.3	0.7	1.2	0.7
laion/larger_clap_music_and_speech	0.9	0.8	0.9	0.3	1.0	1.1	0.7	0.7	0.7	0.4	1.2	0.5	10.9	1.0	1.1	0.5	0.4	1.0	0.8	0.8	0.4	0.7	2.1	0.4	0.6	1.0
laion/clap-htsat-fused	0.4	0.6	0.5	0.6	1.3	0.9	0.7	0.5	1.1	0.8	1.2	0.7	13.0	0.8	0.9	0.5	0.3	0.7	0.5	0.5	1.0	0.2	1.3	0.5	0.9	0.6

Appendix

Table 28: FLEURS A2T retrieval results (languages 27–52 of 102). Best per language in bold.

Model	ln	lo	lt	luo	lv	mi	mk	ml	mn	mr	ms	mt	my	nb	ne	nl	nso	ny	oc	om	or	pa	pl	ps	pt	ro
LCO-Embedding/LCO-Embedding-Omni-3B	43.1	54.3	51.8	66.0	57.9	23.8	95.3	55.8	31.6	80.6	99.6	89.5	6.2	92.2	80.0	100.0	43.4	43.5	91.7	97.6	64.9	85.0	98.3	59.8	100.0	96.5
LCO-Embedding/LCO-Embedding-Omni-7B	47.5	74.6	55.2	68.0	56.5	22.4	92.9	56.5	22.1	82.6	99.9	89.0	6.7	83.8	80.9	99.7	43.3	41.5	93.1	90.2	65.3	89.9	86.3	60.0	100.0	93.2
Qwen/Qwen2-Audio-7B	0.4	4.0	0.6	1.6	2.1	0.8	1.5	0.8	0.9	0.9	1.6	1.9	0.7	4.5	1.1	2.2	0.6	0.8	1.0	14.6	1.5	1.2	1.1	1.0	3.7	1.2
microsoft/speecht5_multimodal	1.7	1.7	0.5	3.1	1.4	0.7	0.8	0.7	0.6	0.9	1.3	0.8	0.6	3.1	0.6	1.9	1.0	1.2	0.7	12.2	0.9	1.2	0.5	2.0	1.0	1.0
laion/larger_clap_music_and_speech	1.0	1.7	0.9	3.1	0.7	1.2	0.7	0.5	0.6	0.1	0.8	0.5	0.6	2.5	0.7	2.7	1.1	0.5	0.7	19.5	0.5	0.3	1.2	1.6	0.8	0.8
laion/clap-htsat-fused	1.5	1.0	0.4	2.0	0.7	0.4	0.2	0.7	0.4	0.2	0.7	0.5	0.1	1.4	0.4	1.4	1.3	0.8	0.8	19.5	0.6	0.7	0.5	1.0	0.4	0.9

Appendix

Table 29: FLEURS A2T retrieval results (languages 53–78 of 102). Best per language in bold.

Model	ru	sd	sk	sl	sn	so	sr	sv	sw	ta	te	tg	th	tr	uk	umb	ur	uz	vi	wo	xh	yo	yue	zu
LCO-Embedding/LCO-Embedding-Omni-3B	100.0	77.3	96.6	78.1	45.1	37.8	89.7	86.6	44.1	32.8	85.8	66.5	99.7	99.1	98.3	35.1	99.0	52.9	99.2	66.8	43.1	29.5	100.0	33.0
LCO-Embedding/LCO-Embedding-Omni-7B	100.0	76.6	92.2	83.8	43.7	33.3	89.6	78.1	48.9	26.2	72.0	69.3	99.7	97.7	98.9	30.1	99.0	54.5	99.1	66.8	39.4	31.5	100.0	27.9
Qwen/Qwen2-Audio-7B	2.2	0.7	1.8	1.0	1.0	1.0	1.7	1.6	1.8	0.5	1.9	1.3	0.7	1.5	1.5	0.8	3.0	0.5	1.9	2.2	0.9	0.5	1.3	1.3
microsoft/speecht5_multimodal	1.3	0.9	0.6	0.5	0.8	1.2	1.0	1.3	1.8	1.4	1.5	1.2	0.6	0.7	1.2	1.1	2.0	1.3	0.6	1.6	0.7	1.1	1.5	0.7
laion/larger_clap_music_and_speech	0.8	0.9	1.3	1.6	0.8	0.5	0.9	1.1	0.8	1.0	1.3	1.5	0.3	0.5	0.7	0.5	2.0	0.5	1.2	1.6	0.8	0.5	0.6	0.8
laion/clap-htsat-fused	0.8	0.5	0.8	0.5	0.9	0.4	0.9	0.9	1.0	1.4	1.1	0.7	0.2	0.9	1.2	0.5	1.3	0.5	0.4	0.8	0.3	0.2	0.9	0.7

Appendix

Table 30: FLEURS A2T retrieval results (languages 79–102 of 102). Best per language in bold.

Model	af	am	ar	as	ast	az	be	bg	bn	bs	ca	ceb	ckb	cmn	cs	cy	da	de	el	en	es	et	fa	ff	fi	fil
LCO-Embedding/LCO-Embedding-Omni-3B	92.4	26.9	97.9	47.9	100.0	82.8	98.1	91.9	64.5	87.1	99.4	88.4	43.2	99.9	90.2	26.1	73.7	99.9	69.2	100.0	100.0	66.9	45.0	65.3	58.9	86.5
LCO-Embedding/LCO-Embedding-Omni-7B	90.2	25.6	98.6	42.5	99.8	76.4	97.5	93.0	62.9	82.2	98.7	81.9	38.0	100.0	79.5	26.3	68.5	99.9	44.5	100.0	100.0	55.2	43.7	54.7	39.8	80.8
Qwen/Qwen2-Audio-7B	3.4	2.3	1.4	0.6	2.5	0.5	1.2	1.5	0.8	1.0	1.3	1.3	1.0	1.7	1.0	0.4	0.5	1.2	1.7	4.3	2.4	0.9	0.9	1.2	0.5	0.4
microsoft/speecht5_multimodal	1.5	1.4	1.2	1.4	0.4	1.1	0.7	1.5	0.9	0.6	0.5	1.1	0.7	1.0	1.4	0.7	1.2	1.2	1.1	3.4	0.6	0.9	0.9	1.1	1.0	0.5
laion/larger_clap_music_and_speech	3.0	1.0	1.4	0.5	1.9	0.8	0.6	0.6	0.5	1.7	1.2	0.9	0.7	0.6	1.9	0.8	0.6	1.0	0.9	1.4	1.0	1.1	0.8	0.8	0.9	1.0
laion/larger_clap_general	4.2	1.0	0.7	0.5	1.5	0.7	0.4	0.8	0.5	1.1	0.5	0.9	0.4	0.4	1.2	0.5	1.0	1.3	0.6	1.9	0.8	0.8	0.6	1.8	0.9	0.8

Appendix

Table 31: FLEURS T2A retrieval results (languages 1–26 of 102). Best per language in bold.

Model	fr	ga	gl	gu	ha	he	hi	hr	hu	hy	id	ig	is	it	ja	jv	ka	kam	kea	kk	km	kn	ko	ky	lb	lg
LCO-Embedding/LCO-Embedding-Omni-3B	99.9	29.8	99.9	91.8	32.5	43.8	98.3	94.6	43.1	39.4	99.9	31.9	71.7	100.0	99.7	91.5	59.4	30.8	97.5	67.8	15.4	72.1	100.0	75.1	92.9	38.3
LCO-Embedding/LCO-Embedding-Omni-7B	100.0	27.2	100.0	92.1	31.4	42.8	98.6	87.3	36.2	32.2	99.9	23.3	76.1	100.0	99.8	93.4	52.8	33.6	97.9	59.8	15.7	73.5	100.0	70.4	92.2	36.9
Qwen/Qwen2-Audio-7B	2.5	0.5	2.3	1.3	0.5	1.3	1.2	1.1	0.4	0.8	1.0	0.5	19.6	2.7	0.9	0.8	0.8	0.6	0.7	0.7	0.9	1.0	2.4	0.5	0.6	1.1
microsoft/speecht5_multimodal	1.0	0.7	0.9	1.2	1.0	1.0	1.9	0.9	1.0	1.0	1.6	1.3	13.0	0.7	0.8	1.2	0.6	1.1	0.9	1.1	0.9	0.7	1.6	0.5	0.7	1.1
laion/larger_clap_music_and_speech	2.5	1.2	1.3	0.5	0.5	0.5	1.2	1.3	0.8	0.4	1.0	0.4	10.9	2.0	0.5	0.8	0.6	1.0	1.4	0.6	0.6	0.6	1.8	0.6	0.9	1.0
laion/larger_clap_general	1.9	1.4	0.4	0.4	0.6	0.6	1.4	1.1	1.2	0.5	0.9	0.8	17.4	1.7	1.2	1.4	0.4	1.0	0.8	0.6	0.6	0.1	1.3	0.1	1.0	1.5

Appendix

Table 32: FLEURS T2A retrieval results (languages 27–52 of 102). Best per language in bold.

Model	ln	lo	lt	luo	lv	mi	mk	ml	mn	mr	ms	mt	my	nb	ne	nl	nso	ny	oc	om	or	pa	pl	ps	pt	ro
LCO-Embedding/LCO-Embedding-Omni-3B	45.6	40.7	63.2	73.0	66.4	23.2	95.0	65.2	24.1	82.5	99.6	89.1	5.2	93.8	82.0	99.7	40.0	38.2	91.0	73.2	62.2	83.4	97.0	63.5	100.0	95.7
LCO-Embedding/LCO-Embedding-Omni-7B	43.7	64.0	58.2	65.2	58.6	21.0	92.2	64.2	21.3	80.5	99.6	86.5	4.3	84.9	80.2	99.7	31.9	35.5	90.4	70.7	62.5	87.3	84.2	59.0	100.0	91.4
Qwen/Qwen2-Audio-7B	1.0	2.0	0.6	2.3	0.9	0.8	0.3	1.1	0.7	0.3	1.2	0.5	0.9	3.1	1.0	1.4	0.5	0.1	0.5	17.1	1.5	1.0	0.9	1.2	2.5	1.2
microsoft/speecht5_multimodal	1.5	1.5	0.8	3.9	0.5	0.5	0.8	0.5	0.4	0.6	0.7	0.6	0.8	2.5	1.5	2.2	1.0	1.2	1.0	19.5	0.7	0.9	1.3	1.2	1.2	0.9
laion/larger_clap_music_and_speech	2.1	1.5	1.1	2.3	1.3	1.0	0.6	0.6	0.4	0.6	0.8	1.1	0.5	1.7	1.0	2.5	0.9	0.7	0.6	12.2	0.7	0.5	2.4	1.2	2.6	1.5
laion/larger_clap_general	1.9	1.5	1.3	3.1	0.5	1.0	0.5	0.3	0.3	0.3	0.8	1.3	0.7	2.0	0.4	2.2	1.0	1.1	0.7	7.3	0.3	0.5	1.2	0.8	1.3	1.4

Appendix

Table 33: FLEURS T2A retrieval results (languages 53–78 of 102). Best per language in bold.

Model	ru	sd	sk	sl	sn	so	sr	sv	sw	ta	te	tg	th	tr	uk	umb	ur	uz	vi	wo	xh	yo	yue	zu
LCO-Embedding/LCO-Embedding-Omni-3B	100.0	75.8	95.6	83.2	35.5	32.1	91.1	83.7	45.0	36.2	85.2	66.3	99.3	99.2	98.3	36.7	100.0	56.8	99.0	58.0	33.6	33.8	99.9	20.8
LCO-Embedding/LCO-Embedding-Omni-7B	100.0	74.8	91.5	83.2	32.3	21.6	91.0	75.4	47.4	33.3	74.8	68.0	99.7	98.0	97.5	29.8	99.0	53.8	99.2	56.1	28.8	31.9	100.0	19.3
Qwen/Qwen2-Audio-7B	1.4	0.4	0.9	1.0	0.5	0.7	1.3	0.5	2.1	0.8	1.7	1.2	0.7	1.1	1.6	1.6	2.0	0.5	1.3	1.9	0.8	1.1	0.7	0.8
microsoft/speecht5_multimodal	1.2	0.6	1.0	1.2	0.6	0.4	1.7	1.3	1.6	1.4	1.3	1.2	0.5	0.5	0.8	1.8	2.3	0.8	0.5	1.3	1.0	1.0	1.1	1.2
laion/larger_clap_music_and_speech	0.8	0.5	1.3	1.0	0.4	0.8	2.4	1.7	1.2	0.7	1.1	0.8	0.5	0.9	1.2	1.3	1.7	0.5	0.8	1.3	0.8	0.7	0.6	0.9
laion/larger_clap_general	0.8	0.5	1.6	0.8	0.8	0.5	2.0	1.3	0.8	1.0	0.8	0.8	0.4	1.8	0.8	2.1	1.3	0.8	0.9	1.6	0.8	0.5	0.7	1.1

Appendix

Table 34: FLEURS T2A retrieval results (languages 79–102 of 102). Best per language in bold.

Model	de	en	es	fr
LCO-Embedding/LCO-Embedding-Omni-3B	73.5	76.4	81.4	72.5
LCO-Embedding/LCO-Embedding-Omni-7B	66.1	71.1	77.2	71.8
microsoft/msclap-2023	0.9	0.7	1.8	1.0
Qwen/Qwen2-Audio-7B	0.7	0.8	1.1	1.5
microsoft/speecht5_multimodal	0.7	1.7	0.8	0.8
laion/larger_clap_music_and_speech	1.0	0.5	0.8	1.6

Appendix

Table 35: JamAlt A2T retrieval results (languages 1–4 of 4). Best per language in bold.

Model	de	en	es	fr
LCO-Embedding/LCO-Embedding-Omni-3B	75.6	79.6	81.7	75.4
LCO-Embedding/LCO-Embedding-Omni-7B	72.4	78.6	80.8	76.0
Qwen/Qwen2-Audio-7B	1.2	2.0	1.7	1.8
microsoft/speecht5_multimodal	1.1	3.0	1.0	0.9
laion/larger_clap_music_and_speech	1.5	1.3	0.6	1.0
laion/clap-htsat-unfused	0.8	1.5	0.9	0.7

Appendix

Table 36: JamAlt T2A retrieval results (languages 1–4 of 4). Best per language in bold.

Model	de	en	es	fr
laion/larger_clap_music_and_speech	99.1	94.3	97.9	96.3
OpenMuQ/MuQ-MuLan-large	97.6	95.2	97.6	94.7
laion/larger_clap_general	97.9	93.2	96.8	94.6
MIT/ast-finetuned-audioset-10-10-0.4593	97.7	90.2	96.2	94.4
laion/clap-htsat-unfused	97.0	88.8	94.2	91.8
microsoft/msclap-2023	96.5	87.3	94.9	92.5

Appendix

Table 37: JamAlt A2A retrieval results (languages 1–4 of 4). Best per language in bold.

Model	ar	ast	be	bg	bn	br	cs	cy	da	de	el	en	es
LCO-Embedding/LCO-Embedding-Omni-3B	94.4	98.3	99.4	69.4	58.0	67.4	86.6	38.4	56.6	100.0	46.4	99.8	100.0
LCO-Embedding/LCO-Embedding-Omni-7B	94.8	99.2	99.4	66.2	56.6	56.6	70.2	27.2	49.0	99.8	21.6	99.6	100.0
Qwen/Qwen2-Audio-7B	2.6	8.4	2.8	2.8	2.0	1.4	2.6	1.4	2.2	4.4	2.6	4.2	6.2
microsoft/speecht5_multimodal	1.2	8.4	0.2	1.0	0.8	1.6	1.8	1.0	1.6	1.2	0.6	1.0	2.0
laion/clap-htsat-unfused	1.0	3.4	1.4	1.4	1.6	2.0	2.2	1.2	1.2	1.2	1.4	1.4	0.4
microsoft/msclap-2023	0.8	4.2	1.4	1.0	0.8	0.8	1.8	1.2	1.6	0.8	1.6	1.2	1.6

Appendix

Table 38: CommonVoiceMini17 A2T retrieval results (languages 1–13 of 50). Best per language in bold.

Model	et	fa	fi	fr	frold	gl	ha	hi	hu	it	ja	ka	ko
LCO-Embedding/LCO-Embedding-Omni-3B	46.8	25.2	39.0	99.8	99.8	100.0	61.6	94.0	50.0	99.6	95.2	61.2	99.1
LCO-Embedding/LCO-Embedding-Omni-7B	39.4	21.4	24.2	99.4	99.4	100.0	51.2	94.4	42.4	99.6	95.8	54.4	98.8
Qwen/Qwen2-Audio-7B	2.6	2.8	2.0	4.6	4.6	3.6	2.4	1.4	2.0	6.0	3.4	2.8	4.4
microsoft/speecht5_multimodal	1.4	1.4	3.2	2.4	2.4	1.8	1.8	0.8	1.8	2.0	1.2	1.2	1.2
laion/clap-htsat-unfused	1.0	1.0	1.2	0.6	0.6	1.4	2.1	1.4	1.2	0.8	1.4	1.2	2.4
microsoft/msclap-2023	1.0	1.2	2.0	1.6	1.6	1.0	1.5	1.0	1.0	1.4	1.0	1.6	2.1

Appendix

Table 39: CommonVoiceMini17 A2T retrieval results (languages 14–26 of 50). Best per language in bold.

Model	lt	lv	mk	ml	mn	mr	nl	oc	pl	pt	ro	ru	sk
LCO-Embedding/LCO-Embedding-Omni-3B	62.8	43.4	76.2	57.8	20.2	78.6	99.0	90.9	88.4	99.0	86.0	99.2	82.2
LCO-Embedding/LCO-Embedding-Omni-7B	59.6	38.8	72.9	40.8	11.8	81.2	99.0	90.2	66.2	98.8	78.8	99.2	80.2
Qwen/Qwen2-Audio-7B	1.8	2.2	3.3	2.2	2.0	2.0	2.8	5.1	2.6	2.4	2.6	3.0	2.4
microsoft/speecht5_multimodal	2.4	2.0	1.5	0.8	1.0	2.4	2.2	2.0	1.8	1.6	1.0	0.6	1.8
laion/clap-htsat-unfused	0.8	1.4	1.5	1.6	1.2	1.4	0.8	2.4	1.8	0.6	1.0	1.6	1.0
microsoft/msclap-2023	1.4	0.4	1.5	1.4	0.8	1.2	1.0	2.4	1.8	1.2	1.4	1.4	0.8

Appendix

Table 40: CommonVoiceMini17 A2T retrieval results (languages 27–39 of 50). Best per language in bold.

Model	sl	sr	sv	sw	ta	te	th	tr	uk	ur	vi
LCO-Embedding/LCO-Embedding-Omni-3B	59.8	58.6	68.2	53.0	42.4	91.8	92.0	72.2	86.6	82.2	84.4
LCO-Embedding/LCO-Embedding-Omni-7B	63.0	54.0	46.8	49.4	21.6	59.2	92.0	63.4	86.6	84.6	86.6
Qwen/Qwen2-Audio-7B	1.2	1.8	2.2	2.0	2.0	16.3	1.4	1.2	2.2	1.2	1.6
microsoft/speecht5_multimodal	1.0	0.4	1.8	2.4	0.6	6.1	0.8	0.8	1.6	1.0	1.0
laion/clap-htsat-unfused	0.8	1.2	1.0	1.4	1.0	12.2	1.2	2.0	1.2	1.2	1.4
microsoft/msclap-2023	2.0	2.0	0.4	1.4	1.4	10.2	1.4	1.2	1.2	1.2	0.8

Appendix

Table 41: CommonVoiceMini17 A2T retrieval results (languages 40–50 of 50). Best per language in bold.

Model	ar	ast	be	bg	bn	br	cs	cy	da	de	el	en	es
LCO-Embedding/LCO-Embedding-Omni-3B	94.2	99.2	98.8	73.2	58.2	51.0	87.4	20.0	63.2	99.8	41.2	99.6	100.0
LCO-Embedding/LCO-Embedding-Omni-7B	95.0	97.5	99.0	65.6	49.4	47.0	72.6	16.2	56.0	99.6	19.4	99.8	100.0
Qwen/Qwen2-Audio-7B	1.6	8.4	3.0	1.0	2.2	1.8	2.0	2.8	2.2	3.0	1.0	5.2	5.6
microsoft/speecht5_multimodal	1.0	9.2	1.0	1.4	1.4	2.0	1.8	1.6	1.4	1.6	0.8	2.0	1.4
microsoft/msclap-2023	0.8	10.1	1.0	1.6	1.0	1.6	1.2	1.8	1.6	1.6	1.0	0.8	1.6
laion/clap-htsat-fused	0.6	4.2	1.6	1.8	0.6	1.2	1.6	1.6	1.6	1.8	1.2	1.2	0.8

Appendix

Table 42: CommonVoiceMini17 T2A retrieval results (languages 1–13 of 50). Best per language in bold.

Model	et	fa	fi	fr	frold	gl	ha	hi	hu	it	ja	ka	ko
LCO-Embedding/LCO-Embedding-Omni-3B	46.6	25.6	41.6	99.6	99.6	99.8	49.7	92.8	48.8	99.6	95.0	58.8	98.8
LCO-Embedding/LCO-Embedding-Omni-7B	41.8	26.0	22.0	99.4	99.4	99.8	40.9	91.8	42.2	99.8	96.2	50.6	98.5
Qwen/Qwen2-Audio-7B	2.8	1.4	0.8	4.2	4.2	4.4	2.4	1.6	1.2	6.4	1.4	2.4	4.4
microsoft/speecht5_multimodal	2.0	1.0	1.8	1.0	1.0	1.4	1.8	1.2	1.4	1.2	1.0	0.8	0.9
microsoft/msclap-2023	1.8	1.2	1.4	1.2	1.2	0.8	0.6	1.2	1.4	1.2	0.8	0.8	2.1
laion/clap-htsat-fused	1.2	1.0	1.8	1.4	1.4	1.2	1.5	1.0	1.2	0.4	1.2	1.0	1.8

Appendix

Table 43: CommonVoiceMini17 T2A retrieval results (languages 14–26 of 50). Best per language in bold.

Model	lt	lv	mk	ml	mn	mr	nl	oc	pl	pt	ro	ru	sk
LCO-Embedding/LCO-Embedding-Omni-3B	66.6	47.8	78.3	52.2	11.6	80.8	98.6	91.7	88.6	98.4	84.0	99.2	80.8
LCO-Embedding/LCO-Embedding-Omni-7B	55.6	40.4	75.0	32.4	6.6	77.0	98.4	90.2	66.4	98.0	73.8	99.0	79.0
Qwen/Qwen2-Audio-7B	1.8	1.4	3.6	2.8	1.6	1.4	2.8	6.3	1.6	3.4	2.8	2.2	2.8
microsoft/speecht5_multimodal	2.0	2.0	1.8	1.0	0.8	0.8	1.6	1.6	1.6	1.4	1.8	1.4	1.0
microsoft/msclap-2023	1.6	1.2	2.1	1.4	1.4	1.0	1.2	2.4	2.0	1.0	1.4	1.0	2.0
laion/clap-htsat-fused	1.4	2.0	1.8	0.8	1.4	0.8	1.2	2.8	1.4	2.0	0.8	1.8	2.4

Appendix

Table 44: CommonVoiceMini17 T2A retrieval results (languages 27–39 of 50). Best per language in bold.

Model	sl	sr	sv	sw	ta	te	th	tr	uk	ur	vi
LCO-Embedding/LCO-Embedding-Omni-3B	65.2	49.8	67.0	47.0	29.8	73.5	90.4	72.2	91.4	80.8	81.8
LCO-Embedding/LCO-Embedding-Omni-7B	66.4	44.6	50.8	41.2	23.2	61.2	90.8	63.6	86.6	83.4	84.6
Qwen/Qwen2-Audio-7B	1.4	1.0	2.4	1.2	0.8	14.3	1.6	1.2	2.2	2.4	1.6
microsoft/speecht5_multimodal	1.0	1.2	1.4	1.8	1.0	14.3	0.8	1.4	0.8	0.8	1.2
microsoft/msclap-2023	2.0	1.8	1.2	0.6	0.8	8.2	1.4	0.8	1.2	0.8	0.8
laion/clap-htsat-fused	0.6	0.2	1.2	1.4	1.6	10.2	1.4	1.8	0.8	1.0	1.8

Appendix

Table 45: CommonVoiceMini17 T2A retrieval results (languages 40–50 of 50). Best per language in bold.

Model	ab	af	am	ar	as	ast	az	ba	bas	be	bg	bn	br	ca	ckb	cnh	cs	cv	cy	da	dav	de	dv	dyu	el	en	eo	es	et
LCO-Embedding/LCO-Embedding-Omni-3B	53.5	85.5	55.0	92.5	41.5	99.4	83.7	52.0	74.5	99.0	83.0	71.5	79.0	98.0	65.5	66.0	83.5	76.5	45.5	63.0	63.5	99.5	14.0	93.7	51.5	98.5	91.0	100.0	52.0
LCO-Embedding/LCO-Embedding-Omni-7B	44.0	85.5	35.5	93.0	26.5	97.5	81.5	46.5	69.5	98.0	79.0	64.5	66.5	96.5	47.5	62.5	69.0	65.5	33.5	56.0	53.5	99.5	5.5	88.9	30.5	98.5	88.5	100.0	49.0
Qwen/Qwen2-Audio-7B	5.0	7.7	5.0	4.5	7.0	9.6	12.0	3.0	5.5	4.0	4.0	4.0	3.5	6.5	6.0	3.5	5.0	4.5	5.5	5.0	4.0	8.0	4.5	6.3	6.0	8.5	6.5	9.5	5.0
microsoft/speecht5_multimodal	2.5	6.0	3.0	3.0	3.5	5.7	7.6	3.0	3.0	3.0	2.0	4.5	2.0	4.0	2.0	4.5	3.0	2.5	5.0	7.0	2.0	4.0	1.5	9.5	1.5	6.0	4.5	2.5	4.0
laion/clap-htsat-unfused	2.5	5.1	1.5	3.5	2.0	3.2	4.3	4.5	4.0	1.5	3.5	2.5	6.0	4.5	3.0	3.5	3.5	1.5	3.5	3.0	4.0	3.5	2.5	12.7	3.0	4.0	3.5	3.0	2.5
laion/clap-htsat-fused	3.0	5.1	2.0	2.5	1.5	5.1	2.2	3.0	3.5	1.5	3.5	1.5	3.5	4.0	1.5	4.5	3.0	3.5	3.5	2.0	1.5	1.5	1.5	9.5	4.0	3.5	4.0	1.5	2.5

Appendix

Table 46: CommonVoiceMini21 A2T retrieval results (languages 1–29 of 114). Best per language in bold.

Model	eu	fa	fi	fr	fy	ga	gl	gn	ha	he	hi	hsb	hu	hy	ia	id	it	ja	ka	kab	kk	kln	kmr	ko	ky	lg	lij	lt	ltg
LCO-Embedding/LCO-Embedding-Omni-3B	75.0	38.5	53.5	100.0	93.0	41.0	100.0	76.0	69.5	40.0	97.0	72.5	56.0	49.5	99.5	96.5	100.0	93.0	72.0	52.5	51.0	68.0	75.0	99.5	61.0	57.5	95.5	61.2	67.0
LCO-Embedding/LCO-Embedding-Omni-7B	69.0	33.5	33.0	100.0	92.0	35.5	100.0	58.0	55.5	31.5	96.0	73.5	45.5	36.5	99.5	95.5	100.0	94.5	63.0	42.5	31.5	57.0	59.0	98.0	57.0	46.0	92.5	63.6	56.4
Qwen/Qwen2-Audio-7B	4.0	5.0	5.5	5.0	4.5	3.0	9.5	5.5	3.5	3.0	5.5	7.5	3.5	4.0	6.0	5.5	8.5	5.5	4.0	3.5	4.5	5.0	5.5	3.5	4.0	5.0	6.0	1.8	2.2
microsoft/speecht5_multimodal	2.0	2.0	7.0	5.5	6.0	3.5	4.0	4.0	5.5	3.0	2.5	4.5	4.0	3.5	4.5	5.0	4.5	3.0	2.5	2.5	3.0	3.5	4.5	1.5	2.5	3.5	5.0	2.4	1.8
laion/clap-htsat-unfused	4.5	1.5	3.0	3.0	3.0	2.5	2.0	4.0	4.0	2.0	1.0	4.5	3.5	2.5	4.0	2.5	2.5	3.5	2.5	4.0	1.5	2.0	2.5	2.5	2.0	3.5	2.5	1.2	2.4
laion/clap-htsat-fused	3.0	2.5	3.0	5.0	3.0	1.5	3.0	3.0	2.5	3.0	2.0	1.5	3.5	3.0	5.0	3.5	3.0	4.0	2.5	5.5	2.5	3.5	4.0	2.5	2.0	2.5	2.5	1.4	2.0

Appendix

Table 47: CommonVoiceMini21 A2T retrieval results (languages 30–58 of 114). Best per language in bold.

Model	luo	lv	mdf	mhr	mk	ml	mn	mr	mrj	mt	myv	nan	ne	nl	nn	oc	or	os	pa	pl	ps	pt	rm	ro	ru	rw	sah	sat	sc
LCO-Embedding/LCO-Embedding-Omni-3B	86.0	40.6	92.5	79.0	78.2	55.8	16.2	78.0	74.4	80.0	88.0	59.2	61.4	98.8	66.5	90.9	63.1	66.9	76.6	87.0	50.8	98.6	98.4	85.0	99.0	46.5	57.5	4.4	94.5
LCO-Embedding/LCO-Embedding-Omni-7B	75.0	35.8	86.0	65.4	74.0	39.8	10.4	79.0	57.0	77.2	81.3	61.8	48.5	99.4	53.6	89.4	58.8	59.2	78.6	66.6	47.6	98.8	97.9	76.6	99.0	36.5	43.0	6.2	93.5
Qwen/Qwen2-Audio-7B	4.5	2.0	7.5	1.8	1.0	2.6	2.0	1.4	3.0	2.2	3.2	1.2	3.7	1.6	1.7	4.7	3.3	7.7	1.6	2.2	2.0	3.8	3.5	2.4	4.2	4.0	5.0	8.8	5.0
microsoft/speecht5_multimodal	4.0	1.0	4.7	1.2	0.8	1.0	1.0	0.4	1.4	1.4	1.9	0.6	2.2	2.6	1.9	1.8	0.7	1.5	0.8	1.0	1.0	2.0	3.0	1.2	1.0	3.5	2.0	5.3	7.0
laion/clap-htsat-unfused	4.5	1.0	8.4	1.4	0.4	2.0	1.6	1.2	0.6	1.4	0.8	0.8	2.2	0.8	1.5	2.2	1.4	3.8	1.0	1.0	1.4	1.2	2.5	0.8	0.4	4.0	3.0	5.3	2.5
laion/clap-htsat-fused	3.5	1.6	5.6	2.2	2.0	1.0	1.8	0.8	1.8	0.8	2.4	1.6	2.9	1.0	1.9	2.2	2.1	5.4	1.6	1.0	1.2	1.6	0.5	1.4	1.0	3.0	4.0	3.5	4.0

Appendix

Table 48: CommonVoiceMini21 A2T retrieval results (languages 59–87 of 114). Best per language in bold.

Model	sk	skr	sl	sq	sr	sv	sw	ta	te	th	tig	tk	tn	tok	tr	tt	ug	uk	ur	uz	vi	yi	yo	yue	zgh	zh	zza
LCO-Embedding/LCO-Embedding-Omni-3B	87.5	61.0	70.0	79.0	62.0	78.0	52.0	56.5	90.2	93.5	59.0	55.5	49.5	85.5	88.0	54.0	66.5	93.0	91.5	66.5	86.5	61.5	49.0	98.5	62.0	99.5	69.0
LCO-Embedding/LCO-Embedding-Omni-7B	82.5	62.0	70.5	74.0	63.5	65.5	56.0	28.5	63.9	95.0	40.5	41.5	35.5	83.0	75.5	42.0	58.5	94.0	90.5	55.0	89.0	62.2	33.0	99.0	66.0	99.5	58.5
Qwen/Qwen2-Audio-7B	6.5	2.5	3.5	5.0	5.5	5.0	5.0	5.0	18.0	5.0	3.0	5.0	4.0	4.5	3.5	4.5	4.0	6.5	6.5	3.5	4.0	4.2	2.0	6.5	3.5	4.5	4.5
microsoft/speecht5_multimodal	4.0	2.5	4.5	3.0	3.0	4.0	4.5	4.0	8.2	3.0	3.5	3.0	4.5	3.0	5.0	3.0	1.5	2.5	4.5	2.0	1.0	2.1	1.0	2.5	2.5	2.5	3.0
laion/clap-htsat-unfused	3.0	4.0	2.5	4.5	4.5	3.5	2.5	2.5	8.2	2.0	3.0	2.5	1.0	3.0	3.5	2.5	2.0	4.5	1.5	3.5	2.0	4.9	4.0	2.5	2.0	2.5	2.5
laion/clap-htsat-fused	3.0	2.5	3.0	3.0	3.0	1.5	2.5	2.0	9.8	1.0	2.5	2.5	3.0	3.5	2.0	3.5	3.0	5.0	3.0	3.5	3.0	2.8	2.0	2.5	2.5	3.0	2.5

Appendix

Table 49: CommonVoiceMini21 A2T retrieval results (languages 88–114 of 114). Best per language in bold.

Model	ab	af	am	ar	as	ast	az	ba	bas	be	bg	bn	br	ca	ckb	cnh	cs	cv	cy	da	dav	de	dv	dyu	el	en	eo	es	et
LCO-Embedding/LCO-Embedding-Omni-3B	22.0	83.8	22.0	92.0	42.0	99.4	83.7	29.0	58.0	99.5	84.0	69.0	60.0	99.0	45.5	48.5	86.0	55.0	23.0	74.0	51.0	99.5	6.5	87.3	50.5	99.0	96.0	100.0	53.5
LCO-Embedding/LCO-Embedding-Omni-7B	19.5	83.8	22.5	91.0	28.0	96.2	79.3	29.5	48.0	98.0	75.5	62.0	54.0	98.0	32.5	42.5	72.5	53.5	28.5	60.5	33.0	99.5	3.5	81.0	23.5	98.5	91.5	100.0	52.0
Qwen/Qwen2-Audio-7B	5.5	10.3	6.5	4.0	2.5	7.6	5.4	3.5	4.0	5.5	5.5	2.5	5.0	7.5	6.0	4.5	3.0	2.0	2.5	2.5	4.0	6.5	2.0	7.9	4.0	11.0	4.5	8.0	2.5
microsoft/speecht5_multimodal	1.5	7.7	2.5	2.5	3.0	5.1	6.5	2.5	3.0	3.5	2.0	2.5	4.5	3.5	1.5	7.5	2.5	2.0	4.5	5.5	3.5	4.0	2.5	14.3	2.5	5.5	3.0	2.5	3.5
laion/clap-htsat-unfused	3.0	5.1	2.0	1.5	2.5	3.2	3.3	3.5	3.5	3.0	3.0	3.0	4.5	5.5	3.0	3.0	1.5	2.0	3.5	0.5	4.5	5.0	2.5	11.1	2.5	2.0	3.5	3.5	3.5
laion/clap-htsat-fused	2.5	6.0	1.5	1.5	4.0	2.5	2.2	2.0	5.0	1.5	2.0	2.5	3.5	3.5	2.5	3.0	3.5	1.0	1.5	2.5	4.5	1.0	2.0	15.9	3.5	2.0	5.0	1.5	3.5

Appendix

Table 50: CommonVoiceMini21 T2A retrieval results (languages 1–29 of 114). Best per language in bold.

Model	eu	fa	fi	fr	fy	ga	gl	gn	ha	he	hi	hsb	hu	hy	ia	id	it	ja	ka	kab	kk	kln	kmr	ko	ky	lg	lij	lt	ltg
LCO-Embedding/LCO-Embedding-Omni-3B	58.5	33.5	54.5	100.0	90.5	34.0	99.5	60.0	55.0	45.0	95.5	77.5	53.5	47.0	99.0	94.5	100.0	93.5	69.5	38.0	25.5	48.5	53.0	98.5	44.5	31.5	94.5	68.4	57.2
LCO-Embedding/LCO-Embedding-Omni-7B	58.5	30.5	33.5	100.0	89.5	20.5	100.0	47.5	44.5	29.5	94.5	72.0	48.5	27.5	98.5	96.0	100.0	92.0	62.0	35.5	20.0	36.5	43.5	97.5	43.0	24.5	90.5	58.4	47.0
Qwen/Qwen2-Audio-7B	4.0	2.0	4.5	5.0	4.0	3.5	8.5	4.0	5.0	2.0	5.0	4.0	4.5	3.5	7.0	4.5	8.0	2.0	4.0	3.5	3.5	3.5	4.5	5.0	4.0	7.0	2.5	1.6	1.8
microsoft/speecht5_multimodal	3.0	2.5	6.0	4.0	3.5	1.5	3.5	4.0	3.0	3.0	3.0	4.5	2.5	3.0	5.0	5.0	5.0	2.5	3.0	3.5	2.0	4.5	3.0	2.5	3.0	4.5	3.0	1.6	1.2
laion/clap-htsat-unfused	2.0	2.5	4.0	3.0	5.0	1.5	2.5	4.5	2.5	3.0	3.0	3.0	3.5	2.5	3.0	2.5	3.5	4.0	2.5	3.5	2.0	2.0	3.5	2.5	2.5	2.5	5.5	1.0	1.4
laion/clap-htsat-fused	4.0	2.5	3.5	3.5	4.0	4.0	3.0	5.5	2.0	2.0	1.5	5.5	2.0	2.5	2.5	4.0	3.0	4.0	2.5	4.5	2.5	2.5	2.5	4.0	3.0	1.5	5.0	1.6	1.0

Appendix

Table 51: CommonVoiceMini21 T2A retrieval results (languages 30–58 of 114). Best per language in bold.

Model	luo	lv	mdf	mhr	mk	ml	mn	mr	mrj	mt	myv	nan	ne	nl	nn	oc	or	os	pa	pl	ps	pt	rm	ro	ru	rw	sah	sat	sc
LCO-Embedding/LCO-Embedding-Omni-3B	61.5	47.0	76.6	50.4	78.2	49.0	8.0	77.6	43.4	74.6	68.3	60.8	50.7	98.4	67.5	91.2	56.0	45.4	67.6	89.4	38.2	97.8	96.8	79.2	98.6	26.5	25.5	3.5	94.5
LCO-Embedding/LCO-Embedding-Omni-7B	62.0	38.8	63.6	41.0	71.8	32.6	5.4	76.0	33.4	70.4	61.1	63.8	39.7	98.2	51.9	87.2	61.9	33.1	70.0	63.8	35.6	98.2	95.4	69.8	98.4	23.0	27.0	3.5	94.0
Qwen/Qwen2-Audio-7B	4.0	1.6	4.7	1.6	1.6	3.4	1.8	0.8	1.8	1.4	2.1	1.6	3.7	2.6	2.4	4.0	1.4	5.4	1.8	1.0	1.4	4.4	3.0	2.6	4.0	3.5	3.0	5.3	3.5
microsoft/speecht5_multimodal	6.5	1.6	4.7	0.8	1.2	1.0	0.4	1.0	1.0	1.8	1.1	1.6	1.8	2.4	1.9	1.1	1.0	2.3	1.0	1.2	1.2	1.2	1.8	1.8	1.0	4.0	1.5	4.4	4.0
laion/clap-htsat-unfused	3.5	1.2	8.4	0.6	1.4	1.2	0.6	1.4	1.4	1.2	1.9	1.6	1.8	1.2	2.2	2.6	1.2	4.6	0.8	2.4	1.4	2.2	1.2	1.6	1.0	3.5	2.0	7.1	5.0
laion/clap-htsat-fused	3.5	2.0	5.6	1.2	1.0	0.6	1.4	1.0	2.0	2.0	1.3	1.8	2.6	0.8	2.7	1.8	0.7	10.0	1.4	1.0	0.6	1.6	1.4	1.4	1.6	2.0	2.0	2.7	4.5

Appendix

Table 52: CommonVoiceMini21 T2A retrieval results (languages 59–87 of 114). Best per language in bold.

Model	sk	skr	sl	sq	sr	sv	sw	ta	te	th	tig	tk	tn	tok	tr	tt	ug	uk	ur	uz	vi	yi	yo	yue	zgh	zh	zza
LCO-Embedding/LCO-Embedding-Omni-3B	89.0	48.0	74.5	72.0	55.5	74.0	52.5	40.5	77.0	92.5	30.0	38.0	29.5	67.5	83.0	32.5	45.0	93.5	88.5	50.0	86.0	44.8	25.5	99.5	26.5	99.0	63.0
LCO-Embedding/LCO-Embedding-Omni-7B	84.5	45.0	75.0	71.0	52.5	63.0	51.5	24.0	57.4	94.5	24.0	32.5	20.5	62.5	76.5	33.0	41.5	92.5	90.5	43.5	87.5	52.4	10.5	99.5	30.0	99.0	50.5
Qwen/Qwen2-Audio-7B	2.5	3.5	4.5	5.0	6.5	4.5	5.0	2.5	14.8	3.0	3.5	3.5	5.0	4.5	3.5	2.0	5.0	6.0	4.5	4.0	5.0	4.9	1.5	6.0	3.5	6.5	2.5
microsoft/speecht5_multimodal	2.5	2.5	2.0	3.5	2.0	3.0	3.5	1.0	8.2	2.5	3.0	4.5	5.5	7.5	4.5	1.5	3.0	1.0	4.0	2.5	3.5	4.2	2.5	2.5	1.5	2.5	5.5
laion/clap-htsat-unfused	3.0	2.5	5.0	5.0	1.5	4.5	2.5	2.0	6.6	2.5	2.5	3.0	2.5	4.5	3.5	4.5	2.5	4.5	3.0	1.0	2.0	4.2	2.0	7.5	2.5	3.0	2.0
laion/clap-htsat-fused	3.0	3.0	4.0	3.0	4.5	2.0	2.5	1.0	8.2	2.5	1.5	2.5	3.5	5.0	4.5	1.5	3.0	2.5	1.5	4.0	2.5	5.6	2.0	2.5	1.5	2.5	5.5

Appendix

Table 53: CommonVoiceMini21 T2A retrieval results (languages 88–114 of 114). Best per language in bold.

Audio Understanding

Audio understanding enables machines to recognize speech, sounds, music, speakers, and events, then answer questions grounded in what they hear.

See all 216 papers

MuLA-Bench: A Multilingual Long-Form Audio Understanding Benchmark via Multi-Tier Auditing
Sep 20, 2026Zeyu Yang, Xinyu Zhang, Zibo Bi +5Audio-Language Model Evaluation Audio Understanding
The Chinese University of Hong Kong, Shenzhen · Alibaba Token Hub, Alibaba Group · Xi’an Jiaotong University
AudioCapBench: Quick Evaluation on Audio Captioning across Sound, Music, and Speech
Feb 27, 2026Jielin Qiu, Jianguo Zhang, Zixiang Chen +13Audio-Language Model Evaluation Audio Captioning
Salesforce AI Research
MMAC: A Massive Multi-dimensional Benchmark for Audio Captioning
Jul 29, 2026Weijie Wu, Junbo Li, Lin Li +2Audio-Language Model Evaluation Audio Captioning
School of Informatics, Xiamen University, China · DiDi Global Inc., Beijing, China · School of Electronic Science and Engineering, Xiamen University, China

Sep 20, 2026J/K move · Enter open · S save

MuLA-Bench: A Multilingual Long-Form Audio Understanding Benchmark via Multi-Tier Auditing

Zeyu Yang, Xinyu Zhang, Zibo Bi, Pei Zhang+4

The Chinese University of Hong Kong, Shenzhen · Alibaba Token Hub, Alibaba Group · Xi’an Jiaotong University

Audio-Language Model Evaluation Audio Understanding Audio-Language Models Temporal Audio Grounding

Open paper AskPDF arXiv

Long-form audio performance is often summarized by context length and aggregate accuracy, obscuring how language, evidence, and task jointly shape difficulty. We introduce MuLA-Bench: 5,038 open-ended questions over 1,769 in-the-wild recordings totaling 1,377.9 hours, covering 16 languages and eight domains. A balanced Language x Domain semantic track supports controlled comparisons, while a complementary acoustic track preserves naturally occurring non-speech evidence. Evidence-grounded generation, shortcut checks, and language-expert review provide auditable questions without translating a shared source set or injecting target sounds. We evaluate ten audio-language models and conduct pooled diagnostics on a fixed eight-model cohort. Language rankings change across domains and tasks; acoustic-semantic performance gaps vary with the requested operation; and temporal errors can persist after the correct event is identified. Long-range retrieval is comparatively strong, while precise clock alignment and factual grounding of natural acoustic events remain fragile. MuLA-Bench thus exposes conditional failure patterns that a single long-context score does not capture.

Abstract

Audio Understanding

Explore similar work

MuLA-Bench: A Multilingual Long-Form Audio Understanding Benchmark via Multi-Tier Auditing

AudioCapBench: Quick Evaluation on Audio Captioning across Sound, Music, and Speech

MMAC: A Massive Multi-dimensional Benchmark for Audio Captioning