We present Qwen-Audio-3.1-ASR, a family of audio-language models that builds upon its predecesor Qwen-Audio-3.0-ASR with improved speech recognition and expanded capabilities in speaker-diarized transcription and general audio understanding. The family retains support for 30 languages and 16 Chinese dialects, together with instruction-controlled capabilities including historical-context conditioning, hierarchical hotword customization, and native transcription polishing. Speaker-diarized ASR produces structured outputs containing speaker labels, timestamps, and recognized text, maintaining consistent speaker labels across turns and the representation of overlapping speech. The general audio understanding capabilities cover audio-, speech-, and music-captioning, temporal grounding and sound event detection, as well as audio question answering and multiple-choice reasoning. These tasks broaden the application scope from recognizing spoken words to interpreting speakers, acoustic events, and their temporal relations. To accommodate different deployment requirements, the family includes variants for asynchronous file transcription, low-latency streaming, interactive voice input, and non-streaming recognition, with speaker-diarized transcription and general audio understanding available in certain variants. Evaluations span conventional ASR, multilingual and dialectal speech, speaker-diarized transcription, and general audio understanding, demonstrating the capabilities of the Qwen-Audio-3.1-ASR family across transcription and broader audio analysis tasks.
Figures & tables
Figure 1: Performance overview of Qwen-Audio-3.1-ASR on selected benchmarks. (a) Multilingual ASR: unweighted mean of language-specific CER or WER (%) across seven languages in GigaSpeechBench. (b) Speaker-diarized ASR: unweighted mean cpWER (%) across AISHELL-4, AliMeeting-test, MLC-SLM, and MagicData-RAMC. (c) General audio understanding: accuracy (%) on MMAU and MMSU, evaluated using the Qwen-Audio-3.1-ASR-Next variant. Lower is better in (a) and (b), and higher is better in (c). Full evaluation results are reported in Section 5 .
Figure 2: Overview of Qwen-Audio-3.1-ASR and its capabilities. The model family supports multilingual and dialectal speech recognition, user-selectable dialect transcription or translation into Chinese, long-audio context and domain entity recognition, hierarchical hotword customization, and native transcription polishing. Speaker-diarized ASR produces transcripts with speaker labels and timestamps, including overlapping speech, while general audio understanding supports captioning, temporal grounding, sound event detection, and audio question answering. Streaming recognition provides incremental transcripts for real-time display, followed by a non-streaming refresh at the utterance end. Capability availability depends on the model variant; streaming does not support speaker diarization or general audio understanding.
Figure 3: Overview of the instruction-controlled decoding workflow of Qwen-Audio-3.1-ASR.
Supported capabilities
Variant
Target scenario
ASR
SD
AU
Context
Hotwords
Polishing
Qwen-Audio-3.1-ASR-Flash-Filetrans
Asynchronous audio-file transcription
✓
–
–
✓
✓
–
Qwen-Audio-3.1-ASR-Flash-Streaming
Real-time transcription
✓
–
–
✓
✓
–
Qwen-Audio-3.1-ASR-Flash-Message
Interactive voice input
✓
–
–
✓
✓
✓
Qwen-Audio-3.1-ASR-Flash
Non-streaming transcription
✓
✓
–
✓
✓
✓
Qwen-Audio-3.1-ASR-Next
Non-streaming transcription and audio analysis
✓
✓
✓
✓
✓
✓
Table 1: Deployment variants of Qwen-Audio-3.1-ASR model family. ✓ indicates a supported capability; a dash indicates an unsupported capability. SD denotes speaker-diarized ASR with recording-local speaker labels and timestamps; AU denotes general audio understanding. Context, hotwords, and polishing refer to historical-context conditioning, hierarchical hotword customization, and native transcription polishing, respectively. SD and AU are available only in the corresponding non-streaming variants.
Figure 4: The pretraining pipeline for the audio encoder.
Figure 5: Speaker-token loss reweighting. Left: token-count and unweighted-loss shares for five categories in the speaker-diarized ASR training data. Other denotes supervised tokens outside the four named categories; its shares are calculated by subtraction. Formatting and other loss shares are approximate. Right: only speaker-ID number tokens receive a cross-entropy weight of 10; timestamp, formatting, transcription, and other supervised tokens retain a weight of 1. The example fields illustrate category membership rather than exact tokenizer segmentation. Target sequences and token counts remain unchanged. In the schematic loss expression, i indexes supervised output tokens, CEi is the cross-entropy loss for token i , and wi is its loss weight: 10 for speaker-ID number tokens and 1 for all other supervised tokens. The proportionality notation emphasizes relative token weights and omits loss normalization.
Figure 6: Frame-aligned timestamp quantization. (a) An illustrative timestamp is mapped from 1 ms precision to the 80 ms output frame grid of the audio encoder and adapter. (b) Training loss with original and quantized timestamps. Curves are digitized from training plots; absolute loss values and iteration counts are omitted. Both curves retain a shared vertical scale and use the same normalization of training progress.
Figure 7: The FunVerl asynchronous RL framework. Audio preprocessing and vLLM rollout generation produce candidate responses, task-specific rewards score the responses, and the Megatron trainer updates the policy using GRPO. A trajectory queue decouples generation from optimization, and updated policy weights are synchronized to the rollout engine.
Figure 8: Overview of multi-teacher on-policy distillation in FunVerl. Student-generated responses are scored by task-routed teachers and the student using the same input and response prefixes. The k3 objective provides token-level distillation supervision, while NCCL weight synchronization refreshes the rollout policy after student updates.
Figure 9: Overview of the Message ASR workflow used in Qwen-Audio-3.1-ASR, following the design of Qwen-Audio-3.0-ASR. Streaming audio is decoded incrementally to provide responsive partial transcripts. Historical context supplies entity and topic cues, native polishing improves readability within the same decoding pass, and configurable right context balances streaming latency and recognition accuracy. At utterance end, the model refreshes the displayed hypothesis using the complete available context.
Panel A: Published Benchmark Results
Test set
Metric
Kimi -Audio
FireRedASR2 -LLM
FireRed Audio
LongCat -Next
Qwen3-ASR -1.7B
Qwen-Audio -3.0-ASR
Qwen-Audio -3.1-ASR
AISHELL-1
CER
0.60
0.64
0.71
1.47
–
1.03
1.08
AISHELL-2
CER
2.56
2.15
2.63
2.82
2.71
2.02
2.05
FLEURS-zh
CER
2.69
–
3.14
3.24
2.41
2.08
2.25
FLEURS-en
WER
4.44
–
2.53
5.24
3.35
4.17
4.72
LibriSpeech-clean
WER
1.28
–
0.67
1.63
1.63
1.19
1.23
Table 2: Performance on public ASR benchmark test sets . We report CER (%) for Chinese test sets and WER (%) for English test sets. For baselines, Panel A presents results taken from the official public repositories or the original technical reports, whereas Panel B presents results obtained using our unified API-based evaluation pipeline on the same test sets. A dash indicates that a result was not reported in the original source. Direct quantitative comparisons are primarily made within Panel B since the results in Panel A may involve source-specific preprocessing and scoring conventions. All WeNetSpeech-net results in Panel B use corrected reference transcripts following WenetSpeech issue #63 . All AISHELL-2 results use the test-ios subset. Lower values indicate better performance, and the best result within each panel on each test set is bold-faced.
Figure 10: CER (%) on public dialect ASR and AST benchmarks, comprising nine KeSpeech AST subsets and two WSYue Cantonese ASR subsets. Lower is better.
Figure 11: CER (%) on the internal ASR test set covering 16 Chinese dialects. All subsets use the ASR setting. Doubao-ASR does not explicitly list Henan, Qingdao, Nanchang, Ningbo, Hangzhou, Wenzhou, Hunan, or Suzhou in its publicly documented dialect coverage. Tencent Hy-ASR-3.0-preview does not explicitly list Shanxi, Shanghai, Ningbo, Hakka, Hangzhou, Wenzhou, Fujian, or Suzhou. Absence from these lists does not necessarily imply lack of recognition capability. Lower is better.
Figure 12: Semantic sentence accuracy (%) on the internal AST test set covering 11 Chinese dialects, with Mandarin translation as the target task. Qwen3.7-Max assesses whether each output preserves the reference meaning, including named entities and key information. Semantically equivalent expressions and minor differences that do not affect meaning are accepted. Higher is better.
Figure 13: Entity recall (%) across 15 industry domains. Radial positions represent percentage-point gaps from the highest recall in each domain, with the outer ring indicating the domain-specific best result. Numerical labels show the original recall values. Higher recall indicates better performance.
Doubao-ASR
Fun-ASR-Flash
Qwen-Audio-3.1-ASR
Category
Tier
w/o
w/
w/o
w/
w/o
w/
Person names
P0
32.27
82.64
37.33
95.26
62.12
99.43
Subject terms
P0
75.14
93.06
71.84
92.53
69.36
99.42
P1
84.36
93.04
86.02
96.73
88.46
95.31
Trending buzzwords
P0
48.24
91.95
43.26
82.53
63.08
99.46
P1
51.24
89.48
53.92
87.00
74.00
88.34
Table 4: Hotword recall (%) by entity category and priority tier. Results are reported without (w/o) and with (w/) hotword conditioning. P0 denotes high-priority hotwords, and P1 denotes the broader candidate set. Higher values indicate better performance. Bold values denote the highest recall among systems with hotword conditioning in each row.
Earlier context cue
Reference
Without long context
With long context
xsltproc style.xsl input.xml occurred earlier
style.xsl data.xml > out.html
xsl data.xml > out.html
style.xsl data.xml > out.html
Earlier explanation: “这叫法条竞合”
法条竞合
划掉即可
法条竞合
The term unigram occurred twice earlier
我们称之为 unigram
我们称之为 unique
我们称之为 unigram
TCC was repeatedly established as the metric name
our TCC will be...
our GGC will be...
our TCC will be...
The name “洛军是阿占的儿子” occurred earlier
洛军是阿占的儿子
洛君 是阿占的儿子
洛军 是阿占的儿子
The technical term softmax occurred earlier
softmax 形式
softmap 形式
softmax 形式
Table 5: Representative corrections enabled by long-audio context. The context cues occurred earlier in the same session. Colored spans highlight recognition errors and their context-conditioned corrections.
Figure 14: Readability and faithfulness of native polishing vs. the raw transcript and vs. an external ASR+LLM cascade.
Figure 15: Representative Chinese and English examples of native transcription polishing. Qwen-Audio-3.1-ASR removes fillers and disfluent repetitions, incorporates self-corrections, and standardizes sentence structure, punctuation, capitalization, and acronyms. The examples illustrate improvements in readability while largely retaining the original meaning.
Metric
Doubao Streaming ASR 2.0
Qwen-Audio-3.1-ASR-Flash-Message
Text-emission latency
530
220
First-character latency
570
100
Last-character latency
307
103
Table 6: Streaming latency of Doubao Streaming ASR 2.0 and Qwen-Audio-3.1-ASR-Flash-Message. All values are in milliseconds; lower is better.
Figure 16: Industry-term recall on nine domain-specific test sets for (a) streaming outputs used for real-time display and (b) non-streaming outputs used for the final transcript refresh. Doubao Streaming ASR 2.0 abbreviates Doubao Streaming Speech Recognition Model 2.0. Each panel compares the two systems within the same output stage. Recall is reported as a percentage; higher is better.
AISHELL-4
AliMeeting-test
MLC-SLM
MagicData-RAMC
System
DER
WER
cpWER
DER
WER
cpWER
DER
WER
cpWER
DER
WER
cpWER
MOSS-Transcribe-Diarize (MOSI)
4.20
14.33
15.23
3.38
19.47
17.23
3.20
11.71
14.06
4.79
10.88
14.00
Muse-Voice-Transcribe-1.0 (Meta)
3.04
17.78
19.04
6.16
26.93
27.97
3.48
12.64
14.60
4.55
12.55
15.47
Gemini-3.5-Transcribe (Google)
13.99
24.38
36.39
21.84
33.18
45.16
12.71
14.47
23.05
11.44
15.29
25.32
Mai-Transcribe-2 (Microsoft)
5.96
14.92
22.25
8.52
23.83
27.65
7.01
12.26
15.63
5.42
11.99
14.98
Fun-ASR-API (Cascaded)
8.45
17.08
21.22
9.03
25.37
34.68
6.17
29.26
33.83
5.37
11.52
20.42
Table 7: Speaker-diarized ASR results on four evaluation sets. To ensure an equitable comparison and mitigate inference instability common in LALMs during long-form audio processing, all test sets are standardized into approximately 4-minute audio chunks. The chunking methodology and evaluation scripts are provided in the SDR Evaluation Toolkit . Fun-ASR is evaluated as a cascaded system. DER, WER, and cpWER are reported as percentages; lower is better. The best result in each column is shown in bold. Systems marked with an asterisk (*) use results taken directly from their respective technical reports.
Benchmark
Gemini-3.1-Pro
Gemini-3.5-Flash
Seed-2.0-Lite
Qwen-Audio-3.1-ASR-Flash-Next
Comprehensive Audio Understanding (Accuracy)
MMAU
80.90
77.20
78.40
81.60
MMSU
85.30
78.32
81.36
82.24
Specialized Audio Understanding (Accuracy)
CochlScene
63.90
55.07
49.82
76.30
Nonspeech7k
90.90
76.69
88.83
93.20
Table 8: General audio understanding and temporal grounding results for Qwen-Audio-3.1-ASR-Next and three baselines. The first seven benchmarks report accuracy (%): MMAU and MMSU assess broad understanding, while CochlScene, Nonspeech7k, MELD, NSynth, and DiDiSpeech evaluate acoustic scenes, non-speech vocal sounds, emotion, musical instruments, and speaker gender, respectively. Clotho-Moment and DESED report temporal mean intersection over union (mIoU, %) for single-interval localization and multi-interval retrieval, respectively. Baseline results are obtained through our API-based evaluation. Higher is better, and the highest value in each row is boldfaced.
In recent years, automatic speech recognition (ASR) has witnessed transformative advancements driven by three complementary paradigms: data scaling, model scaling, and deep integration with large language models (LLMs). However, bridging the gap between academic benchmark performance and real-world production utility remains a persistent challenge, particularly in handling diverse regional dialects, dynamic entities and hotwords, long-range contextual information, and disfluent spontaneous speech. In this report, we present Qwen-Audio-3.0-ASR, a Mixture-of-Experts (MoE) LLM-based ASR system designed to address these production demands through a unified, instruction-following framework. The model is built upon the Qwen backbone, and is trained on tens of millions of hours of large-scale speech data. Qwen-Audio-3.0-ASR supports transcription across 30 languages and 16 Chinese dialectal varieties spanning eight major dialect regions. Beyond multilingual and dialectal recognition, the model provides production-oriented capabilities including industry-domain entity recognition, hierarchical hotword customization, native single-pass transcription polishing, and long-audio contextual modeling. We further develop a dedicated streaming variant, Qwen-Audio-3.0-ASR-Streaming, for latency-sensitive applications. Extensive evaluations on Chinese, English, multilingual, and real-world industrial test sets demonstrate state-of-the-art or highly competitive recognition performance across a broad range of evaluation conditions, with strong performance relative to leading commercial and proprietary systems including GPT-4o Transcribe and Gemini 3.1 Pro.
Recent advances in Audio LLMs have achieved human-level speech recognition, yet existing systems struggle to capture paralinguistic aspects such as speaker traits, expressive variations, and environmental acoustic conditions. To address this, we design a framework of 22 paralinguistic characteristics and create a dataset of over 1.2M Audio-QA pairs. We develop ParA-LLM, trained with a two-stage curriculum: first on single-attribute questions to build foundational knowledge, then on multi-attribute questions for joint reasoning over speaker and acoustic characteristics. We also release ParA-Bench, a benchmark of 6,000 multiple-choice questions across speaker-speech, acoustic, and mixed categories, where frontier models like GPT-4o-Audio achieve only 36% accuracy. ParA-LLM surpasses state-of-the-art Audio LLMs like GPT-4o-Audio by 7.5% on ParA-Bench, with additional gains of 1.13% on MMAU-Pro Speech and 7.49% on MMAR Speech.
Nishit Anand, Jiaqi Su, Ke Chen +5
Adobe Research, USA · University of Maryland, College Park, USA · OpenAI, USA
We frame the system as diagnostic data curation for a large audio-language model: before fine-tuning, we probe Qwen3-Omni-30B-A3B-Instruct under normal, empty-audio, and shuffled-audio conditions to identify how the model's answers change when audio evidence is removed or mismatched. These model confusion patterns are used to bucket training samples into text-prior, shuffle-leak, strong audio-dependent, and hard or misleading cases. Our strongest train-only system fine-tunes only on strong-audio items, where the normal audio-question pair is correct but both counterfactual variants fail, plus a small number of empty-audio negatives and a text-only response normalizer for parse-failed generations. On the official development set, the best train-only system reaches 67.27% accuracy after response normalization, compared with 65.90% for our local Qwen3-Omni baseline. Final submissions additionally include models trained using train+development splits and a three-model ensemble.