Think Before You Transcribe: Context-Steered Reasoning for Hotword-Aware Speech Recognition
Authors: Qihao Wang, Hang Su, Fan Cui, Lichun Fan, Jian Luan, Yue Hu
Organizations: Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences · MiLM Plus, Xiaomi Inc.
Contextual automatic speech recognition (ASR) improves rare-entity recognition by supplying candidate hotwords, but retrieved candidates may be incomplete or incorrect. A reliable system must therefore exploit a relevant candidate without allowing unsupported alternatives to override the acoustic evidence. We propose a context-steered reasoning framework that generates a short, audio-grounded caption before transcription. This task-specific representation lets the model assess candidate relevance before making a lexical decision. Supervised fine-tuning establishes the structured caption-to-transcription behavior, while targeted trajectory injection strengthens group relative policy optimization when difficult utterances yield only unsuccessful rollouts. Experiments across diverse context conditions show that the proposed method consistently improves both transcription quality and hotword recognition while remaining robust to irrelevant candidates.
Figures & tables
Figure 1: Overview of the CAST framework.
WER / NE-WER / NE-FNR (%; ↓ )
Model
Correct-only
Contextless
All-noise
RAG avg.
Models without explicit reasoning
Qwen2-Audio
22.8/19.0/21.3
23.3/50.8/59.5
26.7/51.4/58.8
28.9/50.0/34.9
Qwen2.5-Omni-3B
4.2/14.0/14.8
8.2/45.2/52.8
8.8/42.8/50.5
5.3/22.2/24.7
Qwen2.5-Omni-7B
3.2/11.4/13.2
7.2/40.0/46.4
6.9/37.1/43.2
4.4/20.5/22.6
Qwen3-Omni-30B-Instruct
2.6/9.2/10.3
6.5 / 38.0 / 44.2
6.5 / 35.7 / 42.2
3.6/15.2/17.2
Table 2: Results on RACE -Bench across external audio-language models and controlled 30B variants. Systems are grouped by whether they produce an explicit intermediate reasoning output. All metrics are percentages and lower is better. Bold and underlining mark the best and second-best values in each metric column. RAG is the arithmetic mean over top- k∈{5,10,20,50} .
WER / NE-WER / NE-FNR (%; ↓ )
Model
English Speech
English Dialogue
Mandarin Speech
Mandarin Dialogue
Models without explicit reasoning
Qwen2-Audio
11.5/27.3/35.1
14.0/33.0/32.9
9.9/24.1/30.0
7.0/22.8/26.2
Qwen2.5-Omni-3B
4.0/7.8/9.7
4.8/14.4/12.9
2.1/10.6/14.1
3.1/15.1/15.2
Qwen2.5-Omni-7B
4.0/7.4/8.7
5.3/11.8/9.2
1.8/9.8/12.2
2.4/14.1/13.2
Kimi-Audio
2.9/6.7/8.0
4.7/13.5/11.3
2.0/11.1/15.3
2.9/15.9/16.7
Table 3: Results on ContextASR-Bench for systems with corresponding evaluations, grouped by whether they produce an explicit intermediate reasoning output. All metrics are percentages and lower is better; bold and underlining mark the best and second-best values in each metric column.
Figure 2: WER and NE-FNR as retrieval size increases on RACE -Bench.
Figure 3: WER under contextless and all-noise input on RACE -Bench.
WER / NE-WER / NE-FNR (%; ↓ )
Setting
Inject
Baseline
Δ
Correct-only
1.9 / 3.6 / 3.4
2.5/7.6/8.3
-0.6/-4.0/-4.9
Contextless
6.2 / 34.6 / 40.4
6.9/39.8/46.3
-0.7/-5.2/-6.0
All-noise
6.3 / 33.8 / 40.0
6.8/36.9/43.7
-0.5/-3.2/-3.7
RAG avg.
3.2 / 11.8 / 12.9
3.7/14.7/16.5
-0.5/-2.9/-3.6
Table 4: Direct comparison between trajectory injection and the otherwise matched GRPO baseline on RACE -Bench. All metrics are percentages and lower is better. Bold marks the better system; Δ is inject minus baseline, so negative values favor injection.
Model
Avg. tokens
Est. decode (s)
Ground-truth transcript
27.3
–
Qwen2.5-Omni-3B
27.8
0.2
Qwen2.5-Omni-7B
27.7
0.3
Qwen3-Omni-30B-Instruct
27.9
0.2
Qwen3-Omni-30B-Think
892.4
6.5
Valley3
245.7
3.0
Table 5: Average output length and approximate decoding cost.
MoE Key Lab of Artificial Intelligence, AI Institute, X-LANCE Lab, Shanghai Jiao Tong University · School of Intelligence Science and Technology, Nanjing University, Suzhou, China