cs.CLApr 1, 2026
SaveSpeech LLMs are Contextual Reasoning Transcribers
Organizations: Microsoft Core AI, USA
Abstract
Despite extensions to speech inputs, effectively leveraging the rich knowledge and contextual understanding of large language models (LLMs) in automatic speech recognition (ASR) remains non-trivial, as the task primarily involves direct speech-to-text mapping. To address this, this paper proposes chain-of-thought ASR (CoT-ASR), which constructs a reasoning chain that enables LLMs to first analyze the input speech and generate contextual analysis, thereby fully exploiting their generative capabilities. With this contextual reasoning, CoT-ASR then performs more informed speech recognition and completes both reasoning and transcription in a single pass. Moreover, CoT-ASR naturally supports user-guided transcription: while designed to self-generate reasoning, it can also seamlessly incorporate user-provided context to guide transcription, further extending ASR functionality. To reduce the modality gap, this paper introduces a CTC-guided Modality Adapter, which uses CTC non-blank token probabilities to weight LLM embeddings, efficiently aligning speech encoder outputs with the LLM's textual latent space. Experiments show that, compared to standard LLM-based ASR, CoT-ASR achieves a relative reduction of 8.7% in word error rate (WER) and 16.9% in entity error rate (EER).
Figures & tables
Figure 1: Illustration of CoT-ASR. The text prompt is a fixed template for the ASR task. The output sequence comprises contextual analysis and transcription, marked by the <CONTEXT> and <TRANSCRIPT> tags, respectively. The reasoning-based contextual analysis is highlighted in red, while the subsequent transcription is marked in green. Both are produced sequentially in a single one-pass generation.
Figure 2: Comparison between CoT-ASR and standard LLM-based ASR. During generation, CoT-ASR first performs a contextual reasoning analysis, which then guides the subsequent transcription. This example illustrates how CoT-ASR leverages the rich knowledge of LLMs to ultimately improve transcription quality.
Figure 3: Illustration of the CTC-guided Modality Adapter. The linear output layer projects the encoder outputs to the CTC vocabulary size including the blank token, while the linear layer maps the encoder outputs to the LLM hidden dimension. denotes matrix multiplication and denotes addition. Gate denotes a frame-wise scalar from the linear projection and sigmoid that modulates the residual contribution.
| Metric | Test Set | Phi4MM-style | CoT-ASR |
| Baseline | |||
| WER | LibriSpeech test-clean | 2.41 | 2.20 |
| LibriSpeech test-other | 5.10 | 4.82 | |
| FLEURS | 3.85 | 3.57 | |
| EER | Consumer Goods | 15.45 | 13.46 |
| Nutrition | 7.36 | 6.66 |
Table 1: WER ( ) results on English ASR test sets from public datasets, including LibriSpeech (test-clean and test-other) and FLEURS, along with EER ( ) results on a series of in-house English ASR evaluation sets spanning diverse domains. Note that the training data used for the models constructed in this paper do not include these public datasets.
| Test Set | CoT-ASR | CoT-ASR |
| with User Context | ||
| Consumer Goods | 13.46 | 9.03 |
| Nutrition | 6.66 | 5.24 |
| Gaming | 14.53 | 12.99 |
| Wellness | 11.51 | 8.13 |
| Patient History | 3.51 | 3.31 |
Table 2: EER ( ) results on a series of in-house ASR evaluation sets, comparing CoT-ASR using its self-generated contextual reasoning (the default setting) with scenarios guided by user-provided context.
| Test Set | CoT-ASR | CoT-ASR |
| with Linear Adapter | ||
| Consumer Goods | 13.46 | 14.93 |
| Nutrition | 6.66 | 6.80 |
| Gaming | 14.53 | 16.52 |
| Wellness | 11.51 | 12.87 |
| Patient History | 3.51 | 4.11 |
Table 3: EER ( ) results on a series of in-house ASR evaluation sets across multiple domains, comparing CoT-ASR with the proposed CTC-guided Modality Adapter (the default setting) against a linear-projection-based adapter (with activation function).
| LLM | Phi4 | Whisper | Gemma 3n | Qwen3 | Qwen2.5 | Voxtral | |
| Size | 3.8B | 1.5B | 8B | 30B | 7B | 3B | |
| Test Set | Phi4MM-style Baseline | CoT-ASR | Whisper- Large-v3 | Gemma 3n E4B IT | Qwen3-Omni- 30B-A3B-Instruct | Qwen2.5- Omni-7B | Voxtral-Mini -3B-2507 |
| Consumer. | 15.45 | 13.46 | 14.32 | 22.66 | 15.02 | 23.79 | 14.32 |
| Nutrition | 7.36 | 6.66 | 7.13 | 11.77 | 6.63 | 14.72 | 7.36 |
| Gaming | 17.42 | 14.53 | 15.14 | 23.59 | 15.01 | 28.68 | 14.15 |
| Wellness | 14.27 | 11.51 | 11.88 | 26.15 | 11.51 | 23.76 | 13.31 |
Table 4: EER ( ) results on a series of in-house ASR evaluation sets across multiple domains, comparing CoT-ASR with several open-source foundation models, including Whisper-large-v3, Gemma 3n E4B IT, Qwen3-Omni-30B-A3B-Instruct, Qwen2.5-Omni-7B, and Voxtral-Mini-3B-2507. Note that the comparison is not well controlled and is provided for context, as differences in LLM backbones and training data prevent direct comparison. The models built in this paper were trained on 38k hours of data, whereas the open-source models are trained on substantially larger datasets, for example, Whisper-large-v3 used approximately 5M hours of training data. In all cases, speech serves as the sole input, without any additional context provided.
| Model | test-clean | test-other | ||||
| WER | B-WER | U-WER | WER | B-WER | U-WER | |
| CoT-ASR | 2.20 | 9.31 | 1.29 | 4.82 | 20.01 | 3.02 |
| Phi4MM-style Baseline | 2.41 | 10.41 | 1.40 | 5.10 | 21.33 | 3.18 |
| +Bias List (Size=100) | 2.42 | 8.84 | 1.60 | 5.56 | 18.47 | 4.03 |
| +Bias List (Size=500) | 5.99 | 10.83 | 5.25 | 13.80 | 22.51 | 12.78 |
Table 5: WER / B-WER (biased WER) / U-WER (unbiased WER) ( ) results on LibriSpeech test sets, comparing CoT-ASR with Phi4MM-style baseline using contextual biasing. Note that this comparison is not well-controlled, as CoT-ASR does not receive the biasing list as additional input.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| Prompt Content |
| INSTRUCTION |
| You are a High-Fidelity Context Synthesizer. Your task is to generate a CONTEXT block based on the provided GROUND_TRUTH text. |
| This context will be placed before the ASR transcript to guide a Speech-LLM. It must act as a semantic "warm-up" by defining the environment, the speaker’s intent, and the specific entities or examples mentioned. |
| CORE ATTRIBUTES OF THE CONTEXT: |
| 1. Scenario Identification: Start by identifying the format (e.g., educational lecture, formal briefing, casual discussion). |
| 2. Specific Weaving: Incorporate unique examples, technical terms, or key names found in the text (e.g., if the text mentions "sugar" or "Sherman", the context must include them) to ground the model. |
Table 6: Prompt used in this paper to construct reasoning-augmented ASR data.
| Test Set | Phi4MM-style Baseline | CoT-ASR |
| Consumer Goods | 7.38 | 6.47 |
| Nutrition | 5.00 | 4.67 |
| Gaming | 8.56 | 8.03 |
| Wellness | 9.13 | 8.31 |
| Patient History | 2.42 | 2.08 |
| Pharmacy | 2.54 | 2.22 |
Table 7: WER ( ) results on a series of in-house English ASR evaluations sets.
| Test Set | 0% error | 10% error | 25% error | 50% error |
| Consumer Goods | 9.03 | 11.02 | 11.46 | 12.76 |
| Nutrition | 5.24 | 5.44 | 5.67 | 6.07 |
| Gaming | 12.99 | 13.70 | 14.50 | 14.24 |
| Wellness | 8.13 | 8.50 | 9.70 | 10.65 |
| Patient History | 3.31 | 3.54 | 3.47 | 3.58 |
| Pharmacy | 3.11 | 3.31 | 3.94 | 4.38 |
Table 8: Robustness of CoT-ASR to simulated errors in user-provided context. EER (%) is reported under varying error rates.
| Test Set | CoT-ASR | w/ Linear Adapter | w/ CTC Compressor | w/ SSR |
| Consumer Goods | 13.46 | 14.93 | 21.70 | 21.44 |
| Nutrition | 6.66 | 6.80 | 11.57 | 12.23 |
| Gaming | 14.53 | 16.52 | 24.29 | 25.12 |
| Wellness | 11.51 | 12.87 | 19.12 | 18.37 |
| Patient History | 3.51 | 4.11 | 7.03 | 7.16 |
| Pharmacy | 5.97 | 6.66 | 11.57 | 10.88 |
Table 9: EER ( ) results on on in-house ASR evaluation sets for CoT-ASR with different modality adapters.