Acoustic Progress Propagation for Long-Horizon Speculative Decoding in ASR

Yuanyuan Jia; Qianqian Yang

eess.ASSep 27, 2026

Acoustic Progress Propagation for Long-Horizon Speculative Decoding in ASR

Authors: Yuanyuan Jia, Qianqian Yang

Organizations: Zhejiang University, Hangzhou, China

Abstract

Speculative decoding accelerates autoregressive automatic speech recognition (ASR), but the acceptance length of alignment-aware drafters can saturate as the draft horizon increases. We propose a progress-aware speculative drafter that recurrently propagates an acoustic progress state across draft steps and feeds it back into audio cross-attention to guide token generation. We jointly train the drafter and progress predictor over variable draft horizons. On five ASR test sets, our method achieves lossless, macro-averaged end-to-end speedups of 1.657x and 1.227x over target-only autoregressive decoding with Qwen3-ASR-0.6B and Qwen3-ASR-1.7B, respectively. Relative to AnchorDraft, our method improves the macro-averaged speedup by 34.3% and 9.0%, respectively. Horizon sweeps show continued growth in acceptance length beyond the baselines' saturation. Code is available at https://github.com/yuanyuanjia71-spec/ProgDraft.

Figures & tables

Panel of Figure 1 — Figure 1: Overview of the proposed progress-conditioned speculative ASR decoding. (a) Draft-and-verify pipeline. (b) Acoustic-progress prediction and feedback within one shared draft step.

		LibriSpeech test-clean		LibriSpeech test-other		TED-LIUM 3 test		GigaSpeech test		FLEURS en-us test
Model	Method	Speedup	$\tau$	Speedup	$\tau$	Speedup	$\tau$	Speedup	$\tau$	Speedup	$\tau$
Qwen3-ASR 0.6B	Base Drafter	1.203 $\times$	3.043	1.172 $\times$	2.976	1.227 $\times$	3.102	1.186 $\times$	3.012	1.131 $\times$	2.864
	SpeechSpec-EAGLE3	0.555 $\times$	1.545	0.560 $\times$	1.544	0.580 $\times$	1.616	0.573 $\times$	1.568	0.532 $\times$	1.451
	AnchorDraft	1.260 $\times$	3.241	1.223 $\times$	3.137	1.263 $\times$	3.245	1.246 $\times$	3.178	1.176 $\times$	2.986
	AnchorDraft + RC	1.120 $\times$	3.071	1.105 $\times$	3.009	1.179 $\times$	3.173	1.120 $\times$	3.047	1.065 $\times$	2.854
	Ours	1.786 $\boldsymbol{\times}$	5.055	1.624 $\boldsymbol{\times}$	4.553	1.737 $\boldsymbol{\times}$	4.867	1.633 $\boldsymbol{\times}$	4.533	1.504 $\boldsymbol{\times}$	4.135

Table 1: End-to-end speedup over target-only autoregressive decoding and average acceptance length

\tau

on five ASR test sets with Qwen3-ASR-0.6B and Qwen3-ASR-1.7B. All methods use inference draft length

K=8

Panel of Figure 2 — Figure 2: Average acceptance length $\tau$ (left) and end-to-end speedup over Target-only AR (right) versus inference horizon $K$ , with Qwen3-ASR-0.6B on the 1,000-utterance test set.

Model Variant	Acceptance length $\tau\uparrow$	E2E Speedup $\uparrow$
w/o Acoustic Progress	3.078	1.206 $\times$
w/o Progress Feedback	3.069	1.210 $\times$
Absolute-position prediction	3.214	1.233 $\times$
w/o Joint Training	3.130	1.121 $\times$
Fixed- $K=3$ training	3.760	1.331 $\times$
Fixed- $K=8$ training	4.572	1.611 $\times$

Table 2: Ablations on Qwen3-ASR-0.6B using 1,000 test utterances and inference horizon

K=8

Speculative Decoding

Speculative decoding accelerates generation by drafting multiple tokens and verifying them in parallel with a larger target model.

See all 311 papers

Alignment Drift in Single-Model Speculative Decoding for ASR: Mechanism, Correction, and Cost
Aug 13, 2026Xinyu Wang, Huapeng Zhou, Ziyu Zhao +5Audio-Text Alignment Speculative Decoding
1Boson AI · 2McGill University
Lossless Inference Acceleration for ASR via EAGLE-Style Speculative Decoding
Oct 9, 2026Sen Zhang, Xianghu Yue, Yiming Chen +3Automatic Speech Recognition Autoregressive Decoding
College of Intelligence and Computing, Tianjin University, Tianjin 300350, China · ASUS Intelligent Cloud Services, Singapore 486030 · School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen, Guangdong 518172, China +1
AS $^2$ D: Accelerating On-Demand Audio Understanding on Mobile Devices
Sep 29, 2026Yunzhe Li, Kyoungjun Park, Hongzi Zhu +1On-Device Language Model Inference Language Model Decoding
The University of Texas at Austin · Shanghai Jiao Tong University

Aug 13, 2026J/K move · Enter open · S save

Alignment Drift in Single-Model Speculative Decoding for ASR: Mechanism, Correction, and Cost

Xinyu Wang, Huapeng Zhou, Ziyu Zhao, Silin Meng+4

1Boson AI · 2McGill University

Audio-Text Alignment Speculative Decoding Automatic Speech Recognition

Open paper AskPDF arXiv

Speculative decoding speeds up generation by letting a cheap draft propose several tokens that a target model checks in one pass. In the single-model form, the draft is a lightweight module attached to the target rather than a separate model. Applying this design to Automatic Speech Recognition (ASR) introduces an extra problem. The draft can read the whole audio at every step, yet its proposals get worse as it runs on its own. Access is not localization. The accepted text keeps the transcript position explicit, but the draft must also track the changing audio position. In the primary matched comparison, per-step audio access changes the first proposal modestly but roughly doubles later-proposal acceptance. Fixed-width windows show that the audio position explains part of this gap. A correctly placed window recovers continuation, while an equally narrow window at the wrong position reduces it. Late-draft median error reaches 21 frames in the hardest reported condition, while target attention during verification stays within a 2-frame median. We test two ways to reduce this drift. The first reads the audio position from verification attention and uses it to guide the next draft round. It saves time only when the extra accepted tokens offset the readout cost. The second is AnchorDraft, which teaches the draft to track the audio position during training without changing the inference graph. The trained draft improves end-to-end speed at both tested target scales. These results show that ASR self-speculation depends on token prediction, audio-position tracking, and draft cost.

Abstract

Speculative Decoding

Explore similar work

Alignment Drift in Single-Model Speculative Decoding for ASR: Mechanism, Correction, and Cost

Lossless Inference Acceleration for ASR via EAGLE-Style Speculative Decoding

AS2^22D: Accelerating On-Demand Audio Understanding on Mobile Devices

AS $^2$ D: Accelerating On-Demand Audio Understanding on Mobile Devices