Streaming automatic speech recognition (ASR) underperforms on domain-shifted target audio, where labeled in-domain data is costly to prepare while unlabeled audio is abundant. We present StreamHear, a semi-supervised pipeline that adapts a pretrained streaming student by fine-tuning an offline transducer teacher on the labeled training set, generating pseudo-labels on the unlabeled portion, and fine-tuning the student on the mixture. We further introduce a prior-regularized dynamic-programming realignment step that fixes chunk-level word placement using an ASR-hypothesis anchor. Across four datasets spanning financial calls, prepared read speech, and phone-quality dialogue, StreamHear consistently outperforms supervised student fine-tuning and narrows the gap to the offline teacher.
Figures & tables
Split
Earnings-21
Earnings-22
SPGISpeech
BankCall
Labeled Train
10.72
24.76
19.58
12.96
Labeled Val
3.09
9.16
10.66
2.62
Labeled Test
4.88
12.38
10.23
2.88
Unlabeled
18.85
63.01
58.63
44.33
Total
37.54
109.31
99.10
62.79
Utterance (s)
23.3
23.0
9.2
6.2
Table 1: Detailed statistics of the datasets utilized in experiments, reporting the total duration in hours for each partition split and the average utterance length in seconds.
Method
Earnings-21
Earnings-22
SPGISpeech
BankCall
Test
Unlabeled
Test
Unlabeled
Test
Unlabeled
Test
Teacher
6.51 ±0.00
0 8.35 ±0.00
12.25 ±0.00
10.62 ±0.00
3.97 ±0.00
4.02 ±0.00
12.71 ±0.00
FT Teacher
6.03 ±0.05
0 7.68 ±0.03
11.24 ±0.03
0 9.55 ±0.01
2.20 ±0.02
2.15 ±0.01
0 8.85 ±0.11
Student
8.27 ±0.00
10.36 ±0.00
13.55 ±0.00
12.31 ±0.00
3.27 ±0.00
3.18 ±0.00
13.60 ±0.00
FT Student
7.19 ±0.09
0 9.62 ±0.10
12.46 ±0.06
10.81 ±0.01
2.77 ±0.02
2.70 ±0.03
10.68 ±0.13
StreamHear
6.63 ±0.04
0 7.77 ±0.03
11.76 ±0.01
0 9.56 ±0.01
2.59 ±0.03
2.26 ±0.01
0 9.80 ±0.05
Table 2: Word Error Rate (WER, %) comparison across all datasets. We report the mean and standard deviation (subscript) over 5 seeds on the labeled test and unlabeled splits. BankCall reports labeled test only (unlabeled pool has no transcripts). Bold indicates the best streaming-student result per column.
Dataset
Aligned
Realigned
Change
Earnings-21
17.46
0 8.01
−9.45
Earnings-22
20.54
10.97
−9.57
Table 3: Alignment-quality WER (%) on chunk text against a pretrained Parakeet-TDT-0.6B-v3 audio hypothesis, before and after DP realignment.
Top-K
Earnings-21
Earnings-22
25%
6.92 ±0.03 / 8.65 ±0.07
11.99 ±0.03 / 10.17 ±0.02
50%
6.83 ±0.06 / 8.28 ±0.05
11.86 ±0.02 / 0 9.87 ±0.03
75%
6.72 ±0.08 / 8.01 ±0.02
11.80 ±0.04 / 0 9.68 ±0.03
100%
6.63 ±0.04 / 7.77 ±0.03
11.76 ±0.01 / 0 9.56 ±0.01
Table 4: Pseudo-label pool scaling with a top- K % confidence filter. WER (%, mean ±std ) as labeled test / unlabeled.
Figure 1: Right-context (RC) sweep on Earnings-21 labeled test WER (%) at fixed LC=70 . Algorithmic latency is (1+RC)×80 ms. Error bars are ±1 std; student inference is deterministic, and StreamHear std ( ≤0.06 pp) is smaller than the marker.
Figure 2: Left-context (LC) sweep on Earnings-21 labeled test WER (%) at fixed RC=1 . Left context is LC×80 ms. Error bars are ±1 std; StreamHear std ( ≤0.05 pp) is smaller than the marker.
Method
Nemotron-EN
Nemotron-ML
Student
0 8.27 ±0.00 / 10.36 ±0.00
10.53 ±0.00 / 13.18 ±0.00
FT Student
0 7.19 ±0.09 / 0 9.62 ±0.10
0 8.54 ±0.11 / 10.94 ±0.13
StreamHear
0 6.63 ±0.04 / 0 7.77 ±0.03
0 8.10 ±0.07 / 0 8.66 ±0.05
Table 5: Student-choice ablation on Earnings-21 WER (%, mean ±std ) as labeled test / unlabeled.