Text Grouping Changes Completion-Time Comparisons in Long-Form Simultaneous Speech Translation
Organizations: VinUniversity, Vietnam
Abstract
Does grouping translation updates before speech synthesis change the measured completion gap between two translators? We compare StreamAtt with its fixed-audio-history baseline on ten English-German talks. For each translation, we keep the intended text and its availability times fixed and vary how updates are grouped for synthesis. Four-second grouping increases StreamAtt's mean completion lead from 106.6 to 229.1 seconds with XTTS-v2, but reduces it from 169.5 to 146.5 seconds with Piper. Both produce shorter audio; different changes in playback idle account for the opposite effects. These directions recur on a separately selected twenty-talk panel. In continuing CosyVoice 3 sessions, grouping instead leaves paired audio unchanged and delays completion on nineteen of twenty blocks. A speech-timing comparison therefore needs to specify how text enters synthesis, not just which synthesizer is used. Recognition and automatic word timing describe the outputs, but do not establish translation accuracy or intelligibility.
Figures & tables
| TTS | Translator | Wait (s) | First playback | Audio | Idle | Offset | ASR edits (%) |
|---|---|---|---|---|---|---|---|
| XTTS-v2 | StreamAtt | 0 | 17.44 | 1080.36 | 19.21 | 468.01 | 32.89 |
| StreamAtt | 2 | 19.82 | 834.02 | 30.05 | 232.52 | 26.72 | |
| StreamAtt | 4 | 22.42 | 723.36 | 41.24 | 133.04 | 22.45 | |
| Baseline | 0 | 17.44 | 1159.30 | 46.85 | 574.59 | 30.90 | |
| Baseline | 2 | 19.81 | 989.54 | 64.74 | 422.72 | 24.87 | |
| Baseline | 4 | 22.34 | 919.94 | 73.75 | 362.13 | 24.16 |
| Panel | TTS | (s) | Gap | Gap | Change | 95% interval for |
|---|---|---|---|---|---|---|
| Exploratory (10) | XTTS-v2 | 2 | ||||
| XTTS-v2 | 4 | |||||
| Piper | 4 | |||||
| Confirmation (20) | XTTS-v2 | 4 | ||||
| Piper | 4 |
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
| Talk | Split | Audio (s) |
|---|---|---|
| 805 | Dev | 594.547 |
| 1102 | Common | 421.396 |
| 792 | Dev | 545.901 |
| 2017 | HE | 556.536 |
| 2007 | HE | 551.335 |
| 1939 | HE | 626.521 |
| Talk | |||||
|---|---|---|---|---|---|
| 805 | -100.752 | -175.063 | -175.050 | -74.310 | -74.298 |
| 1102 | -73.747 | -188.574 | -233.912 | -114.827 | -160.165 |
| 792 | +76.843 | -66.650 | -118.590 | -143.493 | -195.433 |
| 2017 | -189.004 | -211.120 | -238.417 | -22.116 | -49.413 |
| 2007 | -87.943 | -90.048 | -121.696 | -2.105 | -33.753 |
| 1939 | -147.164 | -247.972 | -286.235 | -100.808 | -139.071 |
| Talk | Duration part | Idle part | XTTS | |||
|---|---|---|---|---|---|---|
| 792 | ||||||
| 805 | ||||||
| 1102 | ||||||
| 1381 | ||||||
| 1922 | ||||||
| 1939 |
| Talk | Split | Source (s) | Duration part | Idle part | |||
|---|---|---|---|---|---|---|---|
| 783 | Dev | 413.5 | |||||
| 785 | Dev | 1055.9 | |||||
| 769 | Dev | 1318.3 | |||||
| 767 | Dev | 1772.1 | |||||
| 1375 | Common | 974.3 | |||||
| 1161 | Common | 324.8 |
| TTS | Duration part | Idle part | |
|---|---|---|---|
| XTTS-v2 | |||
| Piper |
| TTS | Grouping rule | |||
|---|---|---|---|---|
| XTTS-v2 | 2-second timer | |||
| Punctuation / 2-second cap | ||||
| 8 words / 2-second cap | ||||
| 4-second timer | ||||
| Piper | 2-second timer | |||
| Punctuation / 2-second cap |
| TTS | Mean (s) | 95% interval | Positive talks |
|---|---|---|---|
| XTTS | 6/10 | ||
| Piper | 2/10 |
| Threshold | XTTS | Piper |
|---|---|---|
| 0.3 | ||
| 0.5 | ||
| 0.7 |
| Update statistic | StreamAtt | Baseline |
| Number of updates | 4064 | 2924 |
| Whitespace words | 14430 | 19823 |
| Median words | 3 | 3 |
| 90th-percentile words | 6 | 10 |
| At most two words (%) | 43.36 | 40.97 |
| Final punctuation (%) | 1.77 | 1.44 |
| History | 0 s | 2 s | 4 s |
|---|---|---|---|
| StreamAtt | 4,068 | 1,894 | 1,149 |
| Baseline | 3,064 | 1,586 | 1,096 |
| Backend | History | Intended words | Substitutions | Insertions | Deletions |
|---|---|---|---|---|---|
| XTTS-v2 | StreamAtt | 14,336 | |||
| Baseline | 19,766 | ||||
| Piper | StreamAtt | 14,336 | |||
| Baseline | 19,766 |
| TTS | Translator | ASR edits (%) | Deletions |
|---|---|---|---|
| XTTS | StreamAtt | ||
| Baseline | |||
| Piper | StreamAtt | ||
| Baseline |
| History | Recognizer | Insertions | Deletions |
|---|---|---|---|
| StreamAtt | Whisper | ||
| StreamAtt | Qwen ASR | ||
| Baseline | Whisper | ||
| Baseline | Qwen ASR |
| Threshold | Non-speech (s) | Speech (s) | Share (%) |
|---|---|---|---|
| 0.3 | 138.35 | 218.65 | 38.75 |
| 0.5 | 146.18 | 210.82 | 40.95 |
| 0.7 | 155.04 | 201.96 | 43.43 |
| History | Wait (s) | median | 95% interval | p90 | 95% interval | Lower median |
|---|---|---|---|---|---|---|
| StreamAtt | 2 | -119.03 | [-145.18, -95.48] | -219.00 | [-261.47, -177.58] | 10/10 |
| StreamAtt | 4 | -173.56 | [-207.73, -140.76] | -314.88 | [-377.34, -252.99] | 10/10 |
| Baseline | 2 | -72.70 | [-106.75, -41.46] | -145.13 | [-204.97, -88.56] | 9/10 |
| Baseline | 4 | -105.36 | [-158.36, -57.25] | -203.06 | [-289.41, -120.88] | 9/10 |
| History | first | Later end | ||
|---|---|---|---|---|
| StreamAtt | 0 | 9/10 | ||
| Baseline | 0 | 10/10 |
Simultaneous Speech Translation
Simultaneous speech translation converts speech into another language as audio arrives, balancing translation quality, latency, and fluent delivery.
See all 30 papers