Paper ID: 2409.00965

What does it take to get state of the art in simultaneous speech-to-speech translation?

Vincent Wilmet, Johnson Du

This paper presents an in-depth analysis of the latency characteristics observed in simultaneous speech-to-speech model's performance, particularly focusing on hallucination-induced latency spikes. By systematically experimenting with various input parameters and conditions, we propose methods to minimize latency spikes and improve overall performance. The findings suggest that a combination of careful input management and strategic parameter adjustments can significantly enhance speech-to-speech model's latency behavior.

Submitted: Sep 2, 2024