cs.CLNov 4, 2026

LibriSpeech-Real-QA: A Benchmark for Streaming Question Answering

Authors: Zewen Zhan, Hui Lu, Xueyuan Chen, Huimeng Wang, Yikai Huang, Xixin Wu, Shiyin Kang, Bin Xia, 

Organizations: Tsinghua University, China · The Chinese University of Hong Kong, China · SenseTime Research, China

Abstract

Most speech large language models (SLMs) still process speech in an offline manner. They only start responding after they receive the complete speech segment. However, many real-world applications require streaming speech interaction, for which the audio continuously flows in and the machine is required to answer questions based on the so far received audio chunks. For example, a user may ask the system to continuously count how many times a person's name has been mentioned while they are speaking. We observe a lack of benchmarks and evaluation protocols for real-time speech interaction. Motivated by this, we propose LibriSpeech-Real-QA, a dataset and a benchmark for real-time speech interaction. We construct different types of real-time question-answer pairs from LibriSpeech to evaluate whether speech LLMs can identify target words in streaming audio. We evaluate several popular real-time commercial models and find that they still tend to respond only after the speech turn has finished. We demonstrate that we can enable a pretrained SLM to perform this task, where we explore several modeling designs and establish a solid baseline for future research.

Figures & tables

Explore similar work

CardsList
  1. SPEARBench: A Benchmark for Naturalness Evaluation in Streaming Speech-to-Speech Language Models

    Jul 6, 2026Thomas Thebaud, Yuzhe Wang, Hao Zhang +5Audio-Language Model EvaluationLanguage Model Generation Evaluation

  2. InteractionBench: A Real-Time Interaction Benchmark for Streaming Video Systems

    Oct 5, 2026Enxin Song, Suhao Yu, Yifei Xu +7Streaming Video UnderstandingVideo-Language Model Evaluation