Streaming ASR

ASR: Automatic Speech Recognition

Streaming ASR studies how to turn incoming audio into text before the speaker has finished, which matters for live captions, voice agents, and on-device interfaces. Common approaches include RNN-T and CTC transducers, chunk-wise attention, and cache-aware encoders that limit how much future audio the model must wait for. Newer systems also use language-model context, adaptive commit policies, and distillation or quantization to improve recognition and run on constrained hardware. The central challenge is balancing transcription accuracy against emission delay, memory, and power, especially across languages, noisy speech, and conversational turn boundaries.

Latest papers

39 papers7 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

WhisperPipe: A Resource-Efficient Streaming Architecture for Real-Time Automatic Speech Recognition
Apr 28, 2026Erfan Ramezani, Mohammad Mahdi Giahi, Mohammad Erfan Zarabadipour +2Streaming ASR Speech Processing
Mechatronics Research Laboratory, Dept. of Computer and Electrical Engineering, Qazvin Islamic Azad University, Qazvin, Iran · Department of Electrical Engineering, Qa. C., Islamic Azad University, Qazvin, Iran.
Reducing the Offline-Streaming Gap for Unified ASR Transducer with Consistency Regularization
Apr 21, 2026Andrei Andrusenko, Vladimir Bataev, Lilit Grigoryan +3Streaming ASR Automatic Speech Recognition
NVIDIA, Armenia · NVIDIA, USA
NIM4-ASR: Towards Efficient, Robust, and Customizable Real-Time LLM-Based ASR
Apr 20, 2026Yuan Xie, Jiaqi Song, Guang Qiu +9Streaming ASR Automatic Speech Recognition
Advanced Intelligent Systems Group, NIO
Efficient Punctuation Restoration via Weighted Lookahead Scoring Method for Streaming ASR Systems
Apr 18, 2026Sungmook Woo, Hyungu Kang, Chanwoo KimStreaming ASR
Department of Artificial Intelligence, Korea University, Seoul, Republic of Korea
Pushing the Limits of On-Device Streaming ASR: A Compact, High-Accuracy English Model for Low-Latency Inference
Apr 16, 2026Nenad Banfic, David Fan, Kunal Vaishnavi +5Efficient Neural Network Inference Streaming ASR
CoreAI, Microsoft
Streaming Translation and Transcription Through Speech-to-Text Causal Alignment
Mar 12, 2026Roman Koshkin, Jeon Haesung, Lianbo Liu +4Speech Translation Streaming ASR
SB Intuitions, Tokyo, Japan
Uni-ASR: Unified LLM-Based Architecture for Non-Streaming and Streaming Automatic Speech Recognition
Mar 11, 2026Yinfeng Xia, Jian Tang, Junfeng Hou +2Automatic Speech Recognition Streaming ASR
Qwen Applications Business Group, Alibaba, China · Tongyi AI Lab, Alibaba, China
Chunk-wise Attention Transducers for Fast and Accurate Streaming Speech-to-Text
Feb 27, 2026Hainan Xu, Vladimir Bataev, Travis M. Bartley +1Streaming ASR Attention Mechanisms
NVIDIA Corporation.
SENS-ASR: Semantic Embedding injection in Neural-transducer for Streaming Automatic Speech Recognition
Feb 17, 2026Youness Dkhissi, Valentin Vielzeuf, Elys Allesiardo +1Contextual ASR Automatic Speech Recognition
LIUM · Orange Innovation, 4 Rue du Clos Courtel, 35510 Cesson-Sévigné, France · LIUM, Le Mans Université Avenue Olivier Messiaen,72085 Le Mans CEDEX 9, France

Apr 28, 2026J/K move · Enter open · S save

WhisperPipe: A Resource-Efficient Streaming Architecture for Real-Time Automatic Speech Recognition

Erfan Ramezani, Mohammad Mahdi Giahi, Mohammad Erfan Zarabadipour, Amir Reza Yosefian+1

Mechatronics Research Laboratory, Dept. of Computer and Electrical Engineering, Qazvin Islamic Azad University, Qazvin, Iran · Department of Electrical Engineering, Qa. C., Islamic Azad University, Qazvin, Iran.

Streaming ASR Speech Processing Memory-Efficient Inference Automatic Speech Recognition Voice Activity Detection

Open paper AskPDF arXiv

Real-time automatic speech recognition (ASR) systems face a fundamental trade-off between transcription accuracy and computational efficiency, particularly when deploying large-scale transformer models like Whisper. Existing streaming approaches either sacrifice accuracy through aggressive chunking or incur prohibitive memory costs through unbounded context accumulation. We present WhisperPipe, a novel streaming architecture that achieves bounded memory consumption while maintaining transcription quality through three key innovations a hybrid Voice Activity Detection (VAD) pipeline combining Silero VAD with energy-based filtering to reduce false activations by 34%, a dynamic buffering mechanism with overlapping context windows that prevents information loss at segment boundaries, and an adaptive processing strategy that balances latency and accuracy based on speech characteristics. Evaluated on 2.5 hours of diverse audio data, WhisperPipe demonstrates a median end-to-end latency of 89ms (90th percentile: 142ms) while consuming 48% less peak GPU memory and 80.9% lower average GPU utilization compared to baseline Whisper implementations. The system maintains stable memory usage over extended sessions, with zero growth rate across 150-minute continuous operation. Comparative analysis against related work shows that WhisperPipe achieves competitive accuracy (WER within 2% of offline Whisper) while operating at 3-5x lower latency than existing streaming solutions. The architecture's modular design enables deployment across resource-constrained environments, from edge devices to cloud infrastructure. Our results demonstrate that careful architectural design can reconcile the competing demands of real-time responsiveness and model sophistication in production ASR systems.