Streaming Inference

Follow this topic Ask about this topic

Latest papers

45 papers7 papers in the last four weeks, up 40% on the four weeks before. 0.0% of all new papers.

Don't Pause! Every prediction matters in a streaming video
Apr 27, 2026Dibyadip Chatterjee, Zhanzhong Pang, Fadime Sener +2Streaming Video Understanding Efficient VLM Inference
National University of Singapore · Google Inc.
CodecSight: Leveraging Video Codec Signals for Efficient Streaming VLM Inference
Apr 7, 2026Yulin Zou, Wenyan Chen, Yan Chen +6Efficient VLM Inference VLM Inference
NTU Singapore Singapore · Beihang University China · A*STAR IAIC Singapore +1
A Simple Baseline for Streaming Video Understanding
Apr 2, 2026Yujiao Shen, Shulin Tian, Jingkang Yang +1Streaming Video Understanding Streaming Inference
S-Lab, Nanyang Technological University
STRIDE: When to Speak Meets Sequence Denoising for Streaming Video Understanding
Mar 29, 2026Junho Kim, Hosu Lee, James M. Rehg +2Streaming Video Understanding Streaming Inference
UIUC · KAIST · Google DeepMind
Stream2LLM: Overlap Context Streaming and Prefill for Reduced Time-to-First-Token (TTFT)
Mar 29, 2026Rajveer Bachkaniwala, Chengqi Luo, Richard So +2LLM Inference LLM Inference Serving
Georgia Tech, Atlanta, USA
Em-Garde: A Propose-Match Framework for Proactive Streaming Video Understanding
Mar 19, 2026Yikai Zheng, Xin Ding, Yifan Yang +6Streaming Video Understanding Efficient VLM Inference
Institute for AI Industry Research (AIR), Tsinghua University · University of Science and Technology of China · 4Microsoft Research Asia +1
AvatarForcing: One-Step Streaming Talking Avatars via Local-Future Sliding-Window Denoising
Mar 15, 2026Liyuan Cui, Wentao Hu, Wenyuan Zhang +3Audio-Video Generation Video Diffusion Models
Zhejiang University · Kling Team, Kuaishou Technology · Beijing University of Posts and Telecommunications +1
ActionPlan: Future-Aware Streaming Motion Synthesis via Frame-Level Action Planning
Mar 13, 2026Eric Nazarenus, Chuqiao Li, Yannan He +3Diffusion Models for Motion Generation Text-to-Motion Generation
Tübingen AI Center, University of Tübingen, Germany · Max Planck Institute for Informatics, Saarland Informatics Campus, Germany
Thinking in Streaming Video
Mar 13, 2026Zikang Liu, Longteng Guo, Handong Li +7Streaming Video Understanding Streaming Inference
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · OPPO AI Center, OPPO Inc.
StreamReady: Learning What to Answer and When in Long Streaming Videos
Mar 9, 2026Shehreen Azad, Vibhav Vineet, Yogesh Singh RawatStreaming Video Understanding Video QA
Center for Research in Computer Vision, University of Central Florida · Microsoft Research
Think-as-You-See: Streaming Chain-of-Thought Reasoning for Large Vision-Language Models
Mar 3, 2026Jialiang Zhang, Junlong Tong, Junyan Lin +4VLM Reasoning Streaming Inference
Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Ocean University of China · Shanghai Jiao Tong University +3
Boosting AI Reliability with an FSM-Driven Streaming Inference Pipeline: An Industrial Case
Mar 2, 2026Yutian Zhang, Zhongyi Pei, Yi Mao +3Streaming Inference Finite-State Machine
School of Software, BNRist, Tsinghua University, China · Tianyi Technology Co., Ltd, China
WeaveTime: Stream from Earlier Frames into Emergent Memory in VideoLLMs
Feb 25, 2026Yulin Zhang, Cheng Shi, Sibei YangVideo-Language Models Streaming Video Understanding
ShanghaiTech University · School of Computing and Data Science, The University of Hong Kong · School of Computer Science and Engineering, Sun Yat-sen University
Incremental Transformer Neural Processes
Feb 21, 2026Philip Mortimer, Cristiana Diaconu, Tommy Rochussen +2Streaming Inference Transformer
University of Cambridge · Helmholtz AI · Technical University of Munich +2
Representation Recycling for Streaming Video Analysis
Apr 28, 2022Can Ufuk Ertenli, Ramazan Gokberk Cinbis, Emre AkbasStreaming Video Understanding Video Object Detection
Department of Computer Engineering, Middle East Technical University (METU), Ankara, Turkey · Center for Robotics and Artificial Intelligence (ROMER), Middle East Technical University (METU), Ankara, Turkey

Apr 27, 2026J/K move · Enter open · S save

Don't Pause! Every prediction matters in a streaming video

Dibyadip Chatterjee, Zhanzhong Pang, Fadime Sener, Yale Song+1

National University of Singapore · Google Inc.

Streaming Video Understanding Efficient VLM Inference Video QA Adaptive Computation Streaming Inference

Open paper AskPDF arXiv

Streaming video models should respond the moment an event unfolds, not after the moment has passed. Yet existing online VideoQA benchmarks remain largely retrospective. They pause the video at fixed timestamps, pose questions about current or past events, and score models only at those moments. This protocol leaves streaming predictions untested. To close this gap, we introduce SPOT-Bench, featuring multi-turn proactive queries that evaluate general streaming perception and assistive capabilities required by an always-on, real-time assistant. SPOT-Bench comes with Timeliness-F1, a consolidated metric that measures streaming predictions by their temporal precision and balanced coverage across the entire video. Our benchmark reveals: (i) offline models detect events reliably but spam predictions unprompted; (ii) post-training for silence reduces spamming but induces unresponsiveness; (iii) half of the streaming video expects no response, which we term dead-time - compute spent here does not affect response latency. These findings motivate AsynKV, a training-free streaming adaptation of offline models, that retains their event perception while improving their streaming behavior. AsynKV features a long-short term memory, utilized efficiently by scaling compute during dead-time. It serves as a strong baseline on SPOT-Bench, outperforming existing streaming models, and achieves state-of-the-art on retrospective benchmarks.