Streaming Inference

Follow this topic Ask about this topic

Latest papers

63 papers15 papers in the last four weeks, up 67% on the four weeks before. 0.1% of all new papers.

TimeOmni-Live: Continuous & Live Time Series Understanding with Long-Term Memory
Nov 6, 2026Tong Guan, Alexander Chemeris, Yiqing Guo +4Time-Series Foundation Models Streaming Inference
Griffith University · Zhejiang University · Langotime +2
Turns Matter: Advancing Streaming Video RL via Turn-Level Credit
Nov 4, 2026Shuai Dong, Kanghui Tian, Linghan Chen +20Streaming Video Understanding RL for Video Reasoning
Fudan University · Shanghai Innovation Institute · Shanghai Artificial Intelligence Laboratory +6
Live-OPSD: On-Policy Self-Distillation for Proactive Streaming Large Video Models
Oct 27, 2026Kaixuan Du, Xin Wan, Dai Guan +3On-Policy Distillation Multimodal Agents
Qwen Business Unit of Alibaba
S3-S2ST: Semantically Supervised Streaming Speech-to-Speech Translation
Oct 27, 2026Yuxiang Zhao, Zhiyun Fan, Yu Pu +11Simultaneous Speech Translation Streaming Inference
1X-LANCE Lab, School of Computer Science, Shanghai Jiao Tong University · 2Shanghai Innovation Institute · 3Alibaba Token Foundry +1
One Megakernel is Not Enough: Predicting Where to Cut for Streaming Inference
Oct 26, 2026Aditya Chatterjee, Anand Joseph X M, Sanjay Singh ChauhanEfficient Neural Network Inference Inference-Time Optimization
NVIDIA, Bangalore, India · NVIDIA, Hyderabad, India · NVIDIA, Pune, India
Shared Context, Separate Engines: Stateful Streaming Inference for Multimodal LLMs on Edge SoCs
Oct 26, 2026Yilong Li, Chengpo Yan, Yijing Zeng +3Efficient VLM Inference VLM Inference
University of Wisconsin–Madison · Meta · Carnegie Mellon University
ChronoReplay: One Answer Interface Is Not Enough for Streaming Spatial Intelligence
Oct 25, 2026Yihao Hu, Yitong Lin, Pan Wang +8Multimodal Spatial Reasoning Streaming Inference
Westlake University · ZJU · USTC +4
Efficient Streaming Audio-Visual Target Speaker Extraction for Real-World Acoustic Scenes
Oct 23, 2026Wendi Sang, Kai Li, Yifan Li +2Target Speaker Extraction Streaming Inference
School of Computer Technology and Application, Qinghai University, Xining 810016, China · Department of Computer Science and Technology, Institute for AI, BNRist, Tsinghua Laboratory of Brain and Intelligence (THBI), IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing 100084, China · Intelligent Computing and Application Laboratory of Qinghai Province, Qinghai University, Xining 810016, China +1
SyncPro: Synchronous Reasoning and Responding for Proactive Streaming Video Understanding
Oct 23, 2026Xueyun Tian, Bingbing Xu, Jiaming Liu +6Proactive Video Understanding Streaming Video Understanding
State Key Laboratory of AI Safety, Institute of Computing Technology, CAS · University of Chinese Academy of Sciences · Qwen Business Unit of Alibaba
JIT-MPE: Just-in-Time Multi-Pitch Estimation under Output Deadlines
Oct 23, 2026Yoshiyuki KobayashiMusic Information Retrieval Streaming Inference
Frieve
Cache Locally, Recompute Selectively for Streaming Video Understanding
Oct 21, 2026Junhyeok Lee, Sangyoon Lee, Jaeho LeeStreaming Video Understanding Streaming Inference
Pohang University of Science and Technology (POSTECH)
SSMCGM-Stream: A Streaming State-Space Model for Personalized Diabetes Management
Oct 20, 2026Shakson Isaac, Chirag PatelBlood Glucose Forecasting Explainable Time Series Forecasting
Department of Biomedical Informatics, Harvard Medical School
StyleStream 2.0: Fast and Controllable Streaming Voice Style Conversion
Oct 20, 2026Yisi Liu, Nicholas Lee, Gopala K. AnumanchipalliVoice Conversion Zero-Shot Voice Conversion
University of California, Berkeley
Answering While You Type: Optimizing Time-to-Answer with Streaming Generation
Oct 19, 2026Nikita Dragunov, Matvey Mikhalchuk, Anton RazzhigaevStreaming Inference Language Model Decoding
FusionBrain Lab
StreamAhead: Real-Time Streaming Video Question Answering via Adaptive Precomputation
Oct 18, 2026Haoyang Fan, Dhruv Parikh, Rajgopal Kannan +1Streaming VQA Video Question Answering
University of Southern California (USC), Los Angeles, CA, USA · 2DEVCOM Army Research Office
SENTRY: A Streaming Edge Network Trust Engine for Real-Time Audio Deepfake Detection
Oct 17, 2026Chirantan Banerjee, Soumita Chatterjee, Bhargab B. Bhattacharya +1Audio Deepfake Detection Streaming Inference
Indian Institute of Engineering Science and Technology, Shibpur, India
Accelerated Dataflow: scaling inference in streaming data analytics with efficiency
Oct 15, 2026RuiLong Jiang, Dustin Rhodes, Danny McCormick +2Streaming Inference LLM Inference Acceleration
Google, Sunnyvale, CA, USA
Foresight: planning future perception in streaming VLMs without retraining
Oct 2, 2026Ashok Prasad Neupane, Dipan Bartaula, Ankit Belbase +5Vision-Language Models Streaming Inference
Independent Researcher · NAAMII, Nepal · University College London, UK +2
OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction
Oct 1, 2026Xiangyu Zeng, Yuandong Yang, Zhiqiu Zhang +21Memory-Augmented VLMs Streaming Video Understanding
NJU
FlashBack: Knowing When to Remember in Streaming Vision-Language Models
Oct 1, 2026Yi Chen, MingMing Yu, Rui-Qi Wang +5Memory-Augmented VLMs Vision-Language Models
Rightly Robotics, A4X
StreamDecisionBench: Evaluating Decisions in Force on Evolving Language Streams
Sep 29, 2026Jhen-Ke Lin, Chung Chun WangLLM Evaluation LLM Inference
National Yang Ming Chiao Tung University
MemEvo: Automatic Discovery of Streaming Video Memory Mechanisms
Sep 29, 2026Guohong Liu, Jialei Ye, Shanhui Zhao +2Memory-Augmented VLMs Language Model-Guided Search Heuristics
Institute for AI Industry Research, Tsinghua University · Peking University
AVTR-1: Open Stack for Real-Time Interactive Avatars
Sep 19, 2026Artem Kravtsov, Dmitrii Ziganshin, Vsevolod Poletaev +4Real-Time Interactive Video Generation Audio-Video Generation
Avaturn Live
Intelligence Under Time Constraints: Rethinking Test-Time Compute
Sep 14, 2026Xiaotian ZhangStreaming Inference Inference-Time Scaling
Trooly.AI
X2Streaming-ASR: wait when uncertain, emit when ready for streaming ASR
Sep 8, 2026Zhiwei Lin, Kaiqi Fu, Rime Wen +4Streaming ASR Automatic Speech Recognition
X Square Robot
Streaming Hierarchical Inference with Tabular Foundation Models
Sep 7, 2026Vitor Crista, Afonso Lourenço, Diogo Martinho +1Tabular Foundation Models Efficient Inference
GECAD, Polytechnic of Porto, Portugal
LeanStream: A Speculate-and-Refine Streaming Framework for Efficient on-Device LLM Inference
Sep 2, 2026Renyuan Liu, Yuyang Leng, Kaiyan Liu +7LLM Inference Efficiency On-Device Language Model Inference
Richard · Global Technology Applied Research, JPMorganChase · University of Kansas +1
StreamScout: Learning When to Look Deeper for Streaming Video Understanding
Aug 31, 2026Ce Zhang, Jing Bi, Jinxi He +9Efficient VLM Inference Streaming Video Understanding
Carnegie Mellon University · TikTok · University of Rochester +2
StreamOPD: A Post-Training Recipe with Spatio-Temporal Cue Gating for Streaming Video Understanding
Aug 17, 2026Keming Wu, Baoyi Wang, Kaichen Zhang +7Streaming Video Understanding Streaming Inference
Tsinghua University · Zhejiang University · The University of Hong Kong +3
StreamTTT: Reconciling Real-Time Perception and Long-Term Memory in Streaming VLMs
Aug 13, 2026Joya Chen, Zeyun Zhong, Mike Zheng ShouMemory-Augmented VLMs Streaming Video Understanding
National University of Singapore · Karlsruhe Institute of Technology

Nov 6, 2026J/K move · Enter open · S save

TimeOmni-Live: Continuous & Live Time Series Understanding with Long-Term Memory

Tong Guan, Alexander Chemeris, Yiqing Guo, Warren Jin+3

Griffith University · Zhejiang University · Langotime+2

Time-Series Foundation Models Streaming Inference

Open paper AskPDF arXiv

Existing time series language models predominantly follow an offline, one-shot paradigm: they process fixed-length inputs independently and do not retain past states. This static approach fundamentally misaligns with the continuous nature of real-world streaming applications and fails to maintain the historical context necessary for cross-window reasoning. To bridge this gap, we introduce Continuous & Live Time Series Understanding (CL-TSU), a novel paradigm that shifts the focus toward maintaining a persistent, stateful understanding over open-ended data streams. To systematically evaluate this paradigm, we present LiveTS-Suite, the first comprehensive data suite tailored for CL-TSU, supporting data generation, model training, and systematic evaluation across 18 tasks spanning six core capabilities. It comprises over 5K streams totaling 2.2B timesteps, annotated with 15K specific response events. We further design an evaluation protocol that jointly assesses response timeliness, content accuracy, and computational efficiency. Building on this foundation, we propose TimeOmni-Live, the first live time series understanding model. It integrates a hybrid time series encoding for natively temporal input, a stream-persistent hybrid memory mechanism to effectively preserve exact historical evidence and long-term task states, and an elastic stream-inference synchronization strategy to reduce processing lag. Evaluations show that TimeOmni-Live establishes a strong baseline for CL-TSU, outperforming frontier LLMs (90.0 vs. 79.3 for Opus 5) with only 4B parameters and achieving 206

\times

lower p95 latency than its backbone. Deployment on a Jetson edge device further shows its ability to support real-time continuous temporal understanding locally at the edge.