Streaming Video Understanding

Follow this topic Ask about this topic

Momentum

9 papers in the last four weeks, up 200% on the four weeks before. 0.1% of all new papers.

Jul 6Week of Sep 21

Latest papers 48

All topics

OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction
Oct 1, 2026Xiangyu Zeng, Yuandong Yang, Zhiqiu Zhang +21Streaming Video Understanding Streaming Video
NJU
MEMO: Multi-Level Entity-Aware Memory for Streaming Video Understanding
Sep 30, 2026Yinying Li, Yuqian Fu, Yulin Dai +3Streaming Video Understanding Streaming Video
East China Normal University · King Abdullah University of Science and Technology
Beyond the Timeline: Augmenting Long-Video Memory with Grounded Entity Biographies
Sep 29, 2026Hui Ren, Lei Fan, Henry Pao +5Long Video Question Answering Streaming Video Understanding
University of Illinois Urbana-Champaign · Amazon.com, Inc.
When to Retrieve, When to Stay: Uncertainty-Aware Temporal Evidence Allocation for Streaming Video-LLMs
Sep 29, 2026Xiang Hu, Jiazuo Yu, Lu Zhang +2Streaming Video Understanding Relevance
IIAU Lab, Dalian University of Technology
Watch-Think-Interact: Bootstrapping Long-Horizon Multi-Turn Streaming Video Reasoning with Reinforcement Learning
Sep 29, 2026Ziheng Huang, Yicheng Bao, Xueheng Li +8Streaming Video Understanding Long-Horizon Task Planning
East China Normal University · University of Science and Technology of China · Xiamen University
EGSD: Event-Grounded Self-Distillation for Streaming Video Understanding
Sep 29, 2026Yuwei Miao, Xuesheng Zhang, Wenhao Zou +5Streaming Video Understanding Streaming
Baidu · University of Chinese Academy of Sciences · China University of Petroleum, Beijing
MemEvo: Automatic Discovery of Streaming Video Memory Mechanisms
Sep 29, 2026Guohong Liu, Jialei Ye, Shanhui Zhao +2Streaming Video Understanding Bounded-Memory
Institute for AI Industry Research, Tsinghua University · Peking University
Compress to Remember: Learning Compact Memory via On-Policy Distillation for Long Video Generation
Sep 28, 2026Xiaoyu Wu, Weihang Guo, Yifei Wang +3Long Video Generation Streaming Video Understanding
Carnegie Mellon University · Rice University
Sprout: Building Dynamic Memory While Reasoning for Agentic Video Understanding
Sep 28, 2026Wei Chen, Xuanyu Zheng, Yancheng Long +6Video Understanding Video Agent
HKUST · Kling AI
SubjectAnchor: Subject-Aware Memory-to-Video for Multi-Shot Storytelling
Sep 28, 2026Xinyu Wang, Huafeng Shi, Zian Li +4Interactive Video Generation Streaming Video Understanding
Shenzhen International Graduate School, Tsinghua University · Kling Team · Peking University +1
Perceptual Refinement of an End-to-End Video Streaming Pipeline via Generative AI Layers
Sep 16, 2026Emanuele Artioli, Farzad Tashtarian, Christian TimmererRate-Distortion Theory Video Restoration
Christian Doppler Laboratory ATHENA, Institute of Information Technology (ITEC), Alpen-Adria-Universität Klagenfurt, Austria
Omni-Streaming Thinking
Sep 14, 2026Enjun Du, Siyi Liu, Ziyu Zheng +4Omni-Modal Audio-Visual Reasoning
1The University of Hong Kong · 2The Hong Kong University of Science and Technology (Guangzhou) · University of Sussex +2
Does Video Memory Use What It Retrieves? A Causal Audit of Memory Specificity
Sep 14, 2026Aditi Tiwari, Akshit Bhalla, Darshan Prasad +1Peak Memory Streaming Video Understanding
University of Illinois Urbana-Champaign · Adobe Research
LiveProBench: Can Streaming Video Models Really Interact Like Humans?
Sep 11, 2026Kaixuan Du, Xin Wan, Hang Zhang +4Streaming Video Understanding Proactive
School of Automation Science and Electrical Engineering, Beihang University · Qwen Business Unit of Alibaba
Caption-once, Frames-on-Demand: Visual-Need Routing for Budget-Aware Agentic Long Video Understanding
Sep 11, 2026Weitong Cai, Hang Zhang, Yukai Huang +6Video Understanding Streaming Video Understanding
Queen Mary University of London · Independent Researcher · Durham University +2
Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding
Sep 3, 2026Hongyu Qu, Guangming Yao, Ling Xing +7Streaming Video Understanding Modern Data-Streaming Systems
Nanjing University of Science and Technology · Ant Group · National University of Singapore +1
CoFiE: Coarse-to-Fine Evidence Selection for Efficient Streaming Video Understanding
Sep 3, 2026Jing Jiang, Yiran Ling, Ruonan Li +2Streaming Video Understanding Vision Encoders
Harbin Institute of Technology · State Key Laboratory of Smart Farm Technologies and Systems · Pengcheng Laboratory
StreamScout: Learning When to Look Deeper for Streaming Video Understanding
Aug 31, 2026Ce Zhang, Jing Bi, Jinxi He +9Streaming Video Understanding Bounded-Memory
Carnegie Mellon University · TikTok · University of Rochester +2
Dynamic Hub-and-Spoke Memory for Streaming Video Understanding
Aug 31, 2026Xinru Jiang, Lin Zhao, Xi Xiao +7Streaming Video Understanding Long Videos
Northeastern University · University of Alabama at Birmingham · Tulane University +2
StreamFlow: Dynamic Memory Flows for Streaming Video Understanding
Aug 11, 2026Muxin Fu, Yifan Zhang, Wentao Zhang +5Streaming Video Understanding Long-Video Benchmarks
Tongji University · Nanyang Technological University · University of Michigan +2
StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understanding
Aug 6, 2026Xichen Zhang, Guankai Li, Yinghao Zhu +6Streaming Video Understanding Streaming Video
The Hong Kong University of Science and Technology · Xiaohongshu Inc. · The University of Hong Kong +1
StreamDAM: Presence-Aware Memory for Real-Time Streaming Video Object Segmentation
Aug 4, 2026Xiang ChenVideo Object Segmentation Streaming Video Understanding
Global College Shanghai Jiaotong University Shanghai, China
ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understanding
Jul 30, 2026Mingkang Dong, Muxin Pu, Jie Li +8Streaming Video Understanding Video Understanding
Universiti Malaya · Monash University · SJTU +5
Surprise Forcing: What to Remember, When to Skip in Long Video Generation
Jul 20, 2026Shuwei Shi, Zhen Li, Muyao Niu +4Long Video Generation Autoregressive Diffusion
Alaya Lab · The University of Tokyo
VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understanding
Jul 16, 2026Xinhao Li, Yuhan Zhu, Xiangyu Zeng +24Video Understanding Streaming Video Understanding
Nanjing University · Nanyang Technological University · Shanghai AI Laboratory +1
FOLIO: Focused Semantic Memory for Streaming Video Understanding
Jul 14, 2026Haoyang Fan, Dhruv Parikh, Anvitha Ramachandran +4Streaming Video Understanding Video Understanding
University of Southern California (USC), Los Angeles, CA, USA · Intel Labs, USA · DEVCOM Army Research Office
SLVMBench: Skill Learning from Video Memory
Jul 13, 2026Yudong Yang, Guangzhi Sun, Yixuan Li +1Streaming Video Understanding Skills
Tsinghua University · University of Cambridge
ProtoKV: Streaming Video Understanding under Delayed Query with Summary-State Memory
Jun 25, 2026Le Tu Ngoc Minh, Jinyeong Lim, Dongsu HanStreaming Video Understanding Key-Value Cache
KAIST · School of Electrical Engineering, KAIST, Daejeon, Republic of Korea · Kim Jaechul Graduate School of AI, KAIST, Daejeon, Republic of Korea
Towards a Dynamic and Fixed-budget Memory Bank for Efficient Streaming Video Understanding
Jun 24, 2026Baiyang Song, Yuli Lin, Qiong Wu +5Streaming Video Understanding Video Understanding
1Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China
Next-Turn: Duration-Aware Streaming Endpoint Detection via Time-to-Next-Speech-Onset Prediction
Jun 16, 2026Tristan Tsoi, Jiajun Deng, Yingke Zhu +5Turn-Taking Streaming Video Understanding
Central Media Technology Institute, Huawei · The Chinese University of Hong Kong · Nanyang Technological University
LiveStarPro: Proactive Streaming Video Understanding with Hierarchical Memory for Long-Horizon Streams
Jun 16, 2026Zhenyu Yang, Kairui Zhang, Bing Wang +2Streaming Video Understanding Streaming Video
What Should a Streaming Video Model Remember?
Jun 15, 2026Haonan Ge, Yiwei Wang, Hang Wu +1Streaming Video Understanding Streaming
University of California, Santa Barbara · University of California, Merced · The University of Queensland
FlowNar: Scalable Streaming Narration for Long-Form Videos
May 30, 2026Zeyun Zhong, Manuel Martin, Chengzhi Wu +4Streaming Video Understanding Large Multimodal Models
Karlsruhe Institute of Technology (KIT) · Fraunhofer IOSB · Lamarr Institute for Machine Learning and Artificial Intelligence +1
SlotMemory: Object-Centric KV Memory for Streaming Long-Video Generation
May 29, 2026Weijia Dou, Hui Li, Jiahao Cui +3Interactive Video Generation Streaming Video Understanding
Fudan University · Meta Superintelligence Labs · Baidu
OmniMem: Scalable and Adaptive Memory Retrieval for Long Video Generation
May 28, 2026Lin Zhao, Yushu Wu, Yifan Gong +2Autoregressive Video Generation Long Video Generation
Northeastern University · Adobe Research
StreamOV: Streaming Omni-Video Understanding via Evidence-Guided Memory and Response Triggering
May 25, 2026Ming Xie, Zizheng Huang, Xudong Tan +6Streaming Video Understanding Streaming Video
Shanghai Innovation Institute · Fudan University · Nanjing University +2
OmniPro: A Comprehensive Benchmark for Omni-Proactive Streaming Video Understanding
May 18, 2026Ruixiang Zhao, Jie Yang, Zijie Xin +4Omni-Modal Audio-Visual Reasoning
Renmin University of China · WeChat Vision, Tencent Inc.
An Efficient Streaming Video Understanding Framework with Agentic Control
May 18, 2026Jinming Liu, Jianguo Huang, Zhaoyang Jia +7Streaming Video Understanding Streaming Video
1Shanghai Jiao Tong University · 2Eastern Institute of Technology, Ningbo, China · 3Microsoft Research Asia
CoRDS: Coreset-based Representative and Diverse Selection for Streaming Video Understanding
May 14, 2026Ailar Mahdizadeh, Puria Azadi, Muchen Li +2Streaming Video Understanding Key-Value Cache Compression
University of British Columbia · Vector Institute
StreamPro: From Reactive Perception to Proactive Decision-Making in Streaming Video
May 11, 2026Ao Li, Zihan Xiao, Zihao Yue +7Streaming Video Understanding Streaming Video
1AIM3 Lab, Renmin University of China · 2MiLM Plus, Xiaomi Inc. · 3Peking University
Semantic-Aware Adaptive Visual Memory for Streaming Video Understanding
May 8, 2026Hang Wu, Sherin Mary Mathews, Yujun Cai +2Streaming Video Understanding Modern Data-Streaming Systems
University of California, Merced · US Bank · University of Queensland
Response-G1: Explicit Scene Graph Modeling for Proactive Streaming Video Understanding
May 8, 2026Ke Ma, Jiaqi Tang, Bin Guo +8Streaming Video Understanding Video Understanding
Northwestern Polytechnical University · The Hong Kong University of Science and Technology · Tsinghua University +1
Decouple and Cache: KV Cache Construction for Streaming Video Understanding
May 3, 2026Zhanzhong Pang, Dibyadip Chatterjee, Fadime Sener +1Streaming Video Understanding Streaming Video
National University of Singapore.
Progressive Online Video Understanding with Evidence-Aligned Timing and Transparent Decisions
Apr 20, 2026Kecheng Zhang, Zongxin Yang, Mingfei Han +6Streaming Video Understanding Streaming
University of Science and Technology of China · Harvard University · Department of CV, MBZUAI +3
OASIS: On-Demand Hierarchical Event Memory for Streaming Video Reasoning
Apr 18, 2026Zhijia Liang, Jiaming Li, Weikai Chen +3Streaming Video Understanding Streaming
Sun Yat-sen University · OPPO AI Center · Shenzhen Loop Area Institute +1
Em-Garde: A Propose-Match Framework for Proactive Streaming Video Understanding
Mar 19, 2026Yikai Zheng, Xin Ding, Yifan Yang +6Streaming Video Understanding Streaming
Institute for AI Industry Research (AIR), Tsinghua University · University of Science and Technology of China · 4Microsoft Research Asia +1
Representation Recycling for Streaming Video Analysis
Apr 28, 2022Can Ufuk Ertenli, Ramazan Gokberk Cinbis, Emre AkbasStreaming Video Understanding Rt-Detr
Department of Computer Engineering, Middle East Technical University (METU), Ankara, Turkey · Center for Robotics and Artificial Intelligence (ROMER), Middle East Technical University (METU), Ankara, Turkey
StreamTTT: Reconciling Real-Time Perception and Long-Term Memory in Streaming VLMs
Date pendingJoya Chen, Zeyun Zhong, Mike Zheng ShouStreaming Video Understanding Long Video Question Answering
National University of Singapore · Karlsruhe Institute of Technology

Oct 1, 2026, cs.CVJ/K move · Enter open · S save

OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction

Xiangyu Zeng, Yuandong Yang, Zhiqiu Zhang, Yuhan Zhu+20

NJU

Streaming Video Understanding Streaming Video Proactive Factual Recall Perception

Open paper AskPDF arXiv

Streaming video LLMs must retain evidence before its relevance to future tasks is known and respond when sufficient evidence becomes available. The challenge is to form reusable factual memory without compromising real-time perception. We introduce OneStreamer, which jointly learns query-independent evidence recording and task response through a shared proactive generation process. Its Proactive Hierarchical Caption Memory (PHCM) produces time-grounded local-detail captions and summaries of completed events. Streaming caption targets supervise the interpretation of observed video prefixes during training. At inference, model-generated records complement a recent visual window, providing reusable factual context without revisiting historical visual features. Proactive State Transition Learning (PSTL) reduces the dominance of repeated waiting states by preserving supervision at all output anchors and selecting representative state-change and state-persistence tokens. We further develop a streaming data synthesis pipeline that aligns output content and timing with available evidence. Combining the resulting streaming captions and QA with cleaned open-source data yields OneStreamer-1M, a broad-coverage streaming video interaction dataset with over one million records spanning diverse tasks. Our 4B model achieves the best results among the compared methods across all eight evaluated streaming video understanding benchmarks. Ablations show that retaining generated captions improves historical QA without degrading real-time perception. PSTL also outperforms dense state supervision while supervising only 27.5% of annotated state tokens. Together, these results support proactive generation as a shared learning interface connecting perception, memory formation, and timely response in streaming video interaction.

Momentum

Related topics

Latest papers 48