Streaming Video Understanding
Overview
Streaming video understanding asks how models can interpret frames as they arrive, answer questions about both recent and past events, and sometimes act or speak proactively. Common approaches use causal processing with bounded visual or semantic memory, compressing redundant frames and retrieving relevant evidence when a query arrives; others train models to reason incrementally or decide when they have enough evidence to respond. Benchmarks increasingly test long-range recall, fast events, multi-turn interaction, response timing, and whether a model stays silent when appropriate. A central tension is balancing accurate, grounded memory and timely responses against limits on computation, latency, and storage, while offline video performance often fails to predict real-time reliability.
Latest papers
130 papers25 papers in the last four weeks, up 79% on the four weeks before. 0.1% of all new papers.