StreamSoccer: Event-Driven Memory for Streaming Soccer Commentary

Chenxi Shao; Bozhong Wang; Jiaxin Huang; Zhao Liu; Sunwei Zhu; Tianxin Hang; Gaoqi He; Yang Li

cs.CVAug 20, 2026

StreamSoccer: Event-Driven Memory for Streaming Soccer Commentary

Authors: Chenxi Shao, Bozhong Wang, Jiaxin Huang, Zhao Liu, Sunwei Zhu, Tianxin Hang, Gaoqi He, Yang Li,

Organizations: East China Normal University, Shanghai, China · Migu Video Technology Co., Ltd., Shanghai, China · South China University of Technology, Guangzhou, Guangdong, China

Abstract

Streaming video understanding requires models to causally update state as video arrives and organize growing history into semantic units that can evolve, persist, and be recalled under bounded computation and memory. This challenge is pronounced in live soccer commentary, where a system must describe completed events, summarize recent play, recall earlier events, or remain silent using only information available before each utterance. We present StreamSoccer, an event-driven system that uses event memory as its intermediate representation. A fixed-budget active memory integrates the stream; completed event states are retained locally and consolidated into retrievable historical records. A unified generator uses current, recent, and historical context to produce three commentary modes, while a rule-assisted scheduler selects a mode or silence. Unlike streaming video-language models organized around frames, visual tokens, or caches, and soccer-commentary methods based on predefined clips or output timestamps, StreamSoccer explicitly models event lifecycles. We construct a three-track streaming soccer commentary dataset and a layered evaluation protocol. At common reference anchors, StreamSoccer obtains CIDEr scores of 38.62, 23.96, and 17.39 for current-event, recent-window, and historical-memory commentary, ranking first on the current-event and historical-memory tracks and second on recent-window. Controlled ablations show that local completed events improve all tracks and that the full system performs best on all three. Across 174 raw-video runs on 58 matches, per-minute RTF p95 ranges from 0.10 to 0.22 without sustained growth with match history. These results indicate that event memory supports streaming soccer commentary across temporal scopes while controlling long-history computation.

Figures & tables

Figure 2: Overview of StreamSoccer. Incoming clips are encoded as event tokens $E_{t}$ and update the active event state. When an event closes, its completed memory $M_{j}^{\mathrm{c}}$ supports current-event commentary, enters the Recent Event Buffer, and is converted into Historical Event Records. The rule-assisted schedule selects a commentary mode or silence. For generation, selected latent states are projected into $Z_{t}$ , while retrieved records $\mathcal{R}_{t}$ are provided as text.

Method	Frames	Stream	Soccer FT	Current-event			Recent-window			Historical-memory
Method	Frames	Stream	Soccer FT	CIDEr	B@4	BS	CIDEr	B@4	BS	CIDEr	B@4	BS
Proprietary MLLMs
GPT-5.4	16			5.93	0.0215	0.8586	14.28	0.0585	0.8784	7.75	0.0405	0.8723
Qwen3.6-Max-Preview	16			1.00	0.0092	0.8613	3.23	0.0131	0.8576	3.95	0.0194	0.8655
Streaming VLMs
StreamingVLM (ICLR2026)	2 FPS	$\checkmark$		0.7351	0.0018	0.4337	0.0532	0.0015	0.4398	0.0313	0.0000	0.4464

Table 1: Commentary quality at shared causal output anchors. We report CIDEr, BLEU-4 (B@4), and BERTScore-F1 (BS) for current-event, recent-window, and historical-memory commentary; higher is better. CIDEr uses the

100\times

scale. Bold and underlined values denote the best and second-best result for each metric within each track.

Figure 3: Raw per-minute RTF p95 over 58 source-clean matches and 174 runs (lower is better). TimeChat-Online stops at its context limit.

Representation	Tok-F1	B@4	METEOR	ROUGE-L	CIDEr
Recurrent memory	0.2642	0.2777	0.2358	0.4335	37.40
Event pooling	0.2290	0.2556	0.2192	0.3986	22.20
Fixed-time pooling	0.2108	0.2462	0.2104	0.3916	15.72

Table 2: Matched projector-only comparison for memory-to-record consolidation.

Figure 4: Normalized commentary-conditioned temporal relevance from a frozen offline VLM over a continuous five-minute match interval. Responses persist across adjacent windows and shift among action concepts as play evolves.

StreamSoccer variants
Variant	Current	Recent	Historical
Direct Qwen3-VL	4.06	2.46	3.08
Qwen3-VL Video-SFT	27.73	12.98	11.24
Current memory only	33.03	15.06	13.22
$\hskip 8.50012pt+$ Local event-memory buffer	38.57	17.96	17.22
$\hskip 8.50012pt+$ Historical records (Full)	38.62	23.96	17.39

Table 3: Track-wise CIDEr for direct-video adaptation and event-memory scope.

Figure 5: Historical-memory commentary at a common causal anchor. The masked future stream is unobserved.

Appendix figures & tables27 assets

Supplementary material from the paper’s appendix.

Appendix

Paper claim	Primary evidence	Supported interpretation	Outside the experiment’s scope
Commentary quality at shared causal anchors	Reference-anchored streaming replay and common-anchor baselines	Conditional generation quality at fixed legal output anchors for the three commentary tracks	Free speaking-time selection, naturally available event records, or a purely visual front end
Availability-aware semantic preference at shared anchors	Frozen four-way blinded LLM audit against silver references	Reference-conditioned judge preference over the fixed 3,789-anchor population, including missing-output penalties	Human preference, claim-level factuality, historical-record attribution, free speaking-time selection, or judge-model generalization
Event memory is a useful intermediate representation	Matched recurrent-memory, operational-event-pooling, and fixed-time-pooling comparison	Downstream event-record caption decodability under a matched language readout and adaptation budget	Natural event discovery, full record factuality, or equal upstream training cost
Accessible memory scope changes track-specific commentary quality	Separately trained memory-scope variants in the main text	Observed differences between configurations with different accessible memory scopes and training targets	Statistical significance of every incremental gain or a causal decomposition of every component
Long-history processing remains efficient	Raw-video per-minute RTF scaling over full matches	Growth of end-to-end processing workload as more match history is observed	Scheduler quality or the response latency of an individual commentary utterance

Appendix

Table A.1: Claim–evidence map for the headline results. Each row deliberately limits the interpretation to the corresponding evaluation protocol.

Setting	Visual input and state	Output time / task	Event closure	Action/type metadata	Current-event alignment	Record readiness	Main use
Sample-level conditional	Preconstructed or cached sample; no full-half replay required	Given by sample	Controlled or materialized	Controlled fields	Given by sample IDs	Preconstructed evidence context	Conditional generation and matched ablations
Reference-anchored replay	Continuous cached Qwen tokens at 2 FPS; state persists within each half	Reference anchor and task type	Oracle operational closures	Label-assisted action/type fields	Just-completed event aligned to the current-event target	Historical anchors deterministically drain pending caption jobs before retrieval	Main-paper common-anchor commentary quality
Free-scheduler replay	Continuous cached Qwen tokens; state persists within each half	Rule-assisted scheduler	Predicted transition plus duration rollover	Current audited replay is label-assisted	None from reference targets	Natural asynchronous readiness	Cadence, mode counts, delay, duplicates, and readiness audit
Raw-video scaling	Raw video decoding, online visual encoding, persistent state	Fixed evaluation workload; separate from native free scheduling	Predicted transition plus duration rollover	Predicted front-end metadata	None from reference targets	Runtime-managed records	Per-minute RTF, VRAM, coverage, and history scaling

Appendix

Table A.2: Evaluation protocol matrix. Table 1 uses the reference-anchored row. It evaluates commentary quality under controlled anchors; it does not evaluate the speaking scheduler or natural record readiness.

Method family	Input	Stream state	Soccer FT	Anchor/interface	Coverage treatment	Contract
Proprietary MLLMs	16 frames	No	No	Track-specific causal sparse frames at each anchor	One request per test anchor; invalid calls must be reported	Compatibility
General streaming VLMs	2 FPS	Yes	No	Continuous causal timeline with anchored scoring	Metrics are conditioned on valid outputs; context-limit and invalid-output coverage are disclosed separately	Compatibility
Soccer-specific models	30 frames	No	Yes	Causal sparse frames at each anchor	Retrained current/recent rows use the match-disjoint split; historical rows are test-only sparse-history diagnostics	Current/recent stricter; historical compatibility
StreamSoccer	2 FPS	Yes	Yes	Reference-anchored continuous replay	3,789/3,789 anchors emitted; interventions disclosed in Table A.2	Controlled reference replay

Appendix

Table A.3: Protocol cards for the method families in the main commentary-quality table. FT denotes task-specific fine-tuning.

Metric	Definition	Interpretation boundary
Token F1	Harmonic mean of token-level precision and recall against the reference	Direct lexical-content overlap; insensitive to unsupported facts expressed with overlapping words
BLEU-4	Modified $1$ - to $4$ -gram precision with a brevity penalty	Local phrase agreement; brittle to valid paraphrases and not a factuality metric
METEOR	Unigram alignment combining precision, recall, and an ordering penalty	More recall-sensitive than BLEU, but still reference-dependent
ROUGE-L	Longest-common-subsequence overlap between prediction and reference	Sequence-level content coverage and ordering, not historical-evidence use
CIDEr	TF–IDF-weighted $n$ -gram agreement; displayed on the $100\times$ scale	Main caption-style content score; scales and scorer versions must not be mixed
BERTScore-F1	Contextual-token similarity under the shared RoBERTa-large layer-17 scorer	Semantic reference similarity; does not establish factual correctness

Appendix

Table A.4: Metric definitions and permitted interpretations. No automatic reference-similarity metric is treated as a direct factuality measure.

Data level	Count
SoccerNet matches / half streams	500 / 1,000
Non-overlapping 4-s clips	685,903
Rule-derived operational events	166,419
MatchTime matches used	422
Caption-aligned operational events	15,189
Current-event samples	15,189

Appendix

Table B.1: Frozen data funnel. Commentary alignment covers a salient subset of all operational events rather than a uniform sample of match states.

Split	Matches	Current	Recent	Historical	Total
Train	295	10,853	5,013	3,775	19,641
Validation	64	2,286	1,105	818	4,209
Test	63	2,050	1,009	730	3,789
Total	422	15,189	7,127	5,323	27,639

Appendix

Table B.2: Match-disjoint three-track split.

Group	SoccerNet actions	Operational role
Set piece	Corner, Throw-in, Direct free-kick, Indirect free-kick, Penalty, Kick-off	Closes an existing state before $C_{t}$ and starts a new event at $C_{t}$
Finish	Goal, Shots on target, Shots off target	Included in the active event, then closes it as terminal_finish
Stoppage	Ball out of play, Foul, Offside	Included in the active event, then closes it as terminal_stoppage
Defensive resolution	Clearance	Closes after $C_{t}$ only when the next clip contains no set piece, finish, or stoppage follow-up
Administration / discipline	Substitution, Yellow card, Red card, Yellow-to-red card, End of period	Starts a new event unless a card coincides with an active stoppage; closes after inclusion
Open play	No action from the groups above	Continues the active state; its type is upgraded when the first meaningful action arrives

Appendix

Table B.3: Deterministic action groups and their operational-event effects. These rules organize memory lifecycles; they do not define natural event ground truth.

Track	Construction contract
Current-event	Context: just-completed $M_{j}^{\mathrm{c}}$ and causal match state. Target: one post-event commentary sentence. Provenance: current event ID, cutoff, and alignment metadata.
Recent-window	Context: active context and completed local events from the preceding 180 s. Target: one recent multi-event summary. Provenance: selected local event IDs and cutoff.
Historical-memory	Context: current/local context plus eligible earlier same-match, same-half records. Target: one sentence relating current play to earlier context. Provenance: historical source IDs, candidate set, and cutoff.

Appendix

Table B.4: Construction contract for the three commentary tracks. Every conditioning item must exist no later than the stored causal cutoff.

Figure B.2: Causal commentary-data construction. Action timestamps and deterministic rules define operational-event intervals; MatchTime supplies the main three-track branch and SoccerNet-Caption the Stage 2 supervision branch. Candidate pools and split assignments are fixed before model-assisted rewriting, while selected evidence IDs are validated before the final match-disjoint manifests.

Component / symbol	Reported value	Specification
Backbone	Qwen3-VL-8B-Instruct	Frozen visual encoder and frozen base LLM
Visual/LLM hidden size $d_{v},d_{l}$	4096 / 4096	Pretrained token spaces
Event-memory hidden size $d_{m}$	1024	Adapter, active memory, and completed event memory
Clip Event Adapter $\mathcal{A}$	2 layers, 8 heads	Cross-attention resampler; $K_{m}=9$ output tokens
Memory update $\mathcal{U}$	2 layers, 8 heads	Memory-to-clip attention, self-attention, FFN, and per-token gate $G_{t}$
Memory initializer $\mathcal{I}$	1 layer	Initializes a new active state from $E_{t}$

Appendix

Table C.1: Architecture specification for the paper-facing configuration.

Stage	Trainable path	Per-process batch / accumulation	Learning rate and schedule	Main objective / budget
1: streaming event memory	Clip Event Adapter, event-memory encoder, transition and auxiliary event heads	1 / 16	$10^{-4}$ ; 500-step warmup; AdamW	Teacher-forced reset; up to 50k steps; weights 0.5 clip action, 1.0 transition, 1.0 completed-event action set, 0.3 event type
2: memory-to-record	Memory projector and language LoRA; base LLM frozen	1 / 16	Projector $10^{-4}$ , LoRA $10^{-5}$ ; 500-step warmup	Assistant caption tokens only; projector alignment then LoRA warmup and joint fitting; target length 192
3: multi-context commentary	Language LoRA, memory projector, route/local/retrieval auxiliary heads; Stage-1 memory cached/frozen in the reported run	2 / 8 per process	LoRA $2\!\times\!10^{-5}$ , projector $3\!\times\!10^{-5}$ ; 500-step warmup	Three-track response CE with auxiliary context losses; 10k configured budget; prompt/target/generation limits 2048/128/96

Appendix

Table C.2: Three-stage training configuration. The displayed batch size is per process. Stage 3 uses prompt profile paper_clean_v2 and seed 42 for the configured sampler.

Runtime parameter	Value
Clip duration / sampling	4 s / 2 FPS
Local event-memory buffer horizon / maximum selected	180 s / 4 events
Historical retrieval	embedding top-3
Historical eligibility	same half, gap $\geq 90$ s
Current-event cooldown	12 s
Recent-window tick / minimum events	120 s / 3

Appendix

Table C.3: Default runtime and rule-assisted scheduler settings. The replay in Table 1 fixes output anchors and therefore does not evaluate these scheduler thresholds.

ID	Result	Step / seed	Population
ANCHOR-S1K	Main-paper StreamSoccer commentary-quality row	1,000 / 42	63 matches; 3,789 anchors
REP-MEM	Recurrent event memory representation row	3,000 / 42	49 matches; 1,726 events
REP-EVENT	Operational-event pooling row	3,000 / 42	49 matches; 1,726 events
REP-FIXED	Fixed-time pooling row	3,000 / 42	49 matches; 1,726 events

Appendix

Table C.4: Stable artifact labels for the protocol and representation results in Appendices A–D.

Representation	Tok-F1	B@4	METEOR	ROUGE-L	CIDEr
Recurrent memory	0.2642	0.2777	0.2358	0.4335	37.40
Event pooling	0.2290	0.2556	0.2192	0.3986	22.20
Fixed-time pooling	0.2108	0.2462	0.2104	0.3916	15.72

Appendix

Table D.1: Matched memory-to-record representation comparison on 1,726 events from 49 matches. CIDEr uses the

100\times

scale.

Tok-F1 difference	Mean	95% paired CI
Memory $-$ event pooling	0.0351	[0.0259, 0.0444]
Memory $-$ fixed pooling	0.0533	[0.0448, 0.0625]
Event pooling $-$ fixed pooling	0.0182	[0.0114, 0.0252]

Appendix

Table D.2: Tok-F1 differences from 10,000 paired match-cluster bootstrap replicates.

Track	$N$	Valid	Tok-F1	B@4	METEOR	ROUGE-L	CIDEr	BS
Current-event	2,050	100%	0.3234	0.2929	0.2487	0.4606	38.62	0.9734
Recent-window	1,009	100%	0.3499	0.0652	0.1518	0.2680	23.96	0.9612
Historical-memory	730	100%	0.2292	0.0451	0.1441	0.2146	17.39	0.9636

Appendix

Table E.1: Additional StreamSoccer commentary metrics under reference-anchored replay. CIDEr uses the

100\times

scale, BS denotes BERTScore-F1, and valid coverage is the fraction of requested anchors producing a non-empty output.

Method	Current	Recent	Historical	Overall [95% CI]	Response
StreamSoccer	90.73	96.63	95.21	93.16 [92.32, 94.01]	100.00
StreamingVLM	8.68	3.37	4.38	6.44 [5.65, 7.21]	93.77
TimeChat-Online	0.59	0.00	0.41	0.40 [0.21, 0.61]	41.20
VideoLLM-online	0.00	0.00	0.00	0.00 [0.00, 0.00]	64.61

Appendix

Table E.2: Strict win rate and response rate (%) in the frozen blinded LLM semantic audit. Current, recent, and historical contain 2,050, 1,009, and 730 anchors. Overall intervals use 10,000 match-cluster bootstrap replicates over 63 matches; missing outputs remain in the fixed denominator and cannot win.

Configuration	Current	Recent	Historical
Current memory only	33.03	15.06	13.22
$+$ local event-memory buffer	38.57	17.96	17.22
Full three-scope model	38.62	23.96	17.39

Appendix

Table E.3: Verified CIDEr (

100\times

) results for the separately trained memory-scope configurations.

Historical input	Records	Tok-F1	B@4	CIDEr	BS
No historical record	0	0.1426	0.0169	2.61	0.8543
Matched hard-negative top-3	3	0.2724	0.0726	30.92	0.8922
Runtime-ranked top-3 †	3	0.2682	0.0697	30.15	0.8906
Oracle-ID top-3 ‡	3	0.2819	0.0841	37.96	0.8947

Appendix

Table E.4: Automatic overlap diagnostics for the four-arm historical-record intervention. All arms use the identical 209-anchor population and have 100% valid coverage. CIDEr uses the

100\times

scale and BS denotes BERTScore-F1. † The learned NULL decision is bypassed only to fix the three-record budget. ‡ The positive record is placed at rank 1; this is a controlled oracle, not native online performance.

Method	T15	T30	T45	T90	Peak VRAM
StreamingVLM	0.274	0.285	0.291	0.285	17,072 MiB
TimeChat-Online *	0.138	0.231	0.322	–	77,603 MiB
VideoLLM-online	0.171	0.259	0.328	0.559	50,969 MiB
StreamSoccer	0.110	0.119	0.119	0.122	17,996 MiB

Appendix

Table F.1: Raw-video efficiency. T15–T90 are p95 cumulative RTFs after 15, 30, 45, and 90 minutes of observed match time. Peak VRAM is the p95 value at the last valid horizon. * TimeChat-Online has no T90 value because its supported context ends before that horizon.

Component	p95 RTF
Raw video decode	0.0053
Visual encoding	0.0273
State update	0.0040
External generation wall	0.0201
Native generation wall	0.0692

Appendix

Table F.2: Diagnostic component-wise p95 RTFs for the frozen raw-video StreamSoccer workload. Prompt construction, post-processing, and residual overhead are not separately tabulated.

Audit item	Value
Historical-memory anchors	730
Anchors with pending jobs (forced / shadow)	594 / 603
Reference-triggered drain batches	594
Unique force-drained caption jobs	1,269
Eligible candidate set changed	0 / 730
Retrieved record identities changed	0 / 730

Appendix

Table F.3: Paired record-readiness audit of the 730 historical-memory anchors. The forced condition reproduces reference-anchored replay; the natural-queue shadow disables only anchor-triggered caption draining.

Audit item	Value
Half streams / clips	126 / 86,145
Commentary outputs	5,949
Comments per minute	1.0505
Current / recent / historical	2,976 / 2,820 / 153
Caption jobs	20,658
Ready / empty captions	20,654 / 4

Appendix

Table F.4: Continuous free-scheduler execution audit. The scheduler is rule-assisted and the run retains label-assisted action/type metadata.

Panel of Figure 31 — Figure G.1: Five qualitative rows, each containing one frozen case from the current-event, recent-window, and historical-memory commentary tracks. The three column labels define the task type; no event timestamps or timelines are shown. Full frozen outputs, sample IDs, and coloring reasons are retained in the accompanying case-card audit. Rows 1–3 are shown here and the figure continues on the following pages.

Panel of Figure 32 — Figure G.2: Qualitative gallery, continued (rows 4–5).

Artifact ID	Paper-facing role and frozen evidence records
ANCHOR-S1K	Reference-anchored StreamSoccer quality and retrieval diagnostics. Records: protocol and content hashes; command; traces; predictions; evaluation summary
REP-MEM	Recurrent event-memory representation result. Records: checkpoint hash; predictions; scores; paired bootstrap
REP-EVENT	Operational-event pooling result. Records: checkpoint hash; predictions; scores; paired bootstrap
REP-FIXED	Fixed-time pooling result. Records: checkpoint hash; predictions; scores; paired bootstrap
SCOPE-CUR	Current-memory-only configuration. Records: replay command and evaluation summary
SCOPE-LOCAL	Current plus local event-memory buffer configuration. Records: replay command and evaluation summary

Appendix

Table H.1: Reproduction map for the frozen results and audits. Each stable ID names the evidence records supporting the corresponding paper-facing result.

Artifact ID	Paper-facing role and frozen evidence records
RAW-EWL	Raw-video efficiency, coverage, and stage timing. Records: run manifest; per-request timings; aggregate tables
FREE-S7K	Free-scheduler continuous-execution audit. Records: traces; queue audit; outputs; summary
HIST-CF209	Four-arm historical-record intervention. Records: protocol; 209-anchor manifest; 836 outputs; evaluation summary; match-cluster bootstrap
READY-730	Paired record-readiness audit. Records: protocol; 730 paired rows; drain traces; audit summary
BLIND-3789	Four-way blinded LLM semantic audit. Records: protocol and source hashes; 3,789 permuted ballots; decoded judgments; match-cluster bootstrap
FAIL-S10K	Step-10,000 reference-anchored mechanical failure audit. Records: commentary and record outputs; character, repetition, and encoding audit; aggregate counts

Appendix

Table H.2: Reproduction map, continued.

Streaming Video Understanding

Streaming video understanding studies how AI systems interpret continuous video in real time while retaining useful history and responding at the right moment.

See all 132 papers

Mar 31, 2026

Towards Automatic Soccer Commentary Generation with Knowledge-Enhanced Visual Reasoning

Soccer commentary plays a crucial role in enhancing the soccer game viewing experience for audiences. Previous studies in automatic soccer commentary generation typically adopt an end-to-end method to generate anonymous live text commentary. Such generated commentary is insufficient in the context of real-world live televised commentary, as it contains anonymous entities, context-dependent errors and lacks statistical insights of the game events. To bridge the gap, we propose GameSight, a two-stage model to address soccer commentary generation as a knowledge-enhanced visual reasoning task, enabling live-televised-like knowledgeable commentary with accurate reference to entities (players and teams). GameSight starts by performing visual reasoning to align anonymous entities with fine-grained visual and contextual analysis. Subsequently, the entity-aligned commentary is refined with knowledge by incorporating external historical statistics and iteratively updated internal game state information. Consequently, GameSight improves the player alignment accuracy by 18.5% on SN-Caption-test-align dataset compared to Gemini 2.5-pro. Combined with further knowledge enhancement, GameSight outperforms in segment-level accuracy and commentary quality, as well as game-level contextual relevance and structural composition. We believe that our work paves the way for a more informative and engaging human-centric experience with the AI sports application. Demo Page: https://gamesight2025.github.io/gamesight2025

Zeyu Jin, Xiaoyu Qin, Songtao Zhou +2

Department of Computer Science and Technology, Tsinghua University · BNRist, Tsinghua University

Grounded Text Generation Sports Video Analysis Video Reasoning

Ask PDF

Aug 10, 2026

NBA_Streaming: A Large-Scale Benchmark for Fine-Grained Basketball Commentary Generation in Continuous Streams

Live basketball commentary generation requires determining when an event is sufficiently observable and describing it before subsequent events unfold. However, existing methods are primarily designed for pre-segmented clips or complete videos, making them unsuitable for continuous streams. Existing datasets also provide limited supervision for player identities, fine-grained actions, event attributes, and coherent event chains, restricting the factual richness of generated commentary. To address these limitations, we introduce NBA_Streaming, a large-scale benchmark for online fine-grained basketball commentary generation. It contains 307.5 hours of basketball broadcasts and approximately 35K temporally aligned events, with annotations of event boundaries, player identities, fine-grained actions, event chains, and natural-language commentary. By moving from isolated clips to continuous streams, NBA_Streaming enables unified evaluation of event localization, response reliability, factual grounding, and commentary quality under causal constraints. We further propose a causal two-stage framework that combines completion-first localization with ball-centric semantic grounding, enabling the system to identify complete events from observed streams and organize scene, event, identity, and action cues for commentary generation. Extensive experiments reveal the difficulty of NBA_Streaming, where existing baselines struggle with online timing, factual grounding, and fine-grained description. Our framework consistently improves over strong alternatives, while the remaining gap highlights NBA_Streaming as a valuable benchmark for streaming sports video understanding and generation. The dataset is publicly available at https://github.com/wyy-081/NBA_Streaming.

Lifang Wu, Yuyang Wu, Yangdong Gao +3

School of Information Science and Technology, Beijing University of Technology, Beijing 100124, China · Fudan University, Shanghai, China · Institute of Automation, Chinese Academy of Sciences, Beijing 100864, China

Temporal Video Grounding Streaming Video Understanding Sports Video Analysis

Ask PDF

Jun 16, 2026

LiveStarPro: Proactive Streaming Video Understanding with Hierarchical Memory for Long-Horizon Streams

Despite the remarkable progress of Video Large Language Models (Video-LLMs), current online architectures still struggle to simultaneously process continuous video streams, decide autonomously when to respond, and preserve long-horizon contextual memory. These obstacles undermine real-time responsiveness and cause severe forgetting throughout prolonged interactions. In this work, we introduce LiveStarPro, a live streaming assistant that is designed for proactive video understanding over long-horizon streams. The design of LiveStarPro rests on three complementary components. The first component is Streaming Verification Decoding (SVeD), an inference framework that identifies the appropriate response timing through single-pass perplexity verification, thereby eliminating the dependency on explicit silence tokens. The second component is Streaming Causal Attention Masks (SCAM), a training strategy that enforces incremental video-language alignment over variable-length streams. The third component is Tree-Structured Hierarchical Memory (TSHM), a recursive memory architecture that organizes evicted historical information into event chains and consequently enables efficient retrieval from effectively unbounded video streams. To facilitate a comprehensive evaluation under realistic online conditions, we further present OmniStarPro, a large-scale benchmark that spans 15 diverse real-world scenarios and that extends to hour-scale streams for the assessment of long-term recall. Extensive experiments demonstrate that LiveStarPro consistently surpasses existing methods, attaining a 28.9% improvement in semantic correctness and an 18.2% reduction in timing error, while its streaming key-value cache further yields a 1.58x inference speedup over the same model without caching. The model and the code are publicly available at https://github.com/sotayang/LiveStarPro.

Zhenyu Yang, Kairui Zhang, Bing Wang +2

Streaming Video Understanding Long-Video Understanding Hierarchical Memory VLM Inference

Ask PDF

Abstract

Streaming Video Understanding

Explore similar work

Towards Automatic Soccer Commentary Generation with Knowledge-Enhanced Visual Reasoning

NBA_Streaming: A Large-Scale Benchmark for Fine-Grained Basketball Commentary Generation in Continuous Streams

LiveStarPro: Proactive Streaming Video Understanding with Hierarchical Memory for Long-Horizon Streams