Towards Automatic Soccer Commentary Generation with Knowledge-Enhanced Visual Reasoning

Zeyu Jin; Xiaoyu Qin; Songtao Zhou; Kaifeng Yun; Jia Jia

cs.MMMar 31, 2026

Towards Automatic Soccer Commentary Generation with Knowledge-Enhanced Visual Reasoning

Authors: Zeyu Jin, Xiaoyu Qin, Songtao Zhou, Kaifeng Yun, Jia Jia

Organizations: Department of Computer Science and Technology, Tsinghua University · BNRist, Tsinghua University

Abstract

Soccer commentary plays a crucial role in enhancing the soccer game viewing experience for audiences. Previous studies in automatic soccer commentary generation typically adopt an end-to-end method to generate anonymous live text commentary. Such generated commentary is insufficient in the context of real-world live televised commentary, as it contains anonymous entities, context-dependent errors and lacks statistical insights of the game events. To bridge the gap, we propose GameSight, a two-stage model to address soccer commentary generation as a knowledge-enhanced visual reasoning task, enabling live-televised-like knowledgeable commentary with accurate reference to entities (players and teams). GameSight starts by performing visual reasoning to align anonymous entities with fine-grained visual and contextual analysis. Subsequently, the entity-aligned commentary is refined with knowledge by incorporating external historical statistics and iteratively updated internal game state information. Consequently, GameSight improves the player alignment accuracy by 18.5% on SN-Caption-test-align dataset compared to Gemini 2.5-pro. Combined with further knowledge enhancement, GameSight outperforms in segment-level accuracy and commentary quality, as well as game-level contextual relevance and structural composition. We believe that our work paves the way for a more informative and engaging human-centric experience with the AI sports application. Demo Page: https://gamesight2025.github.io/gamesight2025

Figures & tables

Fig. 1: Overview of the proposed GameSight . It incorporates visual and contextual information to generate soccer commentary that is entity-aligned and knowledge-enhanced.

Fig. 2: Unlike previous end-to-end models, GameSight is a two-stage system that addresses soccer commentary generation as a knowledge-enhanced visual reasoning task. It revises and enriches the anonymized commentary with Stage I : visual reasoning for entity alignment, and Stage II : commentary refinement with knowledge.

Fig. 3: Soccer game view shots distribution in time length and view shots examples.

Fig. 4: Human reliance for the “Player Guessing” game.

Fig. 5: Distribution of commentary at event level.

Model	$Player@1$	$@3$	$Team$	$Player@1_{c}$	$@3_{c}$	$Team_{c}$
Qwen2.5-VL	35.4	39.3	70.9	43.8	48.6	91.4
InternVL3	26.3	39.1	72.2	30.8	47.4	92.0
VideoLLaMA3	17.2	27.5	63.7	19.4	36.4	83.7
LLaVA-OV	18.7	18.9	66.7	21.7	21.9	87.9
Gemini 2.5-pro	52.6	55.9	71.2	61.3	62.4	96.9
GPT-4o	21.2	24.0	49.5	37.7	45.0	98.0

TABLE I: Zero-shot performance of video-LMMs and LLMs. Bolded and underlined for the 1st and 2nd largest value.

Training Strategy	$Player$ $\uparrow$	$Team$ $\uparrow$	$Player_{c}$ $\uparrow$	$Team_{c}$ $\uparrow$
Qwen2.5-VL #1	35.9	69.6	43.9	92.5
#1 + SFT	63.2	81.7	67.0	91.6
#1 + GRPO	43.7	72.9	50.7	92.7
#1 + SFT+GRPO (Ours)	71.1	84.7	75.9	88.7

TABLE II: Experiment results of fine-tuning on Qwen2.5-VL.

Live text as GT
Model	BLEU	METEOR	ROUGE	CIDEr
MatchVoice	15.182	22.142	18.262	13.514
MatchVision	28.311	25.827	25.173	27.143
GameSight (ours)	20.320	27.160	27.968	0.887
Live televised	4.038	11.778	6.272	0.001
Live televised as GT

TABLE III: Comparison with baselines.

	$SoccerRAG$	$SoccerKAG$	$ICL_{goal}$	$ICL_{other}$
Acc. $\uparrow$	64.60	81.80	98.76	90.74

TABLE IV: Results on knowledge reference accuracy.

Method	DC $\uparrow$	AO $\uparrow$	SS $\uparrow$	MOS $\uparrow$
MatchVoice	0.556	0.164	0.159	2.27
MatchVision	0.631	0.156	0.066	2.72
Live Text	0.474	0.174	-0.059	3.14
GameSight (ours)	0.842	0.213	0.194	4.08
Live Televised	0.813	0.234	0.254	4.22

TABLE V: Results in commentary analysis. DC, AO, SS refer to Deep Cohesion , Anaphor Overlap , and Sentiment Score .

Fig. 6: Commentary structural composition in percentage.

Fig. 7: An example of CoT procedure in stage I: visual reasoning for entity alignment with GameSight .

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Fig. 8: Test set frame-level distribution of Left: grounded video segments and Right: close views with valid clues.

Fig. 9: Train set frame-level distribution of Left: grounded video segments and Right: valid close-up shots.

	All	$-C$	${\mathcal{F}}$	$-{\mathcal{J}}$	$-\mathcal{W}$	$-KeyE$	$-HistoryE$	$-$ CoT
$Player$	35.5	34.8	23.5	34.9	34.7	35.5	35.3	26.1
$Team$	69.5	68.7	63.8	70.6	68.8	71.0	69.4	65.6
$Player_{c}$	43.6	42.2	30.2	43.4	42.2	43.8	43.7	38.5
$Team_{c}$	91.9	90.7	89.6	92.9	90.7	91.6	91.5	95.4

Appendix

TABLE VI: Experiment results of the ablation test on the contextual and visual components with Qwen2.5-VL. Underlined results are above the baseline.

C

\mathcal{F}

\mathcal{J}

\mathcal{W}

refer to

Color

{\mathcal{F}ace}

{\mathcal{J}ersey}

\mathcal{W}eight

respectively.

Timestamp Version	$Player$ $\uparrow$	$Team$ $\uparrow$
MatchTime	63.2%	81.7%
SoccerNet-Caption	-1.6%	-2.6%

Appendix

TABLE VII: Experiment results with different version of timestamps.

Fig. 10: Visual Interface of the ‘Player Guessing’ Game.

	$k$ =0	$k$ =1	$k$ =3	$k$ =5	$k$ =10	All
Deep Cohesion	0.804	0.956	0.960	0.540	0.620	0.630
BLEU	17.020	17.409	16.859	17.060	16.812	16.892
METEOR	8.644	9.051	8.815	8.928	8.644	8.827
ROUGE-L	10.556	10.497	10.181	10.356	10.323	10.346
CIDEr	2.780	3.442	2.326	3.025	3.306	3.530

Appendix

TABLE VIII: Evaluation results across different

k

values.

Fig. 11: Good case by analyzing close view recognized face with player role.

Fig. 12: Good case by observation in the long view and analyzing history timeline.

Fig. 13: Good case by analyzing close view recognized face with player role.

Grounded Text Generation

Grounded text generation produces language tied to verifiable evidence, structured data, or perceptual context to limit unsupported claims.

See all 30 papers

StreamSoccer: Event-Driven Memory for Streaming Soccer Commentary
Aug 20, 2026Chenxi Shao, Bozhong Wang, Jiaxin Huang +6Streaming Video Understanding Memory-Augmented Video Understanding
East China Normal University, Shanghai, China · Migu Video Technology Co., Ltd., Shanghai, China · South China University of Technology, Guangzhou, Guangdong, China
An LLM-Based Automatic Sportscast Solution for Robot Soccer Matches
Jul 16, 2026Francesco Petri, Michele Brienza, Daniele Nardi +3Sports Video Analysis Robotics
Dept. of Computer, Control, and Management Engineering Sapienza University of Rome, Rome (Italy) · Dept. of International Humanities and Social Sciences, International University of Rome, Rome (Italy) · Institute for Cognitive Sciences and Technologies (ISTC-CNR), National Research Council, Italy +1
Real-Time Generation of Game Video Commentary with Multimodal LLMs: Pause-Aware Decoding Approaches
Mar 3, 2026Anum Afzal, Yuki Saito, Hiroya Takamura +5Language Model Decoding Multimodal Large Language Models
School of CIT, Technical University of Munich · National Institute of Advanced Industrial Science and Technology (AIST) · The University of Tokyo +3

Aug 20, 2026J/K move · Enter open · S save

StreamSoccer: Event-Driven Memory for Streaming Soccer Commentary

Chenxi Shao, Bozhong Wang, Jiaxin Huang, Zhao Liu+5

East China Normal University, Shanghai, China · Migu Video Technology Co., Ltd., Shanghai, China · South China University of Technology, Guangzhou, Guangdong, China

Streaming Video Understanding Memory-Augmented Video Understanding Sports Video Analysis

Open paper AskPDF arXiv

Streaming video understanding requires models to causally update state as video arrives and organize growing history into semantic units that can evolve, persist, and be recalled under bounded computation and memory. This challenge is pronounced in live soccer commentary, where a system must describe completed events, summarize recent play, recall earlier events, or remain silent using only information available before each utterance. We present StreamSoccer, an event-driven system that uses event memory as its intermediate representation. A fixed-budget active memory integrates the stream; completed event states are retained locally and consolidated into retrievable historical records. A unified generator uses current, recent, and historical context to produce three commentary modes, while a rule-assisted scheduler selects a mode or silence. Unlike streaming video-language models organized around frames, visual tokens, or caches, and soccer-commentary methods based on predefined clips or output timestamps, StreamSoccer explicitly models event lifecycles. We construct a three-track streaming soccer commentary dataset and a layered evaluation protocol. At common reference anchors, StreamSoccer obtains CIDEr scores of 38.62, 23.96, and 17.39 for current-event, recent-window, and historical-memory commentary, ranking first on the current-event and historical-memory tracks and second on recent-window. Controlled ablations show that local completed events improve all tracks and that the full system performs best on all three. Across 174 raw-video runs on 58 matches, per-minute RTF p95 ranges from 0.10 to 0.22 without sustained growth with match history. These results indicate that event memory supports streaming soccer commentary across temporal scopes while controlling long-history computation.

Abstract

Grounded Text Generation

Explore similar work

StreamSoccer: Event-Driven Memory for Streaming Soccer Commentary

An LLM-Based Automatic Sportscast Solution for Robot Soccer Matches

Real-Time Generation of Game Video Commentary with Multimodal LLMs: Pause-Aware Decoding Approaches