A live esports broadcast needs a director who chooses which camera goes on air and a caster who talks over that choice, every second and without knowing what comes next. Streaming video language models have learned to commentate, but only over a broadcast that a human director has already produced. We study live broadcast production, in which a model receives the raw camera of every player and must both direct and commentate from the past alone. We present DirCast, a single streaming video language model that does both. Every second it scans all cameras at low resolution, decides with a hold-or-switch director whether and where to cut, and reads the chosen camera at high resolution before it speaks. Its context keeps what went on air and what was said and lets the other views expire, which bounds the cost of a stream of any length. We build a dataset of 27 professional Dota 2 matches with a camera for every hero, the human director's choices and the casters' commentary. DirCast directs better than rule-based and zero-shot directors including GPT-5, and its commentary wins 29.2 percent of comparisons against GPT-4o under the Proact-VL protocol, against 19.4 percent for the strongest of eleven streaming commentators that watch the human broadcast.
Figures & tables
Figure 2: Framework. Each second a low-res ViT encodes the ten feeds with their state, the director holds or cuts, the chosen feed is read at high resolution, and the commentator speaks. Low-res views expire while the program stays.
Feed agreement
Cuts
Coverage
Director
Top-1 ↑
Tol. ↑
Any-vis. ↑
P ↑
F1 ↑
/min
Kill ↑
Fight ↑
Rules
Random (human rate)
10.6
12.9
20.1
6.4
5.8
6.9
20.9
19.1
Static
17.1
20.4
23.0
0.0
0.0
0.0
20.9
19.2
Action rule
25.4
30.3
36.1
13.9
11.5
5.8
78.5
62.4
Zero-shot VLMs
Table 1: Test-split directing. A cut is correct if within two seconds of a human cut and to the same feed. Fight coverage measures seconds with a fighting hero on air. Cuts/min should approach the human rate of 7.9. DirCast averages three seeds. Best bold, second underlined.
LiveU
FinalQ
Method
Venue
CC ↑
F1 ↑
TimeDiff ↓
Time
Rate
TextU
Fidelity
Contin.
Subst.
PAUC ↑
Zero-shot, human program
LiveCC-7B-Base [ 8 ]
CVPR’25
19.38
64.61
8.32
5.11
4.99
4.53
4.56
3.17
2.36
0.008
LiveCC-7B-Instruct [ 8 ]
CVPR’25
17.73
79.95
1.84
5.75
6.15
5.69
4.01
3.19
2.59
0.015
Dispider [ 28 ]
CVPR’25
6.70
0.69
30.64
1.36
1.94
1.64
4.60
1.34
1.15
0.000
MMDuet [ 39 ]
EMNLP’25
3.81
16.99
9.79
3.13
1.87
2.99
5.74
2.25
1.52
0.001
Table 2: Streaming commentary on the test split under the Proact-VL protocol, judged by GPT-5. CC is the win rate against GPT-4o. † Fine-tuned on our training split. DirCast is the mean of three seeds.
Cost
Dir.
Commentary
Director
Comm.
GB ↓
Time ↓
Top-1 ↑
CC ↑
F1 ↑
TD ↓
Dir-only
Com-only
37.8
0.53
25.17
24.36
76.73
0.31
Ours
Com-only
37.8
0.56
26.56
29.04
76.64
0.31
Ours
Ours
18.9
0.32
27.00
29.21
78.54
0.30
Human
Com-only
–
–
–
25.67
76.23
0.33
Human
Ours
–
–
–
27.84
77.48
0.30
Table 3: Joint versus separate models. TD is TimeDiff.
Directing
Commentary
Variant
Top-1 ↑
Any-vis. ↑
CC ↑
TimeDiff ↓
DirCast
27.00
41.80
29.21
0.30
w/o hold-or-switch pointer
21.65
35.93
–
0.31
w/o state tokens
22.59
37.98
29.21
0.32
w/o program rollouts
22.90
38.01
28.35
0.30
w/o event-anchored targets
26.10
43.06
–
0.32
Table 4: Component ablations on the full test split.
Time (s)
Context
Mean ↓
P95 ↓
Last 36 s ↓
KV ↓
GB ↓
Top-1 ↑
Speech
Every view kept
0.60
1.05
0.86
92.6k
32.0
23.7
67.2
Program memory
0.42
0.68
0.55
47.6k
25.1
24.6
68.1
+ 72 s horizon
0.36
0.58
0.38
12.8k
19.8
24.6
70.1
+ 36 s horizon
0.29
0.52
0.29
6.9k
18.9
25.1
76.7
Table 5: Streaming eight 288-second windows on one GPU. Time: compute per video second; KV: final cache length.
Figure 3: Accuracy vs. training matches.
Figure 4: Four kills that DirCast shows and the human crew misses. Each panel shows DirCast’s program and commentary (top), the cut gate against its threshold (middle), and the human broadcast (bottom). Framed shots are cuts, and the shaded column marks the kill: (a) KotL kills Shadow Fiend off-air, (b) Clockwerk kills Rubick, (c) Doom kills Terrorblade, and (d) Terrorblade kills Tusk.
Split
Matches
Clips
Verified clip-s
Strict clip-s
Speech
Words
Train
20
2,096
50,283
35,815
64%
142.0k
Val
3
320
9,627
7,357
50%
16.3k
Test
4
496
14,630
11,283
72%
38.0k
Table 6: Dataset statistics. Clips are 36 s long with a stride of 18 s, so clip-seconds and words count overlapping clips; unique footage totals 15.2 h. Speech is the share of clip-seconds with polished commentary.
Clip-seconds
Train
Val
Test
All
75,456
11,520
17,856
Live broadcast
63,374
11,143
16,238
without HUD-only labels (training mask)
59,574
10,977
15,894
after verification (verified mask)
50,283
9,627
14,630
Observed on screen (strict mask)
35,815
7,357
11,283
Table 7: Cleaning funnel. Bold marks the seconds used for training and for evaluation. Clip-seconds count overlapping clips.
Figure 5: Additional qualitative examples of off-air kills that DirCast shows and names (1–10 of 20).
Figure 6: Additional qualitative examples of off-air kills that DirCast shows and names (11–20 of 20).