Recent multimodal large language models (MLLMs) achieve strong results on video question answering, yet often fail to recover the temporal structure that makes a video coherent. They may merge neighboring events, miss single-event progression, or infer implausible relations across events. We study structured temporal understanding as the ability to recover event structure from visual evidence, including event boundaries, single-event progression, and multi-event relations. We argue that this gap arises from a mismatch between current post-training objectives and temporal reasoning, where caption-style supervision allows models to rely on language shortcuts rather than visual temporal grounding. To address this, we propose EventLens, a vision-centric reinforcement learning framework that learns video understanding through event-structure recovery. Instead of reconstructing captions, EventLens trains models to recover temporal structure under controlled perturbations, such as altered temporal granularity, reversed progression, and shuffled event order. We instantiate this framework with a three-level temporal ontology and derive three verifiable task families: event segmentation, progression discrimination, and multi-event relation reasoning. These tasks admit deterministic rewards, enabling task-specialized GRPO training without human preference labels or learned reward models. We further introduce multi-teacher on-policy distillation (MT-OPD) to consolidate specialized policies into a unified model. Experiments on Qwen3-VL backbones show consistent improvements on temporal grounding and event-sensitive reasoning benchmarks, while preserving general video-QA performance. Learning curves further demonstrate that EventLens achieves comparable downstream transfer with significantly reduced training cost compared to direct mixed RL optimization.
Figures & tables
Figure 1: Motivation and task view of unified event-structure recovery. Existing video MLLMs often suffer from boundary ambiguity, progression ambiguity, and relation ambiguity when they must recover explicit temporal structure. We address these bottlenecks with three event-structure recovery tasks: event-boundary recovery, single-event progression discrimination, and multi-event relation reasoning through next-event prediction, fill-in, and narrative reordering.
Figure 2: Overview of EventLens. EventLens adopts a two-stage training pipeline for event-structure reinforcement learning. In Stage I, deterministic rewards train task-specialized GRPO teachers for event-boundary localization, single-event progression discrimination, and multi-event relation reasoning. In Stage II, MT-OPD routes each student trajectory to the corresponding teacher and distills dense token-level teacher logits into one deployable student. The teacher is selected by recovery family, so each student trajectory is supervised by the teacher trained with the matching verifier.
Figure 3: Scene-first construction of event structure for a curling news report. (a) Sampled frames and candidate shots. (b) KEEP retains a shot as one event, MERGE combines consecutive shots into one event, and SPLIT introduces event boundaries within a shot. (c) L1 phases contain L2 events, which contain L3 actions. Frames come from the historical report illustration; boundaries and event assignments are schematic.
Model
Temporal Grounding
Complex Reasoning
QV Highlights
Activity Net
Charades STA
Video Holmes
Video MMMU
Video TT
E.T. Bench
Qwen3-VL-4B Backbone
Qwen3-VL-4B [ 4 ]
54.31
41.99
43.64
39.96
55.44
36.90
34.55
+ SFT-Desc
55.63
42.18
43.52
41.26
55.00
41.70
32.57
+ SFT-DVC
57.13
42.36
40.18
41.97
56.22
41.20
37.78
+ EventLens
58.50
45.76
47.99
40.28
55.56
38.80
41.47
Table 1: Temporal grounding and complex reasoning under the shared 256-frame protocol. Temporal grounding uses mIoU; reasoning columns use official benchmark scores. Bold marks the best score within each backbone group.
Model
General Video QA
Long Video QA
MV Bench
Temp Compass
AoT Bench
Vino Ground
Video MME
MLVU
LongVideo Bench
Qwen3-VL-4B Backbone
Qwen3-VL-4B [ 4 ]
62.55
74.96
53.64
37.00
65.00
56.77
60.70
+ SFT-Desc
64.62
76.35
54.39
41.40
64.30
56.77
61.30
+ SFT-DVC
64.70
75.93
54.12
40.20
65.00
58.17
62.10
+ EventLens
63.90
75.12
53.99
45.00
65.30
58.37
61.60
Table 2: General and long-video QA under the shared 256-frame protocol. Columns use official benchmark scores. Bold marks the best score within each backbone group.
Model
Charades STA
QV Highlights
Vino Ground
E.T. Bench
MV Bench
LongVideo Bench
Qwen2.5-VL-7B [ 5 ]
38.34
27.66
29.40
30.96
63.38
59.20
Video-R1-7B [ 16 ]
28.80
22.33
31.20
29.50
60.27
56.20
EventLens-7B
42.70
43.79
41.00
35.85
64.00
58.60
Table 3: Comparison with existing video post-training methods on Qwen2.5-VL-7B. Charades-STA and QVHighlights report mIoU; VinoGround reports the Text score and E.T. Bench the average score.
Variant
TimeLens
E.T. Bench
Video MME
Vino Ground
Video TT
Holmes
MLVU
LongVideo Bench
avg mIoU
Avg
Avg
Score
Score
Acc
Avg
Avg
Base
46.65
34.55
65.00
37.00
36.90
39.96
56.77
60.70
R1 teacher
50.84
40.34
65.30
39.20
38.20
40.12
58.76
61.10
R2 teacher
50.76
40.94
65.40
45.40
39.50
40.61
59.76
61.80
R3 teacher
50.96
39.87
65.00
37.80
37.20
39.19
57.57
61.60
Single-teacher oracle
50.96
40.94
65.40
45.40
39.50
40.61
59.76
61.80
Table 4: Teacher consolidation and task-composition ablation on Qwen3-VL-4B. R1–R3 denote segmentation, progression, and relation teachers. Full MT-OPD is EventLens with all three teachers; the single-teacher oracle selects the best individual score per metric.
Variant
R1 Seg.
R2 Prog.
R3 Rel.
Base
0.399
0.550
0.555
R1+R2
0.589
0.680
0.574
R1+R3
0.592
0.590
0.708
R2+R3
0.444
0.700
0.705
Full MT-OPD
0.588
0.685
0.706
Table 5: Direct validation of recovery capabilities on Qwen3-VL-4B. R1–R3 denote segmentation, progression, and relation scores.
Figure 4: Wall-clock efficiency of EventLens versus direct Mixed-GRPO on four representative transfer metrics. The curves track MT-OPD student optimization and direct Mixed-GRPO training.
Training annotations
VinoGround Group
MVBench
LongVideo Bench
Charades mIoU
QVHighlights mIoU
E.T. Bench Avg.
Instruction base
37.00
62.55
60.70
43.64
54.31
34.55
Clean
39.00
63.38
61.20
45.70
58.16
38.20
10% jitter
38.60
63.20
61.30
45.61
57.99
37.92
20% jitter
38.60
63.40
61.00
45.42
57.58
37.71
Table 6: Robustness to timestamp perturbation in event supervision on Qwen3-VL-4B. Training uses 100 GRPO steps and 2,000 samples per recovery family; percentages denote perturbed chains. VinoGround reports the Group score.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Recovery view
Constructed recovery task
Deterministic target
Event-boundary recovery
Recover phase-, event-, or action-level time spans requested by the prompt.
Event intervals; balanced by level and domain
Single-event progression
Decide whether an event-level clip is played forward or backward.
Direction label; balanced by playback direction and domain
Multi-event relation
Recover the next event, a missing middle event, or the order of shuffled clips.
Choice or ordering; balanced by relation subtask and domain
Appendix
Table 7: Event-structure recovery views used for post-training.
Figure 5: Domain composition of the input videos used to construct proxy-task instances.
\fnum@figure : Additional E.T. Bench qualitative cases covering repeated action localization, temporal event matching, and grounded video QA. Across the three settings, the base model often recognizes the coarse visual category or plausible answer but misaligns it with the relevant temporal evidence, while EventLens with MT-OPD preserves finer event boundaries and support intervals.
\fnum@figure : Additional wall-clock transfer curves complementing the four representative metrics in Fig. 4 .