We present a structured temporal video reasoning pipeline built around a discrete EventGraph, a continuous EventField, and a human-readable EventGlyph view. On a calibrated EPIC-KITCHENS subset of 10 videos and 50 temporal reasoning questions, EventField+Glyph achieves 0.98 overall accuracy, which is higher than the caption baseline by +0.40 (paired p = 1.1 \times 10^{-5}) and direct VLM-only QA by +0.20 (p = 0.0063) on this subset. We further evaluate annotation-source variations, including manual, heuristic, and heuristic+Gemini pipelines, and find that the best structured method stays above the caption baseline across settings. We also include cross-video pair benchmarking and an appendix gallery of glyph outputs for all studied videos. Overall, the results indicate that structured temporal representations can support both performance and inspectability by preserving symbolic structure, capturing temporal continuity, and providing human-readable diagnostics for video reasoning.
Figures & tables
Method
Overall Acc.
Macro Avg.
N
caption
0.580
0.580
50
event_graph
0.800
0.800
50
eventfield
0.780
0.780
50
eventfield_glyph
0.980
0.980
50
vlm_only
0.780
0.780
50
Table 1: Main method comparison on EPIC10 calibrated subset.
Figure 1: Overall accuracy comparison across methods.
Pair
Δ Acc. (A-B)
95% CI
p-value
caption vs event_graph
-0.220
[-0.360, -0.080]
0.0074
caption vs eventfield
-0.200
[-0.340, -0.060]
0.0213
caption vs eventfield_glyph
-0.400
[-0.540, -0.240]
0.0000
caption vs vlm_only
-0.200
[-0.380, -0.020]
0.0525
event_graph vs eventfield
0.020
[0.000, 0.060]
1.0000
event_graph vs eventfield_glyph
-0.180
[-0.300, -0.060]
0.0117
Table 2: Pairwise significance (paired bootstrap CI and McNemar exact p-value).
Figure 2: Operator view for debugging: event graph, EventField, and Glyph from one clip.
Figure 3: Task-wise accuracy by method.
Source
Best method
Best acc.
Caption acc.
Δ vs caption
p-value
Avg events/video
Avg canonical actions/video
manual
eventfield_glyph
0.980
0.580
0.400
0.0000
35.2
24.7
heuristic
eventfield_glyph
0.940
0.660
0.280
0.0026
113.6
2.0
heuristic_gemini
eventfield_glyph
0.920
0.720
0.200
0.0309
46.3
14.2
Table 3: Annotation-source variants on EPIC10 (10 videos, 50 tasks).
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: EventGlyph gallery page 1: P01_13, P02_15, and P06_12.
Figure 5: EventGlyph gallery page 2: P06_14, P07_12, and P07_16.
Figure 6: EventGlyph gallery page 3: P08_14 and P09_08.
Figure 7: EventGlyph gallery page 4: P11_19 and P18_01.
Video reasoning requires a fine-grained understanding of the temporal dependencies and event-level relations between objects and events in videos. Current Multimodal Large Language Models (MLLMs) are prone to severe temporal hallucinations in video reasoning. An underlying cause of these hallucinations is weak visual-temporal grounding and the lack of explicit structure for modelling event relations. Models often rely on auxiliary text, such as dense captions, rather than explicitly anchoring their reasoning in actual visual evidence. However, these textual representations are inherently unstructured and fail to provide explicit causal constraints needed to guide the model's reasoning. In this work, we propose GraphThinker, a reinforcement finetuning method that constructs a structured event representation of a video and enforces visual grounding to jointly reduce reasoning hallucinations. Specifically, we employ an MLLM to construct an Event-based Video Scene Graph (EVSG) that captures both intra- and inter-event relations, guiding a structured video reasoning process. Moreover, we address the weak grounding issue by introducing a novel visual attention reward during reinforcement finetuning that encourages the model to actively attend to reliable visual cues. On the RexTime dataset, GraphThinker achieves an over 4% improvement in IoU=0.3 for moment localisation. On the VidHalluc dataset, GraphThinker achieves a 9.8% improvement in reducing temporal sequence hallucination and a 7.6% gain in Binary QA in reducing action hallucination, compared to the state-of-the-art methods.
Zixu Cheng, Da Li, Jian Hu +4
Queen Mary University of London · Samsung AI Centre Cambridge · Shanghai Artificial Intelligence Laboratory +1
TimeLogicQA evaluates whether video question answering systems can reason over temporal relations such as event existence, ordering, persistence, boundary conditions, and overlap. We address this task with a visual evidence routing pipeline that separates perception from symbolic temporal reasoning. The system first parses each question into event targets, answer mode, candidate options, and temporal operators. It then routes videos according to duration and operator difficulty, using ordered full-frame evidence for short clips and event-focused candidate windows for long videos. A multimodal large language model produces structured visual evidence for the relevant events, while programmatic verifiers recover dense action intervals and a deterministic reducer applies operator-specific temporal rules to produce the final answer. Conservative fusion accepts an answer only when the visual evidence, temporal program, and confidence checks agree, reducing noisy answer flips. On the official test evaluation, our final system achieves an AvgAcc of 81.8.
Yuyang Sun, Yongliang Wu, Xingyu Zhu +8
Southeast University · National University of Singapore · Independent Researcher +2
Reasoning temporal localization (RTL) requires a model to generate an answer that itself contains the time interval supporting it, so high-level reasoning and precise temporal grounding must be produced jointly in a single response. To tackle this challenging task, we propose the first event-centric video chain-of-thought framework, dubbed EventCoT. EventCoT first performs event-centric tokenization of the input video to convert it into compact event tokens, enabling efficient identification of question-relevant events. It then reasons within the identified events to generate the answer, grounding the time interval via embedding matching that aligns placeholder tokens with visual embeddings. EventCoT achieves state-of-the-art results on ActivityNet-RTL for reasoning temporal localization while using substantially fewer visual tokens than previous work. To verify its general performance, we further evaluate EventCoT on the grounded video question answering benchmark ReXTime, where it attains strong zero-shot results.
Youngkil Song, Yoonjae Baek, Dongwon Kim +3
1Pohang University of Science and Technology, Pohang, South Korea · 2Korea Advanced Institute of Science and Technology, Daejeon, South Korea · 3Handong Global University, Pohang, South Korea