Interpretable Temporal Video Reasoning with EventGraph and EventField

Durgendra Narayan Singh

cs.CVSep 6, 2026

Interpretable Temporal Video Reasoning with EventGraph and EventField

Authors: Durgendra Narayan Singh

Organizations: Independent Researcher Twin Falls, ID, USA

Abstract

We present a structured temporal video reasoning pipeline built around a discrete EventGraph, a continuous EventField, and a human-readable EventGlyph view. On a calibrated EPIC-KITCHENS subset of 10 videos and 50 temporal reasoning questions, EventField+Glyph achieves 0.98 overall accuracy, which is higher than the caption baseline by +0.40 (paired p = 1.1 \times 10^{-5}) and direct VLM-only QA by +0.20 (p = 0.0063) on this subset. We further evaluate annotation-source variations, including manual, heuristic, and heuristic+Gemini pipelines, and find that the best structured method stays above the caption baseline across settings. We also include cross-video pair benchmarking and an appendix gallery of glyph outputs for all studied videos. Overall, the results indicate that structured temporal representations can support both performance and inspectability by preserving symbolic structure, capturing temporal continuity, and providing human-readable diagnostics for video reasoning.

Figures & tables

Method	Overall Acc.	Macro Avg.	N
caption	0.580	0.580	50
event_graph	0.800	0.800	50
eventfield	0.780	0.780	50
eventfield_glyph	0.980	0.980	50
vlm_only	0.780	0.780	50

Table 1: Main method comparison on EPIC10 calibrated subset.

Figure 1: Overall accuracy comparison across methods.

Pair	$\Delta$ Acc. (A-B)	95% CI	p-value
caption vs event_graph	-0.220	[-0.360, -0.080]	0.0074
caption vs eventfield	-0.200	[-0.340, -0.060]	0.0213
caption vs eventfield_glyph	-0.400	[-0.540, -0.240]	0.0000
caption vs vlm_only	-0.200	[-0.380, -0.020]	0.0525
event_graph vs eventfield	0.020	[0.000, 0.060]	1.0000
event_graph vs eventfield_glyph	-0.180	[-0.300, -0.060]	0.0117

Table 2: Pairwise significance (paired bootstrap CI and McNemar exact p-value).

Figure 2: Operator view for debugging: event graph, EventField, and Glyph from one clip.

Source	Best method	Best acc.	Caption acc.	$\Delta$ vs caption	p-value	Avg events/video	Avg canonical actions/video
manual	eventfield_glyph	0.980	0.580	0.400	0.0000	35.2	24.7
heuristic	eventfield_glyph	0.940	0.660	0.280	0.0026	113.6	2.0
heuristic_gemini	eventfield_glyph	0.920	0.720	0.200	0.0309	46.3	14.2

Table 3: Annotation-source variants on EPIC10 (10 videos, 50 tasks).

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 4 — Figure 4: EventGlyph gallery page 1: P01_13, P02_15, and P06_12.

Panel of Figure 5 — Figure 5: EventGlyph gallery page 2: P06_14, P07_12, and P07_16.

Panel of Figure 6 — Figure 6: EventGlyph gallery page 3: P08_14 and P09_08.

Panel of Figure 7 — Figure 7: EventGlyph gallery page 4: P11_19 and P18_01.

Video Reasoning

Video reasoning studies how AI systems use video evidence to infer events, actions, temporal relations, causes, and future states.

See all 210 papers

GraphThinker: Reinforcing Temporally Grounded Video Reasoning with Event Graph Thinking
Feb 19, 2026Zixu Cheng, Da Li, Jian Hu +4Video SGG Video Reasoning
Queen Mary University of London · Samsung AI Centre Cambridge · Shanghai Artificial Intelligence Laboratory +1
Temporal Evidence Routing with Structured Visual Evidence for TimeLogicQA
May 31, 2026Yuyang Sun, Yongliang Wu, Xingyu Zhu +8Video QA Structured Reasoning
Southeast University · National University of Singapore · Independent Researcher +2
EventCoT: Event-centric Video Chain-of-thought for Reasoning Temporal Localization
Jul 6, 2026Youngkil Song, Yoonjae Baek, Dongwon Kim +3Temporal Video Grounding Multimodal CoT Reasoning
1Pohang University of Science and Technology, Pohang, South Korea · 2Korea Advanced Institute of Science and Technology, Daejeon, South Korea · 3Handong Global University, Pohang, South Korea

Feb 19, 2026J/K move · Enter open · S save

GraphThinker: Reinforcing Temporally Grounded Video Reasoning with Event Graph Thinking

Zixu Cheng, Da Li, Jian Hu, Yuhang Zang+3

Queen Mary University of London · Samsung AI Centre Cambridge · Shanghai Artificial Intelligence Laboratory+1

Video SGG Video Reasoning VLM Hallucination Mitigation Temporal Video Grounding LLM Reasoning with Graphs RL for VLMs

Open paper AskPDF arXiv

Video reasoning requires a fine-grained understanding of the temporal dependencies and event-level relations between objects and events in videos. Current Multimodal Large Language Models (MLLMs) are prone to severe temporal hallucinations in video reasoning. An underlying cause of these hallucinations is weak visual-temporal grounding and the lack of explicit structure for modelling event relations. Models often rely on auxiliary text, such as dense captions, rather than explicitly anchoring their reasoning in actual visual evidence. However, these textual representations are inherently unstructured and fail to provide explicit causal constraints needed to guide the model's reasoning. In this work, we propose GraphThinker, a reinforcement finetuning method that constructs a structured event representation of a video and enforces visual grounding to jointly reduce reasoning hallucinations. Specifically, we employ an MLLM to construct an Event-based Video Scene Graph (EVSG) that captures both intra- and inter-event relations, guiding a structured video reasoning process. Moreover, we address the weak grounding issue by introducing a novel visual attention reward during reinforcement finetuning that encourages the model to actively attend to reliable visual cues. On the RexTime dataset, GraphThinker achieves an over 4% improvement in IoU=0.3 for moment localisation. On the VidHalluc dataset, GraphThinker achieves a 9.8% improvement in reducing temporal sequence hallucination and a 7.6% gain in Binary QA in reducing action hallucination, compared to the state-of-the-art methods.

Abstract

Video Reasoning

Explore similar work

GraphThinker: Reinforcing Temporally Grounded Video Reasoning with Event Graph Thinking

Temporal Evidence Routing with Structured Visual Evidence for TimeLogicQA

EventCoT: Event-centric Video Chain-of-thought for Reasoning Temporal Localization