Open-Ended Video Game Glitch Detection with Agentic Reasoning and Temporal Grounding

Muyang Zheng; Tong Zhou; Geyang Wu; Zihao Lin; Haibo Wang; Lifu Huang

cs.MAApr 9, 2026

Open-Ended Video Game Glitch Detection with Agentic Reasoning and Temporal Grounding

Authors: Muyang Zheng, Tong Zhou, Geyang Wu, Zihao Lin, Haibo Wang, Lifu Huang

Organizations: University of California, Davis Davis, California, USA · Virginia Polytechnic Institute and State University Blacksburg, Virginia, USA

Abstract

Open-ended video game glitch detection aims to identify glitches in gameplay videos, describe them in natural language, and localize when they occur. Unlike conventional game glitch understanding tasks which have largely been framed as image-level recognition or closed-form question answering, this task requires reasoning about game-specific dynamics such as mechanics, physics, rendering, animation, and expected state transitions directly over continuous gameplay videos and distinguishing true glitches from unusual but valid in-game events. To support this task, we introduce VideoGlitchBench, the first benchmark for open-ended video game glitch detection with temporal localization. VideoGlitchBench contains 5,238 gameplay videos from 120 games, each annotated with detailed glitch descriptions and precise temporal spans, enabling unified evaluation of semantic understanding and temporal grounding. We further propose GliDe, an agentic framework with three key components: a game-aware contextual memory for informed reasoning, a debate-based reflector for multi-perspective glitch detection and verification, and an event-level grounding module that recovers complete glitch intervals from fragmented temporal evidence. We also design a task-specific evaluation protocol that jointly measures semantic fidelity and temporal accuracy. Experiments show that this task remains highly challenging for current multimodal models, while GliDe achieves substantially stronger performance than corresponding vanilla model baselines.

Figures & tables

Figure 1. We introduce VideoGlitchBench , a comprehensive benchmark designed for open-ended video game glitch detection. Examples of our benchmark \dataname.\lifu{to save space, you can probably make this figure denser, and use 1-2 example videos to demonstrate these three cases; also add the category of the bugs to highlight the requirement of diverse types of reasoning for mechanics, physics, rendering, etc.}

Dataset	Scale	Pipeline	Annotation	Localization	Evaluation
GamePhysics ( Taesiri et al., 2022a )	26,954 videos	Semi-Auto	Game name/weak metadata	✗	Retrieval relevance
GameBugDescriptions ( Taesiri et al., 2022b )	167 videos	Manual	Event description & Bug type	✗	Multiple-choice accuracy
GlitchBench ( Taesiri et al., 2024 )	923 images	Manual	Short glitch description	✗	Description quality
VideoGameBunny ( Taesiri and Bezemer, 2025 )	185,259 images	Semi-Auto	Caption & Image-to-JSON & Question-answer pairs	✗	N/A (Designed for instruction-tuning)
GameBench ( Cao et al., 2026 )	880 videos	Semi-Auto	Multiple-choice questions	✗	Multiple-choice accuracy
PhysGame ( Cao et al., 2026 )	38,957 videos	Semi-Auto	Question-answer pairs	✗	N/A (Designed for instruction-tuning)

Table 1. Comparison between VideoGlitchBench and existing game glitch datasets.

Figure 2. The annotation pipeline of our benchmark VideoGlitchBench . Benchmark curation.

Statistics of VideoGlitchBench	Value
Total number of videos	5238
Game genres/subgenres	6 / 21
#Bugs in a single video (avg/max)	1.03 / 6
Video length (seconds, avg/max)	19.11 / 60.00
Bug description length (tokens, avg/max)	35.94 / 164

Table 2. Key statistics of VideoGlitchBench .

Figure 3. Illustration of our GliDe framework. Our pipeline.

Proprietary Models
Model	Description Generation			Temporal Grounding	Overall
Model	Precision (%)	Recall (%)	F1 (%)	mIoU	F1 $\times$ IoU (%)
gemini-2.0-flash ( Google, 2024 )	18.36	25.35	21.29	0.44	10.55
gpt-4o-mini ( Hurst et al., 2024 )	12.33	32.36	17.86	0.35	6.20
claude-3.5-haiku ( Anthropic, 2024 )	21.66	32.54	26.01	0.47	12.91
nova-lite-v1 ( Intelligence, 2024 )	6.66	23.77	10.41	0.28	2.98

Table 3. Comparison of vanilla models and GliDe -enhanced models on VideoGlitchBench across description generation, temporal grounding, and overall performance.

Variant	Precision (%)	Recall (%)	F1 (%)
GliDe	34.55	45.09	39.12
w/o Game-aware memory	30.65	35.80	33.03
w/o Debate-based verification	28.94	36.21	32.17

Table 4. Ablation study on the game-aware memory and debate-based verification mechanism.

Backbone	GliDe	w/o Event-level grounding
Qwen2.5-VL-7B-Instruct	0.53	0.32
InternVL2.5-8B	0.50	0.24
UI-TARS-1.5-7B	0.51	0.36

Table 5. Ablation study on the event-level grounding strategy across different open-source backbones (mIoU).

Figure 4. Qualitative comparison between the vanilla model and GliDe on a Human: Fall Flat gameplay clip. Case analysis.

Figure 5. Qualitative comparison between the vanilla model and GliDe on a ARK: Survival Evolved gameplay clip. Case analysis.

Figure 6. Hyperparameter sensitivity study.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7. Taxonomy of VideoGlitchBench . A pie chart illustrating the percentage breakdown of different video game genres included in the \dataname{} dataset, such as Action, Adventure, and RPG.

Genre	Subgenre	Videos	Share (%)
Action		3834	73.20
	Action Adventure	1533	29.27
	Shooter	1368	26.12
	Survival	264	5.04
	Fighting	260	4.96
	Battle Royale	163	3.11

Appendix

Table 6. Video distribution by genre and subgenre.

Figure 8. Word cloud of all the bug descriptions.

Tool Name	Description
vqa	Submit the full stitched window image with a text question to the VLM and return a free-form answer.
zoom_in	Crop a specified region from one or more raw frames, upscale the result, and chain a VQA call on the cropped output. Used when the anomaly occupies a small portion of the full image.
object_tracking	Track a specified object across frames using SAM3, then automatically run physics analysis on the resulting centroid sequence. Physics analysis computes per-frame velocity and acceleration, and applies four rule-based anomaly detectors: (1) position_jump : inter-frame displacement $>20\%$ of the frame diagonal; (2) velocity_spike : inter-frame speed change $>500$ px/s; (3) motion_freeze : displacement $<1$ px for $\geq 4$ consecutive frames; (4) jittering : direction reversals in $\geq 50\%$ of consecutive frame-pair steps.

Appendix

Table 7. Tool library for our agentic framework.

Figure 9. An example of the annotation process.

Figure 10. Annotation interface used for VideoGlitchBench .

Explore similar work

TempGlitch: Evaluating Vision-Language Models for Temporal Glitch Detection in Gameplay Videos
May 20, 2026Yakun Yu, Ashley Wiens, Adrián Barahona-Ríos +4VLM Evaluation Automated Software Testing
University of Alberta · Sony Interactive Entertainment
RefGlitch-Bench: A Benchmark for Reference-based Gameplay Glitch Detection with Vision-Language Models
Apr 13, 2026Yakun Yu, Ashley Wiens, Adrián Barahona-Ríos +4VLM Evaluation Video Anomaly Detection
University of Alberta · Sony Interactive Entertainment
Glance, Scrutinize, and Think: Advancing Video Anomaly Detection from Training-Free to Agentic Reasoning
Aug 7, 2026Shibo Gao, Peipei Yang, Xu-Yao Zhang +1Anomaly Localization Video Anomaly Detection
School of Electronic and Information Engineering, Beijing Jiaotong University, Beijing 100044, China. · Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China. · State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China.

May 20, 2026J/K move · Enter open · S save

TempGlitch: Evaluating Vision-Language Models for Temporal Glitch Detection in Gameplay Videos

Yakun Yu, Ashley Wiens, Adrián Barahona-Ríos, Benedict Wilkins+3

University of Alberta · Sony Interactive Entertainment

VLM Evaluation Automated Software Testing Video Anomaly Detection Temporal Video Understanding

Open paper AskPDF arXiv

Vision-language models (VLMs) are increasingly being explored for video game quality assurance, especially gameplay glitch detection. Most existing evaluations, however, treat glitches as static visual anomalies, asking models to detect failures from a single frame. We argue that this framing misses a key distinction: some glitches are spatial and visible in an isolated frame, whereas others are temporal and become evident only through changes across ordered frames. A preliminary study confirms this gap, showing that temporal glitches are substantially harder for VLMs to detect than spatial ones. To enable systematic evaluation of this underexplored setting, we introduce TempGlitch, a controlled gameplay video benchmark for temporal glitch detection. TempGlitch covers five temporal glitch types with balanced per-category samples, together with paired glitch-free videos that enable reliable binary evaluation. We evaluate 12 proprietary and open-weight VLMs across multiple frame-sampling settings. Our results show that current VLMs remain near chance on TempGlitch, often collapsing into either overly conservative behavior that misses most glitches or overly sensitive behavior that flags clean videos as glitchy. Moreover, denser frame sampling and larger model size do not reliably resolve these failures. TempGlitch provides a focused testbed for temporal reasoning, robust gameplay understanding, and automated glitch detection with VLMs. Code and data are available at the project website.

Abstract

Explore similar work

TempGlitch: Evaluating Vision-Language Models for Temporal Glitch Detection in Gameplay Videos

RefGlitch-Bench: A Benchmark for Reference-based Gameplay Glitch Detection with Vision-Language Models

Glance, Scrutinize, and Think: Advancing Video Anomaly Detection from Training-Free to Agentic Reasoning