Open-Ended Video Game Glitch Detection with Agentic Reasoning and Temporal Grounding

Vision-language models (VLMs) are increasingly being explored for video game quality assurance, especially gameplay glitch detection. Most existing evaluations, however, treat glitches as static visual anomalies, asking models to detect failures from a single frame. We argue that this framing misses a key distinction: some glitches are spatial and visible in an isolated frame, whereas others are temporal and become evident only through changes across ordered frames. A preliminary study confirms this gap, showing that temporal glitches are substantially harder for VLMs to detect than spatial ones. To enable systematic evaluation of this underexplored setting, we introduce TempGlitch, a controlled gameplay video benchmark for temporal glitch detection. TempGlitch covers five temporal glitch types with balanced per-category samples, together with paired glitch-free videos that enable reliable binary evaluation. We evaluate 12 proprietary and open-weight VLMs across multiple frame-sampling settings. Our results show that current VLMs remain near chance on TempGlitch, often collapsing into either overly conservative behavior that misses most glitches or overly sensitive behavior that flags clean videos as glitchy. Moreover, denser frame sampling and larger model size do not reliably resolve these failures. TempGlitch provides a focused testbed for temporal reasoning, robust gameplay understanding, and automated glitch detection with VLMs. Code and data are available at the project website.

Visual glitches in video games degrade player experience and perceived quality, yet manual quality assurance cannot keep pace with the growing test surface of modern game development. Prior automation efforts, particularly those using vision-language models (VLMs), largely operate on isolated frames without sufficient context to judge whether a glitch is present. We introduce RefGlitch-Bench, a benchmark for reference-based video game glitch detection with VLMs. The key idea is to formulate glitch detection as an explicit within-video comparison problem: given a test frame, a reference frame provides a visual baseline that helps the model distinguish true glitches from benign visual variation. RefGlitch-Bench includes a controlled synthetic dataset with five injected glitch types and manually annotated reference/test frame pairs, enabling an oracle-reference evaluation that isolates the potential benefit of reference guidance. We further establish four initial baselines for automatically selecting references from earlier frames in the same video, with LastCleanFrame performing best and transferring across VLMs. Finally, we evaluate automatic reference guidance on real-world gameplay data, where it improves frame-level glitch detection beyond the controlled setting while revealing reference reliability and error propagation as key challenges. Code and data are available at: https://github.com/PipiZong/RefGlitch-Bench.git.

Video Anomaly Detection (VAD) aims to identify anomalous events and localize their temporal intervals. Existing approaches exhibit a "when-what" dissociation: traditional DNN-based methods localize when anomalies occur but lack semantic understanding, whereas LLM-based methods explain what happens but neglect precise temporal grounding. We attribute this to the absence of a unified reasoning paradigm. Inspired by how humans inspect surveillance videos - glancing globally to form temporal hypotheses, scrutinizing suspicious segments, and thinking iteratively to correct errors - we study this global-to-local paradigm from two perspectives. We first propose Glance then Scrutinize (GtS), a training-free framework using static and dynamic textual guidance for coarse-to-fine anomaly grounding and understanding, balancing accuracy and speed. To break the ceiling imposed by frozen external modules, we further propose a tool-augmented agentic VAD method, where a multimodal large language model learns to invoke a video cropping tool, inspect densely resampled frames, and self-correct mislocalized hypotheses, via cold-start supervised fine-tuning followed by reinforcement learning with a joint answer-grounding reward. For training and evaluation, we extend our prior VAGU benchmark into VAGU-T (Video Anomaly Grounding, Understanding, and Thinking), comprising 7,567 real-world videos over 21 anomaly categories with human-validated grounding, explanations, QA pairs, and chain-of-thought tool-calling traces. We further introduce JeAUG, a metric jointly evaluating semantic interpretability and temporal precision. Experiments show that GtS substantially surpasses training-free baselines, while the agentic model delivers both higher accuracy and faster inference.