On the Evaluation Protocol of Gesture Recognition for UAV-based Rescue Operation based on Deep Learning: A Subject-Independence Perspective

Domonkos Varga

cs.CVFeb 19, 2026

On the Evaluation Protocol of Gesture Recognition for UAV-based Rescue Operation based on Deep Learning: A Subject-Independence Perspective

Authors: Domonkos Varga

Abstract

This paper presents a methodological analysis of the gesture-recognition approach proposed by Liu and Szirányi, with a particular focus on the validity of their evaluation protocol. We show that the reported near-perfect accuracy metrics result from a frame-level random train-test split that inevitably mixes samples from the same subjects across both sets, causing severe data leakage. By examining the published confusion matrix, learning curves, and dataset construction, we demonstrate that the evaluation does not measure generalization to unseen individuals. Our findings underscore the importance of subject-independent data partitioning in vision-based gesture-recognition research, especially for applications - such as UAV-human interaction - that require reliable recognition of gestures performed by previously unseen people.

Figures & tables

Title	Gesture Recognition for UAV-based Rescue Operation based on Deep Learning
Authors	Chang Liu; Tamás Szirányi
Book Title	Improve
Pages	180–187
Publication Year	2021

Table 1: Reference Details of the Original Work [ 5 ] Under Analysis.

Figure 1: Workflow of the gesture-recognition system implemented and proposed by Liu & Szirányi [ 5 ] .

Figure 2: The normalized confusion matrix published in [ 5 ] .

Figure 3: The training curves published in [ 5 ] .

Figure 8: Illustration of the likely incorrect data-splitting procedure used in the original study. Video recordings from all six participants appear to have been merged into a single pool, from which individual frames were randomly divided into a 90% training set and a 10% test set. This frame-level random split necessarily mixes data from each person into both sets, resulting in unavoidable subject leakage.

Figure 9: Correct subject-independent data-splitting protocol. Entire participants are assigned exclusively to the training or test set before any video frames are extracted. This prevents subject overlap and ensures that the test set contains individuals unseen during training, enabling a valid evaluation of generalization.

Figure 10: Excerpt from the official HaGRID benchmark (available at: https://github.com/hukenovs/hagrid ) documentation illustrating its subject-independent data-splitting protocol. The dataset is partitioned by user_id into 76% training, 9% validation, and 15% testing, ensuring that each subject appears exclusively in a single split. This exemplifies current best practice in gesture-recognition research and contrasts sharply with frame-level random splitting applied by Liu and Szirányi [ 5 ] .

Gesture Recognition

Gesture recognition maps body, hand, or muscle movements to communicative or control actions using cameras and wearable or wireless sensors.

See all 56 papers

Can Large Language Models Detect Methodological Flaws? Evidence from Gesture Recognition for UAV-Based Rescue Operation Based on Deep Learning
Mar 23, 2026Domonkos VargaLLM Auditing LLM Evaluation
UAV-OVO: Out-of-Viewpoint Generalization in UAV Action Recognition
May 25, 2026Yu Xia, Zhengbo Zhang, Shuaihu Zhang +1Test-Time Adaptation OOD Generalization
Wuhan University · Singapore University of Technology and Design
FR-GESTURE: An RGBD Dataset For Gesture-based Human-Robot Interaction In First Responder Operations
Feb 19, 2026Konstantinos Foteinos, Georgios Angelidis, Aggelos Psiris +3Human-Robot Interaction Robotic Data Collection
∗Department of Informatics and Telematics, Harokopio University of Athens, Athens, Greece · † Department of Networks and Digital Media, Kingston University, Kingston upon Thames, Surrey, United Kingdom · ‡ Department of Electrical and Computer Engineering, University of Western Macedonia, Kozani, Greece +1

Mar 23, 2026J/K move · Enter open · S save

Can Large Language Models Detect Methodological Flaws? Evidence from Gesture Recognition for UAV-Based Rescue Operation Based on Deep Learning

Domonkos Varga

LLM Auditing LLM Evaluation Data Leakage

Open paper AskPDF arXiv

Reliable evaluation is essential in machine learning research, yet methodological flaws-particularly data leakage-continue to undermine the validity of reported results. In this work, we investigate whether large language models (LLMs) can act as independent analytical agents capable of identifying such issues in published studies. As a case study, we analyze a gesture-recognition paper reporting near-perfect accuracy on a small, human-centered dataset. We first show that the evaluation protocol is consistent with subject-level data leakage due to non-independent training and test splits. We then assess whether this flaw can be detected independently by six state-of-the-art LLMs, each analyzing the original paper without prior context using an identical prompt. All models consistently identify the evaluation as flawed and attribute the reported performance to non-independent data partitioning, supported by indicators such as overlapping learning curves, minimal generalization gap, and near-perfect classification results. These findings suggest that LLMs can detect common methodological issues based solely on published artifacts. While not definitive, their consistent agreement highlights their potential as complementary tools for improving reproducibility and supporting scientific auditing.

Abstract

Gesture Recognition

Explore similar work

Can Large Language Models Detect Methodological Flaws? Evidence from Gesture Recognition for UAV-Based Rescue Operation Based on Deep Learning

UAV-OVO: Out-of-Viewpoint Generalization in UAV Action Recognition

FR-GESTURE: An RGBD Dataset For Gesture-based Human-Robot Interaction In First Responder Operations