cs.CVMar 26, 2026

Dynamic LIBRAS Gesture Recognition via CNN over Spatiotemporal Matrix Representation

Authors: Jasmine Moreira

Organizations: CPGEI Universidade Tecnológica Federal do Paraná Curitiba, Paraná, Brazil

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

This paper proposes a method for dynamic hand gesture recognition based on the composition of two models: the MediaPipe Hand Landmarker, responsible for extracting 21 skeletal keypoints of the hand, and a convolutional neural network (CNN) trained to classify gestures from a spatiotemporal matrix representation of dimensions 90 by 21 of those keypoints. The method is applied to the recognition of LIBRAS (Brazilian Sign Language) gestures for device control in a home automation system, covering 11 classes of static and dynamic gestures. For real-time inference, a sliding window with temporal frame triplication is used, enabling continuous recognition without recurrent networks. Tests achieved 95% accuracy under low-light conditions and 92% under normal lighting. The results indicate that the approach is effective, although systematic experiments with greater user diversity are needed for a more thorough evaluation of generalization.

Figures & tables

Fig. 1: LIBRAS Fingerspelling Alphabet
Fig. 2: MediaPipe Keypoint Scheme
Fig. 3: Static Gesture — Letter A
Fig. 4: Motion Matrices — ”A” and Toggle (On/Off)
Fig. 5: Second Model Architecture — Convolutional Network
Fig. 6: Training Accuracy
Fig. 7: Training Loss
Fig. 8: Sliding Window
Fig. 9: Gesture Detection under Low-Light Conditions
ConditionCorrectMain error
Low light95/100C as neutral
Normal light92/100White (5), C (3)
TABLE I: Test Results by Lighting Condition
Fig. 10: Gesture Detection under Normal Lighting
More in this topic

Gesture Recognition

Gesture recognition maps body, hand, or muscle movements to communicative or control actions using cameras and wearable or wireless sensors.

See all 56 papers

Explore similar work

CardsList

  1. Online Hand Gesture Recognition Using 3D Convolutional Neural Networks

    May 22, 2026Yinghao Qin, Tijana TimotijevicGesture Recognition3D CNNs

    School of Electronic Engineering and Computer Science Queen Mary, University of London London, UK

  2. UniCon-Former: Unified Convolution Transformer is All You Need for Hand Gesture Recognition

    Aug 13, 2026Mallika Garg, Debashis Ghosh, Pyari Mohan PradhanHybrid CNN-Transformer ArchitecturesEfficient ViTs

    Indian Institute of Technology Kharagpur, INDIA · Indian Institute of Technology Roorkee, INDIA

  3. Neural Networks for Temporal Pattern Recognition and Dynamic Arm Gesture Speed Estimation for Robot Control

    Oct 8, 2026Milán Zsolt Bagladi, László GulyásRecurrent Neural NetworksRobotic Perception

    Department of Artificial Intelligence ELTE Eötvös Loránd University Budapest, Hungary