Data augmentation is a crucial technique for training robust deep learning models for human motion, where annotated datasets are often scarce. However, generic augmentation methods often ignore the underlying geometric and kinematic constraints of the human body, risking the generation of unrealistic motion patterns that can degrade model performance. Furthermore, the conventional approach of training a single generalist model on a dataset expanded with a mixture of all available transformations does not fully exploit the unique learning signals provided by each distinct augmentation type. We challenge this convention by introducing a novel training paradigm, EnsAug, that strategically uses augmentation to foster model diversity within an ensemble. Our method involves training an ensemble of specialists, where each model learns from the original dataset augmented by only a single, distinct geometric transformation. Experiments on sign language and human activity recognition benchmarks demonstrate that our diversified ensemble methodology significantly outperforms the standard practice of training one model on a combined augmented dataset and achieves state-of-the-art accuracy on two sign language and one human activity recognition dataset while offering greater modularity and efficiency. Our primary contribution is the empirical validation of this training strategy, establishing an effective baseline for leveraging data augmentation in skeletal motion analysis.
Figures & tables
Fig. 1 : Example of a person performing a sign in the WLASL dataset [ 2 ] and the pose and hands landmarks extracted with MediaPipe [ 4 ] .
Fig. 2 : EnsAug framework pipeline.
Model
Wlasl100
Wlasl300
Signum
MHAD
Jitter
.610
.452
.846
.559
Scale
.619
.468
.838
.562
MagWarp
.526
.423
.830
.509
TimeWarp
.663
.537
.852
.512
WinWarp
.635
.543
.900
.584
WinScale
.639
.539
.889
.612
TABLE I : Performance (Accuracy %) of the baseline model trained with traditional, generic time-series augmentation techniques. The ‘Ensemble’ row shows the result of combining these models with majority voting.
Fig. 3 : The Transformer architecture used for each specialist model. The input landmark sequence is passed through a stack of encoder layers, followed by global average pooling over the time dimension to produce a fixed-size representation for the final classifier.
Fig. 4 : Model results on the WLASL-100 dataset. All metrics are percentages (mean ± 95% CI).
Fig. 5 : Model results on the WLASL-300 dataset. All metrics are percentages (mean ± 95% CI).
Model
Pattern
Hyperparameter Range
CamDepth
Near
Distance: 0.7
Far
Distance: 1.3
TempDepth
Far to near
Distance: (0.5, 1.3)
Near to far
Distance: (0.7, 1.5)
Near far near
Distance: (0.6, 1.4)
HV-Shift
Linear shift
Distance: (−0.15,0.15)
TABLE II : Augmentation patterns and ranges.
TABLE III : Comparison with model performance (in % accuracy) from previous works. Pose-GRU and Pose-TGCN can be found in [ 2 ] , GCN-BERT can be found in [ 13 ] , Procrustes-DTW (P-DTW) can be found in [ 14 ] , Spoter can be found in [ 15 ] , EnsAug (Ours) represents our ensemble augmentation framework.
Fig. 6 : Model results on the SIGNUM dataset. All metrics are percentages (mean ± 95% CI).
Fig. 7 : Model results on the MHAD dataset. All metrics are percentages (mean ± 95% CI).
Dataset
Min
Mean
Max
SIGNUM
0.28
0.37
0.43
MHAD
0.44
0.53
0.59
WLASL-100
0.50
0.58
0.68
WLASL-300
0.59
0.63
0.66
TABLE IV : Pairwise error overlap (Jaccard Index) between specialists. Lower values indicate more diverse error patterns.
Fig. 8 : Accuracy vs. ensemble size k (mean ± 95% CI). MHAD uses M=6 augmentations; others use M=8 .
Skeleton-based human action recognition is a powerful approach for understanding human behaviour from pose data, but collecting large-scale, diverse, and well-annotated 3D skeleton datasets is both expensive and labor-intensive. To address this challenge, we propose a conditional generative pipeline for data augmentation in skeleton action recognition. Our method learns the distribution of real skeleton sequences under the constraint of action labels, enabling the synthesis of diverse and high-fidelity data. Even with limited training samples, it can effectively generate skeleton sequences and achieve competitive recognition performance in low-data scenarios, demonstrating strong generalisation in downstream tasks. Specifically, we introduce a Transformer-based encoder-decoder architecture, combined with a generative refinement module and a dropout mechanism, to balance fidelity and diversity during sampling. Experiments on HumanAct12 and the refined NTU-RGBD (NTU-VIBE) dataset show that our approach consistently improves the accuracy of multiple skeleton-based action recognition models, validating its effectiveness in both few-shot and full-data settings. The source code can be found at here.
Xu Dong, Wanqing Li, Anthony Adeyemi-Ejeye +1
Innovative Media Lab, University of Surrey, Guildford, UK · Advanced Multimedia Research Lab, University of Wollongong, Wollongong, Australia
Automated evaluation of movement quality can enhance physiotherapeutic treatment and sports training by providing objective, real-time feedback. However, deep learning models that assess movements captured by inertial measurement units (IMUs) are often limited by data scarcity, class imbalance, and label ambiguity. We present a data augmentation method that generates IMU data using musculoskeletal simulations integrated with systematic modifications of movement trajectories. The approach enforces anatomically plausible kinematic constraints and enables automatic labeling by combining inverse kinematic parameters with a knowledge-based evaluation strategy. Across four datasets of varying complexity, augmented variants closely resemble real-world data and contribute to gains in classification accuracy, generalization to unseen subjects, and patient-specific fine-tuning from few examples. The magnitude of these gains varies with dataset properties, in particular class balance and label ambiguity. These findings indicate that musculoskeletal simulation-based augmentation can address common challenges faced by deep learning applications in physiotherapeutic exercise evaluation.
Andreas Spilz, Heiko Oppel, Michael Munz
AI for Sensor Data Analytics Research Group, Ulm University of Applied Sciences, Ulm, 89081, Germany
Enhancing the generalization of robotic learning in diverse unseen environments remains a fundamental challenge. Existing approaches often rely on large-scale pretraining, which is labor-intensive and time-consuming, or semantic data augmentation methods that assume flawless upstream object detection in real-world scenarios. In this work, we propose RoboAug, a novel generative data augmentation framework that reduces reliance on large-scale pretraining and perfect visual recognition by requiring only a single image with bounding box annotations for dataset construction. Leveraging this minimal supervision, RoboAug employs pretrained generative models for precise semantic augmentation and introduces a plug-and-play region-contrastive loss to guide attention toward task-relevant regions, thereby enhancing generalization and task success rates. Extensive real-world experiments on UR-5e, AgileX, and Tian Gong 2.0 demonstrate that RoboAug consistently outperforms state-of-the-art augmentation baselines under background, distractor, and lighting shifts. Our project is available at https://x-roboaug.github.io/.
Xinhua Wang, Kun Wu, Zhen Zhao +10
Beijing Innovation Center of Humanoid Robotics · Computation, Information and Technology, Technical University of Munich · City University of Hong Kong +3