Toward Reliable Infant Pose Estimation: A Training-Dynamics Approach to Noisy Annotation Detection

Emanuele Cardinale; Sara Moccia; Alessandro Cacciatore; Lucia Migliorelli

cs.CVOct 5, 2026

Toward Reliable Infant Pose Estimation: A Training-Dynamics Approach to Noisy Annotation Detection

Authors: Emanuele Cardinale, Sara Moccia, Alessandro Cacciatore, Lucia Migliorelli

Organizations: Department of Engineering and Geology, Università degli Studi “G. d’Annunzio” Chieti-Pescara, Pescara, 65127, Italy · Laboratory of Computational Oncology, Department of Oncology, KU Leuven, Leuven, 3000, Belgium · Department of Political Science, Università di Teramo, Teramo, 64100, Italy · Department of Innovative Technologies in Medicine and Dentistry, Università degli Studi “G. d’Annunzio” Chieti-Pescara, Chieti, 66100, Italy

Abstract

Spontaneous movement analysis in preterm infants relies increasingly on markerless pose estimation (PE) to derive clinically relevant motion biomarkers directly from video recordings. Training accurate infant PE models requires large sets of manually annotated keypoints, and human annotation is inherently prone to error. Noisy keypoints (i.e., keypoints mislocalized with respect to their true anatomical position) are especially problematic in this clinical setting, since they can propagate as artificial artifacts into the reconstructed joint trajectories. Building on the small-loss hypothesis and training-dynamics-based sample selection established in the noisy-label learning literature, we propose a novel framework for detecting noisy keypoint annotations. A hybrid convolutional-attention model is trained to predict the anatomical category of each keypoint from its spatial coordinates and local visual features; the resulting cross-entropy training dynamics are then used to derive per-keypoint descriptors, which are partitioned into clean and noisy subsets via unsupervised clustering. We validate the approach on NeoPose, a newly collected dataset of 65 hospitalized preterm infants, under two realistic noise scenarios (random positional perturbation and left-right swapping) across multiple noise levels. Results show that the proposed approach achieves an F1-score of up to 91.9% in noisy-keypoint detection. The framework further generalizes to the heterogeneous COCO benchmark, where filtering CE-detected noisy keypoints from the training set also yields measurable improvements (up to 7.4 AP points) in downstream pose estimation accuracy at moderate-to-high noise levels.

Figures & tables

Figure 1: Our proposed framework to detect noisy keypoints given a set of RGB images and their corresponding annotation set. It comprises two stages. In the first stage, a hybrid convolutional-attention classifier is trained to predict the anatomical category of a keypoint and its lateral position from the input image and its spatial coordinates. In the second stage, keypoint-wise features extracted from the trained model are used as input to an unsupervised clustering algorithm, which separates clean annotations from noisy ones.

$\Delta t_{side}$	$\Delta l_{side}$	$\Delta t_{joint}$	$\Delta l_{joint}$	$F1-score$
				Randomic noise	Swapping noise
✓				$0.749\pm 0.012$	$0.596\pm 0.088$
	✓			$0.521\pm 0.064$	$0.779\pm 0.104$
		✓		$0.896\pm 0.032$	$0.001\pm 0.001$
			✓	$0.491\pm 0.054$	$0.200\pm 0.063$
✓	✓			$0.757\pm 0.044$	$0.812\pm 0.081$

Table 1: Feature selection for noisy-keypoint clustering. F1-scores are reported separately for the random-noise and swapping-noise scenarios, averaged across the four injected noise levels (10%, 20%, 30%, 40%).

Noise (%)	Randomic Noise		Swapping Noise
	MSE-based	CE-based	MSE-based	CE-based
10	0.558	0.906	0.510	0.894
20	0.654	0.909	0.861	0.885
30	0.672	0.911	0.743	0.771
40	0.674	0.904	0.632	0.699

Table 2:

F1

-score on the NeoPose dataset under increasing levels of random and swapping noise (MSE- vs CE-based methods).

Figure 2: Mean loss trajectories for clean (blue) and noisy (red) keypoints under the random-noise scenario for NeoPose dataset, reported with their epoch-wise standard deviation across training. Each column corresponds to an increasing noise level (10%, 20%, 30%, 40%). The first row shows the MSE regression loss trajectories extracted from HRNet, used in the MSE-based baseline. The second row shows the CE classification loss trajectories extracted from the proposed keypoint classifier.

	Noise 10%	Noise 20%	Noise 30%	Noise 40%
MSE-based	0.4524	0.5385	0.5843	0.6199
CE-based	0.2413	0.2475	0.2615	0.2770

Table 3: Separability metric (SeM) between clean and noisy keypoint loss trajectories under the random-noise scenario, for increasing noise levels (10%, 20%, 30%, 40%). The first row reports values obtained from the MSE regression loss trajectories of the MSE-based baseline; the second row from the CE classification loss trajectories of the proposed method.Lower values indicate better separation between clean and noisy dynamics.

Figure 3: Mean side loss trajectories for clean (blue) and noisy (red) keypoints under the swapping-noise scenario, reported with their epoch-wise standard deviation across training. Each column corresponds to an increasing noise level (10%, 20%, 30%, 40%). Trajectories are extracted from the side classification loss of the proposed CE-based method.

Noise (%)	Random Noise
	MSE Based	CE Based
10	0.387	0.510
20	0.560	0.670
30	0.628	0.733
40	0.661	0.784

Table 4:

F1-score

on COCO pose estimation under increasing levels of randomic noise

Figure 4: Mean loss trajectories for clean (blue) and noisy (red) keypoints under the random-noise scenario for COCO dataset, reported with their epoch-wise standard deviation across training. Each column corresponds to an increasing noise level (10%, 20%, 30%, 40%). The first row shows the MSE regression loss trajectories extracted from HRNet, used in the MSE-based baseline. The second row shows the CE classification loss trajectories extracted from the proposed keypoint classifier.

Figure 5: Per-keypoint noisy-label detection F1-score on the COCO dataset under the random-noise scenario, reported for each of the 12 annotated keypoints across the four noise levels (10%, 20%, 30%, 40%).

Explore similar work

Sep 3, 2026cs.CV

The Blind Spot in 2D Infants' Pose Estimation:Robust Learning from Noisy Annotations

Noisy annotations pose a significant challenge for supervised deep learning, as neural networks rely on large-scale, high-quality labeled data whose corruption can severely impair model performance. Although robustness to label noise has been extensively studied for classification tasks, it remains relatively underexplored in Pose Estimation (PE). This limitation becomes critical in clinical contexts, including neonatology, where PE of preterm infants is used to support the assessment of spontaneous motility, a key indicator of neurodevelopmental trajectories. In such settings, infants' images labeling is further hindered by visual challenges (e.g., keypoint self-occlusions, caregiver interference), making the annotation process inherently susceptible to errors. To tackle noisy annotations in PE, we introduce REliable keypoint selection via Memory of traINing Dynamics (REMIND), a clustering-based keypoint-selection strategy that exploits keypoint-wise training dynamics to identify noisy labels without assuming any prior knowledge of the noise distribution, thus enabling noise-free model training. When evaluated on the proprietary NeoPose dataset, comprising 46 videos of 46 preterm infants recorded in real clinical settings, REMIND correctly identifies noisy annotations across multiple corruption scenarios, achieving up to 93% Area Under the Curve (AUC) with three different PE architectures used in the relevant literature. To our knowledge, this is the first study to explicitly address label noise in preterm infants' PE, paving the way for the design of trustworthy learning-based algorithms for infants'monitoring support when data quality cannot be guaranteed.

Emanuele Cardinale, Marco Proietti, Alessandro Cacciatore +3

Department of Engineering and Geology, Università degli Studi “G. d’Annunzio” Chieti-Pescara, Pescara, 65127, Italy · Department of Political Science, Communication and International Relations, University of Macerata, Macerata, 62100, Italy · Laboratory of Computational Oncology, Department of Oncology, KU Leuven, Leuven, 3000, Belgium +3

Noisy Labels Keypoint Detection Deep Learning Learnability Blind Spots Developmental Trajectories

Ask PDF

Sep 1, 2026cs.CV

Cross-Model Distillation of a Human-Pose Foundation Model from Unannotated Infant Video for Markerless 3D Pose Estimation

Spontaneous movement is one of the earliest windows onto an infant's neuromotor health, and structured clinical instruments that score it are validated early predictors of cerebral-palsy risk. However, they require specially trained raters, are time-consuming, and carry inter-rater variability. This motivates automated, video-based markerless assessment, especially as marker-based motion capture is impractical in infants. Yet the foundation models that make markerless capture possible are trained almost entirely on adults: our recent multi-view infant study found that no single model is jointly best, with strong 2D keypoint accuracy and direct 3D body recovery split across different models. While that study identifies this trade-off, it does not resolve it. Here, we perform cross-model distillation from the Sapiens 2 pose model into the SAM 3D Body model, using unannotated infant video alone. A frozen teacher supplies dense pseudo-labels, and a differentiable renderer aligns the predicted mesh to them in the training loop. On eleven held-out infants (18 sessions, 173 recordings) under our prior study's multi-view protocol, fine-tuning improves same-view 2D keypoint agreement with the Sapiens reference (median body percentage of correct keypoints @ 10px 0.22 -> 0.42, face 0.22 -> 0.42) and Procrustes-aligned mean per joint 3D position error (25.5 -> 22.2 mm). This demonstrates how cross-model distillation improves SAM 3D Body model performance on infants.

R. James Cotton, Divya Joshi, Colleen Peyton

1Shirley Ryan AbilityLab · Department of Physical Medicine and Rehabilitation, Northwestern University · Department of Physical Therapy and Human Movement Science, Northwestern University +1

Wifi-Csi 3D Human Pose Estimation 3D Foundation Models Body Wireless Foundation Models Dataset Distillation 3D Generation

Ask PDF

May 16, 2026cs.CV

Markerless Motion Capture for Biomechanical Whole-Body Kinematic Estimation in Infants

arly identification of motor impairment in infancy relies on expert visual assessment of spontaneous movement, motivating the development of automated, objective alternatives. One promising approach is using computer vision, which benefits from high quality pose estimation from video. In this study, we systematically evaluated three state-of-the-art pose estimation frameworks (MeTRAbs-ACAE, SAM 3D Body, and Sapiens) on 100 videos over 13 sessions of 8 infants recorded with a multi-view markerless motion capture system. We quantified keypoint detection accuracy using reprojection error, geometric consistency, and Procrustes-aligned 3D position error, and demonstrated proof-of-concept for fitting an inverse kinematic framework to infant data. While Sapiens achieved the lowest reprojection error and highest geometric consistency of the methods evaluated (22.8 pixels and 0.82, respectively), SAM 3D Body provided the most comprehensive 3D information for kinematic reconstruction with Procrustes-aligned position errors of 19 to 28 mm. We demonstrate in a case comparison example that biomechanical models fit to SAM 3D estimates distinguish representative movement patterns in infants related to motor development, as identified by a clinical expert. Together, these findings highlight both the promise and current limitations of 3D pose estimation for infant biomechanics and establish preliminary groundwork for scalable, video-based assessment of early motor development.

Divya Joshi, J. D. Peiffer, Colleen Peyton +1

Center for Bionic Medicine, Shirley Ryan AbilityLab, Chicago, IL · Dept. of Physical Therapy and Human Movement Science, Northwestern University, Chicago, IL · Dept. of Biomedical Engineering, Northwestern University, Evanston, IL +2

Biomechanics Body Keypoint Detection Motion Inverse Kinematics

Ask PDF