InfantNet: A Large Scale Dataset for Infant Body Pose and Shape Estimation

Zhipeng Hui; Jiahao Wang; Xun Dong; Guofeng Zhang; Xingrui Wang; Jiawei Peng; Qihao Liu; Xiaoding Yuan

cs.CVOct 29, 2026

InfantNet: A Large Scale Dataset for Infant Body Pose and Shape Estimation

Authors: Zhipeng Hui, Jiahao Wang, Xun Dong, Guofeng Zhang, Xingrui Wang, Jiawei Peng, Qihao Liu, Xiaoding Yuan,

Organizations: Johns Hopkins University · Sony Corporation of America · University of Washington · CISPA Helmholtz Center for Information Security

Abstract

Infant pose and shape estimation is essential for applications in childcare, developmental monitoring, and medical diagnosis. However, existing methods and datasets are largely designed for adults, and direct transfer to infants fails due to substantial differences in body proportions, articulation limits, and frequent self-occlusion. To address this gap, we introduce InfantNet, the largest real-image infant dataset to date, comprising 108,902 RGB images of infants aged 6-18 months. Each image is annotated with 2D keypoints, and a curated subset of 11,642 images additionally includes 3D pose and shape annotations with full SMIL parameters, verified through multi-view rendering and reprojection-error filtering. We use an iterative annotation pipeline to ensure high fidelity across both 2D and 3D labels. InfantNet establishes a large-scale, comprehensive benchmark for infant 2D keypoint detection and 3D pose-and-shape recovery. Baseline experiments demonstrate that state-of-the-art adult pose estimators do not generalize well to infants, whereas fine-tuning on InfantNet yields a consistent improvement. The gains are even more pronounced for 3D pose and shape estimation, where SMIL-based estimation demonstrates robust performance with accuracy comparable to results reported for SMPL-based models. By releasing the InfantNet dataset and benchmark, we provide a vital resource for advancing infant pose analysis and related healthcare applications.

Figures & tables

Dataset	Syn/Real	# Images	Keypoints	Segmentation	3D Annotations
BabyPose [ 35 ]	Real	16,000	Yes	No	Depth only
BabyNet [ 12 ]	Real	2,984	No	BBox only	No
MINI-RGBD [ 17 ]	Synthetic	12,000	Yes	No	Depth only
SyRIP [ 19 ]	Syn+Real	1,700	Yes	No	No
InfantNet (ours)	Real	108,902	Yes	Yes	Yes

Table 1: Comparison of InfantNet with existing open-source infant pose datasets. InfantNet is the largest real-world infant dataset to date, providing 108,902 images with keypoint annotations, full segmentation masks, and a curated subset with 3D SMIL parameters. In contrast, prior datasets are limited in scale, modality, or 3D supervision (e.g., MINI-RGBD provides depth only).

Figure 2: Annotation pipeline of our InfantNet Dataset. Frames from YouTube and HomeVisit recordings were annotated using a semi-automatic, inspector-guided workflow. For 2D labels, RTMO-L provided initial keypoints that were refined by annotators with iterative quality control, following the COCO format. For 3D labels, we applied an optimization-based method to fit SMIL to infant images, and annotators further filtered the results to ensure accurate pose and shape annotations.

Figure 3: Visualization of different annotation stages in our InfantNet Dataset: Original image, Segmentation mask, Bounding box, and Annotated Body Keypoints.

Subset	Viewpoint Info	Raw Frames	Filtered Frames
YouTube (Train)	Varied	44,257	36,185
YouTube (Test)	Varied	6,782	3,959
HomeVisit (Train)	4 fixed cameras	123,876	56,968
HomeVisit (Test)	4 fixed cameras	31,521	11,790
Total	–	206,436	108,902

Table 2: Dataset Statistics. “Filtered Frames” denotes the final samples retained after removing low-quality, ambiguous, or non-compliant frames during annotation.

Figure 4: Pose diversity statistics measured by per-joint standard deviation between InfantNet and large-scale human datasets [ 27 , 8 , 29 , 5 , 34 ] . InfantNet exhibits broad variation in 3D joint configurations.

Model	Direct Inference (%)				Fine-tuned (%)
	AP	AP.5	AP.75	AR	AP	AP.5	AP.75	AR
Hourglass 256x256	86.27	95.58	92.37	88.43	92.63	98.99	97.90	93.95
Hourglass 384x384	87.39	96.59	93.43	89.27	93.40	98.98	97.89	94.52
HRFormer-small 256x192	87.65	96.53	93.36	89.82	93.14	98.97	97.87	94.36
HRFormer-small 384x288	88.00	96.54	94.30	90.07	93.51	98.97	97.91	94.60
HRFormer-base 256x192	88.24	95.50	93.33	90.26	93.85	98.98	97.92	95.09

Table 3: Comparison of 2D pose estimation models on the InfantNet benchmark under two settings: direct inference using pretrained models and fine-tuning on InfantNet. We report Average Precision (AP), AP at IoU thresholds 0.5 (AP.5) and 0.75 (AP.75), and Average Recall (AR). Direct transfer of pretrained models leads to a clear performance drop, while fine-tuning consistently improves performance, with an average relative gain of 6.2% in AP and 5.2% in AR . ViTPose-huge achieves the best results after adaptation (AP: 94.77% , AR: 95.82% ), highlighting the importance of domain-specific data for accurate infant pose estimation.

Model	Adult Body Model		Infant Body Model
	MPJPE $\downarrow$	PA-MPJPE $\downarrow$	MPJPE $\downarrow$	PA-MPJPE $\downarrow$
HMR [ 22 ]	393.30	124.15	103.36 ( $\downarrow$ 74% )	58.79 ( $\downarrow$ 53% )
PARE [ 25 ]	364.49	120.06	83.73 ( $\downarrow$ 77% )	63.16 ( $\downarrow$ 47% )
4D-Humans [ 15 ]	526.62	125.02	84.73 ( $\downarrow$ 84% )	46.74 ( $\downarrow$ 63% )

Table 4: Comparison of 3D pose and shape estimation on InfantNet using adult body models (left) vs. infant body models (right). Metrics: MPJPE and PA-MPJPE (mm). Infant models consistently improve accuracy across all architectures, highlighting the importance of infant data and models.

Figure 5: Failure modes and effects of fine-tuning: Without InfantNet FT, keypoint estimators frequently fail, e.g., by mislocalizing the left arm under severe self-occlusion (left) or predicting leg incorrectly in atypical infant poses (right). After FT, models can accurately localize joints in these challenging scenarios.

Method	All	Head	Torso	Arms	Legs
ViTPose-H (Before FT)	0.871	0.962	0.842	0.865	0.793
ViTPose-H (FT on InfantNet)	0.906	0.986	0.890	0.901	0.828

Table 5: PCK (%) breakdown of ViTPose-H before and after fine-tuning on InfantNet. Fine-tuning improves all body parts, but leg joints remain the most challenging, reflecting the unique articulation and occlusion patterns in infant poses.

Figure 6: Visualization of 3D pose and shape estimation. From left to right, the columns show the input image, direct inference with adult body model and data, and training with infant body model and data for HMR [ 22 ] , PARE [ 25 ] , and 4D-Humans [ 15 ] , respectively.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Direct Inference				Source-Specific Fine-tuned				Cross-Subset Fine-tuned
	AP	AP.5	AP.75	AR	AP	AP.5	AP.75	AR	AP	AP.5	AP.75	AR
Hourglass 256x256	0.8588	0.9790	0.9440	0.8868	0.9193	0.9898	0.9678	0.9374	0.8667	0.9896	0.9566	0.8962
Hourglass 384x384	0.8707	0.9794	0.9452	0.8987	0.9284	0.9897	0.9781	0.9450	0.8808	0.9894	0.9566	0.9091
HRFormer-small_256x192	0.8761	0.9779	0.9537	0.9062	0.9247	0.9896	0.9778	0.9422	0.8734	0.9896	0.9549	0.9023
HRFormer-small_384x288	0.8852	0.9887	0.9557	0.9145	0.9307	0.9895	0.9788	0.9475	0.8816	0.9896	0.9658	0.9115
HRFormer-base_256x192	0.8900	0.9883	0.9559	0.9172	0.9325	0.9893	0.9782	0.9489	0.8964	0.9898	0.9673	0.9235

Appendix

Table S1 : Comparison of 2D pose estimation models on the InfantNet Subsets (

\text{YouTube}_{\text{Test}}

) benchmark under three evaluation settings: direct inference on

\text{YouTube}_{\text{Test}}

using COCO-pretrained models (left block), source-specific fine-tuning on

\text{YouTube}_{\text{Train}}

(middle block), and cross-subset fine-tuning on

\text{HomeVisit}_{\text{Train}}

(right block). We report Average Precision (AP), AP at IoU thresholds 0.5 (AP.5) and 0.75 (AP.75), and Average Recall (AR). Across all architectures, fine-tuning substantially improves performance. On average, source-specific adaptation yields the relative gains of 5.0% in AP and 3.7% in AR , whereas cross-subset fine-tuning provides margin relative gains of 0.5% in AP and 0.3% in AR over direct inference.

Model	Direct Inference				Source-Specific Fine-tuned				Cross-Subset Fine-tuned
	AP	AP.5	AP.75	AR	AP	AP.5	AP.75	AR	AP	AP.5	AP.75	AR
Hourglass 256x256	0.8643	0.9570	0.9257	0.8834	0.9222	0.9898	0.9689	0.9357	0.9116	0.9799	0.9687	0.9273
Hourglass 384x384	0.8724	0.9572	0.9358	0.8907	0.9288	0.9897	0.9790	0.9415	0.9177	0.9797	0.9689	0.9329
HRFormer-small_256x192	0.8766	0.9573	0.9359	0.8954	0.9235	0.9896	0.9687	0.9369	0.9194	0.9796	0.9685	0.9350
HRFormer-small_384x288	0.8774	0.9568	0.9352	0.8960	0.9261	0.9893	0.9688	0.9398	0.9219	0.9896	0.9688	0.9369
HRFormer-base_256x192	0.8769	0.9560	0.9346	0.8977	0.9326	0.9897	0.9791	0.9452	0.9260	0.9893	0.9688	0.9423

Appendix

Table S2 : Comparison of 2D pose estimation models on the InfantNet Subsets(

\text{HomeVisit}_{\text{Test}}

) benchmark under three evaluation settings: direct inference using COCO-pretrained models (left block), fine-tuning on

\text{HomeVisit}_{\text{Train}}

(middle block), and fine-tuning on

\text{YouTube}_{\text{Train}}

(right block). We report Average Precision (AP), AP at IoU thresholds 0.5 (AP.5) and 0.75 (AP.75), and Average Recall (AR). Across all architectures, fine-tuning substantially improves performance. In average, source-specific adaptation yields the relative gains of 6.1% in AP and 5.3% in AR , whereas cross-subset fine-tuning offers comparably strong relative gains of 5.2% in AP and 4.7% in AR over direct inference.

Figure S1 : Additional examples of meshes generated via optimization-based SMIL fitting for the InfantNet dataset. The refined SMIL parameters become the 3D annotations used to train end-to-end infant models. InfantNet captures a diverse set of infant shapes and poses with high-quality 2D and 3D annotations.

Model	MPJPE $\downarrow$	PA-MPJPE $\downarrow$
HMR [ 22 ]	118.53	62.30
PARE [ 25 ]	107.52	63.83
4D-Humans [ 15 ]	84.50	42.40

Appendix

Table S3 : Cross-subset 3D pose and shape estimation. The models are trained on the HomeVisit subset and tested on the YouTube subset. Metrics: MPJPE and PA-MPJPE (mm). Infant models consistently improve accuracy across all architectures, even with cross-subset training and evaluation, validating the generalizability of the infant models.

Figure S2 : Examples of occlusion, pose, viewpoint edge cases, respectively.

Model	Avg Time (s / img)
Optimization
SMIL-Opt [ 3 , 45 ]	38.7020
End-to-End Models
HMR [ 22 ]	0.0033
PARE [ 25 ]	0.0091
4D-Humans [ 15 ]	0.0212

Appendix

Table S4 : Comparison of average inference speed between optimization-based SMIL fitting and end-to-end infant modeling models. End-to-end models achieve millisecond-level inference, demonstrating the substantial computational advantage of learning-based 3D infant pose and shape estimation models.

Model	MPJPE $\downarrow$	PA-MPJPE $\downarrow$
HMR [ 22 ]	100.00	55.60
PARE [ 25 ]	78.48	59.41
4D-Humans [ 15 ]	86.57	50.15

Appendix

Table S5 : 3D pose and shape estimation on masked infant inputs. Metrics: MPJPE and PA-MPJPE (mm). Applying masking results in negligible degradation, demonstrating model robustness and dataset viability.

Figure S3 : Visualization of 3D pose and shape estimation model performances. The columns from left to right: the input image, optimization-based SMIL fitting, and inference with end-to-end model (4D-Humans [ 15 ] ) trained with infant body model and data, respectively. The end-to-end 4D-Humans model produces anatomically plausible meshes that better align with the infant silhouette, while optimization-based SMIL fitting produces deformed meshes, illustrating the robustness of end-to-end models in 3D infant estimation compared to optimization-based fitting.

Human Pose Estimation

Human pose estimation recovers people’s body keypoints, 3D skeletons, or meshes from images, video, and other sensor data.

See all 80 papers

Markerless Motion Capture for Biomechanical Whole-Body Kinematic Estimation in Infants
May 16, 2026Divya Joshi, J. D. Peiffer, Colleen Peyton +1Markerless Motion Capture 3D Human Pose Estimation
Center for Bionic Medicine, Shirley Ryan AbilityLab, Chicago, IL · Dept. of Physical Therapy and Human Movement Science, Northwestern University, Chicago, IL · Dept. of Biomedical Engineering, Northwestern University, Evanston, IL +2
Cross-Model Distillation of a Human-Pose Foundation Model from Unannotated Infant Video for Markerless 3D Pose Estimation
Sep 1, 2026R. James Cotton, Divya Joshi, Colleen Peyton3D Foundation Models Markerless Motion Capture
1Shirley Ryan AbilityLab · Department of Physical Medicine and Rehabilitation, Northwestern University · Department of Physical Therapy and Human Movement Science, Northwestern University +1
Toward Reliable Infant Pose Estimation: A Training-Dynamics Approach to Noisy Annotation Detection
Oct 5, 2026Emanuele Cardinale, Sara Moccia, Alessandro Cacciatore +1Markerless Motion Capture Noisy-Label Learning
Department of Engineering and Geology, Università degli Studi “G. d’Annunzio” Chieti-Pescara, Pescara, 65127, Italy · Laboratory of Computational Oncology, Department of Oncology, KU Leuven, Leuven, 3000, Belgium · Department of Political Science, Università di Teramo, Teramo, 64100, Italy +1

May 16, 2026J/K move · Enter open · S save

Markerless Motion Capture for Biomechanical Whole-Body Kinematic Estimation in Infants

Divya Joshi, J. D. Peiffer, Colleen Peyton, R. James Cotton

Center for Bionic Medicine, Shirley Ryan AbilityLab, Chicago, IL · Dept. of Physical Therapy and Human Movement Science, Northwestern University, Chicago, IL · Dept. of Biomedical Engineering, Northwestern University, Evanston, IL+2

Markerless Motion Capture 3D Human Pose Estimation

Open paper AskPDF arXiv

arly identification of motor impairment in infancy relies on expert visual assessment of spontaneous movement, motivating the development of automated, objective alternatives. One promising approach is using computer vision, which benefits from high quality pose estimation from video. In this study, we systematically evaluated three state-of-the-art pose estimation frameworks (MeTRAbs-ACAE, SAM 3D Body, and Sapiens) on 100 videos over 13 sessions of 8 infants recorded with a multi-view markerless motion capture system. We quantified keypoint detection accuracy using reprojection error, geometric consistency, and Procrustes-aligned 3D position error, and demonstrated proof-of-concept for fitting an inverse kinematic framework to infant data. While Sapiens achieved the lowest reprojection error and highest geometric consistency of the methods evaluated (22.8 pixels and 0.82, respectively), SAM 3D Body provided the most comprehensive 3D information for kinematic reconstruction with Procrustes-aligned position errors of 19 to 28 mm. We demonstrate in a case comparison example that biomechanical models fit to SAM 3D estimates distinguish representative movement patterns in infants related to motor development, as identified by a clinical expert. Together, these findings highlight both the promise and current limitations of 3D pose estimation for infant biomechanics and establish preliminary groundwork for scalable, video-based assessment of early motor development.

Abstract

Human Pose Estimation

Explore similar work

Markerless Motion Capture for Biomechanical Whole-Body Kinematic Estimation in Infants

Cross-Model Distillation of a Human-Pose Foundation Model from Unannotated Infant Video for Markerless 3D Pose Estimation

Toward Reliable Infant Pose Estimation: A Training-Dynamics Approach to Noisy Annotation Detection