SIMSPINE: A Biomechanics-Aware Simulation Framework for 3D Spine Motion Annotation and Benchmarking

Muhammad Saif Ullah Khan; Didier Stricker

cs.CVFeb 24, 2026

SIMSPINE: A Biomechanics-Aware Simulation Framework for 3D Spine Motion Annotation and Benchmarking

Authors: Muhammad Saif Ullah Khan, Didier Stricker

Organizations: German Research Center for Artificial Intelligence (DFKI)

Abstract

Modeling spinal motion is fundamental to understanding human biomechanics, yet remains underexplored in computer vision due to the spine's complex multi-joint kinematics and the lack of large-scale 3D annotations. We present a biomechanics-aware keypoint simulation framework that augments existing human pose datasets with anatomically consistent 3D spinal keypoints derived from musculoskeletal modeling. Using this framework, we create the first open dataset, named SIMSPINE, which provides sparse vertebra-level 3D spinal annotations for natural full-body motions in indoor multi-camera capture without external restraints. With 2.14 million frames, this enables data-driven learning of vertebral kinematics from subtle posture variations and bridges the gap between musculoskeletal simulation and computer vision. In addition, we release pretrained baselines covering fine-tuned 2D detectors, monocular 3D pose lifting models, and multi-view reconstruction pipelines, establishing a unified benchmark for biomechanically valid spine motion estimation. Specifically, our 2D spine baselines improve the state-of-the-art from 0.63 to 0.80 AUC in controlled environments, and from 0.91 to 0.93 AP for in-the-wild spine tracking. Together, the simulation framework and SIMSPINE dataset advance research in vision-based biomechanics, motion analysis, and digital human modeling by enabling reproducible, anatomically grounded 3D spine estimation under natural conditions.

Figures & tables

Figure 1: SimSpine annotations. Neutral pose of the simulated spine model (left), divided into three anatomical regions—cervical (pink), thoracic (blue), and lumbar (green)—with 15 annotated landmarks: 9 along the vertebral column, 2 on the skull, 2 at the clavicle joints, and 2 on the shoulder blades (last 4 not shown) . Cervical and thoracic motion is limited to transition junctions (indicated by anatomical axis markers), while intermediate vertebrae remain rigidly coupled. The lumbar segment (L1–L5) is fully articulated, with intervertebral rotations simulated for all degrees of freedom (bottom right). Overall spinal curvature is characterized by thoracic kyphosis ( $\theta_{k}$ ) and lumbar lordosis ( $\theta_{l}$ ) angles. Motion was generated using Beaucage-Gauvreau et al. ’s musculoskeletal model [ 5 ] as a function of full-body movement in Human3.6M, with 5 training and 2 validation subjects performing 15 actions.

Dataset	Year	Input(s)	Label(s)	# Samples	Tasks	Annotation Notes	Avail.
Kim et al. [ 33 ]	2019	Radiographs	Lumbar spine positions (sagittal, 2D)	797 images	Lumbar vertebrae segmentation	2× expert (radiologists)	✗
Horng et al. [ 24 ]	2019	Radiographs	Vertebral segmentation masks (coronal, 2D)	595 images	Vertebral segmentation	Multi-expert (clinical)	$\dagger$
VinDr-SpineXR [ 58 ]	2021	Radiographs	ROI boxes for 13 abnormalities	10k images / 5k cases	Lesion detection	Expert annotations	✓
VerSe [ 64 ]	2021	CT Scans	Vertebra centroid / segmentation masks (sagittal, 2D)	374 scans / 355 patients	Vertebral segmentation	Hybrid (voxel)	✓
Scoliosis1K [ 77 ]	2024	Silhouette Images	Demographics, Scoliosis Diagnosis, 2D Body Pose	1.5k videos (447k frames) from 1k walking subjects	Scoliosis detection	Evaluation by medical professionals	✓
USTC&SYSU-Scoliosis [ 79 ]	2025	Radiographs, RGB	Regional Cobb’s angles	1898 images / 1067 patients aged 10-18	Scoliosis detection	Multi-expert agreement	$\dagger$

Table 1: Comparison of existing spine datasets. Summary of public datasets relevant to spinal imaging, scoliosis analysis, and motion estimation. Our proposed dataset uniquely provides vertebra-level 3D kinematics from RGB videos, bridging clinical imaging, biomechanics, and computer vision domains. Availability: ✓ Public,

\dagger

On request, ✗ Private.

Figure 2: Biomechanics-aware keypoint simulation pipeline. From synchronized multi-view RGB, a 2D detector [ 31 ] predicts spinal landmarks that are robustly triangulated using calibrated cameras to obtain pseudo-3D spinal keypoints. These pseudo labels are temporally aligned and merged with known Human3.6M [ 27 ] 3D markers (GT 3D Pose). OpenSim inverse kinematics (IK) [ 12 ] fits a subject-scaled full-body model [ 53 , 59 , 5 ] to the merged trajectories. We attach virtual markers to vertebral bodies and, using the IK joint angles and subject-specific anthropometrics, generate anatomically consistent spine keypoints via forward kinematics (FK). The pipeline also outputs biomechanical parameters (e.g., per-vertebra rotations).

Panel of Figure 3 — Figure 3: Thoracolumbar spine in SimSpine . Left: Distributions of thoracolumbar curvature across actions, defined by the Lumbar Lordotic Angle (LLA, $\theta_{l}$ ) in the lower back and Thoracic Kyphotic Angle (TKA, $\theta_{k}$ ) in the upper back. LLA and TKA average within 1 SD at 32–41° and 28–39°, respectively, indicating greater curvature in the lower spine but higher variability in the upper. Values fall within reported biomechanical ranges [ 40 , 16 ] , confirming that SimSpine produces anatomically plausible curvatures and captures expected action-specific postural trends [ 4 , 9 , 69 , 66 , 60 ] . Right: Per-vertebra range of motion (ROM) on y-axis for the three lumbar rotational DOFs. Our simulated data (solid) follows similar trends as reported by White and Panjabi (1978) [ 71 ] (dashed).

Figure 4: Cervical spine in SimSpine . The distributions (per action) remain centered near neutral with task-dependent spread, reflecting that our model uses a single 3-DOF aggregate neck joint while keeping the thoracic/cervical bodies rigid beyond the cervicothoracic junction. This is within the neck ROM reported in [ 14 ] with approximately half coverage, which indicates the presence of only small head movements in the dataset.

	Dataset		SpineTrack				Ours
Method	Pretrain	Finetune	AP B	AR B	AP S	AR S	AUC
SpinePose-s	SpineTrack	-	0.792	0.821	0.896	0.908	0.611
SpinePose-m			0.840	0.864	0.914	0.926	0.633
SpinePose-l			0.854	0.877	0.910	0.922	0.633
SpinePose-s-ft	SpineTrack	SpineTrack + Ours	0.788	0.815	0.920	0.929	0.790
SpinePose-m-ft			0.821	0.846	0.928	0.937	0.798

Table 2: Baselines for 2D Spine Pose Estimation. Performance of various CNN and transformer architectures on the SpineTrack and SimSpine benchmarks.

Action	${\mathrm{S_{C}}}$	${\mathrm{S_{T}}}$	${\mathrm{S_{L}}}$	${\mathrm{S}}$	${\mathrm{B}}$	${\mathrm{All}}$
Mean (GT 2D)	12.36	8.80	5.09	9.00	7.07	7.85
Directions	33.69	39.86	40.16	37.88	27.63	31.79
Discussion	36.35	40.55	41.44	39.39	28.33	32.81
Eating	38.47	43.04	35.40	39.48	27.44	32.32
Greeting	30.66	35.19	33.73	33.29	25.57	28.70
Phoning	42.41	44.64	39.14	42.43	27.23	33.40

Table 3: Multiview 3D Spine Reconstruction. MPJPE (mm) across actions and spinal regions using linear triangulation.

S_{C}

: Cervical,

S_{T}

: Thoracic,

S_{L}

: Lumbar,

S

: Full spine,

B

: Body, and All: Complete skeleton.

Action	${\mathrm{S_{C}}}$	${\mathrm{S_{T}}}$	${\mathrm{S_{L}}}$	${\mathrm{S}}$	${\mathrm{B}}$	${\mathrm{All}}$
Mean (GT 2D)	0.33	0.43	0.06	0.67	1.81	1.79
Mean (Finetuned)	19.68	20.67	12.21	26.67	24.21	29.53
Mean (Zero-Shot)	26.88	23.40	9.92	39.01	42.25	46.48

Table 4: Triangulation Baseline: P-MPJPE (mm). Same setup as Table 3 , evaluated after similarity alignment.

Train Set	2D	Direct	Discuss	Eating	Greet	Phone	Photo	Pose	Purchase	Sit	SitD	Smoke	Wait	WalkD	WalkT	Walk	Avg
Spine Only	Det.	16.58	17.92	17.79	18.32	21.81	19.74	15.69	17.70	19.28	22.32	18.43	18.82	18.91	16.24	15.96	18.58
Full-Body	Det.	15.55	16.40	14.45	15.98	19.84	17.66	14.25	15.68	15.97	19.23	16.38	17.17	17.41	12.24	12.12	16.28
Spine Only	GT	15.66	17.01	16.05	17.43	20.64	18.34	14.87	16.40	17.91	22.10	16.92	17.78	17.96	15.39	15.28	17.52
Full-Body	GT	11.56	13.05	11.36	13.61	18.06	14.61	10.60	12.39	13.64	17.36	13.25	14.47	14.51	9.15	9.98	13.48

Table 5: Simple Baselines for Monocular 3D Spine Pose Lifting. Evaluation of Martinez et al. ’s lifting model trained on spine-only (15 joints) and full-body (37 joints) keypoint sets. Reported metric: Procrustes-aligned MPJPE (P-MPJPE, mm) per action. Training on full-body joints improves spine localization accuracy. Evaluation is on 15 spine joints only.

		P-MPJPE				MPJPE
Train Set	2D	${\mathrm{S_{C}}}$	${\mathrm{S_{T}}}$	${\mathrm{S_{L}}}$	${\mathrm{S}}$	${\mathrm{S_{C}}}$	${\mathrm{S_{T}}}$	${\mathrm{S_{L}}}$	${\mathrm{S}}$
Spine Only	Det.	12.98	14.29	3.32	18.58	96.88	75.96	18.83	67.70
Full-Body	Det.	12.81	10.12	3.51	16.28	100.50	72.61	18.16	67.39
Spine Only	GT	12.03	13.45	3.32	17.52	77.93	58.70	15.54	53.60
Full-Body	GT	10.94	7.98	2.03	13.48	39.24	26.97	7.76	25.94

Table 6: Monocular 3D Spine Pose Lifting (Component-Wise). Same experimental setup as Table 5 , but results are aggregated across actions and broken down by spine components. Columns report P-MPJPE (mm) for

S_{C}

(cervical),

S_{T}

(thoracic),

S_{L}

(lumbar), and

S

(full spine). Rows compare training on spine-only (15 joints) vs. full-body (37 joints); 2D inputs are either detected (Det.) or ground truth (GT).

Panel of Figure 5 — Figure 5: Ablation Study: Mixup Composition. We examine how the fraction of SimSpine used in training influences indoor (AUC) and outdoor (AP) performance. Increasing the SimSpine fraction improves indoor performance up to 10%, while outdoor gains saturate by 2–5%. Per-batch (PB) mixup maintains the best balance between indoor and outdoor metrics, whereas per-epoch (PE) alternation favors one domain at the expense of the other. Sampling only 2% of SimSpine achieves near-saturated results on both datasets, indicating diminishing returns from larger fractions.

Explore similar work

MonoMSK: Monocular 3D Musculoskeletal Dynamics Estimation
Nov 24, 2025Farnoosh Koleini, Hongfei Xue, Ahmed Helmy +1Markerless Motion Capture 3D Human Pose Estimation
University of North Carolina at Charlotte
BioHuman: Learning Biomechanical Human Representations from Video
May 14, 2026Yujun Huo, He Zhang, Chentao Song +3Human Pose Estimation
Beihang University, China · Tsinghua University, China
Biomechanically Accurate Gait Analysis: A 3d Human Reconstruction Framework for Markerless Estimation of Gait Parameters
Mar 3, 2026Akila Pemasiri, Ethan Goan, Glen Lichtwark +3Gait Analysis Markerless Motion Capture
Queensland University of Technology, Brisbane · Griffith University, Gold Coast

Nov 24, 2025J/K move · Enter open · S save

MonoMSK: Monocular 3D Musculoskeletal Dynamics Estimation

Farnoosh Koleini, Hongfei Xue, Ahmed Helmy, Pu Wang

University of North Carolina at Charlotte

Markerless Motion Capture 3D Human Pose Estimation 3D Human Reconstruction Inverse Dynamics

Open paper AskPDF arXiv

Reconstructing biomechanically realistic 3D human motion - recovering both kinematics (motion) and kinetics (forces) - is a critical challenge. While marker-based systems are lab-bound and slow, popular monocular methods use oversimplified, anatomically inaccurate models (e.g., SMPL) and ignore physics, fundamentally limiting their biomechanical fidelity. In this work, we introduce MonoMSK, a hybrid framework that bridges data-driven learning and physics-based simulation for biomechanically realistic 3D human motion estimation from monocular video. MonoMSK jointly recovers both kinematics (motions) and kinetics (forces and torques) through an anatomically accurate musculoskeletal model. By integrating transformer-based inverse dynamics with differentiable forward kinematics and dynamics layers governed by ODE-based simulation, MonoMSK establishes a physics-regulated inverse-forward loop that enforces biomechanical causality and physical plausibility. A novel forward-inverse consistency loss further aligns motion reconstruction with the underlying kinetic reasoning. Experiments on BML-MoVi, BEDLAM, and OpenCap show that MonoMSK significantly outperforms state-of-the-art methods in kinematic accuracy, while for the first time enabling precise monocular kinetics estimation.

Abstract

Explore similar work

MonoMSK: Monocular 3D Musculoskeletal Dynamics Estimation

BioHuman: Learning Biomechanical Human Representations from Video

Biomechanically Accurate Gait Analysis: A 3d Human Reconstruction Framework for Markerless Estimation of Gait Parameters