Comprehensive Evaluation of Monocular and Multi-view 3D Human Pose Estimation Models for Fitness Exercise Analysis

Néstor Ortega; Agustín Salgado; Javier Sánchez

cs.CVNov 2, 2026

Comprehensive Evaluation of Monocular and Multi-view 3D Human Pose Estimation Models for Fitness Exercise Analysis

Authors: Néstor Ortega, Agustín Salgado, Javier Sánchez

Organizations: Centro de Tecnologías de la Imagen (CTIM) Instituto Universitario de Cibernética, Empresas y Sociedad (IUCES) University of Las Palmas de Gran Canaria 35017 Las Palmas de Gran Canaria, Spain

Abstract

Three-dimensional human pose estimation has become a fundamental component of computer vision systems for sports analytics, rehabilitation, and motion understanding. Although recent deep learning approaches have achieved remarkable accuracy on standard benchmarks, their performance under realistic sports scenarios remains insufficiently studied. Furthermore, the trade-offs between recent monocular and multi-view approaches, as well as the benefits of post-processing techniques for improving temporal consistency, have not been systematically evaluated on fitness-oriented datasets. This paper evaluates recent state-of-the-art 3D human pose estimation methods on Fit3D, a dataset of synchronized multi-view recordings of a wide variety of fitness exercises with accurate ground-truth annotations. We study two recent monocular lifting approaches, TCPFormer and PoseMamba-L, and a multi-view model, MVGFormer, under a unified evaluation protocol. To investigate the influence of the input data, monocular methods are evaluated using ground-truth keypoints and detections generated by recent 2D pose estimators, RTMPose and YOLO-Pose. Additionally, we analyze the effectiveness of three temporal filters for reducing prediction jitter while preserving motion dynamics. Experimental results provide a detailed comparison of accuracy, temporal consistency, and robustness to imperfect 2D detections across the evaluated methods. Our analysis highlights the impact of detector quality on lifting-based approaches, compares transformer- and state-space-based temporal architectures, and discusses the practical trade-offs between monocular and multi-view systems for fitness motion analysis. Our results show that 2D detection quality, temporal modeling, and post-processing must be considered jointly in pose estimation pipelines for exercise analysis.

Figures & tables

Method	Type	Input	Main idea	Temporal modeling
YOLO-Pose [ 16 ]	2D detector	RGB image	Single-stage pose estimation	No
RTMPose [ 15 ]	2D detector	RGB image/crop	Top-down pose estimation	No
TCPFormer [ 13 ]	Monocular 3D	2D keypoints	Transformer with implicit pose proxy	Yes
PoseMamba-L [ 7 ]	Monocular 3D	2D keypoints	Mamba-based state-space model	Yes
MVGFormer [ 14 ]	Multi-view 3D	Multi-view RGB + calibration	Geometry-aware Transformer	No
One Euro Filter [ 17 ]	Filter	3D trajectories	Adaptive low-pass smoothing	—

Table 1: Summary of the methods and temporal filters evaluated in this work.

Model	Backbone	Input modality	Temporal modeling	Params	MACs/frame	Training
TCPFormer [ 13 ]	Transformer with implicit pose proxy	2D keypoint sequence ( $T=243$ )	Temporal attention, $\mathcal{O}(T^{2})$	35.1 M	449 M	Pretrained on Human3.6M, used as-is
PoseMamba-L [ 7 ]	Bidirectional spatio-temporal state-space model (Mamba)	2D keypoint sequence ( $T=243$ )	State-space scans, $\mathcal{O}(T)$	6.7 M	115 M	Pretrained on Human3.6M, used as-is
MVGFormer [ 14 ]	PoseResNet-50 + Transformer decoder with triangulation	Multi-view RGB images + camera calibration	None (single-frame)	37.6 M	639.7 G	Pretrained on CMU Panoptic, used as-is

Table 2: Summary of the evaluated 3D pose estimation models.

T

is the length of the input sequence. Parameter counts and multiply–accumulate operations (MACs) are those reported by the original authors. For TCPFormer and PoseMamba-L, MACs/frame denotes the computational cost per output frame for a 243-frame sequence. For MVGFormer, the reported MACs correspond to one frame-wise multi-view inference in the original experimental configuration.

Figure 2: Skeleton representations used in the experimental pipeline. YOLO-Pose and RTMPose provide detections in the COCO-17 format. TCPFormer and PoseMamba-L use the 17-joint representation adopted in the monocular pipeline, which contains separate Nose and Head joints. MVGFormer follows a 15-joint Panoptic-style representation that includes Nose but no separate Head joint. For COMMON15 comparison, the monocular Spine and Head joints are therefore excluded, while Nose is retained and matched to the corresponding MVGFormer joint.

Characteristic	Fit3D [ 12 ]
Application domain	Fitness training. Multiple subjects exercising.
Environment	Indoor laboratory
Subjects	13 subjects in the original data collection
Activities	47 exercise types, in 4 families (Warmups, Barbell Exercises, Dumbbell Exercises, Equipment-Free Exercises)
Sequences	611 multi-view sequences ( $\geq 5$ repetitions each), i.e., 2444 videos across the four cameras.
RGB cameras	4 (multiple cameras)

Table 3: Main characteristics of the Fit3D dataset.

Figure 3: Fit3D dataset: Images of the four cameras of frame 400 from the s04 and s08 sequences of the pushup and drag curl exercises, respectively.

Model / input	s03	s04	s05	s07	s08	s09	s10	s11	Mean
TCPFormer – GT 2D	42.17	50.02	42.02	45.91	52.15	39.92	49.56	47.50	46.16
TCPFormer – YOLO-Pose	87.52	94.19	95.21	103.65	95.49	90.16	97.91	84.71	93.61
TCPFormer – RTMPose	69.57	74.63	71.88	70.94	71.23	71.96	69.70	67.20	70.89
PoseMamba-L – GT 2D	34.87	45.79	34.07	36.76	38.78	31.30	39.83	35.57	37.12
PoseMamba-L – YOLO-Pose	78.68	85.83	82.51	94.17	85.16	80.35	92.54	75.32	84.32
PoseMamba-L – RTMPose	71.55	77.07	69.36	69.05	71.75	72.13	69.43	68.81	71.14

Table 4: Mean PA-MPJPE (mm) per subject for TCPFormer and PoseMamba-L using three 2D input sources, evaluated on their original 17-joint representation.

Model / input	s03	s04	s05	s07	s08	s09	s10	s11	Mean
TCPFormer – RTMPose	68.394	71.662	71.473	70.798	72.220	70.441	67.010	64.527	69.566
PoseMamba-L – RTMPose	68.922	74.660	68.557	68.321	71.521	69.209	64.481	64.476	68.768
MVGFormer (multi-view)	61.672	60.299	67.718	70.701	77.136	71.240	55.807	68.952	66.691

Table 5: Mean PA-MPJPE (mm) per subject for TCPFormer, PoseMamba-L, and MVGFormer using the common 15-joint representation. The monocular models use RTMPose as 2D input.

Frame	Ground truth	TCPFormer	PoseMamba-L	MVGFormer
568
1854
1958

Figure 7: Qualitative comparison of 3D pose estimations for the man maker sequence at three representative frames. Ground truth is compared with TCPFormer, PoseMamba-L, and MVGFormer predictions. In frames 1854 and 1958, the person lies on the floor, where the pose is more difficult to estimate, especially for MVGFormer.

Model / version	s03	s04	s05	s07	s08	s09	s10	s11	Mean
TCPFormer – Original	68.394	71.662	71.473	70.798	72.220	70.441	67.010	64.527	69.566
TCPFormer – One Euro filter	68.560	71.779	71.621	70.897	72.427	70.552	67.194	64.679	69.713
TCPFormer – Savitzky–Golay filter	68.386	71.651	71.460	70.787	72.212	70.431	67.001	64.511	69.555
TCPFormer – Median filter	68.394	71.661	71.470	70.794	72.220	70.439	67.008	64.517	69.563
PoseMamba-L – Original	68.922	74.660	68.557	68.321	71.521	69.209	64.481	64.476	68.768
PoseMamba-L – One Euro filter	68.923	74.656	68.555	68.318	71.526	69.207	64.485	64.478	68.769

Table 6: Comparison of mean PA-MPJPE (mm) per subject and model, before and after temporal filtering, evaluated on the common 15-joint skeleton. TCPFormer and PoseMamba-L use RTMPose as the 2D input source.

Model / version	s03	s04	s05	s07	s08	s09	s10	s11	Mean
TCPFormer – Original	2.823	2.867	3.216	3.047	3.325	2.685	2.925	2.659	2.943
TCPFormer – One Euro filter	2.803	2.848	3.187	3.011	3.323	2.632	2.887	2.608	2.913
TCPFormer – Savitzky–Golay filter	2.764	2.825	3.149	2.972	3.285	2.592	2.847	2.557	2.874
TCPFormer – Median filter	2.792	2.848	3.184	3.007	3.308	2.627	2.880	2.593	2.905
PoseMamba-L – Original	2.774	2.824	3.136	2.938	3.240	2.578	2.841	2.539	2.859
PoseMamba-L – One Euro filter	2.775	2.824	3.136	2.938	3.241	2.578	2.841	2.539	2.859

Table 7: Mean Per-Joint Velocity Error (MPJVE) with respect to the ground truth (mm/frame) per subject, comparing the original predictions with the One Euro, Savitzky–Golay, and Median filters, evaluated on the common 15-joint skeleton. TCPFormer and PoseMamba-L use RTMPose as the 2D input source.

Figure 9: Mean body velocity for TCPFormer on the band pull apart sequence of subject s03, evaluated using COMMON15. The top plot shows the complete sequence, while the bottom plot shows a zoomed view between frames 200 and 400, comparing the ground truth, original predictions, and temporally filtered trajectories.

Figure 10: Mean body velocity for MVGFormer on the band pull apart sequence of subject s03, evaluated using COMMON15. The top plot shows the complete sequence, while the bottom plot shows a zoomed view between frames 200 and 400.

Ground truth	MVGFormer	One Euro	Savitzky–Golay	Median Filter
band pull apart – Frame 142

man maker – Frame 2007

Figure 11: Qualitative comparison of MVGFormer predictions before and after temporal filtering in two representative sequences. The first row shows the band pull apart sequence, characterized by a relatively regular standing motion, whereas the second row shows the more challenging man maker sequence, which involves a complex floor-supported body configuration. Ground truth is compared with the original MVGFormer prediction and the outputs obtained after applying the One Euro, Savitzky–Golay, and Median filters. The examples illustrate how temporal filtering can attenuate isolated reconstruction errors and improve the consistency of the predicted pose, particularly in the Median-filtered output.

3D Human Pose Estimation

3D human pose estimation recovers people’s body-joint positions or full-body shape and motion from images, video, and other sensors.

See all 119 papers

STA-TFM: Spatio-Temporal Aggregation Across Views TransForMer for Pose Estimation
Sep 21, 2026Mena Kamel, Natalie Won, Amrut Sarangi +23D ViTs Multi-View Learning
SanofiDigital R&D, K703, Industriepark Höchst, 65929 Frankfurt am Main, Germany · SanofiDigital R&D, Carrer de Rosselló i Pòrcel, 21, Nou Barris, 08016 Barcelona, Spain
MoViD: View-Invariant 3D Human Pose Estimation via Motion-View Disentanglement
Mar 29, 2026Yejia Liu, Hengle Jiang, Haoxian Liu +23D Human Pose Estimation Disentangled Representation Learning
Department of Computer Science and Engineering, Hong Kong University of Science and Technology
Unsupervised 3D Human Pose Estimation via Conditional Multi-view Ancestral Sampling
May 15, 2026Ryohei Goto, Takuya Fujihashi, Shunsuke Saruwatari +1Diffusion Sampling 3D Human Pose Estimation
The University of Osaka

Sep 21, 2026J/K move · Enter open · S save

STA-TFM: Spatio-Temporal Aggregation Across Views TransForMer for Pose Estimation

Mena Kamel, Natalie Won, Amrut Sarangi, Sven Jager+1

SanofiDigital R&D, K703, Industriepark Höchst, 65929 Frankfurt am Main, Germany · SanofiDigital R&D, Carrer de Rosselló i Pòrcel, 21, Nou Barris, 08016 Barcelona, Spain

3D ViTs Multi-View Learning 3D Human Pose Estimation

Open paper AskPDF arXiv

Monocular 3D human pose estimation (HPE) remains challenging due to depth ambiguity, occlu- sions, and the need for temporal consistency. While multi-view methods provide superior accuracy over monocular approaches, they often require complex setups. We introduce STA-TFM, a transformer-based architecture that combines spatial and temporal information for multi-view pose estimation. The approach leverages DSTformer, a monocular feature extractor, to capture long-range pose dependencies within each view. A fusion transformer then aggregates information across views to produce coherent 3D estimates. To address training data scarcity, we use a data generation pipeline that transforms any existing 3D pose dataset into multi-view setups with controllable parameters. Experiments on various datasets demonstrate that STA-TFM outperforms existing camera-parameter-free multi-view methods. STA-TFM achieves 50.9% and 49.5% reductions in mean per joint position error (MPJPE) and mean per joint velocity error (MPJVE) on the DHP19 dataset. Furthermore, it achieves 6.7% and 7.7% respective reductions on HAA4D, and a 15.2% MPJPE reduction on TotalCapture. STA-TFM handles noisy and missing 2D inputs, supporting potential deployment in healthcare monitoring, athletic assessment, and immersive technologies. Code, training checkpoints, and data are available at https://zenodo.org/records/22832620.

Abstract

3D Human Pose Estimation

Explore similar work

STA-TFM: Spatio-Temporal Aggregation Across Views TransForMer for Pose Estimation

MoViD: View-Invariant 3D Human Pose Estimation via Motion-View Disentanglement

Unsupervised 3D Human Pose Estimation via Conditional Multi-view Ancestral Sampling