MoViD: View-Invariant 3D Human Pose Estimation via Motion-View Disentanglement

Yejia Liu; Hengle Jiang; Haoxian Liu; Runxi Huang; Xiaomin Ouyang

cs.CVMar 29, 2026

MoViD: View-Invariant 3D Human Pose Estimation via Motion-View Disentanglement

Authors: Yejia Liu, Hengle Jiang, Haoxian Liu, Runxi Huang, Xiaomin Ouyang

Organizations: Department of Computer Science and Engineering, Hong Kong University of Science and Technology

Abstract

3D human pose estimation is a key enabling technology for applications such as healthcare monitoring, human-robot collaboration, and immersive gaming, but real-world deployment remains challenged by viewpoint variations. Existing methods struggle to generalize to unseen camera viewpoints, require large amounts of training data, and suffer from high inference latency. We propose MoViD, a viewpoint-invariant 3D human pose estimation framework that disentangles viewpoint information from motion features. The key idea is to extract viewpoint information from intermediate pose features and leverage it to enhance both the robustness and efficiency of pose estimation. MoViD introduces a view estimator that models key joint relationships to predict viewpoint information, and an orthogonal projection module to disentangle motion and view features, further enhanced through physics-grounded contrastive alignment across views. For real-time edge deployment, MoViD employs a frame-by-frame inference pipeline with a view-aware strategy that adaptively activates flip refinement based on the estimated viewpoint. Evaluations on nine public datasets and newly collected multiview UAV and gait analysis datasets show that MoViD reduces pose estimation error by over 24.2% compared to state-of-the-art methods, maintains robust performance under severe occlusions with 60% less training data, and achieves real-time inference at 15 FPS on NVIDIA edge devices.

Figures & tables

Figure 1. Comparison of the traditional framework for 3D human pose estimation and our proposed approach MoViD. The key idea of MoViD is to extract and leverage viewpoint information for view-invariant pose estimation.

Figure 2. Pose estimation errors across different views.

Figure 5. Typical application scenarios of MoViD: Activity recognition across diverse deployment settings and human-robot interaction during navigation.

Figure 6. MoViD features a view estimator to predict viewpoint information from intermediate pose estimation features. This information is then leveraged to enable view-invariant pose estimation through view disentanglement and physics-enhanced contrastive learning, as well as enhancing frame-based adaptive inference to reduce latency.

Figure 7. Illustration of the view estimator. It computes angular differences between key bone vectors, and leverages a view encoder to extract spatial-temporal relationship for view prediction.

Figure 8. View-orthogonal contrastive learning. We first disentangles motion and view features through the orthogonal projection module, and then extracts view-invariant motion features through physics-enhanced contrastive learning across different views.

Figure 9. MoViD features a new frame-by-frame inference pipeline with adaptive activation of the flip refinement module to reduce inference latency.

Dataset	Views	Modalities	Usage
Human3.6M ( Ionescu et al., 2014 )	4	RGB, 3D Keypoints, SMPL	Training
MPI-INF-3DHP ( Mehta et al., 2017 )	14	RGB, 3D Keypoints, SMPL	Training
InstaVariety ( Kanazawa et al., 2019 )	Varied	RGB, 2D Keypoints	Training
AMASS ( Mahmood et al., 2019 )	Synthetic	3D Keypoints, SMPL	Training
HuMMan ( Cai et al., 2022 )	10	RGB, Depth, 3D Keypoints, SMPL	Testing
RICH ( Huang et al., 2022 )	8	RGB, Depth, 3D Keypoints, Contact	Testing

Table 1. Summary of datasets used in our experiments. The model is trained on Human3.6M, MPI-INF-3DHP, InstaVariety, and AMASS, and evaluated on HuMMan, RICH, EMDB, 3DPW, EgoHumans and two self-collected datasets.

Category	Approach	HuMMan Dataset (14 joints)			EMDB Dataset (24 joints)			RICH Dataset (24 joints)
		PA-MPJPE	MPJPE	PVE	PA-MPJPE	MPJPE	PVE	PA-MPJPE	MPJPE	PVE
SOTA Pose Estimation Baselines	HMR2.0 ( Goel et al., 2023 )	78.5	115.1	132.8	59.9	98.7	120.2	62.8	103.6	108.3
	ReFit ( Wang and Daniilidis, 2023 )	79.1	117.4	135.2	71.2	104.2	123.9	67.3	93.4	113.4
	HSMR ( Xia et al., 2025 )	65.1	98.2	119.3	52.5	92.3	108.5	57.4	101.2	109.2
	WHAM ( Shin et al., 2024 )	61.1	99.7	112.1	52.8	77.7	93.6	55.7	98.3	111.5
Viewpoint- Invariant Baselines	SPIN ( Kolotouros et al., 2019a )	82.6	122.5	141.7	78.3	110.9	112.6	77.3	113.2	143.4

Table 2. Overall performance comparison of MoViD and various baselines on HuMMan, EMDB, and RICH datasets. Metrics include PA-MPJPE (mm), MPJPE (mm), and PVE (mm). Lower values are better for all metrics (

\downarrow

). Best result is in bold .

Figure 10. The end-to-end system for real-time pose estimation.

Panel of Figure 11 — Figure 11. Accuracy of downstream tasks.

Panel of Figure 12 — Figure 12. Viewpoint estimation performance and its relationship with pose estimation accuracy.

Method	View Acc. (%)	PA-MPJPE (mm)
CNN-based regression ( Ghezelghieh et al., 2016 )	78.3	98.7
Auxiliary prediction ( Wang et al., 2020 )	80.1	84.3
MoViD (Ours)	98.7	49.2

Table 3. Comparison of viewpoint estimation methods on HuMMan.

Setting	Latency (ms)	Power Peak (W)	PA-MPJPE	ACCEL	Stream
WHAM	154.7	13.8	56.10	30.00	✗
MoViD	66.2	8.8	46.88	27.51	✓

Table 4. Overall system performance comparison on the HuMMan dataset.

Module	2D Pose	Preprocessing	Inference	Total
Latency (ms)	26.9	10.3	29.0	66.2

Table 5. Runtime breakdown on Nvidia Jetson Orin NX.

Scenario	MoViD Latency (ms)	End-to-End (ms)	FPS
Good view (flip off)	25.0	62.2	16.1
Bad view (flip on)	62.9	100.1	10.0

Table 6. Latency under good and bad views on Nvidia Jetson Orin NX.

Figure 13. Effect of flip augmentation on different views.

Panel of Figure 14 — Figure 14. Understanding MoViD performance.

Figure 15. UAV trajectories including side-follow, horizontal orbit, and vertical motion, with subjects performing walking, jogging, hops, lunges, and mixed actions.

Panel of Figure 16 — Figure 16. Performance on the self-collected UAV dataset across different trajectories.

Figure 17. The self-collected multimodal multi-view dataset for gait analysis.

		Front View		Left View		Right View		Back View
Action	Metric ( $\downarrow$ )	W	M	W	M	W	M	W	M
Walk	PA-MPJPE (mm)	97.9	87.6	105.2	91.6	108.5	92.3	102.5	86.3
Stairs		65.3	58.2	71.1	55.0	64.8	47.9	65.0	50.4
TUG		50.7	49.7	52.4	53.2	59.0	53.1	67.5	62.4
Bend		56.0	46.5	58.3	51.7	57.5	48.6	46.1	47.3
Squat		53.2	47.9	51.9	46.6	68.6	49.2	55.3	46.2

Table 7. Performance comparison on pose estimation error (PA-MPJPE) and gait analysis error. Lower is better for all metrics. Best result is in bold . W: WHAM, M: MoViD.

Method	Fisheye	Partial occlusion	Low light
WHAM	85mm	50mm	32mm
Ours	69mm	43mm	28mm

Table 8. Robustness across different conditions (PA-MPJPE error).

Explore similar work

STA-TFM: Spatio-Temporal Aggregation Across Views TransForMer for Pose Estimation
Sep 21, 2026Mena Kamel, Natalie Won, Amrut Sarangi +23D ViTs Multi-View Learning
SanofiDigital R&D, K703, Industriepark Höchst, 65929 Frankfurt am Main, Germany · SanofiDigital R&D, Carrer de Rosselló i Pòrcel, 21, Nou Barris, 08016 Barcelona, Spain
VIMCAN: Visual-Inertial 3D Human Pose Estimation with Hybrid Mamba-Cross-Attention Network
May 8, 2026Zepeng Yang, Junxuan Bai, Hao Li +43D Human Pose Estimation Human Pose Estimation
Beihang University · Capital University of Physical Education and Sports · Peng Cheng Laboratory +1
From Sparse to Dense: Spatio-Temporal Fusion for Multi-View 3D Human Pose Estimation with DenseWarper
May 14, 2026Ling Li, Changjie Chen, Yuyan Wang +43D Reconstruction 3D Human Pose Estimation
Department of Computer Science, THUAI, BNRist, Tsinghua University, Beijing, China · Dalian University of Technology, Dalian, China · University of Manchester, Manchester, UK +2

Sep 21, 2026J/K move · Enter open · S save

STA-TFM: Spatio-Temporal Aggregation Across Views TransForMer for Pose Estimation

Mena Kamel, Natalie Won, Amrut Sarangi, Sven Jager+1

SanofiDigital R&D, K703, Industriepark Höchst, 65929 Frankfurt am Main, Germany · SanofiDigital R&D, Carrer de Rosselló i Pòrcel, 21, Nou Barris, 08016 Barcelona, Spain

3D ViTs Multi-View Learning 3D Human Pose Estimation

Open paper AskPDF arXiv

Monocular 3D human pose estimation (HPE) remains challenging due to depth ambiguity, occlu- sions, and the need for temporal consistency. While multi-view methods provide superior accuracy over monocular approaches, they often require complex setups. We introduce STA-TFM, a transformer-based architecture that combines spatial and temporal information for multi-view pose estimation. The approach leverages DSTformer, a monocular feature extractor, to capture long-range pose dependencies within each view. A fusion transformer then aggregates information across views to produce coherent 3D estimates. To address training data scarcity, we use a data generation pipeline that transforms any existing 3D pose dataset into multi-view setups with controllable parameters. Experiments on various datasets demonstrate that STA-TFM outperforms existing camera-parameter-free multi-view methods. STA-TFM achieves 50.9% and 49.5% reductions in mean per joint position error (MPJPE) and mean per joint velocity error (MPJVE) on the DHP19 dataset. Furthermore, it achieves 6.7% and 7.7% respective reductions on HAA4D, and a 15.2% MPJPE reduction on TotalCapture. STA-TFM handles noisy and missing 2D inputs, supporting potential deployment in healthcare monitoring, athletic assessment, and immersive technologies. Code, training checkpoints, and data are available at https://zenodo.org/records/22832620.

Abstract

Explore similar work

STA-TFM: Spatio-Temporal Aggregation Across Views TransForMer for Pose Estimation

VIMCAN: Visual-Inertial 3D Human Pose Estimation with Hybrid Mamba-Cross-Attention Network

From Sparse to Dense: Spatio-Temporal Fusion for Multi-View 3D Human Pose Estimation with DenseWarper