Kinematics-Induced Multimodal 3D Human Pose Estimation with Subject-Level Privacy

Kaushik Bhargav Sivangi; Fani Deligianni

cs.CVOct 2, 2026

Kinematics-Induced Multimodal 3D Human Pose Estimation with Subject-Level Privacy

Authors: Kaushik Bhargav Sivangi, Fani Deligianni

Organizations: Bio-AIm Lab School of Computing Science University of Glasgow

Abstract

Multimodal 3D Human Pose Estimation (3D HPE) combines complementary information from RGB, LiDAR, and mmWave radar, but models trained on correlated observations from the same individuals, raise privacy risks overlooked by record level analysis. We present a unified framework for multimodal 3D HPE that couples kinematics-induced sensor fusion with subject level privacy auditing and private training. First, our multimodal model aligns modality specific joint representation, injects skeletal structure and adaptively aggregates complementary sensor evidence for accurate pose prediction. Second, we formulate a black-box subject membership inference attack for 3D HPE, complemented by an empirical pointwise maximal leakage analysis, which characterizes how individual attack score outcomes change inference about the membership outcome. Third, we instantiate user-level differential privacy via Action Temporal Stratification, a population weighted within-subject sampling strategy that enforces action and temporal coverage. We evaluate our framework on the MM-Fi dataset across three diverse experimental protocols. Source-code will be released upon acceptance.

Figures & tables

Figure 1: Overall framework of the proposed privacy aware multimodal 3D HPE method. RGB, LiDAR, and mmWave radar keypoint representations are integrated through the kinematics induced fusion module to predict the 3D pose. The model is trained on Action Temporal Stratified Subject level DP, while the final model is attacked through subject membership inference and empirical pointwise maximal leakage.

Protocol 3: All Activities
Model	Modality	Random		Cross-Environment		Cross-Subject
		MPJPE $\downarrow$	PA-MPJPE $\downarrow$	MPJPE $\downarrow$	PA-MPJPE $\downarrow$	MPJPE $\downarrow$	PA-MPJPE $\downarrow$
Existing Methods
MM-Fi [ 19 ]	RGB	279.0	81.2	288.6	84.1	285.3	81.9
MM-Fi [ 19 ]	LiDAR	92.5	61.5	303.8	133.0	103.8	61.5
MM-Fi [ 19 ]	Radar	117.0	57.3	161.6	73.7	129.7	60.0

Table 1: Comparison on Protocol 3 using all activities. Lower MPJPE and PA-MPJPE indicate better performance.

$\epsilon$	MPJPE (mm) $\downarrow$	ROC-AUC	Empirical PML (nats)
1	1081.32	0.48	0.0
16	314.85	0.59	0.14
64	278.58	0.64	0.14
100	269.86	0.65	0.29
200	247.72	0.73	0.29
$\infty$	37.65	0.83	0.44

Table 2: Privacy–utility and empirical leakage results under the MM-Fi Protocol 3 cross-subject setting. Each row corresponds to a subject-level differentially private model trained at the indicated privacy budget

\epsilon

and non-dp baseline is also provided. MPJPE measures 3D pose estimation utility, ROC-AUC measures subject-membership discrimination, and empirical PML characterizes leakage in the corresponding membership-evidence channel. Lower MPJPE and PML are better, while ROC-AUC closer to

0.5

indicates weaker membership distinguishability.

Protocol 1: Daily Activities
Model	Modality	Random		Cross-Environment		Cross-Subject
		MPJPE $\downarrow$	PA-MPJPE $\downarrow$	MPJPE $\downarrow$	PA-MPJPE $\downarrow$	MPJPE $\downarrow$	PA-MPJPE $\downarrow$
Existing Methods
Point Transformer [ 54 , 40 ]	Radar	73.09	55.60	88.28	68.79	75.96	58.70
DiffPose [ 55 , 40 ]	Radar	73.44	56.83	86.35	66.87	70.31	54.12
mmDiff [ 40 ]	Radar	65.26	50.35	82.73	63.87	65.62	50.23

Table 3: Comparison on Protocol 1 (Daily Activities) and Protocol 2 (Rehabilitation Activities). Lower MPJPE and PA-MPJPE indicate better performance.

Protocol	Split	Bimodal Fusion	MPJPE $\downarrow$	PA-MPJPE $\downarrow$
Protocol 1: Daily Activities
P1	Random	RGB+LiDAR	42.24	34.02
P1	Random	RGB+Radar	46.17	35.69
P1	Random	LiDAR+Radar	44.89	34.02
P1	Cross-Environment	RGB+LiDAR	55.68	42.68
P1	Cross-Environment	RGB+Radar	60.91	44.73

Table 4: Bimodal fusion ablation on MM-Fi across the three activity protocols and evaluation splits. Results are reported for RGB+LiDAR, RGB+Radar, and LiDAR+Radar combinations. MPJPE and PA-MPJPE are reported in millimeters; lower is better.

Split	Method	MPJPE (mm) $\downarrow$
Random	BEV Baseline	67.03
Random	Our Fusion	43.21
Cross-Environment	BEV Baseline	73.37
Cross-Environment	Our Fusion	67.03
Cross-Subject	BEV Baseline	56.44
Cross-Subject	Our Fusion	45.83

Table 5: Comparison with the BEV-based fusion baseline [ 57 ] on MM-Fi Protocol 3 using the LiDAR+Radar bimodal setting across the random, cross-environment, and cross-subject splits. MPJPE is reported in millimeters; lower is better.

Explore similar work

HYPERPOSE: Hyperbolic Kinematic Phase-Space Attention for 3D Human Pose Estimation
May 11, 2026Vinduja Thekkath, Ashish Musale, Ajay Waghumbare +13D Human Pose Estimation Attention Mechanisms
WiFi-JEPA: Self-supervised Learning for WiFi-CSI 3D Human Pose Estimation
Jul 13, 2026Doeon Kim, Jungyoon Lee, Seongsin Kim +1Self-Supervised Learning Joint-Embedding Predictive Architecture
Department of Intelligent Semiconductors, Soongsil University, Republic of Korea · Department of AI Convergence Security, Soongsil University, Republic of Korea · School of AI Software, Soongsil University, Republic of Korea
RePos: Relative-to-Absolute Output Factorization for Cross-Environment WiFi-Based 3D Human Pose Estimation
Jul 3, 2026Zhangcheng Hou, Tomoaki OhtsukiRelative Pose Estimation Domain Generalization
Faculty of Science and Technology, Keio University, Yokohama 223-8522, Japan

May 11, 2026, cs.CVJ/K move · Enter open · S save

HYPERPOSE: Hyperbolic Kinematic Phase-Space Attention for 3D Human Pose Estimation

Vinduja Thekkath, Ashish Musale, Ajay Waghumbare, Upasna Singh

3D Human Pose Estimation Attention Mechanisms Hyperbolic Representation Learning

Open paper AskPDF arXiv

We introduce HYPERPOSE, a novel 3D human pose estimation framework that performs spatio-temporal reasoning entirely within the Lorentz model of hyperbolic space

\mathbb{H}^d

to natively preserve the hierarchical tree topology of the human skeleton. Current state-of-the-art pose estimators aim to capture complex joint dynamics by relying on transformers and graph convolutional networks. Since these architectures operate exclusively in Euclidean space which fundamentally mismatches the inherent tree structure of the human body, these methods inevitably suffer from exponential volume distortion and struggle to maintain structural coherence. To this end, we depart from flat spaces and aim to improve geometric fidelity with Hyperbolic Kinematic Phase-Space Attention (HKPSA), natively embedding complex joint relationships without distortion, alongside a multi-scale windowed hyperbolic attention mechanism that efficiently models temporal dynamics in

O(TW)

complexity. Furthermore, to overcome the well-known instability of training non-Euclidean manifolds, HYPERPOSE introduces a novel Riemannian loss suite and an uncertainty-weighted curriculum, enforcing physical geodesic constraints like bone length and velocity consistency. Extensive evaluations on the Human3.6M and MPI-INF-3DHP datasets demonstrate that HYPERPOSE achieves state-of-the-art structural and temporal coherence, significantly reducing both volume distortion and velocity error, while establishing new state-of-the-art benchmarks in overall positional accuracy.

Abstract

Explore similar work

HYPERPOSE: Hyperbolic Kinematic Phase-Space Attention for 3D Human Pose Estimation

WiFi-JEPA: Self-supervised Learning for WiFi-CSI 3D Human Pose Estimation

RePos: Relative-to-Absolute Output Factorization for Cross-Environment WiFi-Based 3D Human Pose Estimation