SEAL-pose: Enhancing 3D Human Pose Estimation via a Learned Loss for Structural Consistency

Yeonsung Kim; Junggeun Do; Seunguk Do; Sangmin Kim; Jaesik Park; Jay-Yoon Lee

cs.CVFeb 23, 2026

SEAL-pose: Enhancing 3D Human Pose Estimation via a Learned Loss for Structural Consistency

Authors: Yeonsung Kim, Junggeun Do, Seunguk Do, Sangmin Kim, Jaesik Park, Jay-Yoon Lee

Organizations: Seoul National University, South Korea · Texas A&M University, College Station, Texas, USA

Abstract

3D human pose estimation (HPE) is characterized by intricate local and global dependencies among joints. Conventional supervised losses are limited in capturing these correlations because they treat each joint independently. Previous studies have attempted to promote structural consistency through manually designed priors or rule-based constraints; however, these approaches typically require manual specification and are often non-differentiable, limiting their use as end-to-end training objectives. We propose SEAL-pose, a data-driven framework in which a learnable loss-net trains a pose-net by evaluating structural plausibility. Rather than relying on hand-crafted priors, our joint-graph-based design enables the loss-net to learn complex structural dependencies directly from data. Extensive experiments on three 3D HPE benchmarks with eight backbones show that SEAL-pose reduces per-joint errors and improves pose plausibility compared with the corresponding backbones across all settings. Beyond improving each backbone, SEAL-pose also outperforms models with explicit structural constraints, despite not enforcing any such constraints. Finally, we analyze the relationship between the loss-net and structural consistency, and evaluate SEAL-pose in cross-dataset and in-the-wild settings.

Figures & tables

Figure 1: SEAL-pose improves structural consistency by preserving skeletal symmetry and kinematic connectivity; under the same 2D input, KTPFormer (baseline) can exhibit local joint inconsistencies (e.g., ankle/wrist) that propagate to the overall limb configuration.

Figure 2: (a) Overview of SEAL-pose. SEAL-pose combines a pose-net $F_{\phi}$ that lifts 2D keypoints $x_{i}$ to a predicted 3D keypoints $\tilde{y}_{i}$ with a loss-net $E_{\theta}$ (graph-based or MLP) that predicts an energy score, where $y_{i}$ denotes the ground-truth 3D pose. It adopts alternating optimization : it first freezes $E_{\theta}$ and optimizes $F_{\phi}$ to minimize the energy of predicted poses. Then, it freezes $F_{\phi}$ and trains $E_{\theta}$ so that its energy better reflects pose quality. (b-c) Loss-net Architecture Variants.

Table 3

Method	Whole-body	Body	Face/Aligned $\ddagger$	Hand/Aligned $\ddagger$
Jointformer $\dagger$	88.3	84.9	66.5 / 17.8	125.3 / 43.7
3D-LFM	64.1	60.8	56.6 / 10.4	78.2 / 28.2
SimpleBaseline ( Martinez et al., 2017 )	67.4	63.3	49.9 / 14.1	98.0 / 34.8
+ SEAL-pose (MLP)	62.8	61.1	46.3 / 13.7	90.7 / 34.2
+ SEAL-pose (Graph)	64.8	61.6	47.6 / 13.3	94.0 / 34.5
VideoPose ( Pavllo et al., 2019 )	61.5	57.4	48.8 / 11.9	84.1 / 30.3

Table 3: Performance on the Human3.6M WholeBody . SEAL-pose reduces P-MPJPE across all body parts, resulting in more coherent predictions.

\dagger

from H3WB’s official benchmark.

\ddagger

nose-aligned MPJPE for face and wrist-aligned MPJPE for hands.

Figure 3: Qualitative Comparison of Predicted Poses on H36M . Predictions from SEAL-pose (bottom, blue) demonstrate clear improvements over the baseline (top, red) by producing structures closer to the ground-truth human pose (black).

Dataset	Metric	MPJPE $\downarrow$	LSE $\downarrow$	LLE $\downarrow$	BSLE $\downarrow$
H36M	Ground Truth		0.00	0.00	0.00
	SimpleBaseline ( Martinez et al., 2017 )	43.8	4.85	5.09	6.12
	+ Constraint ( Cao and Zhao, 2021 )	41.5	4.35	4.54	6.17
	+ SEAL-pose	40.7	3.68	3.94	5.49
3DHP	Ground Truth		1.21	0.00	0.00
	SimpleBaseline ( Martinez et al., 2017 )	80.9	10.14	11.60	8.13

Table 4: Structural Consistency Evaluation Across Datasets . SEAL-pose reduces structural error metrics such as LSE, LLE, and BSLE (see § 4.2 for the definitions), improving plausibility.

Figure 4: Comparison of structural consistency in MPI-INF-3DHP. Average structural inconsistency measures (from left to right: LSE, BSLE, LLE) are displayed for predictions of baselines ( red ) and SEAL-pose ( blue ) binned by P-MPJPE. SEAL-pose consistently achieves lower structural errors even under similar P-MPJPEs.

Figure 5: Gradient-Based Inference results on MPI-INF-3DHP . P-MPJPE, LSE, LLE, and BSLE all decrease steadily over iterations, indicating that the loss-net effectively captures structural plausibility and provides meaningful corrective feedback to the pose-net.

Train $\rightarrow$ Test	MPJPE $\downarrow$		P-MPJPE $\downarrow$
Train $\rightarrow$ Test	Baseline	SEAL-pose	Baseline	SEAL-pose
H36M $\rightarrow$ 3DHP	111.4	97.3	83.2	70.3
3DHP $\rightarrow$ H36M	157.0	151.9	109.7	107.9

Table 5: Cross-dataset evaluation between Human3.6M (H36M) and MPI-INF-3DHP (3DHP) using PoseFormerV2. SEAL-pose does not introduce additional degradation beyond the baseline under dataset shift, suggesting no harmful loss-net overfitting.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Gradient-Based Inference results on Human3.6M . P-MPJPE, LSE, LLE, and BSLE all decrease steadily over iterations, indicating that the loss-net effectively captures structural plausibility and provides meaningful corrective feedback to the pose-net.

Setting	SimpleBaseline	SemGCN	VideoPose
SEAL-pose (Graph)	68.2 (-12.7)	62.9 (-11.6)	62.2 (-4.2)
SEAL-pose (Graph + pair-loss)	65.6 (-15.3)	62.3 (-12.2)	64.1 (-2.3)

Appendix

Table 6: SEAL-pose on 3DHP (single-frame models). In parentheses:

\Delta

MPJPE vs. each backbone baseline.

Setting	$\bm{\Lambda}$	$\bm{K}$	$T$	MPJPE $\downarrow$	P-MPJPE $\downarrow$
MixSTE (baseline)	—	—	243	20.8	16.1
SEAL-pose (Graph margin)	$10^{-3}$	—	243	20.3 (-0.5)	15.8 (-0.3)
SEAL-pose (Graph margin + pair-loss)	$10^{-3}$	3	243	20.0 (-0.8)	15.7 (-0.4)

Appendix

Table 7: SEAL-pose on H36M with fixed

\Lambda

and

K

columns. In parentheses:

\Delta

vs. MixSTE (

T{=}243

); negative is better.

Figure 7: Convergence comparison between the baseline and SEAL-pose during training. Left: MPJPE, Right: P-MPJPE across epochs. Compared to the baseline (red), SEAL-pose (blue) exhibits similarly stable training dynamics while converging slightly faster, reaching comparable or lower error levels earlier.

Figure 8: Energy convergence across hyperparameter settings. Across hyperparameters, the energy score consistently converges to a stable regime, suggesting that training remains stable and proceeds reliably after reaching a steady sub-optimal solution.

pose-net lr	loss-net lr	energy weight	MPJPE
0.00010	0.00010	0.005	62.178
0.00010	0.00010	0.002	62.308
0.00005	0.00003	0.004	62.476
0.00005	0.00003	0.005	62.505
0.00010	0.00003	0.004	62.602
0.00010	0.00003	0.005	62.719

Appendix

Table 8: Hyperparameter sweep over learning rates(pose-net/loss-net) and energy weight.

Figure 9: Qualitative Comparisons of SEAL-pose with PoseFormerV2 ( Zhao et al., 2023 ) on In-The-Wild images. Provides visualizations on unseen In-The-Wild images, illustrating that our method produces structurally plausible poses in out-of-distribution settings.

Figure 10: Qualitative Comparisons of SEAL-pose with KTPFormer ( Peng et al., 2024 ) on an In-The-Wild Images. Provides visualizations on unseen In-The-Wild images, illustrating that our method produces structurally plausible poses in out-of-distribution settings.

Backbone	Baseline	M1	M2	M3	M4	Min–Max (Range)
SimpleBaseline	80.9	66.6	67.0	68.2	66.8	66.6–68.2 (1.6)
SemGCN	74.5	64.3	63.2	62.9	67.3	62.9–67.3 (4.4)
VideoPose	66.4	66.5	66.5	62.2	65.9	62.2–66.5 (4.3)
Avg. (3 backbones)	73.9	65.8	65.6	64.4	66.7	–

Appendix

Table 9: MPJPE sensitivity to loss-net input designs (M1–M4). MPJPE is reported for four input designs with the backbone baseline. Best results among M1–M4 are boldfaced; Min–Max (Range) summarizes variation across designs.

Panel of Figure 11 — Figure 11: Cross-dataset error distributions in PoseFormerV2. SEAL-pose closely follows the baseline distribution and does not introduce additional degradation under dataset shift, while typically reducing the high-error tail.

Explore similar work

HYPERPOSE: Hyperbolic Kinematic Phase-Space Attention for 3D Human Pose Estimation
May 11, 2026Vinduja Thekkath, Ashish Musale, Ajay Waghumbare +13D Human Pose Estimation Attention Mechanisms
DSR-Net: Learning Dynamic Skeletal Relations for 3D Human Pose Estimation
Oct 10, 2026Fan Li, Jian Xiang, LinLin Xue +13D Human Pose Estimation Dynamic Graph Learning
School of Computer Science and Technology, Zhejiang University of Science and Technology, Liuxia, Hangzhou 310023, China
PRGCN: A Graph Memory Network for Cross-Sequence Pattern Reuse in 3D Human Pose Estimation
Oct 22, 2025Zhuoyang Xie, Yibo Zhao, Hui Huang +2Memory-Augmented Neural Networks Graph Neural Networks
College of Computer Science and Artificial Intelligence, Wenzhou University, Wenzhou, 325035, P.R China. · the Key Laboratory of Computer Vision and System, Ministry of Education, Tianjin University of Technology, Tianjin, 300384, China. · School of Data Science and Artificial Intelligence, Wenzhou University of Technology, Wenzhou, 325000, P.R China. +1

May 11, 2026J/K move · Enter open · S save

HYPERPOSE: Hyperbolic Kinematic Phase-Space Attention for 3D Human Pose Estimation

Vinduja Thekkath, Ashish Musale, Ajay Waghumbare, Upasna Singh

3D Human Pose Estimation Attention Mechanisms Hyperbolic Representation Learning

Open paper AskPDF arXiv

We introduce HYPERPOSE, a novel 3D human pose estimation framework that performs spatio-temporal reasoning entirely within the Lorentz model of hyperbolic space

\mathbb{H}^d

to natively preserve the hierarchical tree topology of the human skeleton. Current state-of-the-art pose estimators aim to capture complex joint dynamics by relying on transformers and graph convolutional networks. Since these architectures operate exclusively in Euclidean space which fundamentally mismatches the inherent tree structure of the human body, these methods inevitably suffer from exponential volume distortion and struggle to maintain structural coherence. To this end, we depart from flat spaces and aim to improve geometric fidelity with Hyperbolic Kinematic Phase-Space Attention (HKPSA), natively embedding complex joint relationships without distortion, alongside a multi-scale windowed hyperbolic attention mechanism that efficiently models temporal dynamics in

O(TW)

complexity. Furthermore, to overcome the well-known instability of training non-Euclidean manifolds, HYPERPOSE introduces a novel Riemannian loss suite and an uncertainty-weighted curriculum, enforcing physical geodesic constraints like bone length and velocity consistency. Extensive evaluations on the Human3.6M and MPI-INF-3DHP datasets demonstrate that HYPERPOSE achieves state-of-the-art structural and temporal coherence, significantly reducing both volume distortion and velocity error, while establishing new state-of-the-art benchmarks in overall positional accuracy.

Abstract

Explore similar work

HYPERPOSE: Hyperbolic Kinematic Phase-Space Attention for 3D Human Pose Estimation

DSR-Net: Learning Dynamic Skeletal Relations for 3D Human Pose Estimation

PRGCN: A Graph Memory Network for Cross-Sequence Pattern Reuse in 3D Human Pose Estimation