ER-Pose: Rethinking Keypoint-Driven Representation Learning for Real-Time Human Pose Estimation

Nanjun Li; Pinqi Cheng; Zean Liu; Minghe Tian; Xuanyin Wang

cs.CVMar 9, 2026

ER-Pose: Rethinking Keypoint-Driven Representation Learning for Real-Time Human Pose Estimation

Authors: Nanjun Li, Pinqi Cheng, Zean Liu, Minghe Tian, Xuanyin Wang

Abstract

Single-stage multi-person pose estimation aims to jointly perform human localization and keypoint prediction within a unified framework, offering advantages in inference efficiency and architectural simplicity. Consequently, multi-scale real-time detection architectures, such as YOLO-like models, are widely adopted for real-time pose estimation. However, these approaches typically inherit a box-driven modeling paradigm from object detection, in which pose estimation is implicitly constrained by bounding-box supervision during training. This formulation introduces biases in sample assignment and feature representation, resulting in task misalignment and ultimately limiting pose estimation accuracy. In this work, we revisit box-driven single-stage pose estimation from a keypoint-driven perspective and identify semantic conflicts among parallel objectives as a key source of performance degradation. To address this issue, we propose a keypoint-driven learning paradigm that elevates pose estimation to a primary prediction objective. Specifically, we remove bounding-box prediction and redesign the prediction head to better accommodate the high-dimensional structured representations for pose estimation. We further introduce a keypoint-driven dynamic sample assignment strategy to align training objectives with pose evaluation metrics, enabling dense supervision during training and efficient NMS-free inference. In addition, we propose a smooth OKS-based loss function to stabilize optimization in regression-based pose estimation. Based on these designs, we develop a single-stage multi-person pose estimation framework, termed ER-Pose. On MS COCO and CrowdPose, ER-Pose-n achieves AP improvements of 3.2/6.7 without pre-training and 7.4/4.9 with pre-training respectively compared with the baseline YOLO-Pose. These improvements are achieved with fewer parameters and higher inference efficiency.

Figures & tables

Panel of Figure 1 — Fig. 1 : Performance comparison between ER-Pose and representative human pose estimation methods on the COCO test-dev set. (a) Accuracy vs. inference time. (b) Accuracy vs. number of parameters. ER-Pose demonstrates superior inference efficiency while maintaining competitive accuracy.

Fig. 2 : Illustration of the typical box-driven pipeline: the bounding-box-centered single-stage pose estimation paradigm.

Fig. 3 : Overview of the ER-Pose framework. ER-Pose adopts keypoint-driven single-stage architecture. Feature extraction design follows YOLOv8, with CSPDarkNet [ 57 , 7 ] used as backbone and PAN [ 58 ] employed as neck to produce multi-scale features. Head consists of human confidence branch and pose branch. Confidence branch adopts DSConv [ 59 ] to reduce computational cost, while keypoint regression employs standard convolution to preserve sufficient representational capacity. Network is trained under MAH-SAH scheme.

Fig. 4 : Suboptimal keypoint feature selection induced by box-driven modeling. The upper part shows positive samples selected by the network, while the lower part presents samples from the same network that yield more accurate keypoint estimates but are suppressed during inference.

Panel of Figure 6 — Fig. 6 : Comparison of Gaussian, Laplace, and SOKS distributions.

Method	Backbone	Input Size	Params (M)	Latency (ms)	AP	$AP_{50}$	$AP_{75}$
HigherHRNet [ 21 ]	HRNet-W32 [ 17 ]	512 $\times$ 512	28.6	34.398	66.4	87.5	72.8
CenterNet [ 45 ]	Hourglass-3x [ 23 ]	512 $\times$ 512	194.9	45.855	63.0	86.8	69.6
DEKR [ 51 ]	HRNet-W32 [ 17 ]	512 $\times$ 512	29.6	33.935	68.0	86.7	74.5
DEKR [ 51 ]	HRNet-W48 [ 17 ]	640 $\times$ 512	65.7	36.585	71.0	88.3	77.4
CID [ 47 ]	HRNet-W32 [ 17 ]	512 $\times$ 512	29.4	45.678	68.9	88.5	76.6
ED-Pose [ 42 ]	ResNet-50 [ 63 ]	800 $\times$ 1333	50.6	171.971	69.8	90.2	77.2

TABLE I : Comparison results of ER-Pose with advanced methods on the COCO test-dev set. Inference latency was measured on one Tesla V100 GPU under identical hardware conditions, with evaluation performed using onnxruntime.

Input size	Latency(ms)	AP	$\mathrm{AP}_{50}$	$\mathrm{AP}_{75}$	AR
640	10.587	57.1	84.6	62.0	62.4
800	11.844	59.5	86.2	65.0	64.5
960	13.412	60.9	86.8	68.8	65.9

TABLE II : Comparison of model performance under different input Resolutions

Box branch	Keypoint-driven Assignment	AP	$AP_{50}$	$AP_{75}$	AR
$\checkmark$	$\times$	50.4	79.5	54.2	57.8
$\checkmark$	$\checkmark$	54.7	83.2	58.8	60.2
$\times$	$\checkmark$	57.1	84.6	62.0	62.4

TABLE III : Semantic ablation experiments on the box-based branch

Panel of Figure 10 — Fig. 7 : Gradient heatmaps of confidence prediction branches for models of tiny and small variants. From left to right, results correspond to models using one, two, and three feature scales. (a) Grad-CAM visualization of ER-Pose-n. (b) Grad-CAM visualization of ER-Pose-s.

scale	features	AP	$AP_{m}$	$AP_{l}$	AR	$AR_{m}$	$AR_{l}$
n	{P5}	55.9	46.7	69.1	60.4	51.2	73.2
	{P4, P5}	57.1	49.1	68.8	62.4	54.9	73.0
	{P3, P4, P5}	56.4	47.5	68.9	61.6	53.4	73.0
s	{P5}	64.0	56.1	75.9	68.4	60.5	79.7
s	{P4, P5}	64.0	58.9	72.1	68.6	63.4	75.9
	{P3, P4, P5}	65.7	58.7	75.9	70.5	63.9	79.7

TABLE IV : Branch ablation experiments

Panel of Figure 12 — Fig. 8 : Gradient heatmaps of confidence prediction branches for models of tiny and small variants. From left to right, results correspond to models using one, two, and three feature scales. (a) Grad-CAM visualization of ER-Pose-n. (b) Grad-CAM visualization of ER-Pose-s.

scale	features
n	{P4, P5}
s	{P3, P4, P5}
m	{P3, P4, P5}
l	{P3, P4, P5}

TABLE V : Features maps protocol

IoU thr.	ER-Pose(OKS-IoU)			Box-driven(IoU)
IoU thr.	AP	AR	NMS(ms)	AP	AR	NMS(ms)
0.4	57.1	61.5	7.434	49.9	56.3	0.978
0.5	57.0	61.7	8.769	50.1	56.7	1.085
0.6	56.9	61.8	9.746	50.3	57.1	1.034
0.7	56.6	61.9	12.199	50.4	57.4	1.040
0.8	56.2	62.2	13.791	50.4	57.8	0.951

TABLE VI : Model performance comparison under different positive sample assignment centers and NMS strategies

model	AP	$AP_{50}$	$AP_{75}$	AR	latency(ms)
SAH	53.1	81.8	56.7	58.8	10.587
MAH+NMS(OKS)	56.6	83.4	61.4	60.8	13.544
MAH+SAH	57.1	84.6	62.0	62.4	10.721

TABLE VII : Ablation results of multi-head structures. SAH denotes models trained using single-head assignment, where inference relies exclusively on SAH. Models using only MAH adopt OKS-based NMS during inference to suppress redundant predictions.

Fig. 9 : Grad-CAM visualizations of dual-head structures.

Fig. 10 : Visualization of human target confidence maps.

$\alpha$	$\beta$	AP	$AP_{50}$	$AP_{75}$	AR
0.5	0	50.7	78.9	54.6	56.7
0.5	2	46.2	80.6	46.4	52.0
0.5	4	54.8	82.9	58.8	60.4
0.5	6	57.1	84.4	62.4	62.2
0.5	8	56.7	84.4	61.2	62.1
1.0	0	48.7	77.7	52.2	54.1

TABLE VIII : Comparison experiments under different

\alpha

and

\beta

settings.

Fig. 11 : Task alignment deviation under different

\alpha

and

\beta

settings. Red bars denote alignment deviation of box-driven models, where positive samples are assigned based on box localization. Blue bars represent alignment deviation of ER-Pose-n architecture.

Fig. 12 : Scatter Plot of TAE vs. AP

Fig. 13 : Comparison of task alignment, illustrating effects of keypoint-driven assignment on sample selection.

Loss	ER-Pose-n		ER-Pose-s		ER-Pose-m
Loss	AP	AR	AP	AR	AP	AR
Lap	55.7	60.6	62.2	67.2	68.9	73.0
Gau(OKS)	56.1	61.4	65.3	70.1	69.1	73.3
SOKS	57.1	61.6	65.8	70.2	69.7	73.8

TABLE IX : Performance evaluation under different loss functions.

model	Backbone	Input size	Params(M)	AP	$\mathrm{AP_{50}}$	$\mathrm{AP_{75}}$	$\mathrm{AP_{E}}$	$\mathrm{AP_{M}}$	$\mathrm{AP_{H}}$
SimpleBaseline [ 22 ]	ResNet-101	256 $\times$ 192	34.0	60.8	81.4	65.7	71.4	61.2	51.2
DEKR [ 22 ]	HRNet-W32	512 $\times$ 512	65.7	67.3	86.4	72.2	74.6	68.1	58.7
CID [ 47 ]	HRNet-W32	512 $\times$ 512	29.4	71.2	89.8	76.7	77.9	71.9	63.8
HigherHRNet [ 21 ]	HRNet-W32	512 $\times$ 512	63.8	65.9	86.4	70.6	73.3	66.5	57.9
YOLO-Pose-s [ 6 ]	CSPDarknet	640 $\times$ 640	11.6	61.9	87.6	67.8	69.3	62.5	53.9
YOLO-Pose-m [ 6 ]	CSPDarknet	640 $\times$ 640	26.4	66.6	89.4	73.7	73.8	67.3	58.8

TABLE X : Evaluation on Crowdpose

Panel of Figure 24 — Fig. 14 : Visualization comparison of ER-Pose and advanced regression methods on partial images of CrowdPose. (a) YOLO-Pose-s test results. (b) KAPAO-s test results. (c) ER-Pose-s test results. ER-Pose can better localize humans in complex and occluded scenes, and can achieve better accuracy.

Setting	Dataset	YOLO-Pose-n	ER-POSE-n	$\Delta AP$
From scratch	COCO	47.5	50.7	+3.2
Pretrained	COCO	50.4	57.1	+6.7
From scratch	CrowdPose	45.5	52.9	+7.4
Pretrained	CrowdPose	53.4	58.3	+4.9
Pretrained(COCO)	OCHuman	30.7	37.1	+6.4

TABLE XI : Comparison of AP with and without pretraining on COCO, CrowdPose, and OCHuman, relative to baseline YOLO-Pose.

Fig. 15 : Visualization examples of ER-Pose-n predictions across diverse scenarios.

Human Pose Estimation

Human pose estimation recovers people’s body keypoints, 3D skeletons, or meshes from images, video, and other sensor data.

See all 79 papers

Jun 16, 2025

DETRPose: Real-Time End-to-End Multi-Person Pose Estimation via Modified Transformer Decoder and Novel Denoising Keypoints

Multi-person pose estimation (MPPE), which involves detecting body joint positions (keypoints) for every person in an image, is a fundamental task in computer vision. Despite recent advances, no transformer-based model currently achieves real-time performance. This work addresses the latency challenge by introducing DETRPose, the first family of real-time, end-to-end transformer models for multi-person 2D pose estimation. DETRPose significantly enhances the GroupPose decoder, enabling real-time inference. For training, a novel denoising keypoint technique is proposed to accelerate convergence. The varifocal loss is also extended for keypoints, termed Keypoint Similarity VariFocal loss, to improve query quality. Extensive evaluation demonstrates that DETRPose models achieve accuracy comparable to or exceeding that of leading alternatives while requiring five to ten times fewer training epochs. DETRPose-S matches the accuracy of YOLOv8-Pose-X and YOLO11-Pose-X on the COCO dataset (67.0 vs 67.3 and 67.2 in AP) with 81% fewer parameters (11.5M vs 69.4M and 58.8M) and 52% faster inference speed (2.39ms vs 5.23ms and 4.93ms). On the CrowdPose dataset, DETRPose-X has

13.1\times

fewer FLOPs (232.3G vs 3048.1G) and only

2%

fewer precision (75.1 vs 76.6 in AP) than ED-Pose-SwinL-5S. On the OCHuman dataset, DETRPose-S surpasses all previous models, showing the robustness of DETRPose on out-of-distribution datasets. Code is available at https://github.com/SebastianJanampa/DETRPose

Sebastian Janampa, Marios Pattichis

The University of New Mexico

Detection Transformer Efficient ViTs Human Pose Estimation Keypoint Detection

Ask PDF

Mar 4, 2026

Yolo-Key-6D: Single Stage Monocular 6D Pose Estimation with Keypoint Enhancements

Estimating the 6D pose of objects from a single RGB image is a critical task for robotics and extended reality applications. However, state-of-the-art multi stage methods often suffer from high latency, making them unsuitable for real time use. In this paper, we present Yolo-Key-6D, a novel single stage, end-to-end framework for monocular 6D pose estimation designed for both speed and accuracy. Our approach enhances a YOLO based architecture by integrating an auxiliary head that regresses the 2D projections of an object's 3D bounding box corners. This keypoint detection task significantly improves the network's understanding of 3D geometry. For stable end-to-end training, we directly regress rotation using a continuous 9D representation projected to SO(3) via singular value decomposition. On the LINEMOD and LINEMOD-Occluded benchmarks, YOLO-Key-6D achieves competitive accuracy scores of 96.24% and 69.41%, respectively, with the ADD(-S) 0.1d metric, while proving itself to operate in real time. Our results demonstrate that a carefully designed single stage method can provide a practical and effective balance of performance and efficiency for real world deployment.

Kemal Alperen Çetiner, Hazım Kemal Ekenel

ASELSAN, Aerospace Technologies, Türkiye · Istanbul Technical University, Department of Computer Engineering, Türkiye · New York University Abu Dhabi, Division of Engineering, UAE

Object Pose Estimation

Ask PDF

Aug 15, 2026

TinyDETR-Pose: Towards End-to-End Real-Time Single-Stage 6DoF Object Pose Estimation with Lightweight Transformers

Real-time 6DoF object pose estimation on resource-constrained hardware remains challenging, as accurate correspondence-based and refinement pipelines typically rely on non-differentiable PnP/RANSAC stages or costly iterative refinement, while recent foundation-model-based approaches incur inference costs that are prohibitive for edge deployment. We present TinyDETR-Pose, a lightweight, end-to-end, single-stage framework that jointly detects objects and regresses their full 6D pose in a single forward pass. Built on the efficient LW-DETR architecture, TinyDETR-Pose formulates detection and pose estimation as a set-prediction problem and attaches dedicated MLP heads for rotation, monocular depth, and projected object center regression to each decoder query, eliminating the need for PnP, NMS (non-maximum suppression), or iterative pose refinement. Object symmetries are handled through a ADD-S loss applied uniformly to all objects, without the need for object-specific loss schedules or separate geodesic/ADD supervision. In addition, predictions are assigned to ground truth using a symmetry-safe Hungarian matcher based on class and 2D spatial cues, yielding stable assignment under symmetry and depth ambiguity. On YCB-V, TinyDETR-Pose achieves a comparable ADD-S AUC of 85.9, while requiring up to 72.7% fewer parameters than other DETR-based single-stage pose-estimation approaches. Due to its compact design, TinyDETR-Pose runs in real time and achieves an inference latency of only ~4.5 ms per frame on an NVIDIA Jetson Nano using TensorRT, demonstrating that accurate end-to-end transformer-based 6D pose estimation can be made practical for edge deployment.

Paul Julius Kühn, Duc Anh Nguyen, Saptarshi Neil Sinha +2

Fraunhofer IGD 64283 Darmstadt, Germany · TU Darmstadt, 64289 Darmstadt, Germany · TU Delft 2628 CD Delft, Netherlands

Object Pose Estimation Edge Inference Detection Transformer

Ask PDF

Abstract

Human Pose Estimation

Explore similar work

DETRPose: Real-Time End-to-End Multi-Person Pose Estimation via Modified Transformer Decoder and Novel Denoising Keypoints

Yolo-Key-6D: Single Stage Monocular 6D Pose Estimation with Keypoint Enhancements

TinyDETR-Pose: Towards End-to-End Real-Time Single-Stage 6DoF Object Pose Estimation with Lightweight Transformers