GEKey: Learning Geodesic Eccentric 3D Keypoints via Self-Supervision

Pierre Onghena; Santiago Velasco-Forero; Beatriz Marcotegui

cs.CVNov 5, 2026

GEKey: Learning Geodesic Eccentric 3D Keypoints via Self-Supervision

Authors: Pierre Onghena, Santiago Velasco-Forero, Beatriz Marcotegui

Organizations: Mines Paris – PSL University Centre for Statistics and Images (STIM)

Abstract

Point clouds represent 3D object geometry as sets of spatial coordinates. A fundamental task in their analysis is keypoint detection, i.e., identifying a sparse subset of points that captures the most salient features. While supervised methods rely on manual annotations, unsupervised approaches leverage intrinsic geometric properties to localize keypoints without human input. We introduce GEKey, a self-supervised framework that uses the geodesic eccentricity transform, an SE(3)-invariant geometric measure, to identify instance-specific salient points. These points serve as pseudo-labels to train a model that aligns its predictions with the object's intrinsic structure. GEKey is simple, efficient, and robust, preserving intra-class variation by balancing instance-specific and category-level structure. Extensive experiments on ShapeNetCoreV2, KeypointNet, CAPE, ITOP, and DeformingThings4D show state-of-the-art performance in both keypoint detection and alignment, with strong resilience to noise, downsampling, deformations, and real-world conditions. The code and pretrained models are available at https://github.com/Pierreoo/GEKey.

Figures & tables

Figure 2: The geometric analysis of GEKey-PL, outlining the process from geodesic eccentricity to the derivation of pseudo-labels.

Figure 3: Overview of self-supervised GEKey to generalize keypoint prediction. The blue workflow shows a backbone model predicting keypoints $\hat{Y}$ , which are compared, via the bipartite matching loss, to pseudo-labels $\mathcal{Y}^{*}$ generated by the green pipeline.

Method	Airplane	Bed	Bottle	Cap	Car	Chair	Guitar	Helmet	Knife	Motorcycle	Mug	Table	Vessel	Avg.
ULCS [ 8 ]	71.0	67.0	75.4	57.5	71.3	68.5	50.1	64.1	52.1	78.4	47.4	60.1	76.9	64.6
Skeleton Merger [ 28 ]	72.1	71.9	72.8	59.5	72.0	69.7	69.3	72.4	92.0	95.3	65.9	79.1	94.2	75.9
SC3K [ 50 ]	87.2	80.0	77.4	56.3	76.1	56.7	96.5	55.0	98.3	85.0	46.3	79.2	92.9	75.9
GEKey	98.5	87.0	61.3	65.0	92.7	90.5	99.7	51.3	99.8	98.2	90.9	91.2	96.3	86.3

Table 1: Keypoint inclusivity on KeypointNet, comparing unsupervised learning-based methods.

Method	Airplane	Bed	Bottle	Cap	Car	Chair	Guitar	Helmet	Knife	Motorcycle	Mug	Table	Vessel	Avg.
FPS [ 27 ]	97.1	98.3	97.1	93.5	94.0	97.1	97.8	89.7	98.3	94.2	95.2	98.0	97.1	95.9
ISS [ 48 ]	77.0	95.6	96.0	91.2	92.4	93.7	87.6	88.0	85.2	89.1	93.4	95.4	89.9	90.3
GEKey-PL	99.2	98.7	96.7	93.6	95.5	97.6	98.7	88.6	99.6	94.8	96.0	98.5	98.2	96.6
ULCS [ 8 ]	88.6	94.2	80.9	60.8	83.7	83.9	79.8	79.9	76.8	78.9	89.6	83.0	78.8	81.5
Skeleton Merger [ 28 ]	92.6	84.3	91.4	85.0	90.7	85.9	85.7	82.1	77.4	86.1	83.2	91.3	85.3	86.2
SC3K [ 50 ]	96.3	98.2	98.0	94.6	89.8	95.3	97.6	90.5	98.8	94.3	95.2	97.4	97.2	95.6

Table 2: Keypoint coverage on KeypointNet, comparing hand-crafted (top) detectors and unsupervised learning-based (bottom) methods.

Method	Airplane	Bed	Bottle	Cap	Car	Chair	Guitar	Helmet	Knife	Motorcycle	Mug	Table	Vessel	Avg.
FPS [ 27 ]	67.6	27.9	38.5	17.4	40.4	41.1	64.6	22.9	45.2	37.1	23.0	38.0	48.5	39.4
ISS [ 48 ]	64.0	29.4	38.8	14.3	39.3	38.8	63.7	20.8	38.0	41.2	22.6	32.4	49.7	37.9
GEKey-PL	80.5	45.2	39.6	27.4	31.8	57.1	67.1	23.7	50.0	46.8	26.2	62.6	49.6	46.7
KeypointDeformer [ 13 ]	53.3	40.7	34.0	9.3	53.8	62.0	52.6	13.5	38.1	45.7	12.6	62.0	51.2	40.7
Skeleton Merger [ 28 ]	73.5	59.9	36.5	15.5	53.3	60.0	49.7	16.7	45.0	39.7	23.6	60.9	58.9	45.6
SC3K [ 50 ]	74.9	19.9	38.0	11.3	33.2	39.8	68.0	17.6	47.9	39.7	18.4	46.4	47.5	38.7

Table 3: Keypoint detection on KeypointNet using mIoU metric, comparing hand-crafted (top) detectors and unsupervised learning-based (bottom) methods.

Figure 4: Keypoint detection by GEKey for an example in each category, showing accurate localization at salient regions.

Method	Airplane	Bed	Bottle	Cap	Car	Chair	Guitar	Helmet	Knife	Motorcycle	Mug	Table	Vessel	Avg.
KeypointDeformer [ 13 ]	73.4	69.5	60.3	56.7	80.2	84.3	62.9	55.3	59.8	61.5	69.8	72.6	73.9	67.7
Skeleton Merger [ 28 ]	77.8	61.3	59.9	53.1	79.5	76.8	63.2	57.0	60.4	57.8	67.2	70.0	72.4	65.9
SC3K [ 50 ]	82.9	64.9	62.7	59.7	75.2	87.0	65.7	58.6	63.0	59.4	75.3	76.0	76.0	69.7
Key-Grid [ 11 ]	84.7	70.6	66.2	58.5	84.5	93.4	69.1	62.5	60.9	63.7	79.4	78.6	74.6	72.8
GEKey	87.6	72.8	59.6	63.5	90.3	97.1	43.4	72.9	58.0	83.8	73.6	91.8	54.1	73.0

Table 4: Keypoint alignment on KeypointNet using

\text{DAS}_{\text{SC3K}}

metric, comparing unsupervised learning-based methods.

Figure 5: Keypoint alignment by GEKey, with keypoints mapped to consistent regions, including highly varied geometries.

Figure 6: Robustness of GEKey to noise (top) and downsampling (bottom) perturbations, visualized at noise scales of 0.10 and 0.20 and downsampling ratios of 16 and 32.

Dataset	Method	Inclusivity $\uparrow$	Coverage $\uparrow$	$T_{con}$ $\uparrow$
CAPE [ 22 , 26 ]	ULCS [ 8 ]	47.8	67.6	88.1
	SC3K [ 50 ]	79.3	86.5	70.7
	SelfGeo [ 49 ]	85.9	91.9	90.4
	GEKey	99.5	96.7	93.3
ITOP [ 10 ]	ULCS [ 8 ]	42.1	64.3	76.5
	SC3K [ 50 ]	76.4	83.1	42.9

Table 5: Keypoint quality under non-rigid deformations on the CAPE, ITOP, and DeformingThings4D datasets.

Figure 7: Without temporal supervision in training, GEKey detects keypoints that remain stable under non-rigid human deformations in CAPE (top), and transfers to the real-world ITOP dataset without fine-tuning (bottom), despite self-occlusion and sensor noise.

Figure 8: Keypoints detected by GEKey on animal shapes from DeformingThings4D, obtained from geometry alone without temporal supervision. Placements are precise, stable across frames, and consistent across species.

Figure 9: Comparison of $\mathcal{L}_{\text{match}}$ and $\mathcal{L}_{\text{Sinkhorn}}$ for the airplane and chair categories.

Method	Airplane	Bed	Bottle	Cap	Car	Chair	Guitar	Helmet	Knife	Motorcycle	Mug	Table	Vessel	Avg.
PointNet++ [ 27 ]	86.2	53.0	52.8	27.7	42.6	66.9	63.9	31.2	53.3	66.4	24.7	71.1	57.2	53.6
PTv1 [ 45 ]	84.4	48.3	46.8	29.6	46.1	65.7	69.0	31.1	52.0	60.6	25.6	69.4	55.5	52.6
PTv2 [ 38 ]	86.5	51.9	46.1	32.1	43.9	65.6	71.6	36.5	53.0	66.2	28.6	70.7	55.8	54.5

Table 6: Keypoint detection on KeypointNet using mIoU metric, comparing different backbone architectures with GEKey.

Method	Airplane	Bed	Bottle	Cap	Car	Chair	Guitar	Helmet	Knife	Motorcycle	Mug	Table	Vessel	Avg.
PointNet++ [ 27 ]	83.7	70.8	52.2	62.4	92.8	94.0	48.1	79.6	59.8	82.2	55.2	91.6	53.5	71.2
PTv1 [ 45 ]	81.0	69.9	56.1	60.8	90.3	96.9	37.0	76.3	60.7	74.4	63.9	91.6	56.0	70.4
PTv2 [ 38 ]	87.6	72.8	59.6	63.5	90.3	97.1	43.4	72.9	58.0	83.8	73.6	91.8	54.1	73.0

Table 7: Keypoint alignment on KeypointNet using

\text{DAS}_{\text{SC3K}}

metric, comparing different backbone architectures with GEKey.

Method	Airplane	Bed	Bottle	Cap	Car	Chair	Guitar	Helmet	Knife	Motorcycle	Mug	Table	Vessel	Avg.
FPS [ 27 ]	13.2	10.9	2.8	1.0	2.7	7.4	7.1	2.4	10.7	6.2	2.3	10.7	6.7	6.5
ISS [ 48 ]	1.7	1.3	1.2	1.2	1.5	1.7	1.2	1.4	3.0	1.4	1.1	1.8	1.1	1.5
GEKey-PL	20.0	19.3	2.5	2.5	1.4	13.6	10.8	1.8	14.9	7.8	2.9	21.0	6.3	9.6
GEKey	15.0	12.4	1.0	1.3	0.2	8.7	9.7	1.1	10.9	5.4	1.3	12.0	3.7	6.4

Table 8: Keypoint detection using mIoU metric at a distance threshold of 0.01.

Figure 10: Per-category mIoU (%) curves of GEKey over Euclidean distance thresholds.

Figure 11: Evolution of mIoU-0.10, mIoU-0.01, and $\text{DAS}_{\text{SC3K}}$ over training for the airplane and chair categories, illustrating the negative correlation between detection and alignment.

	Airplane		Chair
	DAS	mIoU	DAS	mIoU
Skeleton Merger [ 28 ]	77.8	73.5	76.8	60.0
GEKey	73.3	86.5	74.9	65.6

Table 9: Keypoint alignment on KeypointNet using

\text{DAS}_{\text{SM}}

metric.

Figure 12: Ground-truth keypoints color-coded by semantic ID, with instance-specific circled to highlight intra-class variability: not all instances share the same semantic label subset, constraining the $\text{DAS}_{\text{SM}}$ metric when predictions adapt to per-shape geometry.

		$k$ -NN	$\operatorname{ecc}_{100}(u)$		$\operatorname{ecc}_{10}(u)$		$\operatorname{ecc}(u)-\delta(\operatorname{ecc}(u))$
	#Obj	$T_{\text{Tot}}$	$T_{\text{Tot}}$	mIoU	$T_{\text{Tot}}$	mIoU	$T_{\text{Tot}}$
Airplane	1022	7.6	79.3	80.5	9.1	80.3	11.6
Bed	146	1.0	11.7	45.2	1.4	42.3	2.4
Bottle	380	2.5	30.2	39.6	3.5	39.1	14.6
Cap	38	0.3	3.0	27.4	0.4	24.9	0.9
Car	1002	7.7	84.8	31.8	10.1	34.8	39.6

Table 10: Processing times (in seconds) for the

k

-NN graph, geodesic eccentricity, and pseudo-label computation stages of GEKey-PL.

Figure 13: Geodesic distances to the wingtip point on the Delaunay triangulation, alongside geodesic distances on the $k$ -NN graph for comparison.

Figure 14: The local maxima of the geodesic eccentricity transform on the Delaunay triangulation and the $k$ -NN graph.

	$10$ -NN		$20$ -NN		$30$ -NN		$40$ -NN
	Cov.	mIoU	Cov.	mIoU	Cov.	mIoU	Cov.	mIoU
Airplane	98.7	79.4	99.2	80.3	99.3	78.3	99.3	77.1
Bed	98.5	37.5	98.7	42.3	98.6	39.6	98.6	38.6
Bottle	96.4	44.4	97.0	39.1	97.1	38.0	97.2	37.5
Cap	93.2	22.0	93.9	24.9	95.0	23.6	95.4	25.1
Car	94.2	42.2	95.4	34.8	95.8	35.5	95.9	36.0

Table 11: Comparison of the

k

-parameter in the

k

-NN graph of GEKey-PL on pseudo-label Coverage (Cov., %) and mIoU (%), computed with

\operatorname{ecc}_{10}(u)

Figure 16: GEKey exhibits strong synthetic-to-real generalization on real-world examples from the OBJ-BG variant of ScanObjectNN, despite challenges such as background clutter, partial occlusions, sensor noise, and varying point density.

Method	Repeatability (%) $\uparrow$	Consistency ( $\times 10^{-2}$ ) $\downarrow$
FPS [ 27 ]	11.6	12.0
ISS [ 48 ]	9.0	11.0
GEKey-PL	32.3	4.4

Table 12: Repeatability and consistency error between SMPL-deformed shapes for ten keypoints.

Figure 15: GEKey-PL with stable keypoints across deformations of the SMPL human model.

Keypoint Detection

Keypoint detection locates distinctive image points or meaningful landmarks for matching, pose estimation, tracking, and geometric localization.

See all 30 papers

GKDT: General Keypoint Detection Transformer
Jul 1, 2026Changsheng Lu, Yuxin Chen, Haokun Gui +5Keypoint Detection
Hong Kong University of Science and Technology, Hong Kong, China · Australian National University, Canberra, Australia · Tencent Inc., China +1
From Pairs to Sequences: Track-Aware Policy Gradients for Keypoint Detection
Feb 24, 2026Yepeng Liu, Hao Li, Liwen Yang +8Keypoint Detection
School of Computer Science, Wuhan University · Xiaomi EV
GESS: Multi-cue Guided Local Feature Learning via Geometric and Semantic Synergy
Apr 7, 2026Yang Yi, Xieyuanli Chen, Jinpu Zhang +2Keypoint Detection Feature Fusion
Y. Yi, X. Chen, J. Zhang, H. Shen, and D. Hu are with the College of Intelligence Science and Technology, National University of Defense Technology, China.

Jul 1, 2026J/K move · Enter open · S save

GKDT: General Keypoint Detection Transformer

Changsheng Lu, Yuxin Chen, Haokun Gui, Rong Wang+4

Hong Kong University of Science and Technology, Hong Kong, China · Australian National University, Canberra, Australia · Tencent Inc., China+1

Keypoint Detection

Open paper AskPDF arXiv

With the emergence of various pre-trained vision and language models, computer vision is shifting from narrow-domain to open-domain recognition. The construction of a more powerful yet general keypoint detection (GKD) model to support diverse tasks has become increasingly important in the field. To this end, we firstly present a large-scale unified keypoint dataset called MegaKPT. The dataset is composed of over 1.3 million diverse object instances from twenty-nine existing datasets, and enjoys high-quality unified annotations with keypoint text descriptions. Based on MegaKPT, we develop GKDT, a simple, flexible and powerful DINOv3 based Transformer model for General Keypoint Detection. Our GKDT supports visual prompts, text prompts, or both. To enhance model training, we also propose a suite of useful strategies such as mix-modal prompted training and dynamic importance sampling. By testing over 22 test sets with seen or unseen objects, our single GKDT model shows strong performance and generality in detecting keypoints on broad categories, with most categories over 90% [email protected] accuracy, offering high practical applicability to real-world problems. The dataset, models, and codes will be released at https://github.com/AlanLuSun/General-Keypoint-Detection.

Abstract

Keypoint Detection

Explore similar work

GKDT: General Keypoint Detection Transformer

From Pairs to Sequences: Track-Aware Policy Gradients for Keypoint Detection

GESS: Multi-cue Guided Local Feature Learning via Geometric and Semantic Synergy