GEKey: Learning Geodesic Eccentric 3D Keypoints via Self-Supervision

Pierre Onghena; Santiago Velasco-Forero; Beatriz Marcotegui

cs.CVNov 5, 2026

GEKey: Learning Geodesic Eccentric 3D Keypoints via Self-Supervision

Authors: Pierre Onghena, Santiago Velasco-Forero, Beatriz Marcotegui

Organizations: Mines Paris – PSL University Centre for Statistics and Images (STIM)

Abstract

Point clouds represent 3D object geometry as sets of spatial coordinates. A fundamental task in their analysis is keypoint detection, i.e., identifying a sparse subset of points that captures the most salient features. While supervised methods rely on manual annotations, unsupervised approaches leverage intrinsic geometric properties to localize keypoints without human input. We introduce GEKey, a self-supervised framework that uses the geodesic eccentricity transform, an SE(3)-invariant geometric measure, to identify instance-specific salient points. These points serve as pseudo-labels to train a model that aligns its predictions with the object's intrinsic structure. GEKey is simple, efficient, and robust, preserving intra-class variation by balancing instance-specific and category-level structure. Extensive experiments on ShapeNetCoreV2, KeypointNet, CAPE, ITOP, and DeformingThings4D show state-of-the-art performance in both keypoint detection and alignment, with strong resilience to noise, downsampling, deformations, and real-world conditions. The code and pretrained models are available at https://github.com/Pierreoo/GEKey.

Figures & tables

Figure 2: The geometric analysis of GEKey-PL, outlining the process from geodesic eccentricity to the derivation of pseudo-labels.

Figure 3: Overview of self-supervised GEKey to generalize keypoint prediction. The blue workflow shows a backbone model predicting keypoints $\hat{Y}$ , which are compared, via the bipartite matching loss, to pseudo-labels $\mathcal{Y}^{*}$ generated by the green pipeline.

Method	Airplane	Bed	Bottle	Cap	Car	Chair	Guitar	Helmet	Knife	Motorcycle	Mug	Table	Vessel	Avg.
ULCS [ 8 ]	71.0	67.0	75.4	57.5	71.3	68.5	50.1	64.1	52.1	78.4	47.4	60.1	76.9	64.6
Skeleton Merger [ 28 ]	72.1	71.9	72.8	59.5	72.0	69.7	69.3	72.4	92.0	95.3	65.9	79.1	94.2	75.9
SC3K [ 50 ]	87.2	80.0	77.4	56.3	76.1	56.7	96.5	55.0	98.3	85.0	46.3	79.2	92.9	75.9
GEKey	98.5	87.0	61.3	65.0	92.7	90.5	99.7	51.3	99.8	98.2	90.9	91.2	96.3	86.3

Table 1: Keypoint inclusivity on KeypointNet, comparing unsupervised learning-based methods.

Method	Airplane	Bed	Bottle	Cap	Car	Chair	Guitar	Helmet	Knife	Motorcycle	Mug	Table	Vessel	Avg.
FPS [ 27 ]	97.1	98.3	97.1	93.5	94.0	97.1	97.8	89.7	98.3	94.2	95.2	98.0	97.1	95.9
ISS [ 48 ]	77.0	95.6	96.0	91.2	92.4	93.7	87.6	88.0	85.2	89.1	93.4	95.4	89.9	90.3
GEKey-PL	99.2	98.7	96.7	93.6	95.5	97.6	98.7	88.6	99.6	94.8	96.0	98.5	98.2	96.6
ULCS [ 8 ]	88.6	94.2	80.9	60.8	83.7	83.9	79.8	79.9	76.8	78.9	89.6	83.0	78.8	81.5
Skeleton Merger [ 28 ]	92.6	84.3	91.4	85.0	90.7	85.9	85.7	82.1	77.4	86.1	83.2	91.3	85.3	86.2
SC3K [ 50 ]	96.3	98.2	98.0	94.6	89.8	95.3	97.6	90.5	98.8	94.3	95.2	97.4	97.2	95.6

Table 2: Keypoint coverage on KeypointNet, comparing hand-crafted (top) detectors and unsupervised learning-based (bottom) methods.

Method	Airplane	Bed	Bottle	Cap	Car	Chair	Guitar	Helmet	Knife	Motorcycle	Mug	Table	Vessel	Avg.
FPS [ 27 ]	67.6	27.9	38.5	17.4	40.4	41.1	64.6	22.9	45.2	37.1	23.0	38.0	48.5	39.4
ISS [ 48 ]	64.0	29.4	38.8	14.3	39.3	38.8	63.7	20.8	38.0	41.2	22.6	32.4	49.7	37.9
GEKey-PL	80.5	45.2	39.6	27.4	31.8	57.1	67.1	23.7	50.0	46.8	26.2	62.6	49.6	46.7
KeypointDeformer [ 13 ]	53.3	40.7	34.0	9.3	53.8	62.0	52.6	13.5	38.1	45.7	12.6	62.0	51.2	40.7
Skeleton Merger [ 28 ]	73.5	59.9	36.5	15.5	53.3	60.0	49.7	16.7	45.0	39.7	23.6	60.9	58.9	45.6
SC3K [ 50 ]	74.9	19.9	38.0	11.3	33.2	39.8	68.0	17.6	47.9	39.7	18.4	46.4	47.5	38.7

Table 3: Keypoint detection on KeypointNet using mIoU metric, comparing hand-crafted (top) detectors and unsupervised learning-based (bottom) methods.

Figure 4: Keypoint detection by GEKey for an example in each category, showing accurate localization at salient regions.

Method	Airplane	Bed	Bottle	Cap	Car	Chair	Guitar	Helmet	Knife	Motorcycle	Mug	Table	Vessel	Avg.
KeypointDeformer [ 13 ]	73.4	69.5	60.3	56.7	80.2	84.3	62.9	55.3	59.8	61.5	69.8	72.6	73.9	67.7
Skeleton Merger [ 28 ]	77.8	61.3	59.9	53.1	79.5	76.8	63.2	57.0	60.4	57.8	67.2	70.0	72.4	65.9
SC3K [ 50 ]	82.9	64.9	62.7	59.7	75.2	87.0	65.7	58.6	63.0	59.4	75.3	76.0	76.0	69.7
Key-Grid [ 11 ]	84.7	70.6	66.2	58.5	84.5	93.4	69.1	62.5	60.9	63.7	79.4	78.6	74.6	72.8
GEKey	87.6	72.8	59.6	63.5	90.3	97.1	43.4	72.9	58.0	83.8	73.6	91.8	54.1	73.0

Table 4: Keypoint alignment on KeypointNet using

\text{DAS}_{\text{SC3K}}

metric, comparing unsupervised learning-based methods.

Figure 5: Keypoint alignment by GEKey, with keypoints mapped to consistent regions, including highly varied geometries.

Figure 6: Robustness of GEKey to noise (top) and downsampling (bottom) perturbations, visualized at noise scales of 0.10 and 0.20 and downsampling ratios of 16 and 32.

Dataset	Method	Inclusivity $\uparrow$	Coverage $\uparrow$	$T_{con}$ $\uparrow$
CAPE [ 22 , 26 ]	ULCS [ 8 ]	47.8	67.6	88.1
	SC3K [ 50 ]	79.3	86.5	70.7
	SelfGeo [ 49 ]	85.9	91.9	90.4
	GEKey	99.5	96.7	93.3
ITOP [ 10 ]	ULCS [ 8 ]	42.1	64.3	76.5
	SC3K [ 50 ]	76.4	83.1	42.9

Table 5: Keypoint quality under non-rigid deformations on the CAPE, ITOP, and DeformingThings4D datasets.

Figure 7: Without temporal supervision in training, GEKey detects keypoints that remain stable under non-rigid human deformations in CAPE (top), and transfers to the real-world ITOP dataset without fine-tuning (bottom), despite self-occlusion and sensor noise.

Figure 8: Keypoints detected by GEKey on animal shapes from DeformingThings4D, obtained from geometry alone without temporal supervision. Placements are precise, stable across frames, and consistent across species.

Figure 9: Comparison of $\mathcal{L}_{\text{match}}$ and $\mathcal{L}_{\text{Sinkhorn}}$ for the airplane and chair categories.

Method	Airplane	Bed	Bottle	Cap	Car	Chair	Guitar	Helmet	Knife	Motorcycle	Mug	Table	Vessel	Avg.
PointNet++ [ 27 ]	86.2	53.0	52.8	27.7	42.6	66.9	63.9	31.2	53.3	66.4	24.7	71.1	57.2	53.6
PTv1 [ 45 ]	84.4	48.3	46.8	29.6	46.1	65.7	69.0	31.1	52.0	60.6	25.6	69.4	55.5	52.6
PTv2 [ 38 ]	86.5	51.9	46.1	32.1	43.9	65.6	71.6	36.5	53.0	66.2	28.6	70.7	55.8	54.5

Table 6: Keypoint detection on KeypointNet using mIoU metric, comparing different backbone architectures with GEKey.

Method	Airplane	Bed	Bottle	Cap	Car	Chair	Guitar	Helmet	Knife	Motorcycle	Mug	Table	Vessel	Avg.
PointNet++ [ 27 ]	83.7	70.8	52.2	62.4	92.8	94.0	48.1	79.6	59.8	82.2	55.2	91.6	53.5	71.2
PTv1 [ 45 ]	81.0	69.9	56.1	60.8	90.3	96.9	37.0	76.3	60.7	74.4	63.9	91.6	56.0	70.4
PTv2 [ 38 ]	87.6	72.8	59.6	63.5	90.3	97.1	43.4	72.9	58.0	83.8	73.6	91.8	54.1	73.0

Table 7: Keypoint alignment on KeypointNet using

\text{DAS}_{\text{SC3K}}

metric, comparing different backbone architectures with GEKey.

Method	Airplane	Bed	Bottle	Cap	Car	Chair	Guitar	Helmet	Knife	Motorcycle	Mug	Table	Vessel	Avg.
FPS [ 27 ]	13.2	10.9	2.8	1.0	2.7	7.4	7.1	2.4	10.7	6.2	2.3	10.7	6.7	6.5
ISS [ 48 ]	1.7	1.3	1.2	1.2	1.5	1.7	1.2	1.4	3.0	1.4	1.1	1.8	1.1	1.5
GEKey-PL	20.0	19.3	2.5	2.5	1.4	13.6	10.8	1.8	14.9	7.8	2.9	21.0	6.3	9.6
GEKey	15.0	12.4	1.0	1.3	0.2	8.7	9.7	1.1	10.9	5.4	1.3	12.0	3.7	6.4

Table 8: Keypoint detection using mIoU metric at a distance threshold of 0.01.

Figure 10: Per-category mIoU (%) curves of GEKey over Euclidean distance thresholds.

Figure 11: Evolution of mIoU-0.10, mIoU-0.01, and $\text{DAS}_{\text{SC3K}}$ over training for the airplane and chair categories, illustrating the negative correlation between detection and alignment.

	Airplane		Chair
	DAS	mIoU	DAS	mIoU
Skeleton Merger [ 28 ]	77.8	73.5	76.8	60.0
GEKey	73.3	86.5	74.9	65.6

Table 9: Keypoint alignment on KeypointNet using

\text{DAS}_{\text{SM}}

metric.

Figure 12: Ground-truth keypoints color-coded by semantic ID, with instance-specific circled to highlight intra-class variability: not all instances share the same semantic label subset, constraining the $\text{DAS}_{\text{SM}}$ metric when predictions adapt to per-shape geometry.

		$k$ -NN	$\operatorname{ecc}_{100}(u)$		$\operatorname{ecc}_{10}(u)$		$\operatorname{ecc}(u)-\delta(\operatorname{ecc}(u))$
	#Obj	$T_{\text{Tot}}$	$T_{\text{Tot}}$	mIoU	$T_{\text{Tot}}$	mIoU	$T_{\text{Tot}}$
Airplane	1022	7.6	79.3	80.5	9.1	80.3	11.6
Bed	146	1.0	11.7	45.2	1.4	42.3	2.4
Bottle	380	2.5	30.2	39.6	3.5	39.1	14.6
Cap	38	0.3	3.0	27.4	0.4	24.9	0.9
Car	1002	7.7	84.8	31.8	10.1	34.8	39.6

Table 10: Processing times (in seconds) for the

k

-NN graph, geodesic eccentricity, and pseudo-label computation stages of GEKey-PL.

Figure 13: Geodesic distances to the wingtip point on the Delaunay triangulation, alongside geodesic distances on the $k$ -NN graph for comparison.

Figure 14: The local maxima of the geodesic eccentricity transform on the Delaunay triangulation and the $k$ -NN graph.

	$10$ -NN		$20$ -NN		$30$ -NN		$40$ -NN
	Cov.	mIoU	Cov.	mIoU	Cov.	mIoU	Cov.	mIoU
Airplane	98.7	79.4	99.2	80.3	99.3	78.3	99.3	77.1
Bed	98.5	37.5	98.7	42.3	98.6	39.6	98.6	38.6
Bottle	96.4	44.4	97.0	39.1	97.1	38.0	97.2	37.5
Cap	93.2	22.0	93.9	24.9	95.0	23.6	95.4	25.1
Car	94.2	42.2	95.4	34.8	95.8	35.5	95.9	36.0

Table 11: Comparison of the

k

-parameter in the

k

-NN graph of GEKey-PL on pseudo-label Coverage (Cov., %) and mIoU (%), computed with

\operatorname{ecc}_{10}(u)

Figure 16: GEKey exhibits strong synthetic-to-real generalization on real-world examples from the OBJ-BG variant of ScanObjectNN, despite challenges such as background clutter, partial occlusions, sensor noise, and varying point density.

Method	Repeatability (%) $\uparrow$	Consistency ( $\times 10^{-2}$ ) $\downarrow$
FPS [ 27 ]	11.6	12.0
ISS [ 48 ]	9.0	11.0
GEKey-PL	32.3	4.4

Table 12: Repeatability and consistency error between SMPL-deformed shapes for ten keypoints.

Figure 15: GEKey-PL with stable keypoints across deformations of the SMPL human model.

Keypoint Detection

Keypoint detection locates distinctive image points or meaningful landmarks for matching, pose estimation, tracking, and geometric localization.

See all 30 papers

Jul 1, 2026

GKDT: General Keypoint Detection Transformer

With the emergence of various pre-trained vision and language models, computer vision is shifting from narrow-domain to open-domain recognition. The construction of a more powerful yet general keypoint detection (GKD) model to support diverse tasks has become increasingly important in the field. To this end, we firstly present a large-scale unified keypoint dataset called MegaKPT. The dataset is composed of over 1.3 million diverse object instances from twenty-nine existing datasets, and enjoys high-quality unified annotations with keypoint text descriptions. Based on MegaKPT, we develop GKDT, a simple, flexible and powerful DINOv3 based Transformer model for General Keypoint Detection. Our GKDT supports visual prompts, text prompts, or both. To enhance model training, we also propose a suite of useful strategies such as mix-modal prompted training and dynamic importance sampling. By testing over 22 test sets with seen or unseen objects, our single GKDT model shows strong performance and generality in detecting keypoints on broad categories, with most categories over 90% [email protected] accuracy, offering high practical applicability to real-world problems. The dataset, models, and codes will be released at https://github.com/AlanLuSun/General-Keypoint-Detection.

Changsheng Lu, Yuxin Chen, Haokun Gui +5

Hong Kong University of Science and Technology, Hong Kong, China · Australian National University, Canberra, Australia · Tencent Inc., China +1

Keypoint Detection

Ask PDF

Feb 24, 2026

From Pairs to Sequences: Track-Aware Policy Gradients for Keypoint Detection

Keypoint-based matching is a fundamental component of modern 3D vision systems, such as Structure-from-Motion (SfM) and SLAM. Most existing learning-based methods are trained on image pairs, a paradigm that fails to explicitly optimize for the long-term trackability of keypoints across sequences under challenging viewpoint and illumination changes. In this paper, we reframe keypoint detection as a sequential decision-making problem. We introduce TraqPoint, a novel, end-to-end Reinforcement Learning (RL) framework designed to optimize the Track-quality (Traq) of keypoints directly on image sequences. Our core innovation is a track-aware reward mechanism that jointly encourages the consistency and distinctiveness of keypoints across multiple views, guided by a policy gradient method. Extensive evaluations on sparse matching benchmarks, including relative pose estimation and 3D reconstruction, demonstrate that TraqPoint significantly outperforms some state-of-the-art (SOTA) keypoint detection and description methods.The code will be available at https://github.com/xiaomi-research/traqpoint.

Yepeng Liu, Hao Li, Liwen Yang +8

School of Computer Science, Wuhan University · Xiaomi EV

Keypoint Detection

Ask PDF

Apr 7, 2026

GESS: Multi-cue Guided Local Feature Learning via Geometric and Semantic Synergy

Robust local feature detection and description are foundational tasks in computer vision. Existing methods primarily rely on single appearance cues for modeling, leading to unstable keypoints and insufficient descriptor discriminability. In this paper, we propose a multi-cue guided local feature learning framework that leverages semantic and geometric cues to synergistically enhance detection robustness and descriptor discriminability. Specifically, we construct a joint semantic-normal prediction head and a depth stability prediction head atop a lightweight backbone. The former leverages a shared 3D vector field to deeply couple semantic and normal cues, thereby resolving optimization interference from heterogeneous inconsistencies. The latter quantifies the reliability of local regions from a geometric consistency perspective, providing deterministic guidance for robust keypoint selection. Based on these predictions, we introduce the Semantic-Depth Aware Keypoint (SDAK) mechanism for feature detection. By coupling semantic reliability with depth stability, SDAK reweights keypoint responses to suppress spurious features in unreliable regions. For descriptor construction, we design a Unified Triple-Cue Fusion (UTCF) module, which employs a semantic-scheduled gating mechanism to adaptively inject multi-attribute features, improving descriptor discriminability. Extensive experiments on four benchmarks validate the effectiveness of the proposed framework. The source code and pre-trained model will be available at: https://github.com/yiyscut/GESS.git.

Yang Yi, Xieyuanli Chen, Jinpu Zhang +2

Y. Yi, X. Chen, J. Zhang, H. Shen, and D. Hu are with the College of Intelligence Science and Technology, National University of Defense Technology, China.

Keypoint Detection Feature Fusion

Ask PDF

Abstract

Keypoint Detection

Explore similar work

GKDT: General Keypoint Detection Transformer

From Pairs to Sequences: Track-Aware Policy Gradients for Keypoint Detection

GESS: Multi-cue Guided Local Feature Learning via Geometric and Semantic Synergy