From Pairs to Sequences: Track-Aware Policy Gradients for Keypoint Detection

Yepeng Liu; Hao Li; Liwen Yang; Fangzhen Li; Xudi Ge; Yuliang Gu; kuang Gao; Bing Wang

cs.CVFeb 24, 2026

From Pairs to Sequences: Track-Aware Policy Gradients for Keypoint Detection

Authors: Yepeng Liu, Hao Li, Liwen Yang, Fangzhen Li, Xudi Ge, Yuliang Gu, kuang Gao, Bing Wang, +3 more

Organizations: School of Computer Science, Wuhan University · Xiaomi EV

Abstract

Keypoint-based matching is a fundamental component of modern 3D vision systems, such as Structure-from-Motion (SfM) and SLAM. Most existing learning-based methods are trained on image pairs, a paradigm that fails to explicitly optimize for the long-term trackability of keypoints across sequences under challenging viewpoint and illumination changes. In this paper, we reframe keypoint detection as a sequential decision-making problem. We introduce TraqPoint, a novel, end-to-end Reinforcement Learning (RL) framework designed to optimize the Track-quality (Traq) of keypoints directly on image sequences. Our core innovation is a track-aware reward mechanism that jointly encourages the consistency and distinctiveness of keypoints across multiple views, guided by a policy gradient method. Extensive evaluations on sparse matching benchmarks, including relative pose estimation and 3D reconstruction, demonstrate that TraqPoint significantly outperforms some state-of-the-art (SOTA) keypoint detection and description methods.The code will be available at https://github.com/xiaomi-research/traqpoint.

Figures & tables

Figure 1: Multi-view reconstruction: Our TraqPoint vs. RDD [ 5 ] . Keypoints that successfully generate landmarks are marked in blue . Failed keypoints are marked in red . Our TraqPoint generates more keypoints in structurally significant areas with higher cross-view consistency, yielding more landmarks.

Figure 2: Following the architectural design of RDD [ 5 ] , we adopt an identical network structure. Specifically, we replace the feature extractor employed in RDD [ 5 ] with DINOv3-ConvNeXt [ 45 ] . The keypoint branch serves as the policy network ( $\pi_{\theta}$ ) .

Figure 3: Overview of our proposed Sequence-Aware Keypoint Policy Learning framework: First, we select a reference frame from the input image sequence and perform hybrid keypoint sampling on it. Next, we leverage geometric mapping to locate the corresponding positions of the reference frame’s keypoints in other frames of the sequence. We then count the number of these keypoints visible across the entire sequence. After that, we compute per-frame rewards for each sampled point and aggregate these into a final track reward. Finally, we update the policy network’s gradients.

Figure 4: Qualitative results on the MegaDepth dataset [ 23 ] and the ETH benchmark [ 44 ] . For feature matching, keypoints are plotted in orange . Green lines indicate correct matches, while red lines denote incorrect ones. For multi-view reconstruction, keypoints that successfully generate point cloud landmarks are marked in blue ; failed ones are marked in red . Our TraqPoint produces keypoints with better structural saliency and high cross-view consistency, which effectively improves feature matching and reconstruction tasks.

Method	MegaDepth-1500			ScanNet
	AUC $\uparrow$			AUC $\uparrow$
	@5°	@10°	@20°	@5°	@10°	@20°
Sparse with Learned Matcher
SP [ 9 ] +SG [ 40 ] CVPR’19	49.7	67.1	80.6	16.1	33.4	50.2
SP [ 9 ] +LG [ 24 ] ICCV’23	49.9	67.0	80.1	16.2	33.0	49.7
Sparse with MNN

Table 1: Comparison on the MegaDepth [ 23 ] and ScanNet [ 8 ] . Top 4,096 features are used for all sparse matching methods. Best in bold, second best underlined.

Methods	Day	Night
Methods	(0.25m,2°) / (0.5m,5°) / (1.0m,10°)
SuperPoint [ 9 ] CVPRW’18	87.4 / 93.2 / 97.0	77.6 / 85.7 / 95.9
DISK [ 48 ] NeurIps’20	86.9 / 95.1 / 97.8	83.7 / 89.8 / 99.0
ALIKE [ 55 ] TM’22	85.7 / 92.4 / 96.7	81.6 / 88.8 / 99.0
ALIKED [ 54 ] TIM’23	86.5 / 93.4 / 96.8	85.7 / 91.8 / 96.9
XFeat [ 33 ] CVPR’24	84.7 / 91.5 / 96.5	77.6 / 89.8 / 98.0

Table 2: Visual Localization on Aachen day-night [ 41 ] . Best in bold, second best underlined.

Methods	ATE $\downarrow$	MTE $\downarrow$	AKTL $\uparrow$
Seq-01
RootSIFT [ 1 ] CVPR’12	60.4	102.1	4.4
XFeat [ 33 ] CVPR’24	55.0	92.7	3.9
RDD [ 5 ] CVPR’25	35.3	78.3	4.6
RIPE [ 19 ] ICCV’25	43.7	90.6	4.1
TraqPoint (Ours)	29.9	51.4	7.3

Table 3: Visual Odometry test results on sequence (01-03) of KITTI [ 14 ] . ATE: Average Trajectory Error; MTE: Maximum Trajectory Error; AKTL: Average Keypoint Tracking Length.

Methods	Reg. Img $\uparrow$	Sparse Pts $\uparrow$	Track Len $\uparrow$	Reproj Err $\downarrow$
Madrid Metropolis (1344 imgs)
RootSIFT [ 1 ] CVPR’12	500	116k	6.32	0.60px
SuperPoint [ 9 ] CVPRW’18	438	29k	9.03	1.05px
D2-Net [ 10 ] CVPR’19	501	84k	6.33	1.28px
ASLFeat [ 27 ] CVPR’20	613	96k	8.76	0.90px
PosFeat [ 22 ] CVPR’22	419	72k	9.18	0.86px

Table 4: Results on ETH benchmark [ 44 ] for 3D reconstruction. Best in bold, second best underlined.

Variant	AUC@5° $\uparrow$	AKTL $\uparrow$
TraqPoint-Full (Ours)	55.8	6.6
1. RL Paradigm
TraqPoint – Pairwise	53.3	4.3
Match Reward – Pairwise	49.7	2.8
2. Reward Function Design
w/o Ranking Reward	52.6	4.0

Table 5: Ablation study. We report matching capability (AUC@5° on MegaDepth [ 23 ] ) and tracking stability (average value of AKTL on KITTI [ 14 ] sequence 01-03).

Figure 5: Ablation study on sequence length and the number of sampled keypoints.

Explore similar work

RIPE++: Reinforced Keypoint Learning from Positive Pairs Only
Aug 20, 2026Johannes Künzel, Peter Eisert, Anna HilsmannRepresentation Learning Reinforcement Learning
Fraunhofer Heinrich-Hertz-Institute, HHI, Germany · Humboldt University Berlin, Germany
RaCo: Ranking and Covariance for Practical Learned Keypoints
Feb 17, 2026Abhiram Shenoi, Philipp Lindenberger, Paul-Edouard Sarlin +1Keypoint Detection Local Feature Matching
ETH Zurich · Google · Microsoft Mixed Reality & AI Lab
Towards Complementary Keypoint Detection via Mixture of Detectors
Oct 12, 2026Xiaoyong Lu, Guobao Xiao, Dong Liang +1Keypoint Detection Mixture of Experts
Southeast University · Tongji University · Nanjing University of Aeronautics and Astronautics

Aug 20, 2026J/K move · Enter open · S save

RIPE++: Reinforced Keypoint Learning from Positive Pairs Only

Johannes Künzel, Peter Eisert, Anna Hilsmann

Fraunhofer Heinrich-Hertz-Institute, HHI, Germany · Humboldt University Berlin, Germany

Representation Learning Reinforcement Learning Image Matching Keypoint Detection Self-Supervised Visual Representation Learning

Open paper AskPDF arXiv

Sparse keypoint extraction and matching underpin core tasks in geometric computer vision, including structure-from-motion, visual SLAM, augmented reality, and medical image registration. Learning robust local feature representations, however, typically requires accurate camera poses or depth supervision, which are often unavailable in real-world settings. Reinforcement learning (RL) has recently emerged as a promising alternative, requiring only the information if two images show the same scene or not. However, existing RL formulations such as RIPE rely on coarse binary rewards and carefully constructed negative training pairs, limiting training stability and descriptor discriminability. In this paper, we revisit RL-based keypoint learning and propose a reward that fully exploits the geometric consistency signal, deriving both reward and penalty from a single positive pair without contrasting against negatives. This richer signal provides sufficient supervisory contrast to learn discriminative detectors and descriptors from positive image pairs alone, enabling representation learning under extremely limited supervision. Furthermore, we show that the same RL objective can be extended to the matching stage by adapting LightGlue, raising AUC@5 on MegaDepth1500 from 56.58 to 59.65 and enabling weakly-supervised training of the full sparse matching pipeline from image pairs with partial visual overlap. We validate our approach on established benchmarks, demonstrating competitive results compared to fully-supervised methods. We further show that the method can be even trained on low texture medical video sequences, where camera poses are usually unavailable and standard SfM pipelines often fail. Code and data are available at https://github.com/fraunhoferhhi/RIPEpp .

Abstract

Explore similar work

RIPE++: Reinforced Keypoint Learning from Positive Pairs Only

RaCo: Ranking and Covariance for Practical Learned Keypoints

Towards Complementary Keypoint Detection via Mixture of Detectors