cs.CVAug 20, 2026

RIPE++: Reinforced Keypoint Learning from Positive Pairs Only

Authors: Johannes Künzel, Peter Eisert, Anna Hilsmann

Organizations: Fraunhofer Heinrich-Hertz-Institute, HHI, Germany · Humboldt University Berlin, Germany

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Sparse keypoint extraction and matching underpin core tasks in geometric computer vision, including structure-from-motion, visual SLAM, augmented reality, and medical image registration. Learning robust local feature representations, however, typically requires accurate camera poses or depth supervision, which are often unavailable in real-world settings. Reinforcement learning (RL) has recently emerged as a promising alternative, requiring only the information if two images show the same scene or not. However, existing RL formulations such as RIPE rely on coarse binary rewards and carefully constructed negative training pairs, limiting training stability and descriptor discriminability. In this paper, we revisit RL-based keypoint learning and propose a reward that fully exploits the geometric consistency signal, deriving both reward and penalty from a single positive pair without contrasting against negatives. This richer signal provides sufficient supervisory contrast to learn discriminative detectors and descriptors from positive image pairs alone, enabling representation learning under extremely limited supervision. Furthermore, we show that the same RL objective can be extended to the matching stage by adapting LightGlue, raising AUC@5 on MegaDepth1500 from 56.58 to 59.65 and enabling weakly-supervised training of the full sparse matching pipeline from image pairs with partial visual overlap. We validate our approach on established benchmarks, demonstrating competitive results compared to fully-supervised methods. We further show that the method can be even trained on low texture medical video sequences, where camera poses are usually unavailable and standard SfM pipelines often fail. Code and data are available at https://github.com/fraunhoferhhi/RIPEpp .

Figures & tables

Figure 1 : We propose a reward formulation for weakly-supervised keypoint extraction training that enables training from positive pairs only and is directly applicable to the training of learned matching architectures, demonstrated with LightGlue [ 25 ] .
Figure 2 : Schematic overview of the proposed weakly-supervised keypoint extraction framework. During training, the framework operates on pairs of images, from which a neural network generates heatmaps H\mathbf{H} from which keypoint locations K\mathbf{K} with a probability p\mathbf{p} are sampled. Descriptors D\mathbf{D} are extracted from the encoder, as Hypercolumn Features. Using these descriptors, keypoints are matched and filtered using epipolar constraints to calculate the reward matrix R\mathbf{R} , which weights the combined log-probabilities L\mathbf{L} of the keypoint locations, resulting in an approximation of the gradients, using REINFORCE [ 49 ] .
Figure 3 : Visual comparison of the reward matrix construction to shape the log-probabilities in L\mathbf{L} for the gradient computation ( Eq. 1 ). In RIPE (left) rewards are assigned at image pair level: geometric consistent matches are rewarded for positive pairs and penalized for negative sample pairs, outliers are ignored. In RIPE ++ (right), rewards are defined at correspondence level within positive pairs only. Geometrically consistent matches are rewarded and geometrically inconsistent matches are explicitly penalized. This finer-grained reward makes negative pairs unnecessary and yields a more informative training signal.
Figure 4 : Schematic overview of the proposed weakly-supervised training framework for LightGlue. Given keypoint locations and descriptors from two images, LightGlue refines the representations through multiple layers of self- and cross-attention, incorporating positional and cross-image context. Rather than supervising with ground-truth correspondences derived from depth or pose data, we formulate matching as a reinforcement learning problem: matches deemed geometrically consistent by RANSAC receive a positive reward, while outliers are penalized. This enables end-to-end training of the matcher using only image pairs.
 MethodAUC @5∘@5^{\circ}AUC @10∘@10^{\circ}AUC @20∘@20^{\circ}Supervision
Geometric GTALIKED [ 53 ] TIM’2356.6669.6469.6479.3579.35Pose+Homog.
DaD [ 12 ] CoRR’2556.4656.4670.0780.18Depth
DeDoDe-B [ 13 ] CVPRW’2456.0156.0169.2069.2078.7278.72Pose
DISK [ 45 ] NeurIPS’2050.6950.6964.3564.3574.7474.74Pose/ Depth
RACO [ 43 ] 3DV’2657.9671.2881.00Homog. (Det.) + Pose (Desc.)
No geo. GTSuperPoint [ 8 ] CVPRW’1847.2647.2660.8960.8970.6570.65Homography
Table 1 : Relative pose estimation on MegaDepth1500. Methods are grouped by whether training requires geometric ground truth (pose, depth) for the detector or descriptor. RIPE ++ trains both detector and descriptor from positive image pairs alone, without geometric supervision, yet remains comparable to geometric ground truth group and outperforms RIPE despite discarding its negative pairs. Best and second-best are highlighted within each group.
Figure 5 : Qualitative results of raw matches for RIPE ++ and RIPE ++ in combination with our weakly-supervised LightGlue matcher on the MegaDepth1500 benchmark dataset on the left and right respectively. The middle image shows qualitative results for RIPE ++ Medical on the SCARED1500 test dataset. For more qualitative results please refer to the supplementary material.
MethodAUC @5∘@5^{\circ}AUC @10∘@10^{\circ}AUC @20∘@20^{\circ}Supervision
ALIKED [ 53 ] TIM’2318.2418.2440.6140.6161.2361.23Pose+Homography
DaD [ 12 ] CoRR’2518.4418.4440.5640.5661.4061.40Depth
DeDoDe-B [ 13 ] CVPRW’2418.8518.8541.6941.6962.8062.80Pose
DISK [ 45 ] NeurIPS’2016.5816.5836.7736.7755.6355.63Pose/ Depth
RACO [ 43 ] 3DV’2619.3542.3642.3663.3463.34Homography
SIFT [ 27 ] IJCV’0413.2413.2432.1732.1751.6651.66—
Table 2 : Evaluation of the relative pose estimation on Scared 1500, showing the benefits of our proposed training scheme. It allows to easily train a specialized keypoint extractor, simply from video frames. The best and second-best performances are highlighted.
ParameterMetrics
pos onlyentropy# RANSAC inlier% RANSAC inlierAUC @5∘@5^{\circ}AUC @10∘@10^{\circ}AUC @20∘@20^{\circ}
 –297.534.451.8365.3775.94
✓–427.541.552.4266.7878.03
✓1\times10−41\text{\times}{10}^{-4}–––––
✓1\times10−51\text{\times}{10}^{-5}26633.649.6262.772.8
✓1\times10−61\text{\times}{10}^{-6}366.540.656.5969.4479.18
Table 3 : Ablation study of our proposed methods on the MegaDepth1500 test set.
Figure 6 : Effect of our entropy-based regularization on keypoint localization. Without it, heatmaps tend to become diffuse, particularly at low input resolutions (top-left). Regularization encourages sharper, more peaked responses (bottom).
MethodAUC @5∘@5^{\circ}AUC @10∘@10^{\circ}AUC @20∘@20^{\circ}
RIPE ++56.5869.5379.33
↓\downarrow + LightGlue+3.07+4.16+4.53
Matched-RIPE ++59.6573.6983.86
    
Table 4 : Improvements on MegaDepth1500 from training LightGlue with our proposed approach on positive image pairs only.
ExtractorMatcher
NameValuePurposeNameValuePurpose
ρin\rho_{\text{in}}1.0reward Eq. 3νin\nu_{\text{in}}1.0reward matcher training
ρout\rho_{\text{out}}-0.1penalty Eq. 3νout\nu_{\text{out}}-1.0penalty matcher training
λ\lambda-1e-7penalty Eq. 3λ\lambda-1e-07penalty no match
ψ\psi5.0weight Ldesc\mathcal{L}_{\text{desc}}η\eta0.0001weight Lnm\mathcal{L}_{\text{nm}} Eq. 10
ω\omega1e-6weight LH\mathcal{L}_{H}   
Table 5 : Overview of training hyperparameters for the keypoint extractor (left) and the matcher (right).
Figure 7 : Illustration Distance-based Reward
ParameterMetrics 
pos onlycontrastiveinfoncedistance rewardcurriculumentropy# RANSAC inlier% RANSAC inlierAUC @5∘@5^{\circ}AUC @10∘@10^{\circ}AUC @20∘@20^{\circ}
 ✓    2973451.8365.3775.94
✓✓    4274152.4266.7878.03
✓✓ ✓  3844054.7568.3378.92
✓✓  ✓ 3473852.5765.8476.43
✓ ✓   4304551.7265.075.9
Table 6 : Ablation of the different components on the MegaDepth1500 benchmark set with the longer side of the input rescaled to 1200 pixels and 2048 keypoints detected.
Figure 8 : Number of false NN matches (left) and RANSAC inliers (right) for negative image pairs.
MethodDayNight
.25m/ 2 ​ °.5m/ 5 ​ °5m/ 10 ​ °.25m/ 2 ​ °.5m/ 5 ​ °5m/ 10 ​ °
ALIKED85.592.095.866.080.692.7
DaD88.993.297.069.685.996.9
DeDoDe81.889.193.455.568.178.0
DISK81.991.495.461.875.986.9
RaCo87.493.497.172.886.996.9
Table 7 : Results on outdoor visual localization using Aachen Day-Night v1.1.
Figure 9 : Qualitative results on the MegaDepth1500 benchmark dataset for, RIPE [ 19 ] (top), RIPE ++ (ours, middle) and RaCo [ 43 ] (bottom).
Figure 10 : Qualitative results on MegaDepth1500 for RIPE ++ paired with our weakly-supervised LightGlue matcher.
Figure 11 : Qualitative results on the Scared1500 benchmark dataset for RIPE ++ (ours, top), RIPE ++ Medical (ours, middle) and RaCo [ 43 ] (bottom).

Explore similar work

CardsList
Jul 2, 2026cs.CV

GeoMix: Descriptor-Free Visual Localization via Global Context and Multi-Detector Training

Descriptor-free visual localization eliminates high-dimensional descriptor storage, preserves scene privacy, and simplifies map maintenance, yet its accuracy still lags far behind descriptor-based pipelines. We identify this gap to insufficient geometric discriminability in geometry-only matching. Without visual appearance, current methods underutilize local geometry cues, lack the global context among keypoints, and overfit to a single keypoint detector. We further observe that descriptor-free matching naturally enables multi-detector training, as heterogeneous keypoints can be optimized in a shared geometry-only space without aligning descriptor spaces. Building on these insights, we propose GeoMix, a descriptor-free 2D-3D matching framework that strengthens geometric discriminability at three levels. Locally, directional and distance-aware embeddings enrich neighborhood aggregation with fine-grained spatial structure. Globally, learnable context nodes aggregate and redistribute scene-wide information via cross-attention to resolve ambiguities beyond local receptive fields. At the training level, Mix-Training exploits this detector-agnostic geometry space to learn representations across multiple keypoint detectors. Extensive experiments on MegaDepth, Cambridge Landmarks, 7Scenes, and Aachen Day-Night show that GeoMix sets a new state of the art among descriptor-free methods, reducing 75th-percentile rotation error by 89% and translation error by up to 90% over the previous best, while generalizing zero-shot to unseen detectors and narrowing the gap to descriptor-based pipelines. Code is available at \href\href{https://github.com/YejunZhang/Geomix}{\text{this links}}.

Yejun Zhang, Xinjue Wang, Zihan Wang +2

Aalto University, Espoo, Finland · Tampere University, Tampere, Finland · University of Oulu, Oulu, Finland

Sequential Visual LocalizationKeypoint Detection
AskPDF
Apr 20, 2026cs.CV

MARCO: Navigating the Unseen Space of Semantic Correspondence

Recent advances in semantic correspondence rely on dual-encoder architectures, combining DINOv2 with diffusion backbones. While accurate, these billion-parameter models generalize poorly beyond training keypoints, revealing a gap between benchmark performance and real-world usability, where queried points rarely match those seen during training. Building upon DINOv2, we introduce MARCO, a unified model for generalizable correspondence driven by a novel training framework that enhances both fine-grained localization and semantic generalization. By coupling a coarse-to-fine objective that refines spatial precision with a self-distillation framework, which expands sparse supervision beyond annotated regions, our approach transforms a handful of keypoints into dense, semantically coherent correspondences. MARCO sets a new state of the art on SPair-71k, AP-10K, and PF-PASCAL, with gains that amplify at fine-grained localization thresholds (+8.9 PCK@0.01), strongest generalization to unseen keypoints (+5.1, SPair-U) and categories (+4.7, MP-100), while remaining 3x smaller and 10x faster than diffusion-based approaches. Code is available at https://github.com/visinf/MARCO .

Claudia Cuttano, Gabriele Trivigno, Carlo Masone +1

Politecnico di Torino · TU Darmstadt · ELIZA

Ground-Truth CorrespondenceDual-Encoder ArchitecturesFine-Grained Perception
AskPDF
Sep 8, 2026cs.CV

RoMa-ΩΩ: What Feed-Forward 3D Models Know About Image Matching

Learned image matching has experienced significant progress in recent years, culminating in robust and accurate matchers such as RoMa, whose robustness is often attributed to its use of frozen DINO features. In a parallel development, feed-forward reconstruction models, such as VGGT, have been trained on ever-growing datasets to accurately regress dense 3D point maps and camera poses. The distinction between matchers and feed-forward reconstruction models has become increasingly blurred with the introduction of matching losses in models such as MASt3R and VGGT-ΩΩ. This raises a natural question: what do feed-forward 3D models know about image matching? In this work, we answer this question by analyzing three scenarios: (i) zero-shot matching of patch features, (ii) direct matching of 3D point predictions, and (iii) training a full matcher on top of the learned representations. We find that, despite performing poorly in zero-shot matching, especially in later layers, feed-forward reconstruction models provide strong representations for linear probing and full matching pipelines. We further show that, even without any training, their raw predictions alone enable competitive matching, albeit only under moderate viewpoint changes and modality gaps. Based on these insights, we retrain RoMa v2 by replacing its DINO backbone with VGGT-ΩΩ. Our resulting model, \ours, outperforms state-of-the-art matchers on a wide range of benchmarks, e.g. +8.1 mAA compared to RoMa v2 on WxBS.

David Nordström, Xinyue Zhang, Thibaut Loiseau +2

Chalmers University of Technology, Sweden · Mobile Perception Lab, ShanghaiTech University, China · LIGM, Ecole des Ponts, Univ. Gustave Eiffel, CNRS, France

Feed-Forward 3D Reconstruction3D GenerationDino
AskPDF