Finding matching keypoints between images is a core problem in 3D computer vision. However, modern matchers struggle with large in-plane rotations. A straightforward mitigation is to learn rotation invariance via data augmentation. However, it remains unclear at which stage rotation invariance should be incorporated. In this paper, we study this in the context of a modern sparse matching pipeline. We perform extensive experiments by training on a large collection of 3D vision datasets and evaluating on popular image matching benchmarks. Surprisingly, we find that incorporating rotation invariance already in the descriptor yields similar performance to handling it in the matcher. However, rotation invariance is achieved earlier in the matcher when it is learned in the descriptor, allowing for a faster rotation-invariant matcher. Further, we find that enforcing rotation invariance does not hurt upright performance when trained at scale. Finally, we study the emergence of rotation invariance through scale and find that increasing the training data size substantially improves generalization to rotated images. We release two matchers robust to in-plane rotations that achieve state-of-the-art performance on e.g. multi-modal (WxBS), extreme (HardMatch), and satellite image matching (SatAst). Code is available at https://github.com/davnords/loma.
Figures & tables
Figure 1: Who Handles the Orientation? We compare incorporating rotation invariance in (a) the matcher and (b) both the matcher and descriptor, by reporting the AUC@20 on MegaDepth-1500 [ 27 , 41 ] (rotated and upright) for different stopping layers using N=2048 keypoints. We average over 5 runs and include a 95% confidence interval.
Figure 2: Experimental setup. We train descriptors and matchers with and without rotation augmentation, resulting in the three different models used for our main experiments. NoRot corresponds to the LoMa-B baseline.
Figure 3: Training batch. Illustration of a training batch with rotation augmentation.
Datasets
Type / GT Source
Weight
ScanNet++ v2 [ 52 ]
Indoor / Mesh
1
BlendedMVS [ 51 ]
Aerial / Mesh
1
Map-Free [ 2 ]
Object-centric / MVS
1
Hypersim [ 37 ]
Indoor / Graphics
1
MegaScenes [ 44 ]
Outdoor / MVS
1
MegaDepth [ 27 ]
Outdoor / MVS
1
Table 1: Training data . For the main experiments we use a large collection of 3D datasets for training.
Method
MegaDepth
ScanNet
AUC @→
5∘
10∘
20∘
5∘
10∘
20∘
Rotated evaluation
SP+SG
13.3
19.1
23.8
4.6
9.6
15.0
SP+LG
13.2
19.0
23.9
3.7
7.8
12.3
ALIKED+LG
11.2
15.4
18.8
3.4
6.9
10.6
NoRot
52.4
68.8
81.2
16.8
31.2
44.3
Table 2: Relative pose estimation on MegaDepth-1500 [ 27 , 41 ] and ScanNet-1500 [ 12 , 39 ] . We evaluate both with and without random rotations.
Figure 4: ScanNet-1500 performance. We plot the AUC@20 for our different models.
Method
WxBS
SatAst
Non-rotated
Rotated
SP+SG
45.6
7.3
19.8
SP+LG
46.0
10.9
12.8
ALIKED+LG
43.9
8.0
12.1
NoRot
68.7
50.0
18.8
RotMatch
71.2
62.8
42.4
Table 3: Challenging matching on WxBS [ 32 ] (mAA@10px) and SatAst [ 19 ] (AUC@10 ∘ ). SatAst includes arbitrary rotations by default.
Method
Rotated
Non-rotated
mAA@ →
5px
10px
5px
10px
SP+SG
7.7
10.9
26.3
36.0
SP+LG
9.8
14.3
24.8
34.8
ALIKED+LG
6.5
9.3
26.4
35.7
NoRot
29.1
39.2
37.7
50.6
RotMatch
36.0
48.6
37.9
51.1
Table 4: Taking matching to the next level on HardMatch [ 34 ] (mAA@10px).
Figure 5: HardMatch group breakdown. We report mAA@10px for different subgroups of the non-rotated version of HardMatch. We find that our rotation-invariant models achieve significant gains on celestial matching.
Figure 6: Qualitative HardMatch pair. Rotation invariance significantly improves robustness to image pairs that lack a canonical orientation, such as star constellations. NoRot fails to find any matches above the threshold τ .
Training Data
Non-Rotated
Rotated
AUC @→
5∘
10∘
20∘
5∘
10∘
20∘
Only MegaDepth
53.2
69.5
82.0
16.1
22.9
29.1
All Data
55.0
71.3
83.2
52.2
68.7
81.0
Table 5: Effect of increased data. Relative pose estimation on MegaDepth-1500 [ 27 , 41 ] .
Figure 7: Qualitative impact of rotation invariance. Descriptor features (a) trained with rotation augmentation and (b) without, for an image and its 180∘ rotation. We visualize the features using SVD, mapping the top three components to RGB.
Method
Non-Rotated
Rotated
AUC @→
5∘
10∘
20∘
5∘
10∘
20∘
Joint Rot. Desc.
54.9
71.3
83.3
54.7
71.1
83.2
RotMatch
55.3
71.5
83.4
54.5
70.9
82.9
RotDesc&Match
55.0
71.1
83.2
54.2
70.6
82.8
Table 6: Effect of joint rotations. Relative pose estimation on MegaDepth-1500 [ 27 , 41 ] . We compare a descriptor with joint rotations and a matcher with independent rotations with our other models (trained with independent rotations).
Figure 8: Performance for arbitrary rotations. We report AUC@20 on MegaDepth-1500 at different rotation angles. We average over 5 runs and include a 95% confidence interval.
Figure 9: Visualizing an arbitrary rotation. Qualitative matches using circular cropping with αA=0∘ and αB=210∘ .
Vision Foundation Models (VFMs) have significantly advanced dense feature matching, yet severe in-plane rotation remains a critical challenge. Existing solutions face a fundamental dilemma: data-driven methods require inefficient parameter scaling to implicitly learn rotations, whereas strictly equivariant networks lack the semantic capacity of modern VFMs. Consequently, current frameworks typically freeze VFMs and shift the entire burden of rotation generalization to the downstream decoder. To break this architectural bottleneck, we propose REDI-Match, an efficient framework driven by a novel Rotation-Equivariant Distillation (REDI) paradigm. Instead of relying on rotation data augmentation to establish rotational correspondences, REDI distills the non-equivariant semantic representations of a VFM into a lightweight, strictly rotation-equivariant encoder, leveraging an equivariant geometric architecture to constrain robust high-dimensional semantics. To fully exploit these features, we equip the decoder with an entropy-driven spatial alignment module. By evaluating discrete rotation hypotheses, this mechanism explicitly locks onto the canonical coordinate system, eliminating global ambiguity before continuous refinement. Extensive experiments demonstrate that REDI-Match establishes a new state-of-the-art (SOTA) across multiple benchmarks. Notably, it achieves a 13.89% absolute pose accuracy improvement on the highly challenging SatAst dataset while operating 1.9x faster than the current SOTA (RoMa v2), enabling real-time inference (~41 FPS) on a single RTX 4090 GPU. Code: https://github.com/YinjiGe/REDI-Match.
Yinji Ge, Guixu Zheng, Wulong Guo +5
1Tsinghua University · 2Southern University of Science and Technology · 3Beihang University +1
Point cloud registration critically depends on local features that are both distinctive and robust to arbitrary 3D rotations. Existing learning-based methods typically approximate rotation invariance via fragile local reference frames or extensive data augmentation, providing only empirical invariance and often degrading under unseen rotational transformations. In this paper, we propose SHReg, a strictly rotation-equivariant point cloud registration framework grounded in the representation theory of SO(3). By representing local geometric features as irreducible representations of SO(3), SHReg guarantees exact equivariance under arbitrary rotations without relying on local reference frames. Built upon a spherical-harmonics-based equivariant backbone, SHReg jointly learns rotation-invariant descriptors for robust correspondence matching and rotation-equivariant features that preserve fine-grained orientation information. The preserved equivariant structure enables each correspondence to directly hypothesize a rigid transformation, reducing reliance on large-scale hypothesis sampling in conventional RANSAC-based pipelines and leading to improved robustness under challenging rotational variations. Extensive experiments on 3DMatch, 3DLoMatch, and KITTI demonstrate that SHReg consistently outperforms state-of-the-art methods in registration accuracy, particularly under large rotational perturbations.
Chongjian Wang, Junjie Gao
College of Mathematics and Systems Science, Shandong University of Science and Technology, Qingdao, China · School of Artificial Intelligence, Shandong Women’s University, Jinan, China
Dense feature matching aims to estimate all correspondences between two images of a 3D scene and has recently been established as the gold standard due to its high accuracy and robustness. However, existing dense matchers still fail or perform poorly for many hard real-world scenarios, and high-precision models are often slow, limiting their applicability. In this paper, we attack these weaknesses on a wide front through a series of systematic improvements that together yield a significantly better model. In particular, we construct a novel matching architecture and loss, which, combined with a curated diverse training distribution, enables our model to solve many complex matching tasks. We further make training faster through a decoupled two-stage matching-then-refinement pipeline, and at the same time, significantly reduce refinement memory usage through a custom CUDA kernel. Finally, we leverage the recent DINOv3 foundation model along with multiple other insights to make the model more robust and unbiased. In our extensive set of experiments, we show that the resulting novel matcher sets a new state-of-the-art, being significantly more accurate than its predecessors. Code is available at https://github.com/Parskatt/romav2
Johan Edstedt, David Nordström, Yushan Zhang +7
Linköping University · Chalmers University of Technology · University of Amsterdam +1