Pair-Relative Partial Equivariance for Rotation-Robust Keypoint Descriptor Matching

Jongmin Lee

cs.CVNov 2, 2026

Pair-Relative Partial Equivariance for Rotation-Robust Keypoint Descriptor Matching

Authors: Jongmin Lee

Organizations: Chung-Ang University

Abstract

Learned keypoint descriptors trained on upright imagery often fail under large in-plane rotation. Steerers address this by applying a single exact SO(2) representation to the descriptor space, but this global group constraint can be too rigid for keypoint descriptors, whose angular responses need not follow a single group representation. We propose pair-relative partial equivariance, which measures equivariance under the empirical distribution of relative rotations in image pairs, and PaSteer, a relaxed descriptor-steering framework that allows the steering map to leave the exact representation manifold while retaining the lightweight, backbone-free design of steering. PaSteer instantiates this relaxation with two variants: an identity-preserving residual steerer (PaSteer-soft) and a free-form Fourier steerer (PaSteer-free). We show that the per-angle optimal linear steerer for a fixed non-equivariant descriptor generally does not obey the group composition law, explaining why exact representations can be suboptimal in the frozen-descriptor regime. On Roto-360, MegaDepth-1500, AIMS, and IMC-PT, PaSteer consistently improves rotation-robust matching over an exact steerer on frozen DeDoDe descriptors, especially under large relative rotations, with minimal upright degradation.

Figures & tables

Figure 1: PaSteer-free recovers correspondences that exact steering misses. A MegaDepth-1500 [ 19 ] pair under a large $112^{\circ}$ in-plane rotation. Both panels share the same frozen DeDoDe-B [ 9 ] descriptor; only the steering changes. Exact Steerers [ 5 ] finds 148 correct matches (left), PaSteer-free 242 (right). Green: correct ; blue: recovered ( $+113$ correct matches); orange: rejected .

Figure 2: PaSteer matching pipeline. An

\mathrm{SO}(2)

keypoint detector (DeDoDe-L [ 9 ] ) and a keypoint descriptor (DeDoDe-B) feed a steered max-similarity matcher: the query descriptors are steered by

\tilde{\rho}(\theta)

over the

C_{8}

probe grid under partial equivariance, and the best-scoring angle is kept per keypoint.

Figure 3: Exact representation vs. relaxed steering. The gray dashed curve denotes the true angular response of the keypoint descriptor. The orange solid curve shows the rigid exact-representation orbit, which misses this response, while the blue solid curve shows the relaxed PaSteer-soft orbit, $\tilde{\rho}(\theta)=\mathrm{expm}(\theta D+\Delta(\theta))$ , which follows the true response more closely.

Figure 4: The relaxed map approaches the optimal floor. Steering error is

1-\cos

of the cosine similarity between a steered upright descriptor

\rho(\theta)f(I)

and the truly rotated one

f(R_{\theta}I)

(lower is better,

0

is exact recovery). On fixed DeDoDe-B [ 9 ] descriptor, PaSteer tracks the optimal per-angle floor, while the exact representation [ 5 ] falls short and its error grows with rotation angle.

benchmark / metric	Exact	PaSteer-free	$\Delta$
Roto-360 @5px	92.0	93.4	$+1.4$
MD-1500 C4 AUC@5	32.5	36.4	$+3.9$
MD-1500 SO2 AUC@5	20.0	22.9	$+2.8$
AIMS AP	48.6	50.1	$+1.5$
IMC-PT C4 AUC@20	67.2	75.2	$+8.0$
IMC-PT SO2 AUC@20	41.1	50.6	$+9.5$

Table 1: Main result. On frozen DeDoDe-B descriptors [ 9 ] , we measure absolute scores for the exact steerer and PaSteer-free with their difference

\Delta

, on Roto-360 [ 17 ] correct matches @5px; MegaDepth-1500 [ 19 ] pose AUC@5 ∘ ; AIMS [ 31 ] average precision on the full FMAP set (323 frames); IMC-PT [ 13 ] pose AUC@20 ∘ . PaSteer-free improves over the exact steerer on all four benchmarks, by the widest margin under large relative rotation.

steerer	Roto	upr.	C4	SO2	AIMS
exact expm	92.0	49.0	32.5	20.0	48.6
PaSteer-free (ours)	93.4	49.3	36.4	22.9	50.1
$\Delta$ (free $-$ exact)	$+1.4$	$+0.3$	$+3.9$	$+2.8$	$+1.5$

Table 2: Full-scale MegaDepth training. PaSteer-free versus the exact steerer, both trained on the full MegaDepth set on a frozen DeDoDe-B [ 9 ] with identical scenes, budget, generator core, and initialization. Roto denotes Roto-360 correct matches @5px [ 17 ] ; upr., C4, SO2 denote MegaDepth-1500 pose AUC@5 ∘ (upright/C4/SO2) [ 19 ] ; AIMS denotes average precision on AIMS [ 31 ] . The

\Delta

row is the relaxed-over-exact margin.

desc.	metric	Exact	PS-free	$\Delta$
DeDoDe-B	Roto-360 @5px	92.0	93.4	$+1.4$
	MD-1500 C4 AUC@5	32.5	36.4	$+3.9$
	MD-1500 SO2 AUC@5	20.0	22.9	$+2.8$
DeDoDe-G	Roto-360 @5px	51.1	80.6	$\mathbf{+29.5}$
	MD-1500 C4 AUC@5	12.9	21.6	$\mathbf{+8.7}$
	MD-1500 SO2 AUC@5	11.8	20.1	$\mathbf{+8.3}$

Table 3: The cost of the exact constraint grows with descriptor capacity. Comparison with the exact steerer and PaSteer-free (PS-free) on DeDoDe-B [ 9 ] and the larger DeDoDe-G, on Roto-360 [ 17 ] correct matches @5px and MegaDepth-1500 [ 19 ] pose AUC@5 ∘ (C4/SO2). The margin widens markedly on DeDoDe-G across every rotation axis; scores are means over three training seeds (two for exact on DeDoDe-G).

Matched-budget controlled comparison
steering	3px	5px	10px
none (unsteered)	19.7	20.5	21.6
exact expm (our budget)	77.3	78.3	79.2
PaSteer-soft ( $r{=}12$ , ours)	89.1	90.1	91.0
PaSteer-free (ours)	93.9	95.1	96.0
Released reference models, not controlled

Table 4: Matched-budget three-way comparison on Roto-360 [ 17 ] , correct matches % @{3,5,10}px, frozen DeDoDe-B [ 9 ] . The top block is the matched-budget controlled comparison (ordering exact

<

PaSteer-soft

<

PaSteer-free); the bottom block lists released models as absolute references only, not controlled comparisons. Per-benchmark MegaDepth-1500 and AIMS tables are in supp. Sec. B .

Figure 5: Continuous-angle robustness. Per-angle MD-1500 [ 19 ] pose AUC@5 ∘ as the relative in-plane rotation

\Delta\theta

is swept continuously, on a frozen DeDoDe-B [ 9 ] descriptor with

C_{8}

probes. Unsteered (gray dashed) is the frozen descriptor without steering; exact Setting-A (orange) is the released Steerers [ 5 ] Setting-A model; PaSteer-free (blue) is our relaxed steerer. PaSteer-free is at or above both across nearly the whole sweep, with the largest gains at the

C_{8}

discretization-worst angles.

$\theta$	$\rho^{\star}$	exact	PaSteer-soft	PaSteer-free
$30^{\circ}$	0.229	0.305	0.311	0.239
$90^{\circ}$	0.264	0.360	0.348	0.280
$180^{\circ}$	0.236	0.452	0.366	0.248
comp. violation	$0.60$ – $0.70$
orthog. deviation	$0.85$ – $1.04$

Table 5: Steering error against the per-angle optimum. Steering error

1-\cos

is reported on HPatches [ 1 ] evaluation set using a frozen DeDoDe-B [ 9 ] descriptor. The top rows compare the closed-form per-angle optimum

\rho^{\star}

, the exact steerer, PaSteer-soft, and PaSteer-free at selected rotation angles. The bottom rows report the composition violation and orthogonality deviation of

\rho^{\star}

: the former measures how far

\rho^{\star}(\theta+\phi)

is from

\rho^{\star}(\theta)\rho^{\star}(\phi)

, and the latter measures how far

\rho^{\star\top}\rho^{\star}

is from the identity.

config	+params%	in-band	out-band	err(0)
exact expm	$0$	0.152	0.410	0.000
PaSteer-soft ( $r{=}12$ )	$+75$	0.156	0.374	0.000
PaSteer-soft ( $r{=}24$ )	$+150$	0.155	0.379	0.000
PaSteer-free	$+1200$	0.123	0.297	0.006

Table 6: The gain is functional form, not parameter count. Steerer mechanism comparison on a frozen DeDoDe-B [ 9 ] descriptor (HPatches [ 1 ] evaluation set): steering error

1-\cos

in and out of the partial-equivariance band, with err(0) the error at

0^{\circ}

. The out-of-band gain of PaSteer-soft over exact saturates with rank despite added parameters, so the remaining gap to PaSteer-free reflects leaving the representation and orthogonality constraints, not capacity.

steerer	params (%)	Roto@5	C4@5	SO2@5	$\Delta$ C4	$\Delta$ SO2
exact	0.48	92.1	32.3	20.3	–	–
$K{=}1$	1.5	89.0	20.8	12.4	$-11.6$	$-7.9$
$K{=}2$	2.4	92.2	29.9	19.2	$-2.4$	$-1.0$
$K{=}3$	3.4	93.1	34.6	22.6	$+2.3$	$+2.3$
$K{=}4$	4.4	93.4	35.1	23.5	$+2.7$	$+3.2$
$K{=}6$	6.3	93.6	37.9	23.9	$\mathbf{+5.6}$	$\mathbf{+3.7}$

Table 7: Free-Fourier capacity sweep over harmonic order

K

. Frozen DeDoDe-B [ 9 ] Setting-A: Roto@5 is Roto-360 [ 17 ] correct matches % @5px; C4@5 and SO2@5 are MegaDepth-1500 [ 19 ] pose AUC@5 ∘ ;

\Delta

C4 and

\Delta

SO2 are PaSteer-free

(K)

minus exact; params (%) is steerer parameters as a fraction of the descriptor. More parameters alone do not beat exact: a sufficient harmonic order in the free-form parameterization is needed to cross it.

steering	T1 handheld	T3 uniform
dual softmax (none)	50.8	10.7
exact expm (ours)	50.7	19.7
PaSteer-soft ( $r{=}12$ , ours)	50.7	24.7
PaSteer-free (ours)	49.9	35.0
Steerers Setting-A (release, ref.)	50.7	30.6

Table 8: The useful degree of equivariance depends on

q(\Delta\theta)

. RotoReal MD-1500 [ 19 ] pose AUC@5 ∘ on a frozen DeDoDe-B [ 9 ] , resampling pairs under a measured handheld distribution (T1) versus uniform over

\mathrm{SO}(2)

(T3); only the steering changes. Steering is near-irrelevant on T1 and separates the methods only on T3. The release row is MegaDepth-trained and shown for reference.

model	MD upright	MD C4	MD SO2
exact steerer	50.3	47.0	43.3
PaSteer-free (ours)	49.7	47.1	43.6

Table 9: Joint training reaches parity, isolating the frozen-descriptor advantage. Full-scale joint descriptor–steerer training on DeDoDe-B [ 9 ] , pose AUC@5 ∘ on MegaDepth-1500 [ 19 ] , with identical scenes, schedule, generator core, and initialization so only the steerer mechanism differs. Exact and PaSteer-free reach parity once the descriptor co-adapts, so the advantage of the relaxation is concentrated in the frozen-descriptor regime (supp. Tab. 10 ).

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Matched-budget controlled comparison
steering	upright	C4	SO2
dual softmax (none)	49.8	12.4	11.4
exact expm (ours, HPatches)	49.8	28.5	19.4
PaSteer-soft ( $r{=}12$ , ours)	49.8	37.5	24.4
PaSteer-free (ours)	49.0	43.1	33.0
Released reference models, not controlled

Appendix

Table 10: MegaDepth-1500 [ 19 ] matched-budget absolutes. Pose AUC@5 ∘ on a frozen DeDoDe-B [ 9 ] . The top block is the matched-budget controlled comparison (exact

<

PaSteer-soft

<

PaSteer-free); the bottom block lists released models as absolute references only.

steering	AP
Matched-budget controlled comparison
unsteered (dual softmax)	25.5
exact expm (ours, HPatches)	37.1
PaSteer-soft ( $r{=}12$ , ours)	44.4
PaSteer-free (ours)	51.2
Released reference models, not controlled

Appendix

Table 11: AIMS [ 31 ] matched-budget absolutes. Average precision on the FMAP evaluation subset, frozen DeDoDe-B [ 9 ] , with identical candidate set and positives across rows. The top block is the matched-budget controlled comparison (same ordering as Roto-360 [ 17 ] ); the bottom block lists released models as absolute references only.

IMC-PT [ 13 ]	upright	C4	SO2
exact	66.4/80.7/89.5	41.5/55.6/67.2	22.2/31.3/41.1
PaSteer-free	66.0/80.6/89.6	47.3/62.7/75.2	28.7/40.1/50.6

Appendix

Table 12: IMC-PT full-scale Setting-A controlled comparison. IMC-PT 2020 [ 13 ] validation (1500 pairs/axis), frozen DeDoDe-B [ 9 ] , pose AUC@{5,10,20} ∘ . PaSteer-free improves over exact on the large-rotation C4/SO2 axes (up to

+8.0

+9.5

AUC) while staying within a fraction of a point of exact on upright.

steerer	AP All	AP A.O.
exact expm	48.63	43.40
PaSteer-free (ours)	50.11	45.24
margin	$+1.48$	$+1.85$

Appendix

Table 13: AIMS [ 31 ] full-FMAP controlled comparison. Exact versus PaSteer-free on the full FMAP evaluation set (

323

frames,

21\,777

pairs), frozen DeDoDe-B [ 9 ] , full-scale Setting-A,

C_{8}

maximum-similarity matching with identical candidate set and positives. AP All is over all pairs and AP A.O. is the large-rotation (any-orientation) split; PaSteer-free improves on both. Protocol caveats are in Sec. C .

model	Roto	MD upr.	MD C4	MD SO2
exact steerer (joint)	91.9	49.7	46.1	42.3
PaSteer-free (ours, joint)	90.7	45.8	41.5	38.2
PaSteer-free (pinned, ours, joint)	–	47.7	42.2	39.2

Appendix

Table 14: Joint training on DeDoDe-G. Full-scale joint descriptor–steerer training on DeDoDe-G [ 9 ] . We report Roto-360 [ 17 ] correct matches @5px and MegaDepth-1500 [ 19 ] pose AUC@5 ∘ for upright, C4, and SO2 rotations. All rows use the same scenes, schedule, generator core, and initialization; only the steerer mechanism differs. Unlike the frozen-descriptor setting, the exact steerer keeps a small edge after the descriptor co-adapts, while pinning

\tilde{\rho}(0)=I

recovers part of the PaSteer-free upright gap.

Image Matching

Image matching finds corresponding points, regions, or objects across images despite changes in viewpoint, scale, lighting, or sensor.

See all 93 papers

Who Handles Orientation? Investigating Invariance in Feature Matching
Apr 13, 2026David Nordström, Johan Edstedt, Fredrik Kahl +1Image Matching Local Feature Matching
Chalmers University of Technology · Linköping University · University of Amsterdam
SHReg: Strictly Rotation-Equivariant Point Cloud Registration via Spherical Harmonics
Jul 25, 2026Chongjian Wang, Junjie GaoEquivariant Neural Networks Geometric Representation Learning
College of Mathematics and Systems Science, Shandong University of Science and Technology, Qingdao, China · School of Artificial Intelligence, Shandong Women’s University, Jinan, China
REDI-Match: Rotation-Equivariant Distillation for Efficient and Robust Dense Matching
Jun 23, 2026Yinji Ge, Guixu Zheng, Wulong Guo +5Image Matching Vision Foundation Model Adaptation
1Tsinghua University · 2Southern University of Science and Technology · 3Beihang University +1

Apr 13, 2026J/K move · Enter open · S save

Who Handles Orientation? Investigating Invariance in Feature Matching

David Nordström, Johan Edstedt, Fredrik Kahl, Georg Bökman

Chalmers University of Technology · Linköping University · University of Amsterdam

Image Matching Local Feature Matching Invariant Representation Learning

Open paper AskPDF arXiv

Finding matching keypoints between images is a core problem in 3D computer vision. However, modern matchers struggle with large in-plane rotations. A straightforward mitigation is to learn rotation invariance via data augmentation. However, it remains unclear at which stage rotation invariance should be incorporated. In this paper, we study this in the context of a modern sparse matching pipeline. We perform extensive experiments by training on a large collection of 3D vision datasets and evaluating on popular image matching benchmarks. Surprisingly, we find that incorporating rotation invariance already in the descriptor yields similar performance to handling it in the matcher. However, rotation invariance is achieved earlier in the matcher when it is learned in the descriptor, allowing for a faster rotation-invariant matcher. Further, we find that enforcing rotation invariance does not hurt upright performance when trained at scale. Finally, we study the emergence of rotation invariance through scale and find that increasing the training data size substantially improves generalization to rotated images. We release two matchers robust to in-plane rotations that achieve state-of-the-art performance on e.g. multi-modal (WxBS), extreme (HardMatch), and satellite image matching (SatAst). Code is available at https://github.com/davnords/loma.

Abstract

Image Matching

Explore similar work

Who Handles Orientation? Investigating Invariance in Feature Matching

SHReg: Strictly Rotation-Equivariant Point Cloud Registration via Spherical Harmonics

REDI-Match: Rotation-Equivariant Distillation for Efficient and Robust Dense Matching