ImCorr: Sub-pixel Semantic Correspondence via Implicit Feature Decoding

Yusung Choi

cs.CVSep 24, 2026

ImCorr: Sub-pixel Semantic Correspondence via Implicit Feature Decoding

Authors: Yusung Choi

Organizations: Pukyong National University, Busan, Republic of Korea

Abstract

The strong performance that modern semantic correspondence methods achieve at standard thresholds plateaus sharply at fine-grained thresholds. We argue that this plateau stems not from the representational capacity of backbone features, but from a grid-tied readout. Patch-based vision transformers tokenize images onto discrete grids, introducing two forms of quantization error: querying nearest patch features instead of exact keypoints on the source side, and the absence of grid features representing precise ground-truth locations on the target side. We quantify this quantization ceiling across all 499,188 keypoints in SPair-71k: under the standard 448x448, patch-14 setting, 84.9% of ground-truth keypoints have no grid feature representing their precise location at PCK@0.01. This is a structural limitation at the representation level, independent of the matching strategy. We address this with ImCorr: Sub-pixel Semantic Correspondence via Implicit Feature Decoding, which formulates correspondence estimation over a continuous feature field queryable at arbitrary continuous coordinates. A FiLM-conditioned decoder is trained to embed sub-pixel positional information into the feature field. Querying the field directly at exact keypoint coordinates theoretically eliminates representation-level quantization error on the source side, while decoding onto a grid denser than the backbone grid substantially reduces quantization error on the target side. On SPair-71k and AP-10K (intra-species, cross-species, and cross-family), ImCorr improves performance at fine-grained thresholds (PCK@0.01-0.05), achieving a 6.2 percentage point gain over the prior state of the art at PCK@0.01 on SPair-71k. These results demonstrate that representational continuity is an effective solution for precise semantic correspondence. Code is available at https://github.com/YusungChoi/ImCorr.

Figures & tables

Figure 1 : Illustration of grid quantization error in semantic correspondence. (a) Source-side quantization : the same patch feature is used in matching regardless of where within the patch a keypoint lies, so the keypoint’s exact positional information is not reflected in the representation. The distance to the nearest patch center, $\|\mathbf{x}-\hat{\mathbf{x}}\|$ , quantifies the extent of positional information disregarded. (b) Target-side quantization : when the ground-truth location falls between grid points, no grid feature represents its precise location, making correct localization structurally impossible within the PCK threshold.

Image Size	Patch Size	PCK@0.1	PCK@0.05	PCK@0.01
224 $\times$ 224	14	2.70%	27.56%	96.23%
224 $\times$ 224	16	5.30%	36.72%	97.22%
448 $\times$ 448	14	0.26%	2.88%	84.91%
448 $\times$ 448	16	0.28%	5.41%	88.59%
896 $\times$ 896	14	<0.01%	0.22%	44.06%
896 $\times$ 896	16	0.03%	0.34%	55.16%

Table 1 : Proportion of keypoints unreachable within the PCK threshold (relative to target bounding box) when only feature map grid points are considered as candidates (SPair-71k, 499,188 keypoints). This represents a structural ceiling independent of backbone quality.

Figure 2 : Overview of the FiLM-conditioned decoder and local ensemble. (Left) Zoomed-in view of the decoding process for grid point $\mathbf{z}^{*}_{10}$ : the offset $\mathbf{\Delta}=[w,h]$ relative to the query coordinate is transformed by an MLP $\varphi$ into channel-wise modulation parameters $[\boldsymbol{\gamma};\boldsymbol{\beta}]$ (Eq. ( 5 )), which modulate the grid feature (Eq. ( 6 )) and are subsequently refined by a nonlinear rendering MLP $\rho$ to produce the per-grid-point feature $f_{\theta}(\mathbf{z}^{*},\mathbf{\Delta})$ (Eq. ( 7 )). (Right) The four per-grid-point features surrounding the query coordinate (red star) are combined via area-based weighting, added to a bilinear residual, and projected through a linear layer $\psi$ to produce the final feature $F_{\mathbf{I}}(\mathbf{x})$ (Eq. ( 8 )).

Unsupervised/Weakly Supervised
		SPair-71k			AP-10K (I.S.)			AP-10K (C.S.)			AP-10K (C.F.)
Method	Backbone	0.01	0.05	0.10	0.01	0.05	0.10	0.01	0.05	0.10	0.01	0.05	0.10
DINOv2+NN [ 20 ]	ViT-B	6.3	38.4	53.9	6.4	41.0	60.9	5.3	37.0	57.3	4.4	29.4	47.4
DIFT [ 17 ]	SD	7.2	39.7	52.9	6.2	34.8	50.3	5.1	30.8	46.0	3.7	22.4	35.0
SD+DINO [ 20 ]	SD+ViT-B	7.9	44.7	59.9	7.6	43.5	62.9	6.4	39.7	59.3	5.2	30.8	48.3
DIY-SC [ 4 ]	SD+ViT-B	10.1	53.8	71.6	-	-	70.6	-	-	69.8	-	-	57.8

Table 2 : Quantitative comparison on standard semantic correspondence benchmarks. We report per-image PCK (%,

\uparrow

) at multiple thresholds on SPair-71k and three splits of AP-10K (intra-species, cross-species, cross-family). The best and second-best results per column are bolded and underlined , respectively.

\lx@sectionsign

uses depth maps at training;

\dagger

uses object masks at training;

\ddagger

uses object masks at inference.

Figure 3 : Qualitative comparison on SPair-71k (PCK@0.01, per-image). Green and red dots indicate correct and incorrect correspondences, respectively. While GeoAware-SC and MARCO capture the approximate location of most keypoints, they tend to deviate slightly from the ground truth under the fine-grained threshold. In contrast, ImCorr matches the same keypoints with substantially higher precision.

Setting	Src	Tgt	PCK@0.01	PCK@0.05	PCK@0.10
Baseline			19.1	72.9	83.3
+ Src Query	✓		25.6	77.2	86.7
+ Tgt Decoding		✓	30.1	76.1	84.1
Full model (Ours)	✓	✓	33.2	78.9	86.2
Gain over baseline			+14.1	+6.0	+2.9

Table 3: Ablation on source-side query and target-side decoding. We report per-image PCK (%,

\uparrow

) on SPair-71k. ✓denotes the component is enabled.

$r$	PCK@0.01	PCK@0.05	PCK@0.10
2	30.3	76.2	86.9
4	33.2	78.9	86.2
6	32.9	78.0	85.6
8	31.7	77.1	85.1

Table 4: Ablation on the densification factor

r

of the dense search grid

\mathcal{G}^{\prime}

. We report per-image PCK (%,

\uparrow

) on SPair-71k.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4 : Additional results on SPair-71k across object categories.

Figure 5 : Additional results on SPair-71k across object categories.

Figure 6 : Additional results on SPair-71k across object categories.

Explore similar work

Apr 20, 2026cs.CV

MARCO: Navigating the Unseen Space of Semantic Correspondence

Recent advances in semantic correspondence rely on dual-encoder architectures, combining DINOv2 with diffusion backbones. While accurate, these billion-parameter models generalize poorly beyond training keypoints, revealing a gap between benchmark performance and real-world usability, where queried points rarely match those seen during training. Building upon DINOv2, we introduce MARCO, a unified model for generalizable correspondence driven by a novel training framework that enhances both fine-grained localization and semantic generalization. By coupling a coarse-to-fine objective that refines spatial precision with a self-distillation framework, which expands sparse supervision beyond annotated regions, our approach transforms a handful of keypoints into dense, semantically coherent correspondences. MARCO sets a new state of the art on SPair-71k, AP-10K, and PF-PASCAL, with gains that amplify at fine-grained localization thresholds (+8.9 PCK@0.01), strongest generalization to unseen keypoints (+5.1, SPair-U) and categories (+4.7, MP-100), while remaining 3x smaller and 10x faster than diffusion-based approaches. Code is available at https://github.com/visinf/MARCO .

Claudia Cuttano, Gabriele Trivigno, Carlo Masone +1

Politecnico di Torino · TU Darmstadt · ELIZA

Ground-Truth Correspondence Dual-Encoder Architectures Fine-Grained Perception

Ask PDF

May 23, 2025cs.CV

Semantic Correspondence: Unified Benchmarking and a Strong Baseline

Establishing semantic correspondence is a challenging task in computer vision, aiming to match keypoints with the same semantic information across different images. Benefiting from the rapid development of deep learning, remarkable progress has been made over the past decade. However, a comprehensive review and analysis of this task remains absent. In this paper, we present the first extensive survey of semantic correspondence methods. We first propose a taxonomy to classify existing methods based on the type of their method designs. These methods are then categorized accordingly, and we provide a detailed analysis of each approach. Furthermore, we aggregate and summarize the results of methods in literature across various benchmarks into a unified comparative table, with detailed configurations to highlight performance variations. Additionally, to provide a detailed understanding on existing methods for semantic matching, we thoroughly conduct controlled experiments to analyse the effectiveness of the components of different methods. Finally, we propose a simple yet effective baseline that achieves state-of-the-art performance on multiple benchmarks, providing a solid foundation for future research in this field. We hope this survey serves as a comprehensive reference and consolidated baseline for future development. Code is publicly available at: https://github.com/Visual-AI/Semantic-Correspondence.

Kaiyan Zhang, Xinghui Li, Jingyi Lu +1

University of Hong Kong, Hong Kong · Department of Engineering, University of Oxford, OX1 3PJ Oxford, U.K.

Ground-Truth Correspondence Computer Vision Strong Baseline Deep Learning

Ask PDF

Jun 2, 2026cs.CV

SAMatcher: Co-Visibility Modeling with Segment Anything for Robust Feature Matching

Reliable correspondence estimation is a fundamental problem in image processing, underpinning applications such as Structure from Motion, visual localization, and image registration. Existing learning-based methods have significantly improved local feature representations, yet most still operate at the pixel or patch level and lack explicit modeling of regions that are jointly visible across views. We propose SAMatcher, a feature matching framework that formulates correspondence estimation through co-visibility modeling. Instead of directly matching local features, SAMatcher first predicts co-visible region masks and bounding boxes as structured priors for correspondence estimation. Built upon the Segment Anything Model (SAM), it introduces a symmetric cross-view interaction mechanism that enables bidirectional feature exchange and cross-view semantic alignment. We further develop a unified supervision scheme that jointly optimizes mask prediction and box localization through mask learning, box regression, and mask-box consistency constraints. Extensive experiments on challenging benchmarks demonstrate substantial improvements over existing matching pipelines, particularly under large viewpoint and scale variations. Our results show that foundation models originally designed for monocular segmentation can be effectively extended to multi-view correspondence reasoning through explicit co-visibility modeling, offering a new perspective on structured representation learning for image matching. Code and project page: https://xupan.top/Projects/samatcher

Xu Pan, Qiyuan Ma, Mingyue Dong +3

State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University, Wuhan, P. R. China · National Key Laboratory of Space Target Awareness, Space Engineering University, Beijing, P. R. China

Harder Better Faster Denser Feature Matching Segment Anything Model

Ask PDF