ImCorr: Sub-pixel Semantic Correspondence via Implicit Feature Decoding

Yusung Choi

cs.CVSep 24, 2026

ImCorr: Sub-pixel Semantic Correspondence via Implicit Feature Decoding

Authors: Yusung Choi

Organizations: Pukyong National University, Busan, Republic of Korea

Abstract

The strong performance that modern semantic correspondence methods achieve at standard thresholds plateaus sharply at fine-grained thresholds. We argue that this plateau stems not from the representational capacity of backbone features, but from a grid-tied readout. Patch-based vision transformers tokenize images onto discrete grids, introducing two forms of quantization error: querying nearest patch features instead of exact keypoints on the source side, and the absence of grid features representing precise ground-truth locations on the target side. We quantify this quantization ceiling across all 499,188 keypoints in SPair-71k: under the standard 448x448, patch-14 setting, 84.9% of ground-truth keypoints have no grid feature representing their precise location at PCK@0.01. This is a structural limitation at the representation level, independent of the matching strategy. We address this with ImCorr: Sub-pixel Semantic Correspondence via Implicit Feature Decoding, which formulates correspondence estimation over a continuous feature field queryable at arbitrary continuous coordinates. A FiLM-conditioned decoder is trained to embed sub-pixel positional information into the feature field. Querying the field directly at exact keypoint coordinates theoretically eliminates representation-level quantization error on the source side, while decoding onto a grid denser than the backbone grid substantially reduces quantization error on the target side. On SPair-71k and AP-10K (intra-species, cross-species, and cross-family), ImCorr improves performance at fine-grained thresholds (PCK@0.01-0.05), achieving a 6.2 percentage point gain over the prior state of the art at PCK@0.01 on SPair-71k. These results demonstrate that representational continuity is an effective solution for precise semantic correspondence. Code is available at https://github.com/YusungChoi/ImCorr.

Figures & tables

Figure 1 : Illustration of grid quantization error in semantic correspondence. (a) Source-side quantization : the same patch feature is used in matching regardless of where within the patch a keypoint lies, so the keypoint’s exact positional information is not reflected in the representation. The distance to the nearest patch center, $\|\mathbf{x}-\hat{\mathbf{x}}\|$ , quantifies the extent of positional information disregarded. (b) Target-side quantization : when the ground-truth location falls between grid points, no grid feature represents its precise location, making correct localization structurally impossible within the PCK threshold.

Image Size	Patch Size	PCK@0.1	PCK@0.05	PCK@0.01
224 $\times$ 224	14	2.70%	27.56%	96.23%
224 $\times$ 224	16	5.30%	36.72%	97.22%
448 $\times$ 448	14	0.26%	2.88%	84.91%
448 $\times$ 448	16	0.28%	5.41%	88.59%
896 $\times$ 896	14	<0.01%	0.22%	44.06%
896 $\times$ 896	16	0.03%	0.34%	55.16%

Table 1 : Proportion of keypoints unreachable within the PCK threshold (relative to target bounding box) when only feature map grid points are considered as candidates (SPair-71k, 499,188 keypoints). This represents a structural ceiling independent of backbone quality.

Figure 2 : Overview of the FiLM-conditioned decoder and local ensemble. (Left) Zoomed-in view of the decoding process for grid point $\mathbf{z}^{*}_{10}$ : the offset $\mathbf{\Delta}=[w,h]$ relative to the query coordinate is transformed by an MLP $\varphi$ into channel-wise modulation parameters $[\boldsymbol{\gamma};\boldsymbol{\beta}]$ (Eq. ( 5 )), which modulate the grid feature (Eq. ( 6 )) and are subsequently refined by a nonlinear rendering MLP $\rho$ to produce the per-grid-point feature $f_{\theta}(\mathbf{z}^{*},\mathbf{\Delta})$ (Eq. ( 7 )). (Right) The four per-grid-point features surrounding the query coordinate (red star) are combined via area-based weighting, added to a bilinear residual, and projected through a linear layer $\psi$ to produce the final feature $F_{\mathbf{I}}(\mathbf{x})$ (Eq. ( 8 )).

Unsupervised/Weakly Supervised
		SPair-71k			AP-10K (I.S.)			AP-10K (C.S.)			AP-10K (C.F.)
Method	Backbone	0.01	0.05	0.10	0.01	0.05	0.10	0.01	0.05	0.10	0.01	0.05	0.10
DINOv2+NN [ 20 ]	ViT-B	6.3	38.4	53.9	6.4	41.0	60.9	5.3	37.0	57.3	4.4	29.4	47.4
DIFT [ 17 ]	SD	7.2	39.7	52.9	6.2	34.8	50.3	5.1	30.8	46.0	3.7	22.4	35.0
SD+DINO [ 20 ]	SD+ViT-B	7.9	44.7	59.9	7.6	43.5	62.9	6.4	39.7	59.3	5.2	30.8	48.3
DIY-SC [ 4 ]	SD+ViT-B	10.1	53.8	71.6	-	-	70.6	-	-	69.8	-	-	57.8

Table 2 : Quantitative comparison on standard semantic correspondence benchmarks. We report per-image PCK (%,

\uparrow

) at multiple thresholds on SPair-71k and three splits of AP-10K (intra-species, cross-species, cross-family). The best and second-best results per column are bolded and underlined , respectively.

\lx@sectionsign

uses depth maps at training;

\dagger

uses object masks at training;

\ddagger

uses object masks at inference.

Figure 3 : Qualitative comparison on SPair-71k (PCK@0.01, per-image). Green and red dots indicate correct and incorrect correspondences, respectively. While GeoAware-SC and MARCO capture the approximate location of most keypoints, they tend to deviate slightly from the ground truth under the fine-grained threshold. In contrast, ImCorr matches the same keypoints with substantially higher precision.

Setting	Src	Tgt	PCK@0.01	PCK@0.05	PCK@0.10
Baseline			19.1	72.9	83.3
+ Src Query	✓		25.6	77.2	86.7
+ Tgt Decoding		✓	30.1	76.1	84.1
Full model (Ours)	✓	✓	33.2	78.9	86.2
Gain over baseline			+14.1	+6.0	+2.9

Table 3: Ablation on source-side query and target-side decoding. We report per-image PCK (%,

\uparrow

) on SPair-71k. ✓denotes the component is enabled.

$r$	PCK@0.01	PCK@0.05	PCK@0.10
2	30.3	76.2	86.9
4	33.2	78.9	86.2
6	32.9	78.0	85.6
8	31.7	77.1	85.1

Table 4: Ablation on the densification factor

r

of the dense search grid

\mathcal{G}^{\prime}

. We report per-image PCK (%,

\uparrow

) on SPair-71k.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4 : Additional results on SPair-71k across object categories.

Figure 5 : Additional results on SPair-71k across object categories.

Figure 6 : Additional results on SPair-71k across object categories.

Explore similar work

MARCO: Navigating the Unseen Space of Semantic Correspondence
Apr 20, 2026Claudia Cuttano, Gabriele Trivigno, Carlo Masone +1Ground-Truth Correspondence Dual-Encoder Architectures
Politecnico di Torino · TU Darmstadt · ELIZA
Semantic Correspondence: Unified Benchmarking and a Strong Baseline
May 23, 2025Kaiyan Zhang, Xinghui Li, Jingyi Lu +1Ground-Truth Correspondence Computer Vision
University of Hong Kong, Hong Kong · Department of Engineering, University of Oxford, OX1 3PJ Oxford, U.K.
SAMatcher: Co-Visibility Modeling with Segment Anything for Robust Feature Matching
Jun 2, 2026Xu Pan, Qiyuan Ma, Mingyue Dong +3Harder Better Faster Denser Feature Matching Segment Anything Model
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University, Wuhan, P. R. China · National Key Laboratory of Space Target Awareness, Space Engineering University, Beijing, P. R. China

Apr 20, 2026, cs.CVJ/K move · Enter open · S save

MARCO: Navigating the Unseen Space of Semantic Correspondence

Claudia Cuttano, Gabriele Trivigno, Carlo Masone, Stefan Roth

Politecnico di Torino · TU Darmstadt · ELIZA

Ground-Truth Correspondence Dual-Encoder Architectures Fine-Grained Perception

Open paper AskPDF arXiv

Recent advances in semantic correspondence rely on dual-encoder architectures, combining DINOv2 with diffusion backbones. While accurate, these billion-parameter models generalize poorly beyond training keypoints, revealing a gap between benchmark performance and real-world usability, where queried points rarely match those seen during training. Building upon DINOv2, we introduce MARCO, a unified model for generalizable correspondence driven by a novel training framework that enhances both fine-grained localization and semantic generalization. By coupling a coarse-to-fine objective that refines spatial precision with a self-distillation framework, which expands sparse supervision beyond annotated regions, our approach transforms a handful of keypoints into dense, semantically coherent correspondences. MARCO sets a new state of the art on SPair-71k, AP-10K, and PF-PASCAL, with gains that amplify at fine-grained localization thresholds (+8.9 PCK@0.01), strongest generalization to unseen keypoints (+5.1, SPair-U) and categories (+4.7, MP-100), while remaining 3x smaller and 10x faster than diffusion-based approaches. Code is available at https://github.com/visinf/MARCO .

Abstract

Explore similar work

MARCO: Navigating the Unseen Space of Semantic Correspondence

Semantic Correspondence: Unified Benchmarking and a Strong Baseline

SAMatcher: Co-Visibility Modeling with Segment Anything for Robust Feature Matching