Beyond Spatio-Temporal Priors: A Generalizable Approach for Dense Correspondence Matching

Luping Liu; Bingyi Kang; Yifan Wang; Dong Xu

cs.CVOct 8, 2026

Beyond Spatio-Temporal Priors: A Generalizable Approach for Dense Correspondence Matching

Authors: Luping Liu, Bingyi Kang, Yifan Wang, Dong Xu

Organizations: The University of Hong Kong · ByteDance Seed · Zhejiang University

Abstract

Dense correspondence matching has historically been bounded by simplifying spatio-temporal priors, such as smooth motion and rigid geometry. While effective for classical tasks, these assumptions break down in image editing and reference-guided generation (IEG), where transformations can preserve visual identity while breaking physical continuity. To establish identity-preserving correspondence across such transformations, we introduce FreeMatching, a generalizable framework combining generative and semantic foundation representations with heterogeneous supervision from classical datasets, tracked videos, and synthetic scenes. Teacher-guided iterative refinement further improves correspondence in IEG without dense correspondence annotations. Experimentally, a single FreeMatching model substantially improves correspondence quality on challenging IEG image pairs while retaining competitive performance on classical benchmarks. Furthermore, we demonstrate its utility as a quantitative metric for evaluating identity preservation, with scores that correlate with human judgment. The code is available at https://github.com/luping-liu/FreeMatching.

Figures & tables

Figure 1: Extending Dense Correspondence to IEG. FreeMatching matches across optical flow, geometric matching, and IEG. Each block shows input pairs above and our bidirectional warps below. The classical examples vary in temporal separation (top) and geometric complexity (bottom).

Figure 2: FreeMatching Architecture. Our transformer is initialized from FLUX.2-klein-base-4B. Each image is encoded into latent tokens and augmented with projected DINOv3 features. The two token sequences are concatenated for joint attention, and a task-specific decoder predicts the dense correspondence map and covisibility mask in a single forward pass.

Dataset	Ratio	Dataset	Ratio
ScanNet++ [ 33 ]	15%	Spring [ 34 ]	5%
BlendedMVS [ 35 ]	10%	PointOdyssey [ 36 ]	5%
FlyingThings [ 18 ]	10%	FlyingChairs [ 37 ]	5%
DynamicReplica [ 38 ]	10%
Video (Ours)	20%	Blender (Ours)	20%

Table 1: Supervised Training Data. We combine classical correspondence datasets with our tracked-video and Blender data. Ratios indicate sampling proportions, rounded to multiples of 5%.

Figure 3: Teacher-Guided Iterative Refinement. An exponential moving average (EMA) copy of the student predicts the correspondence used to produce the Initial Warp. A frozen teacher estimates a local correction; composing it with the initial correspondence produces the Refined Warp and a correspondence target for training the student. The EMA copy evolves with the student while the teacher remains fixed.

	Sintel (Clean)				Sintel (Final)				KITTI-2015
Method	EPE $\downarrow$	1px $\downarrow$	2px $\downarrow$	5px $\downarrow$	EPE $\downarrow$	1px $\downarrow$	2px $\downarrow$	5px $\downarrow$	EPE $\downarrow$	1px $\downarrow$	2px $\downarrow$	5px $\downarrow$
GMFlow [ 21 ]	1.21	41.3	12.6	1.9	1.56	42.7	14.4	3.2	4.32	60.3	35.5	16.1
SEA-RAFT [ 32 ]	0.43	5.9	2.7	0.9	0.81	8.7	4.8	2.1	1.97	29.7	16.8	7.2
UniMatch [ 6 ]	0.40	5.4	2.4	0.8	0.75	8.9	4.9	2.2	1.52	24.1	12.5	5.1
RoMa [ 5 ]	1.24	25.2	8.1	4.5	2.49	35.6	16.6	8.7	5.91	41.4	27.5	18.4
UFM [ 7 ]	0.92	14.2	6.4	2.1	1.48	16.9	9.0	3.8	4.75	39.6	24.5	15.5

Table 3: Optical Flow Estimation. Comparison with representative optical-flow and dense-matching methods on Sintel [ 43 ] and KITTI-2015 [ 44 ] , evaluating the cross-task generalization of the same FreeMatching model. Best and second-best results are bold and underlined , respectively.

Model	Ref-MSE $\downarrow$	LPIPS $\downarrow$	SigLIP 2 $\uparrow$	DINOv3 $\uparrow$
Identity	23.65	81.97	86.67	52.38
SEA-RAFT [ 32 ]	18.49	72.45	85.18	58.14
UniMatch [ 6 ]	11.07	58.42	87.86	66.35
RoMa [ 5 ]	8.08	48.91	87.73	70.15
UFM [ 7 ]	10.41	53.05	86.97	66.77
FreeMatching (Ours)	6.22	45.41	90.77	80.01

Table 4: IEG-Bench Evaluation. We report the mean reconstruction similarity scores (higher is better) using SigLIP 2 [ 45 ] and DINOv3 [ 14 ] features, along with reference-mask MSE (Ref-MSE) and LPIPS [ 41 ] (lower is better) between the warped and target images. The identity baseline compares the unwarped source image with the target using an identity correspondence map. Best and second-best results are bold and underlined , respectively.

Figure 4: Visual comparisons on IEG-Bench using UNO examples. The references in Image B are composed into Image A ( $B\rightarrow A$ ). Each method warps Image A toward Image B to evaluate correspondence quality.

Method	Ref-MSE $\downarrow$	Full-MSE $\downarrow$	Ref-DINOv2 $\uparrow$	OpenCLIP $\uparrow$
FreeMatching (10k-IEG)	6.22	10.99	54.31	70.56
UFM (10k-IEG) [ 7 ]	8.55	13.48	39.65	56.48
RoMa (10k-IEG) [ 5 ]	9.70	16.57	47.14	61.18

Table 5: IEG-Bench comparison after 10k adaptation updates. All variants use UNO+OmniGen2 refinement data. Ref-MSE and Ref-DINOv2 use the shared frozen reference mask; Full-MSE and OpenCLIP use the full image. All metrics are scaled by 100. DINOv2 [ 22 ] and OpenCLIP [ 46 ] complement the DINOv3 metric used in the stage-wise analysis.

Figure 5: Fine-Grained Identity Assessment. Nano Banana edits Image A into Image B ( $A\rightarrow B$ ). DINOv3 similarity heatmaps compare the warped source with the target within covisible regions, revealing local appearance inconsistencies. Red boxes highlight affected regions, including the altered balloon text.

ID	FLUX Init	DINOv3	Video & Blender	Refine	DINOv3 $\uparrow$
1					67.31
2	✓				69.28
3	✓	✓			72.76
4	✓	✓	✓		76.52
5	✓	✓	✓	✓	80.01

Table 6: Stage-wise System Construction. We progressively add components and report DINOv3 similarity (

\times 100

) on IEG-Bench. The checkmark (✓) indicates the component is included. FLUX Init: FLUX generative initialization; DINOv3: DINOv3 semantic features; Video & Blender: Video & Blender datasets; Refine: Teacher-guided iterative refinement.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Overview of the constructed datasets. We utilize Real-World Video data for learning natural deformations, Synthetic Blender data for precise geometric supervision, and the IEG benchmark for evaluating identity-preserving correspondence between source or reference images and generated outputs.

Method	Training cost (H20 GPU-hours)
FreeMatching	42.0
UFM [ 7 ]	16.8
RoMa [ 5 ]	30.8

Appendix

Table 7: Training cost of 10k IEG adaptation on 16 H20 GPUs with global batch size 32. GPU-hours are approximate totals across the training GPUs.

Method	Latency (ms/pair)	Peak memory (GiB)
FreeMatching	121.9	15.18
UFM [ 7 ]	51.3	2.35
RoMa [ 5 ]	102.5	2.58

Appendix

Table 8: Inference cost on one H20 GPU at external resolution

288\times 512

and batch size 1. Latency is averaged over 50 calls after warm-up; memory is peak allocated GPU memory.

Method	Sintel Clean	Sintel Final	KITTI	ScanNet	ETH3D
FreeMatching (10k-IEG)	1.05	1.36	4.95	8.56	10.23
UFM (10k-IEG) [ 7 ]	0.92	1.48	4.75	12.03	12.98
RoMa (10k-IEG) [ 5 ]	1.24	2.49	5.91	12.03	11.26

Appendix

Table 9: Classical-task EPE after 10k IEG adaptation. Lower is better; best results within this comparison are bold.

Checkpoint	Ref-MSE	Sintel Clean	Sintel Final	KITTI	ScanNet	ETH3D
FreeMatching-S1	8.08	1.11	1.73	6.26	8.85	11.68
FreeMatching (10k-Flow)	7.26	0.98	1.59	6.46	8.91	11.64
FreeMatching (10k-IEG)	6.22	1.05	1.36	4.95	8.56	10.23

Appendix

Table 10: Effect of the refinement data source. We report IEG-Bench Ref-MSE (

\times 100

) and EPE on classical benchmarks. Lower is better; best results within this comparison are bold.

Model	Human $\uparrow$	LPIPS $\downarrow$	DINOv3 $\uparrow$
SEA-RAFT [ 32 ]	0.49	72.45	58.14
RoMa [ 5 ]	1.39	48.91	70.15
UFM [ 7 ]	1.13	53.05	66.77
FreeMatching (Ours)	2.25	45.41	80.01

Appendix

Table 11: Quantitative comparison on IEG-Bench. We report the Human Mean Opinion Score (MOS) and objective metrics. Human (MOS) is rated on a scale of 0-3 (higher is better). LPIPS [ 41 ] and DINOv3 [ 14 ] are scaled by 100. Our method achieves the highest ratings across all metrics. Best results are highlighted in bold .

Figure 7: Comparison with Semantic Matching. In this multi-object example, the displayed SD-DINO [ 47 ] reconstructions, obtained with the prompts “book” and “vase”, exhibit incomplete shapes and mixed object appearance. FreeMatching preserves more complete object shapes and local details without text prompts.

Explore similar work

SAMatcher: Co-Visibility Modeling with Segment Anything for Robust Feature Matching
Jun 2, 2026Xu Pan, Qiyuan Ma, Mingyue Dong +3Multi-View Geometry Image Matching
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University, Wuhan, P. R. China · National Key Laboratory of Space Target Awareness, Space Engineering University, Beijing, P. R. China
MARCO: Navigating the Unseen Space of Semantic Correspondence
Apr 20, 2026Claudia Cuttano, Gabriele Trivigno, Carlo Masone +1Semantic Correspondence
Politecnico di Torino · TU Darmstadt · ELIZA
FAST: Flow Any Scene Transformer
Sep 30, 2026Yongjian Zhang, Longguang Wang, Zhuo Song +3Optical Flow Vision Transformer
School of Electronics and Communication Engineering, the Shenzhen Campus of Sun Yat-sen University, Sun Yat-sen University, Shenzhen, China · Hong Kong Polytechnic University, HKSAR, China · School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China

Jun 2, 2026, cs.CVJ/K move · Enter open · S save

SAMatcher: Co-Visibility Modeling with Segment Anything for Robust Feature Matching

Xu Pan, Qiyuan Ma, Mingyue Dong, He Chen+2

State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University, Wuhan, P. R. China · National Key Laboratory of Space Target Awareness, Space Engineering University, Beijing, P. R. China

Multi-View Geometry Image Matching Local Feature Matching

Open paper AskPDF arXiv

Reliable correspondence estimation is a fundamental problem in image processing, underpinning applications such as Structure from Motion, visual localization, and image registration. Existing learning-based methods have significantly improved local feature representations, yet most still operate at the pixel or patch level and lack explicit modeling of regions that are jointly visible across views. We propose SAMatcher, a feature matching framework that formulates correspondence estimation through co-visibility modeling. Instead of directly matching local features, SAMatcher first predicts co-visible region masks and bounding boxes as structured priors for correspondence estimation. Built upon the Segment Anything Model (SAM), it introduces a symmetric cross-view interaction mechanism that enables bidirectional feature exchange and cross-view semantic alignment. We further develop a unified supervision scheme that jointly optimizes mask prediction and box localization through mask learning, box regression, and mask-box consistency constraints. Extensive experiments on challenging benchmarks demonstrate substantial improvements over existing matching pipelines, particularly under large viewpoint and scale variations. Our results show that foundation models originally designed for monocular segmentation can be effectively extended to multi-view correspondence reasoning through explicit co-visibility modeling, offering a new perspective on structured representation learning for image matching. Code and project page: https://xupan.top/Projects/samatcher

Abstract

Explore similar work

SAMatcher: Co-Visibility Modeling with Segment Anything for Robust Feature Matching

MARCO: Navigating the Unseen Space of Semantic Correspondence

FAST: Flow Any Scene Transformer