Self-Calibrating Dense Displacement Fields for Reliable Co-Registration of Large Optical Satellite Imagery

Shoukun Sun; Zhe Wang; Sanaz Salati; Jiyin Zhang; Hui Wang; Xiaogang Ma

cs.CVAug 23, 2026

Self-Calibrating Dense Displacement Fields for Reliable Co-Registration of Large Optical Satellite Imagery

Authors: Shoukun Sun, Zhe Wang, Sanaz Salati, Jiyin Zhang, Hui Wang, Xiaogang Ma

Organizations: Department of Computer Science, University of Idaho, Moscow, ID 83844 USA · National Center for Ecological Analysis and Synthesis (NCEAS), University of California, Santa Barbara, CA 93101 USA · Research Computing and Data Services (RCDS), University of Idaho, Moscow, ID 83844 USA · Department of Geography and Planning, Appalachian State University, Boone, NC 28608 USA

Abstract

Co-registration underlies nearly every multi-temporal and multi-sensor use of optical satellite imagery, and operational products still carry documented offsets well above the fraction-of-a-pixel scale at which change detection, time series, and data fusion degrade. Real image pairs differ along several axes at once (sensor response, scene content, viewing geometry, resolution, mosaic seams), and the last of these is not a single global motion. Existing tools embed a motion model and constants tuned to their development data; a pair that fits is registered precisely, while one that does not either fails to match or returns a result wrong by tens of pixels with no failure reported. Learned matchers add a GPU requirement and carry no accuracy guarantee outside their training distribution. We present SCDF (self-calibrating displacement fields), a training-free, GPU-free estimator whose motion model is the dense per-pixel displacement field itself, so no scene motion falls outside the model. A single predict--measure--filter loop runs over a resolution pyramid: the accumulated field predicts where each patch of the moving image falls in the reference, RootSIFT matching and a correlation pass measure the displacement there to sub-pixel precision, and filters whose thresholds are all calibrated on the image pair itself decide what survives. One configuration, with no per-dataset tuning, processes full $8192^2$ scenes on a single CPU core. On 584 constructed-ground-truth pairs built from real Sentinel-2, Landsat-8/9, and NAIP imagery, against seven classical baselines and two zero-shot pretrained matchers, SCDF registers every pair with zero failures, reduces the best baseline's real-pair median end-point error from 6.83 to 4.17m, and cuts its 90th percentile from 17.8 to 7.77m.

Figures & tables

Fig. 1: Misalignment drawn as a warped grid, exaggerated $4\times$ . (a) A real Sentinel-2 pair arrives misaligned by a constructed viewing tilt. (b) What GeFolki, the baseline with the lowest real-pair median error, leaves behind at its worst benchmark block. (c) What SCDF leaves on the same pair. (d) Block-median error on the 20 real-pair blocks as a ratio to the best method per block ( $\circ$ this block, $\times$ declined, right column the worst case); SCDF stays within $1.39\times$ everywhere.

Family	Motion model	Decision constants	Output
Phase-correlation grids AROSICS; COSI-Corr	translation per window, then a global shift or affine fit	thresholds tuned per product	tie-point grid $\to$ fitted model
Dense chip tracking autoRIFT	translation per chip, fixed search range	fixed ${\sim}25$ px window, quality thresholds	dense grid
Optical flow GeFolki	small, smooth flow	tuned (rank filter, radii)	dense per-pixel field
Sparse features $+$ model SIFT+RANSAC; RIFT, RIFT2, SRIF	single global affine or homography	ratio and inlier thresholds	sparse matches $\to$ one global transform
Learned matchers LightGlue, LoFTR; RoMa	learned prior; fixed working resolution, tiled at scene scale	learned weights, tuned acceptance	sparse/semi-dense matches ( $\to$ fitted model)
SCDF (ours)	none — the dense per-pixel field is the product	self-calibrated per pair	dense field $+$ confidence

TABLE I: Tool Families for Optical Co-Registration and Their Embedded Motion Models

Panel of Figure 3 — Fig. 2: How SCDF estimates the field: one seeded point 1 ; a predict–measure–filter loop 2 – 4 , repeated coarse to fine with the patch halving at each scale and only the field carrying over; a correlation pass 5 that re-measures every point and densifies the set; interpolation 6 to a dense displacement field; and the warp 7 that resamples MOV through it. Figure 3 details the measurement inside 5 .

Stage — the decision it makes	Where its threshold comes from
Global magnitude filter — is this displacement grossly out of family?	$\mathrm{med}+3\,\hat{\sigma}$ of the displacement magnitudes at this level , $\hat{\sigma}=1.4826\times$ MAD
LOO local-affine filter — is this point inconsistent with its own 12 neighbors?	$\mathrm{med}+3.5\,\hat{\sigma}$ of this level’s own studentized residual distribution
Response gate — did this window pair lock, or is the peak spurious?	the maximum of $K{=}8$ null correlations of the same REF window against far-away MOV windows, recomputed for every window
Error gate — do the aligned windows agree well enough for this pair?	a quantile of the pair’s own error distribution ( $q{=}0.5$ re-measuring, $0.4$ densifying) 1
Fill interpolator — which of 7 interpolators suits this scene?	5-fold cross-validation over the pair’s own control points
Fixed by design, and not thresholds on measured quantities:

TABLE II: Self-Calibration Audit: Every Decision Threshold and Its Source

Panel of Figure 5 — Fig. 3: The correlation pass. Left: one measurement — the accumulated field predicts where each MOV window sits in REF, the correlation surface $\Gamma(\mathbf{s})$ scores every trial shift at once, and its peak $\mathbf{s}^{\star}$ is the sub-pixel residual, added to and bounded by the prior. Right: the same primitive used twice — re-measure replaces the value at every matched point, densify adds surviving points on a fresh lattice; what fails the acceptance gates (Supplement S-I) keeps its old value or is never added.

							MOV px (m)
Leg	REF	MOV (distorted)	GSD	$\Delta t$	Sites	Cells	A1/2/4	A3
E1/b1	S2 2024-07, 10 m	same scene	1:1	—	25	100	10	30
E2/b1	S2 2024-07	S2 2024-01 (cross-season)	1:1	${\sim}0.5$ yr	25	100	10	30
E2/b2	S2 2024-07	S2 2023-07	1:1	${\sim}1$ yr	25	100	10	30
E2/b3	S2 2024-07	S2 2019-07	1:1	${\sim}5$ yr	25	100	10	30
E2/b4	S2 10 m	Landsat-8/9 30 m	1:3	$\leq 50$ d	25	100	30	90

TABLE III: Benchmark Composition: The Six Legs and Their 584 Cells

Fig. 4: The 60 benchmark sites, marked by the image pairs acquired over them. A site shared between the Sentinel-2 and Landsat legs shows as a dot inside a ring; NAIP is a US program, so its sites are CONUS-only.

	Block median / p90 EPE (m)				Overall	Cells
Method	A1 radiom.	A2 homog.	A3 resol.	A4 seam	median / p90 (m)	scored
SCDF (ours)	0.80 / 1.85	0.50 / 1.03	0.96 / 1.47	0.57 / 1.02	0.63 / 1.27	100/100
GeFolki	2.21 / 3.97	0.56 / 1424	1.17 / 2.04	0.26 / 0.64	1.02 / 3.73	100/100
AROSICS	0.39 / 0.72	874.5 / 1561	0.28 / 0.48	528.6 / 741.3	1.29 / 4.07	99/100
SIFT+RANSAC	0.67 / 1.59	0.06 / 0.50	7.11 / 7.51	446.2 / 907.1	2.53 / 7.03	100/100
autoRIFT	3.06 / 7.15	579.3 / 1276	1.83 / 2.65	FAIL	4.02 / 28.0	78/100

TABLE IV: E1, the Structural Test: Block Median / p90 EPE (m)

	by leg (median / p90, m)					by construction (median / p90, m)
Method	b1	b2	b3	b4	b5	A1	A2	A3	A4
	0.5 yr	1 yr	5 yr	$\times$ LS	$\times$ NAIP	radiom.	homog.	resol.	seam
SCDF (ours)	3.26 / 6.58	2.44 / 4.54	5.10 / 7.60	11.5 / 19.1	4.67 / 7.77	4.37 / 7.85	4.07 / 7.62	3.89 / 6.62	4.36 / 9.25
GeFolki	6.57 / 17.1	4.65 / 11.0	7.38 / 16.2	13.7 / 55.9	5.43 / 9.34	5.67 / 11.1	13.0 / 1366	7.07 / 15.5	4.73 / 9.72
AROSICS	19.1 / 69.6	15.6 / 44.5	15.8 / 69.4	80.3 / 378	8.13 / 15.6	4.01 / 4.92	823.5 / 1511	4.48 / 6.24	482.1 / 534.3
SIFT+RANSAC	9.23 / 19.0	6.87 / 10.1	11.5 / 16.8	39.2 / 74.8	2032 / 3292	8.53 / 17.1	6.69 / 9.39	11.2 / 19.6	458.4 / 918.5

TABLE V: E2, the Realism Test: Block Median / p90 EPE (m) by Leg and by Construction

Fig. 5: Per-pixel EPE ECDFs of four methods on one cell (E1/b1

\cdot

\cdot

africa_south_1 ). The axis is in MOV pixels because the figure stays inside a single cell; 1 px

=

10 m on this leg.

Fig. 6: All 24 blocks, each method colored by its ratio to the best result on that block ( $R_{m}(B)$ , Equation ( 6 )); absolute meters are in Tables IV and V . Hatched: no field produced (a different outcome from a large error).

	Cells	Overall (m)		Placement across blocks
Method	scored	median	p90	best in	top-2 in	worst ratio	$\geq 10\times$ in
E1 — 100 cells, 4 blocks, exact ground truth
SCDF (ours)	100/100	0.63	1.27	0	2	8.0 $\times$	0
GeFolki	100/100	1.02	3.73	1	1	8.9 $\times$	0
AROSICS	99/100	1.29	4.07	2	2	13993 $\times$	2
SIFT+RANSAC	100/100	2.53	7.03	1	1	1697 $\times$	2

TABLE VI: Placement Across Blocks: The Bounded Worst Case

Fig. 7: ECDF over the 484 E2 cells of each method’s per-cell median EPE divided by the best obtained on that same cell (

r_{m}(c)

, Section IV-F ). Left is better; dotted line: 2

\times

; declined cells enter at the right edge, producing the plateaus below 1.0. E1 is omitted: seven of its cells have a best of exactly zero.

	Block score (m)		Ratio
Configuration	median	p90	median	p90
Full stack	2.832	4.809	1.000	1.000
$-$ sub-pixel re-measurement	2.828	5.093	0.974	1.003
$-$ lattice densification	2.899	5.484	1.014	1.093
$-$ cross-validated fill	2.838	4.830	1.035	1.079
$-$ all three	3.713	7.615	1.171	1.410

TABLE VII: Factorial Ablation of the Measurement Stack on a Stratified 60-Cell Subset

Method	$n/40$	Median (s)	Min–max (s)
SIFT+RANSAC	40	29	12–45
autoRIFT	30	73	12–98
AROSICS	40	150	57–1911
SRIF †	40	316	139–538
RIFT	40	330	48–394
SCDF (ours)	40	963	96–1540

TABLE VIII: Single-Core Runtime on E2/b3

Deformable Image Registration

Deformable image registration aligns images by estimating local spatial transformations that account for anatomical change, motion, and differences between scans.

See all 74 papers

Apr 11, 2026

Are Pretrained Image Matchers Good Enough for SAR-Optical Satellite Registration?

Cross-modal optical--SAR (Synthetic Aperture Radar) registration is a bottleneck in remote-sensing disaster response. Modern image matchers are developed and benchmarked almost exclusively on natural-image domains. We evaluate twenty-four pretrained matcher configurations in a zero-shot setting, with no fine-tuning or domain adaptation on satellite or SAR data. The evaluation spans SpaceNet9 and two additional cross-modal benchmarks under a deterministic protocol that uses tiled large-image inference, robust geometric filtering, and tie-point-grounded metrics. Our results show uneven transfer: matchers with explicit cross-modal training do not uniformly outperform those without it. XoFTR (trained for visible--thermal matching) and RoMa achieve the lowest reported mean tie-point error at

3.0

px on the labeled SpaceNet9 training scenes. RoMa achieves this result without any cross-modal training. MatchAnything-ELoFTR (

3.4

px), trained on synthetic cross-modal pairs, is close behind. XoFTR also runs roughly an order of magnitude faster than the RoMa family (

0.4

s vs.

5.0

s per pair). RoMa's performance is consistent with the hypothesis that frozen DINOv2 features confer robustness to large appearance shifts. Deployment protocol choices (geometry model, tile size, inlier gating) change mean error by up to

33\times

for a single matcher. This shift can exceed the effect of swapping matchers within our evaluated protocol ablations. Affine geometry alone reduces mean error from

12.3

9.7

px.

Isaac Corley, Alex Stoken, Gabriele Berton

Taylor Geospatial · Independent Researcher

Cross-Modal Image Registration Cross-Modal Feature Matching Cross-View Image Matching Remote Sensing Remote Sensing Benchmarks

Ask PDF

Aug 4, 2026

LoRetta: A Foundation Model and Extensive Dataset for Global-Scale Remote Sensing Dense Image Matching

Dense image matching establishes pixel-wise correspondences and underpins broad applications in computer vision and photogrammetry. However, extending dense matching to global-scale remote sensing remains challenging because image pairs may differ in acquisition time, season, viewpoint, spatial resolution, and land-cover state. The resulting large geometric offsets, partial overlap, and intrinsically unmatchable regions make direct dense correspondence prediction unreliable and inefficient. We thus reformulate dense matching as localization-and-registration: first localizing the matchable overlap and affine geometry, then refining dense residuals within the aligned frame. Based on this formulation, we propose LoRetta, a foundation model coupling matchability-aware affine localization with guided dense registration. We also introduce LEVIR-GM, a global-scale multi-temporal optical matching benchmark with dataset-native matchability labels (103K aligned, 827K augmented pairs, six continents, five years, 0.5-1024 m resolution). We further establish a unified evaluation protocol for sparse, semi-dense, and dense matchers. On LEVIR-GM, LoRetta achieves an area under the curve (AUC) of 83.3%, outperforming the strongest baseline RoMa v2 by 1.6 points, with larger percentage of correct keypoints (PCK) gains of 6.5 and 8.2 points at 1 and 2 pixels, while reducing inference latency by 47.8%. Astronaut-to-satellite and unmanned aerial vehicle (UAV)-to-satellite geolocalization experiments further demonstrate its transferability as a reusable geometric aligner.

Siwei Yu, Han Guo, Zhenwei Shi +1

Department of Aerospace Intelligent Science and Technology, School of Astronautics, Beihang University, Beijing 100191, China, and also with the Key Laboratory of Spacecraft Design Optimization and Dynamic Simulation Technologies, Ministry of Education

Image Matching Remote Sensing Benchmarks

Ask PDF

Aug 26, 2026

Spaceborne differential photogrammetry for control-free measurement of large-gradient deformation with structural immunity and a predictable accuracy envelope

Optical satellite image correlation measures wide-area deformation in regimes where coherent interferometric synthetic aperture radar fails because displacement gradients are too large. However, standard pairwise workflows lack a pre-acquisition error budget and rely on extensive stable terrain. We formulate repeat-pass optical correlation as a differential estimation problem without surveyed ground control. Nominal georeferencing defines the coordinate frame, stable-area constraints and displacement priors resolve the datum, and surface displacement is estimated jointly with inter-epoch revisit-bias coefficients. The model yields a predictive accuracy envelope and calibrated per-point posterior uncertainty, bounds along-track uncertainty through a displacement prior, and represents pushbroom jitter using per-line revisit offsets. Simulations and Sentinel-2 and WorldView-2 experiments on the 2019 Ridgecrest earthquake, the 2023 Kahramanmaraş earthquake, and the Baltoro glacier validate the predicted noise floor, control-free accuracy margin, and leakage caused by view-angle and digital elevation model errors. The measured noise floor reaches approximately

0.05

pixel at

10

,m ground sampling distance. With only five stable tiles, conventional destriping changes the estimated Baltoro trunk velocity from

106

1251

myr

^{-1}

, whereas the prior-constrained estimate remains

87

myr

^{-1}

. Closure analysis attributes approximately

88\%

of pair-error variance to individual scenes, consistent with

25{,}354

ITS_LIVE glacier-velocity triplets. Three matching methods lead to the same conclusions. The framework therefore turns pairwise correlation into a robust measurement with a predictive error budget, reduced dependence on stable terrain, and conclusions independent of the matching method.

Yueqiang Zhang, Chang Ma, Shuixin Pan +1

State Key Laboratory of Radio Frequency Heterogeneous Integration; Key Laboratory of Optoelectronic Devices and Systems of Ministry of Education and Guangdong Province; Shenzhen Key Laboratory of Intelligent Optical Measurement and Detection; College of Physics and Optoelectronic Engineering, Shenzhen University, Shenzhen, 518060, China · School of Artificial Intelligence and Robotics, Hunan University, Changsha, 410082, China

Remote Sensing Uncertainty Quantification

Ask PDF

Abstract

Deformable Image Registration

Explore similar work

Are Pretrained Image Matchers Good Enough for SAR-Optical Satellite Registration?

LoRetta: A Foundation Model and Extensive Dataset for Global-Scale Remote Sensing Dense Image Matching

Spaceborne differential photogrammetry for control-free measurement of large-gradient deformation with structural immunity and a predictable accuracy envelope