ELViS: Efficient Visual Similarity from Local Descriptors that Generalizes Across Domains

Pavel Suma; Giorgos Kordopatis-Zilos; Yannis Kalantidis; Giorgos Tolias

cs.CVMar 30, 2026

ELViS: Efficient Visual Similarity from Local Descriptors that Generalizes Across Domains

Authors: Pavel Suma, Giorgos Kordopatis-Zilos, Yannis Kalantidis, Giorgos Tolias

Organizations: VRG, FEE, Czech Technical University in Prague · NAVER LABS Europe

Abstract

Large-scale instance-level training data is scarce, so models are typically trained on domain-specific datasets. Yet in real-world retrieval, they must handle diverse domains, making generalization to unseen data critical. We introduce ELViS, an image-to-image similarity model that generalizes effectively to unseen domains. Unlike conventional approaches, our model operates in similarity space rather than representation space, promoting cross-domain transfer. It leverages local descriptor correspondences, refines their similarities through an optimal transport step with data-dependent gains that suppress uninformative descriptors, and aggregates strong correspondences via a voting process into an image-level similarity. This design injects strong inductive biases, yielding a simple, efficient, and interpretable model. To assess generalization, we compile a benchmark of eight datasets spanning landmarks, artworks, products, and multi-domain collections, and evaluate ELViS as a re-ranking method. Our experiments show that ELViS outperforms competing methods by a large margin in out-of-domain scenarios and on average, while requiring only a fraction of their computational cost. Code available at: https://github.com/pavelsuma/ELViS/

Figures & tables

Figure 1: Performance vs. time. Average performance across 8 datasets and multiple domains for fixed numbers of re-ranked images indicated with text labels. All models are trained on the landmarks domain (GLDv2). Runtime is estimated from model latencies reported in Table 5 .

Figure 2: Detailed overview of ELViS. The similarity matrix is refined using optimal transport with descriptor-dependent dustbin gains. The strongest local similarities per descriptor are then selected and transformed element-wise by a learned function $f$ , before being sum-aggregated into a scalar global similarity. During training, a modified BCE loss with a learnable function $g$ reshapes the penalty curve; $g$ is used only for training and is expandable at inference.

Figure 3: Shape of the learned univariate functions

f

(left) and

g

(right). Although parameterized as MLPs, both functions learn well-behaved scalar transformations that effectively separate matching and non-matching distributions. The distributions of input values are visualized separately for positive and negative image pairs, sampled during training.

Panel of Figure 4 — Figure 4: Visualization of the 25 strongest correspondences (votes) among $s_{i},s_{j}$ (left) and $s_{i}^{\prime},s_{j}^{\prime}$ (right) before (left) and after (right) refinement with optimal transport. Red (yellow) represents high (low) similarity. Raw similarity values in $\mathbf{S}$ (left) and values in $\mathbf{S}^{\prime}$ after passing them through $f$ (right) are used. Heatmaps represent the dustbins values by evaluating $h$ densely for all patches in both images; bright values indicate large dustbin gain and uninformative descriptors.

landmarks domain (GLDv2)
Method	$\mathcal{R}$ OP+1M	GLDv2	ILIAS	INSTRE	MET	Prod1M	RP2K	SOP-1k	ID	OOD	avg
No re-ranking	57.7	27.3	9.4	65.3	61.6	24.7	39.0	33.7	42.5	38.9	39.8
Chamfer †	56.7	23.8	6.2	55.0	37.3	17.8	55.8	48.6	40.2	36.8	37.6
Chamfer+OT †	60.6	23.8	14.3	76.0	74.0	35.1	55.4	46.2	42.2	50.2	48.2
RRT ⋆	69.2	33.1	13.1	72.4	64.1	29.3	60.7	52.1	51.1	48.6	49.2
$R^{2}$ Former †	68.5	32.6	15.2	77.6	72.0	35.6	47.7	43.7	50.6	48.6	49.1

Table 1: Domain generalization performance (mAP) . Training performed either on landmarks (GLDv2) or household items (SOP). Results reported per dataset and as average over in-domain (ID), out-of-domain (OOD), and all datasets (avg). Local descriptors extracted with DINOv2 ( Oquab et al., 2024 ) . Gray indicates in-domain results. Green ( Red ) highlights gain (loss) of ELViS over the second best method.

\dagger

\star

indicate similarity-based and descriptor-based models, respectively.

Method ID OOD avg No re-ranking 49.6 60.5 57.8 Chamfer 42.3 44.5 44.0 Chamfer+OT 41.7 51.4 49.0 RRT 57.1 56.2 56.4 $R^{2}$ Former 56.0 63.9 61.9 AMES 58.6 62.8 61.8 ELViS 56.9 (-1.7) 67.4 (+3.5) 64.8 (+2.9)	Method ID OOD avg No re-ranking 25.4 57.8 49.7 Chamfer 32.8 58.7 52.2 Chamfer+OT 29.3 62.3 54.1 RRT 37.5 58.4 53.2 $R^{2}$ Former 35.2 63.0 56.0 AMES 37.1 62.7 56.3 ELViS 36.4 (-1.1) 68.7 (+5.7) 60.6 (+4.3)
(a) DINOv3 ( Siméoni et al., 2025 )	(b) SigLIP2 ( Tschannen et al., 2025 )

Table 2: Performance (mAP) comparison using local descriptors from additional foundational models . Training performed on landmarks (GLDv2). Results reported per dataset and as averages over in-domain (ID), out-of-domain (OOD), and all datasets (avg). Green ( Red ) highlights gain (loss) of ELViS over the second best method.

Method	ID		OOD		avg
No re-ranking	42.5		38.9		39.8
ELViS	50.5		55.0		53.9
w/o dustbin	23.1	(-27.4)	32.8	(-22.2)	30.4	(-23.5)
w/o descriptor-dependent gain	48.8	(-1.7)	52.4	(-2.6)	51.5	(-2.4)
w/o $f$ function	50.8	(+0.3)	53.8	(-1.2)	53.1	(-0.8)
w/o $g$ function	45.6	(-4.9)	49.5	(-5.5)	48.5	(-5.4)

Table 3: Ablation study on method components . Average, ID, and OOD performance, when training on landmarks. Green ( Red ) highlights gain (loss) over ELViS.

Table 8

Figure 5: Effectiveness with small training sets. Performance of ELViS and AMES when trained on subsets of the full training set (754k images), ranging from 0.2% to 100%. No re-ranking performance is indicated at 0 training set size. Training hyperparameters are tuned for each subset size based on validation performance.

dataset	train set	validation set		test set		domain	evaluation metric
dataset	train set	queries	database	queries	database	domain	evaluation metric
GLDv2	754K	379	762K	750	762K	landmark	mAP@100
SOP	48.9K	1K	10.6K	1K	60.5K	household	mAP@100
$\mathcal{R}$ Oxford	–	–	–	70	5K+1M	landmark	mAP
$\mathcal{R}$ Paris	–	–	–	70	6.3K+1M	landmark	mAP
Product1M	–	–	–	6.2K	38.7K	retail	mAP@100

Table F: Dataset statistics and metrics used . For each dataset, we use the most commonly used metric. Train and validation set statistics are reported only for the two training datasets that are used in this work.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 11 — Figure F: Benchmark examples . Three random samples are shown for each of the 9 datasets. All images are resized to a square aspect ratio for better visualization.

Method	Alps			California			Gobi Desert			Amazon			Toshka Lakes
Method	R@1	R@10	R@100	R@1	R@10	R@100	R@1	R@10	R@100	R@1	R@10	R@100	R@1	R@10	R@100
AnyLoc	40.7	70.8	92.0	48.7	75.0	91.6	28.7	57.0	81.7	38.6	63.8	86.2	63.7	84.5	96.3
AstroLoc	98.1	99.5	99.8	97.4	99.2	99.8	94.6	99.2	99.9	93.0	96.9	99.1	99.0	99.6	99.9
No re-ranking	35.7	65.6	91.1	50.1	78.2	93.9	26.2	53.3	84.0	31.4	61.1	83.9	52.4	78.1	94.9
AMES	44.0	74.5	93.7	57.4	83.0	95.7	36.0	64.7	87.0	40.4	69.2	87.8	61.5	85.4	96.8
ELViS	55.1	80.0	94.6	62.0	82.7	94.7	49.7	71.2	86.9	54.4	76.2	89.0	77.0	90.9	97.6

Appendix

Table G: Generalization to extreme domain shifts. Evaluation on retrieval for Astronaut Photography Localization (APL) ( Berton et al., 2024 ) . Performance is measured in Recall@

k

(R@

k

). APL covers different geographical areas of extreme visual environments. Top: reference methods AnyLoc ( Keetha et al., 2023 ) , a universal Visual Place Recognition (VPR) approach and the specialist AstroLoc ( Berton et al., 2025 ) trained for the task. Bottom: Re-ranking 400 images on top of a DINOv3 backbone. Bold indicates the best performance.

Method	$\mathcal{R}$ OP+1M	GLDv2	ILIAS	INSTRE	MET	Prod1M	RP2K	SOP-1k	ID	OOD	avg
No re-ranking	57.7	27.3	9.4	65.3	61.6	24.7	39.0	33.7	42.5	38.9	39.8
Spatial verification	58.5	27.6	12.3	70.4	68.7	28.7	49.7	37.3	43.3	43.7	43.6
SuperGlue	61.3	28.3	15.1	73.7	72.9	38.8	56.2	41.5	44.8	49.7	48.5
OmniGlue	59.1	27.8	13.6	73.0	70.8	19.4	44.4	41.1	43.4	43.7	43.7
ELViS	68.8	32.2	18.8	80.4	77.9	41.5	59.2	52.3	50.5	55.0	53.9

Appendix

Table H: Comparison with image matching methods . Landmark datasets are treated as in-domain (ID) for all methods, while the remaining datasets are evaluated as out-of-domain (OOD).

$\mathbf{\lambda}$	ID	OOD	avg
0.01	46.4	47.7	47.4
0.1	50.5	55.0	53.9
0.5	50.7	52.1	51.7
1.0	48.9	48.2	48.4
learnable	51.2	54.9	54.0

Appendix

Table I: Ablation study of the OT hyperparameters . Models are trained on the Landmarks (GLDv2) domain. We report average performance over the in-domain (ID), out-of-domain (OOD), and all benchmarks (avg). We assess the impact of the regularization parameter

\lambda

and the number of OT iterations, varying one hyperparameter at a time while keeping all others at their default values.

Figure G: Average precision per query. Each marker corresponds to the AP of a single query evaluated on different test datasets using DINOv2 as a representation model. x-axis: performance using ELViS trained on SOP dataset. y-axis: performance using ELViS trained on GLDv2 dataset.

Figure H: Average precision per query. Each marker corresponds to the AP of a single query evaluated on INSTRE using DINOv2 as a representation model. y-axis: performance using ELViS trained on SOP dataset. x-axis: performance of the global descriptor without any re-ranking.

Method	$\mathcal{R}$ Oxford	$\mathcal{R}$ Paris	GLDv2	ILIAS	INSTRE	MET	Prod1M	RP2K	SOP-1k	ID	OOD	avg
DINOv2 descriptors ( Oquab et al., 2024 )
No re-ranking	47.3	67.9	27.3	9.4	65.3	61.6	24.7	39.0	33.7	42.5	38.9	39.8
Chamfer	43.0	54.8	23.8	6.2	55.0	37.3	17.8	55.8	48.6	40.2	36.8	37.6
Chamfer+OT	55.6	65.5	23.8	14.3	76.0	74.0	35.1	55.4	46.2	42.2	50.2	48.2
RRT	64.1 $\pm 0.6$	74.4 $\pm 0.0$	33.1 $\pm 0.2$	13.1 $\pm 0.6$	72.4 $\pm 1.1$	64.1 $\pm 0.7$	29.3 $\pm 1.7$	60.7 $\pm 0.9$	52.1 $\pm 0.6$	51.1 $\pm 0.3$	48.6 $\pm 0.9$	49.2 $\pm 0.8$
$R^{2}$ Former	63.7 $\pm 0.2$	73.4 $\pm 0.1$	32.6 $\pm 0.0$	15.2 $\pm 0.1$	77.6 $\pm 0.1$	72.0 $\pm 0.3$	35.6 $\pm 0.3$	47.7 $\pm 0.6$	43.7 $\pm 0.1$	50.6 $\pm 0.1$	48.6 $\pm 0.2$	49.1 $\pm 0.2$

Appendix

Table J: mAP mean and standard deviation for training on landmarks (GLDv2). Three different backbones used as a representation model.

Method	$\mathcal{R}$ Oxford	$\mathcal{R}$ Paris	GLDv2	ILIAS	INSTRE	MET	Prod1M	RP2K	SOP-1k	ID	OOD	avg
No re-ranking	47.3	67.9	27.3	9.4	65.3	61.6	24.7	39.0	33.7	33.7	40.7	39.8
Chamfer	29.7	62.6	15.4	6.7	63.2	45.2	24.7	50.3	50.2	50.2	35.9	37.7
Chamfer+OT	43.7	60.8	18.6	11.7	75.9	71.6	37.5	52.7	45.8	45.8	45.7	45.7
RRT	28.2 $\pm 1.7$	58.7 $\pm 1.3$	10.8 $\pm 1.1$	12.2 $\pm 1.5$	68.4 $\pm 1.8$	25.5 $\pm 2.8$	32.2 $\pm 1.8$	46.9 $\pm 3.9$	57.1 $\pm 0.3$	57.1 $\pm 0.3$	34.2 $\pm 2.0$	37.1 $\pm 1.5$
$R^{2}$ Former	43.4 $\pm 1.0$	67.5 $\pm 0.3$	23.7 $\pm 0.5$	12.9 $\pm 0.3$	73.4 $\pm 0.4$	59.3 $\pm 0.7$	32.8 $\pm 1.9$	42.0 $\pm 0.7$	51.1 $\pm 0.3$	51.1 $\pm 0.3$	42.8 $\pm 0.7$	43.8 $\pm 0.7$
AMES	46.2 $\pm 1.0$	65.1 $\pm 1.5$	17.2 $\pm 1.1$	12.4 $\pm 0.5$	72.2 $\pm 1.0$	44.2 $\pm 1.8$	36.7 $\pm 1.9$	51.8 $\pm 2.5$	56.7 $\pm 0.6$	56.7 $\pm 0.6$	41.4 $\pm 1.5$	43.3 $\pm 1.2$

Appendix

Table K: mAP mean and standard deviation for training on household items (SOP). DINOv2 used as a representation model.

Panel of Figure 19 — Figure I: The impact of re-ranking with ELViS. We show a single query and its database positives per row for each test set, with the exception of the MET dataset where three queries are shown. The text above each query denotes the change in Average Precision. The text above the positive images denotes the number of negative images ranked before the positive using only global similarity (1) and after re-ranking with ELViS (2), denoted as (1)→(2). The positives are ordered based on the difference (1)-(2) in descending order.

Panel of Figure 20 — Figure J: Visualization of strong correspondences before (left) and after (right) refinement with optimal transport.

Panel of Figure 21 — Figure K: Visualization of similarity matrices before (left) and after (right) refinement with optimal transport with individual values passed through function $f$ . A subset of 100 descriptors is used. These examples correspond to the ones of Figure J .

Image Matching

Image matching finds corresponding points, regions, or objects across images despite changes in viewpoint, scale, lighting, or sensor.

See all 87 papers

Human Knowledge Integrated Multi-modal Learning for Single Source Domain Generalization
Mar 12, 2026Ayan Banerjee, Kuntal Thakur, Sandeep GuptaDomain Generalization Cross-Domain Generalization
Impact Lab, Arizona State University
BiCLIP: Domain Canonicalization via Structured Geometric Transformation
Mar 9, 2026Pranav Mantini, Shishir K. ShahVLM Adaptation Domain Adaptation
University of Houston, Department of Computer Science · The University of Oklahoma, School of Computer Science
Indexing Multimodal Language Models for Large-scale Image Retrieval
Apr 14, 2026Bahey Tharwat, Giorgos Kordopatis-Zilos, Pavel Suma +2Image Retrieval Image Matching
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · VRG, FEE, Czech Technical University in Prague, Prague, Czech Republic

Mar 12, 2026J/K move · Enter open · S save

Human Knowledge Integrated Multi-modal Learning for Single Source Domain Generalization

Ayan Banerjee, Kuntal Thakur, Sandeep Gupta

Impact Lab, Arizona State University

Domain Generalization Cross-Domain Generalization Multimodal Domain Generalization Medical VLMs VLM Adaptation

Open paper AskPDF arXiv

Generalizing image classification across domains remains challenging in critical tasks such as fundus image-based diabetic retinopathy (DR) grading and resting-state fMRI seizure onset zone (SOZ) detection. When domains differ in unknown causal factors, achieving cross-domain generalization is difficult, and there is no established methodology to objectively assess such differences without direct metadata or protocol-level information from data collectors, which is typically inaccessible. We first introduce domain conformal bounds (DCB), a theoretical framework to evaluate whether domains diverge in unknown causal factors. Building on this, we propose GenEval, a multimodal Vision Language Models (VLM) approach that combines foundational models (e.g., MedGemma-4B) with human knowledge via Low-Rank Adaptation (LoRA) to bridge causal gaps and enhance single-source domain generalization (SDG). Across eight DR and two SOZ datasets, GenEval achieves superior SDG performance, with average accuracy of 69.2% (DR) and 81% (SOZ), outperforming the strongest baselines by 9.4% and 1.8%, respectively.

Abstract

Image Matching

Explore similar work

Human Knowledge Integrated Multi-modal Learning for Single Source Domain Generalization

BiCLIP: Domain Canonicalization via Structured Geometric Transformation

Indexing Multimodal Language Models for Large-scale Image Retrieval