Evaluating the Impact of Data Anonymization on Image Retrieval

Marvin Chen; Manuel Eberhardinger; Johannes Maucher

cs.LGFeb 23, 2026

Evaluating the Impact of Data Anonymization on Image Retrieval

Authors: Marvin Chen, Manuel Eberhardinger, Johannes Maucher

Organizations: Institute of Applied AI, Stuttgart Media University, Nobelstr. 10, 70569 Stuttgart, Germany

Abstract

With the growing importance of privacy regulations such as the General Data Protection Regulation, anonymizing visual data is becoming increasingly relevant across institutions. However, anonymization can negatively affect the performance of Computer Vision systems that rely on visual features, such as Content-Based Image Retrieval (CBIR). Despite this, the impact of anonymization on CBIR has not been systematically studied. This work addresses this gap, motivated by the DOKIQ project, an artificial intelligence-based system for document verification actively used by the State Criminal Police Office Baden-Württemberg. We propose a simple evaluation framework: retrieval results after anonymization should match those obtained before anonymization as closely as possible. To this end, we systematically assess the impact of anonymization using two public datasets and the internal DOKIQ dataset. Our experiments span three anonymization methods, four anonymization degrees, and four training strategies, all based on the state of the art backbone Self-Distillation with No Labels (DINO)v2. Our results reveal a pronounced retrieval bias in favor of models trained on original data, which produce the most similar retrievals after anonymization. The findings of this paper offer practical insights for developing privacy-compliant CBIR systems while preserving performance.

Figures & tables

Source	Anonymization methods	Anonymized entities	Retraining on anonymized data	Validation dataset	Vision tasks
[ 7 ] , 2022	Synthetic, traditional	Humans, license plates	Yes	Original	Semantic segmentation
[ 8 ] , 2023	Synthetic, traditional	Humans	Yes	Original	Instance/ semantic segmentation, pose estimation
[ 9 ] , 2024	Traditional	Humans, vehicles	Yes	Original	Instance/ semantic segmentation, pose estimation, object detection
[ 10 ] , 2024	Synthetic, traditional	Humans	No	Anonymized	Instance/ semantic segmentation, keypoint detection, face detection
Ours, 2025	Traditional	Humans, documents	Yes	Anonymized	Image retrieval/ classification

TABLE I: Comparison of previous work and this paper systematically evaluating the impact of data anonymization on computer vision performance. Adopted from Stenger et al. [ 10 ] and extended by two additional columns.

Model	Anonymization method $a\in\mathcal{A}$
With original query images
Unadapted	$M_{\text{CelebA-GT}}(\mathcal{V})\rightarrow M_{\text{CelebA-GT}}(\hat{\mathcal{V}}_{a})$
Adaption A	$M_{a,A}(\mathcal{V})\rightarrow M_{a,A}(\hat{\mathcal{V}}_{a})$
Adaption B	$M_{a,B}(\mathcal{V})\rightarrow M_{a,B}(\hat{\mathcal{V}}_{a})$
Adaption C	$M_{a,C}(\mathcal{V})\rightarrow M_{a,C}(\hat{\mathcal{V}}_{a})$
With anonymized query images

TABLE II: Retrieval scenarios for CelebA.

Model	Anonymization method $a\in\mathcal{A}$ and anonymization degree $p\in\mathcal{P}$
With original query images
Unadapted	$M_{\text{RVL-CDIP-GT}}(\mathcal{V})\rightarrow M_{\text{RVL-CDIP-GT}}(\hat{\mathcal{V}}_{a,p})$
Adaption A	$M_{a,p,A}(\mathcal{V})\rightarrow M_{a,p,A}(\hat{\mathcal{V}}_{a,p})$
Adaption B	$M_{a,p,B}(\mathcal{V})\rightarrow M_{a,p,B}(\hat{\mathcal{V}}_{a,p})$
Adaption C	$M_{a,p,C}(\mathcal{V})\rightarrow M_{a,p,C}(\hat{\mathcal{V}}_{a,p})$
With anonymized query images

TABLE III: Retrieval scenarios for RVL-CDIP.

Model	Anonymization method $a\in\mathcal{A}$
With original query images
Unadapted	$M_{\text{DOKIQ-GT}}(\mathcal{V})\rightarrow M_{\text{DOKIQ-GT}}(\hat{\mathcal{V}}_{a,\text{max}})$
Adaption A	$M_{a,\text{max},A}(\mathcal{V})\rightarrow M_{a,\text{max},A}(\hat{\mathcal{V}}_{a,\text{max}})$
Adaption B	$M_{a,\text{max},B}(\mathcal{V})\rightarrow M_{a,\text{max},B}(\hat{\mathcal{V}}_{a,\text{max}})$
Adaption C	$M_{a,\text{max},C}(\mathcal{V})\rightarrow M_{a,\text{max},C}(\hat{\mathcal{V}}_{a,\text{max}})$
With anonymized query images

TABLE IV: Retrieval scenarios for DOKIQ.

	mAP			mnDCG 989
Model	Pixel	Blur	Mask	Pixel	Blur	Mask
With original query images
Unadapted	73.2	67.3	31.0	94.2	92.6	77.1
Adaption A	57.0	50.6	13.7	89.3	86.8	64.2
Adaption B	38.5	50.8	13.6	81.6	86.9	64.1
Adaption C	45.4	23.8	10.2	84.7	71.3	58.6

TABLE V: CelebA retrieval qualities.

	k-NN			Linear
Model	Pixel	Blur	Mask	Pixel	Blur	Mask
Unadapted	87.9	86.3	84.2	90.6	89.9	85.5
Adaption A	85.9	86.3	71.1	89.4	89.7	77.3
Adaption B	80.4	85.6	71.6	84.4	88.7	77.9
Adaption C	85.1	89.8	78.9	89.0	93.4	82.8

TABLE VI: CelebA gender detection accuracies.

	mAP			mnDCG 1998
Model	Pixel	Blur	Mask	Pixel	Blur	Mask
With original query images
Unadapted	71.5	85.4	55.4	93.2	97.2	87.5
Adaption A	57.2	60.7	23.2	88.8	90.1	68.6
Adaption B	53.3	59.0	38.0	87.3	89.5	79.3
Adaption C	53.9	59.6	28.3	87.5	89.7	72.9

TABLE VII: RVL-CDIP retrieval qualities @ 25% anonymization.

	k-NN			Linear
Model	Pixel	Blur	Mask	Pixel	Blur	Mask
Unadapted	64.7	67.8	62.7	60.2	62.4	59.9
Adaption A	67.1	68.0	47.0	62.0	62.8	50.5
Adaption B	63.9	67.6	62.7	59.8	62.7	60.0
Adaption C	65.5	67.6	57.7	60.9	62.0	57.9

TABLE VIII: RVL-CDIP document classification accuracies @ 25% anonymization.

	mAP			mnDCG 1998
Model	Pixel	Blur	Mask	Pixel	Blur	Mask
With original query images
Unadapted	34.6	50.5	28.0	75.4	85.7	71.7
Adaption A	16.3	39.9	6.7	62.4	80.2	51.4
Adaption B	24.2	17.8	18.0	67.4	62.3	61.2
Adaption C	22.5	16.2	14.8	66.2	58.7	57.4

TABLE IX: RVL-CDIP retrieval qualities @ 100% anonymization.

	k-NN			Linear
Model	Pixel	Blur	Mask	Pixel	Blur	Mask
Unadapted	55.2	67.8	58.3	54.1	62.6	56.0
Adaption A	40.1	66.0	34.9	45.1	61.8	40.8
Adaption B	52.1	56.8	59.8	52.9	55.7	56.3
Adaption C	54.4	52.5	61.9	55.0	54.0	57.6

TABLE X: RVL-CDIP document classification accuracies @ 100% anonymization.

	mAP			mnDCG 750
Model	Pixel	Blur	Mask	Pixel	Blur	Mask
With original query images
Unadapted	65.7	50.8	43.2	92.9	88.2	85.4
Adaption A	45.6	43.7	36.9	87.9	86.8	83.9
Adaption B	46.1	43.9	37.5	88.0	87.1	84.1
Adaption C	42.8	36.8	33.9	86.4	83.1	81.6

TABLE XI: DOKIQ retrieval qualities.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

	mAP			mnDCG 1998
Model	Pixel	Blur	Mask	Pixel	Blur	Mask
With original query images
Unadapted	51.7	71.6	37.5	85.2	93.6	77.9
Adaption A	40.7	55.4	15.0	80.8	88.1	58.8
Adaption B	36.4	44.4	22.9	78.8	83.2	66.3
Adaption C	37.2	43.0	20.7	79.2	82.5	64.2

Appendix

TABLE XII: RVL-CDIP retrieval qualities @ 50% anonymization.

	k-NN			Linear
Model	Pixel	Blur	Mask	Pixel	Blur	Mask
Unadapted	60.6	66.1	58.1	58.1	61.7	57.0
Adaption A	59.2	66.8	38.6	57.1	62.2	46.3
Adaption B	56.3	62.0	56.5	55.5	58.7	56.6
Adaption C	58.2	60.9	56.0	56.7	58.1	55.8

Appendix

TABLE XIII: RVL-CDIP document classification accuracy @ 50% anonymization.

	mAP			mnDCG 1998
Model	Pixel	Blur	Mask	Pixel	Blur	Mask
With original query images
Unadapted	41.0	59.5	31.4	79.4	89.4	73.8
Adaption A	36.2	49.3	7.0	77.7	85.3	51.5
Adaption B	29.4	32.0	20.1	72.9	75.5	63.9
Adaption C	27.4	29.4	17.6	71.5	73.5	60.8

Appendix

TABLE XIV: RVL-CDIP retrieval qualities @ 75% anonymization.

	k-NN			Linear
Model	Pixel	Blur	Mask	Pixel	Blur	Mask
Unadapted	57.0	65.2	56.8	55.6	61.5	55.7
Adaption A	57.6	66.0	34.6	55.7	62.2	41.7
Adaption B	54.9	58.5	57.0	55.1	56.8	55.9
Adaption C	53.2	57.3	56.5	54.0	56.6	55.5

Appendix

TABLE XV: RVL-CDIP document classification accuracy @ 75% anonymization.

Image Retrieval

Image retrieval finds and ranks images from visual, textual, or combined queries across large and specialized collections.

See all 85 papers

Geometric Analysis of Self-Supervised Vision Representations for Semantic Image Retrieval
Apr 27, 2026Esteban Rodríguez-Betancourt, Edgar Casasola-MurilloNeural Representation Geometry Image Retrieval
Posgrado en Computación e Informática Universidad de Costa Rica · Escuela de Ciencias de la Computación Universidad de Costa Rica
Towards Vision-Free CIR: Attribute-Augmented Scoring and LLM-Based Reranking for Zero-Shot Composed Image Retrieval
Jul 14, 2026Ryotaro Shimada, Yu-Chieh Lin, Yuji Nozawa +3Composed Image Retrieval
The University of Tokyo · Kioxia Corporation
Never Seen Before: Benchmarking Genuine Zero-Shot Composed Image Retrieval with Consistent Video-Sourced Datasets
Jun 5, 2026Zhenyu Yang, Zemin Du, Shengsheng Qian +1Composed Image Retrieval

Apr 27, 2026J/K move · Enter open · S save

Geometric Analysis of Self-Supervised Vision Representations for Semantic Image Retrieval

Esteban Rodríguez-Betancourt, Edgar Casasola-Murillo

Posgrado en Computación e Informática Universidad de Costa Rica · Escuela de Ciencias de la Computación Universidad de Costa Rica

Neural Representation Geometry Image Retrieval Self-Supervised Visual Representation Learning Approximate Nearest Neighbor Search

Open paper AskPDF arXiv

Content-based image retrieval (CBIR) systems enable users to search images based on visual content instead of relying on metadata. The text domain has benefited from vector search of representations created with unsupervised methods such as BERT. However, modern self-supervised learning methods for vision are mostly not reported in CBIR-related literature, instead relying on supervised models or multi-modal methods that align text and vision. We evaluate how the representations learned by modern self-supervised learning methods for vision perform under typical retrieval stacks that leverage vector databases and nearest neighbor search. Our evaluation reveals that the latent space geometry impacts approximate nearest neighbor (ANN) indexing. Specifically, highly anisotropic representations with high skewness produced by several modern SSL methods degrade the performance of partition-based and hashing-based search, even if their own linear probe or K-NN accuracy is not affected. In contrast, representations with higher isotropy and local purity better satisfy the distance-based assumptions of ANN indexes, leading to improved semantic retrieval performance.

Abstract

Image Retrieval

Explore similar work

Geometric Analysis of Self-Supervised Vision Representations for Semantic Image Retrieval

Towards Vision-Free CIR: Attribute-Augmented Scoring and LLM-Based Reranking for Zero-Shot Composed Image Retrieval

Never Seen Before: Benchmarking Genuine Zero-Shot Composed Image Retrieval with Consistent Video-Sourced Datasets