Camouflage-aware Image-Text Retrieval via Expert Collaboration

Yao Jiang; Zhongkuan Mao; Xuan Wu; Keren Fu; Qijun Zhao

cs.CVApr 1, 2026

Camouflage-aware Image-Text Retrieval via Expert Collaboration

Authors: Yao Jiang, Zhongkuan Mao, Xuan Wu, Keren Fu, Qijun Zhao

Organizations: College of Computer Science, Sichuan University · National Key Lab of Fundamental Science on Synthetic Vision, Sichuan University

Abstract

Camouflaged scene understanding (CSU) has attracted significant attention due to its broad practical implications. However, in this field, robust image-text cross-modal alignment remains under-explored, hindering deeper understanding of camouflaged scenarios and their related applications. To this end, we focus on the typical image-text retrieval task, and formulate a new task dubbed ``camouflage-aware image-text retrieval'' (CA-ITR). We first construct a dedicated camouflage image-text retrieval dataset (CamoIT), comprising $\sim$ 10.5K samples with multi-granularity textual annotations. Benchmark results conducted on CamoIT reveal the underlying challenges of CA-ITR for existing cutting-edge retrieval techniques, which are mainly caused by objects' camouflage properties as well as those complex image contents. As a solution, we propose a camouflage-expert collaborative network (CECNet), which features a dual-branch visual encoder: one branch captures holistic image representations, while the other incorporates a dedicated model to inject representations of camouflaged objects. A novel confidence-conditioned graph attention (C\textsuperscript{2}GA) mechanism is incorporated to exploit the complementarity across branches. Comparative experiments show that CECNet achieves $\sim$ 29% overall CA-ITR accuracy boost, surpassing seven representative retrieval models. The dataset and code will be available at https://github.com/jiangyao-scu/CA-ITR.

Figures & tables

Figure 1: Qualitative results of three state-of-the-art (SOTA) retrieval methods ( i.e . , CLIP [ 50 ] , AVSE [ 40 ] , and D2S-VSE [ 39 ] ) on CA-ITR. Camouflaged objects in the images are marked with red bounding boxes. Below the dotted line are samples from the general ITR datasets ( i.e . , MS-COCO [ 36 ] and Flickr30K [ 63 ] ).

Figure 2: Data annotation process, an example, and statistical analyses of CamoIT.

Figure 3: The overall pipeline of the proposed CECNet and C 2 GA.

Models	Pub.	FG	BU	MS-COCO [ 36 ]				CamoIT				Flickr30K [ 63 ]				CamoIT
				I2T		T2I		I2T		T2I		I2T		T2I		I2T		T2I
				$R@1$	$R@10$	$R@1$	$R@10$	$R@1$	$R@10$	$R@1$	$R@10$	$R@1$	$R@10$	$R@1$	$R@10$	$R@1$	$R@10$	$R@1$	$R@10$
CFM [ 60 ]	22	✗	✓	59.6	92.9	42.7	83.4	10.7	26.0	10.7	26.7	82.7	98.2	62.6	92.0	5.4	16.5	6.5	18.9
HREM [ 18 ]	23	✗	✓	62.5	93.4	44.0	83.5	11.3	28.1	10.5	27.1	84.8	97.9	62.4	92.2	6.7	19.2	5.7	17.9
CHAN [ 46 ]	23	✓	✓	59.8	93.3	44.9	84.2	10.9	29.7	13.2	30.1	80.6	97.8	63.9	92.6	6.9	20.0	7.8	21.7
DBL [ 7 ]	24	✓	✓	57.5	91.6	41.3	80.9	7.1	20.0	8.7	23.5	78.2	97.4	58.7	89.5	4.5	14.2	3.7	13.6

Table 1: Quantitative results (

R@K

, %) of models trained on MS-COCO [ 36 ] (left) and Flickr30K [ 63 ] (right). I2T means image-to-text retrieval, while T2I denotes text-to-image retrieval. “FG” and “BU” mean whether the model employs local alignment or utilizes the BUTD framework [ 2 ] . “N/A” means that the result is unavailable. The highest scores are marked in bold . “Pub.” denotes the publication year.

Models	I2T			T2I
Models	$R@1$	$R@5$	$R@10$	$R@1$	$R@5$	$R@10$
CFM [ 60 ]	30.8	59.9	70.3	28.9	57.2	68.3
HREM [ 18 ]	34.3	62.7	74.0	31.5	59.9	71.4
CUSA [ 22 ]	23.9	53.5	66.7	23.5	51.3	64.3
LAPS [ 19 ]	27.8	62.0	73.9	28.2	58.0	70.7
D2S-VSE [ 39 ]	37.1	68.4	79.5	35.5	67.5	78.4

Table 2: Quantitative results (

R@K

, %) on CA-ITR. All models are retrained on CamoIT. The highest scores are marked in bold .

Figure 4: Qualitative results of CECNet (top, light purple) and CLIP (bottom, light orange) on sentence retrieval (left) and image retrieval (right). For each query, we present the top three relevant cross-modal instances. To enhance readability, we retain only the essential components of the sentence. The accurate, inaccurate, and ambiguous portions of the sentences in the search results are highlighted in green, red, and gray, respectively. Camouflaged objects are marked with red bounding boxes.

Settings	CamoIT
	I2T			T2I
	R@1	R@5	R@10	R@1	R@5	R@10
Baseline	41.3	69.2	79.0	41.1	67.7	78.4
A1	28.5	58.3	70.9	30.0	58.0	70.4
A2	42.1	69.9	79.2	41.2	69.5	79.0
A3	41.8	69.2	79.3	42.2	68.9	79.5

Table 3: Ablation study of

R@K

(%) on dual-branch visual encoder and C 2 GA. The highest scores are marked in bold .

COD Models	Retrieval				Segmentation
	I2T		T2I		$S_{\alpha}$	$MAE$
	R@1	R@5	R@1	R@5	$S_{\alpha}$	$MAE$
White	41.6	69.6	41.3	69.4	0.443	0.120
SINet [ 13 ]	42.3	70.4	42.1	69.9	0.808	0.049
SINet-v2 [ 12 ]	43.3	72.7	43.1	71.7	0.843	0.037
ZoomNet [ 47 ]	44.0	71.6	42.9	70.5	0.851	0.033

Table 4: Retrieval performance of CECNet and segmentation performance of corresponding COD models. “White” denotes the replacement of COD detection results with a white image. Segmentation performance is measured by

S_{\alpha}

and

MAE

(see [ 11 ] ).

Image-Text Retrieval

Image-text retrieval matches images with natural-language queries, enabling search across visual collections using either modality.

See all 38 papers

TEMA: Anchor the Image, Follow the Text for Multi-Modification Composed Image Retrieval
Apr 23, 2026Zixu Li, Yupeng Hu, Zhiheng Fu +3Cross-Modal Alignment Image Retrieval
School of Software, Shandong University · Department of Computing, Hong Kong Polytechnic University · School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen)
Language-Guided Structure-Aware Network for Camouflaged Object Detection
Mar 25, 2026Min ZhangUnsupervised Camouflaged Object Detection Camouflaged Object Detection
School of Artificial Intelligence, Chongqing University of Technology, Chongqing, 401120, China
Concepts Complement Dense Semantics: Learning Compact Sparse Spaces for Text-Image Retrieval
Sep 26, 2026Yoonseo Kim, Jungwoo Choi, Cheonyoung Park +3Cross-Modal Retrieval Vision-Language Retrieval
Sungkyunkwan University South Korea · KT Corporation South Korea

Apr 23, 2026J/K move · Enter open · S save

TEMA: Anchor the Image, Follow the Text for Multi-Modification Composed Image Retrieval

Zixu Li, Yupeng Hu, Zhiheng Fu, Zhiwei Chen+2

School of Software, Shandong University · Department of Computing, Hong Kong Polytechnic University · School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen)

Cross-Modal Alignment Image Retrieval Composed Image Retrieval

Open paper AskPDF arXiv

Composed Image Retrieval (CIR) is an important image retrieval paradigm that enables users to retrieve a target image using a multimodal query that consists of a reference image and modification text. Although research on CIR has made significant progress, prevailing setups still rely simple modification texts that typically cover only a limited range of salient changes, which induces two limitations highly relevant to practical applications, namely Insufficient Entity Coverage and Clause-Entity Misalignment. In order to address these issues and bring CIR closer to real-world use, we construct two instruction-rich multi-modification datasets, M-FashionIQ and M-CIRR. In addition, we propose TEMA, the Text-oriented Entity Mapping Architecture, which is the first CIR framework designed for multi-modification while also accommodating simple modifications. Extensive experiments on four benchmark datasets demonstrate that TEMA's superiority in both original and multi-modification scenarios, while maintaining an optimal balance between retrieval accuracy and computational efficiency. Our codes and constructed multi-modification dataset (M-FashionIQ and M-CIRR) are available at https://github.com/lee-zixu/ACL26-TEMA/.

Abstract

Image-Text Retrieval

Explore similar work

TEMA: Anchor the Image, Follow the Text for Multi-Modification Composed Image Retrieval

Language-Guided Structure-Aware Network for Camouflaged Object Detection

Concepts Complement Dense Semantics: Learning Compact Sparse Spaces for Text-Image Retrieval