Composed image retrieval (CIR) addresses the task of retrieving a target image by jointly interpreting a reference image and a modification text that specifies the intended change. Most existing methods are still built upon contrastive learning frameworks that treat the ground truth image as the only positive instance and all remaining images as negatives. This strategy inevitably introduces relevance suppression, where semantically related yet valid images are incorrectly pushed away, and semantic confusion, where different modification intents collapse into overlapping regions of the embedding space. As a result, the learned query representations often lack discriminativeness, particularly at fine-grained attribute modifications. To overcome these limitations, we propose distinctive query embeddings through learnable attribute weights and target relative negative sampling (DQE-CIR), a method designed to learn distinctive query embeddings by explicitly modeling target relative relevance during training. DQE-CIR incorporates learnable attribute weighting to emphasize distinctive visual features conditioned on the modification text, enabling more precise feature alignment between language and vision. Furthermore, we introduce target relative negative sampling, which constructs a target relative similarity distribution and selects informative negatives from a mid-zone region that excludes both easy negatives and ambiguous false negatives. This strategy enables more reliable retrieval for fine-grained attribute changes by improving query discriminativeness and reducing confusion caused by semantically similar but irrelevant candidates.
Figures & tables
Figure 1: Example of attribute-aware relevance separation in CIR. Given a reference image and the modification text specifying green and short-sleeved, candidate images are grouped according to their attribute alignment. The red box shows irrelevant images that violate key attributes and commonly appear in top-ranked results of existing CIR models. In contrast, the blue box shows relevant images that satisfy both attributes—results that DQE-CIR retrieves consistently through attribute-aware query embeddings and distinctive ranking constraints.
Figure 2: Overview of the proposed DQE-CIR framework. DQE-CIR first encodes the reference image, modification text, and candidate images using BLIP-2 to obtain base query representations. Learnable Attribute Weights then enhance the color- and shape-specific sub-queries, which are combined to form the final attribute-aware query embedding and optimized through KL-divergence and attribute-aware margin losses. In parallel, Target Relative Negative Sampling (TRNS) selects a single negative from the Δ -score–based mid-zone, enabling distinctive pairwise ranking training. Together, these components strengthen fine-grained attribute sensitivity and improve retrieval discriminativeness.
Figure 3: Illustration of Target Relative Negative Sampling (TRNS). The total query embedding is compared with the target embedding and all image corpus embeddings using cosine similarity to derive relevance scores. Subsequently, the Δ -score is computed as the difference between the target similarity and each candidate similarity. Candidate images whose Δ -score falls within a predefined mid-zone are designated as informative negatives for training.
Method
Dress
Shirt
Toptee
Average
R@10
R@50
R@10
R@50
R@10
R@50
R@10
R@50
CoSMo [ 22 ]
23.60
49.18
18.11
43.18
24.63
54.31
22.11
48.89
MGUR [ 8 ]
23.15
48.74
18.99
43.47
25.55
52.83
22.56
48.35
CLIP4Cir [ 6 ]
38.32
63.90
44.31
65.41
47.27
70.98
43.30
66.76
Bi-BLIP4CIR [ 31 ]
39.12
62.92
39.21
62.81
44.37
67.06
40.90
64.26
CoVR [ 38 ]
44.55
69.03
48.43
67.42
52.60
74.31
48.53
70.25
Table 1: Retrieval quantitative performance of diverse CIR models on the FashionIQ validation dataset. Results are reported for the Dress, Shirt, and Toptee categories using Recall@10 and Recall@50. Bold indicates the best performance for each metric.
Method
Recall@K
Recallsubset@K
Average
K=1
K=5
K=10
K=50
K=1
K=2
K=3
R@5 + Recall subset @1
CoSMo
6.48
23.11
34.63
67.33
20.29
40.22
60.80
43.55
MGUR
5.78
21.45
33.42
67.06
20.29
40.22
60.80
42.91
CLIP4Cir
44.12
77.23
86.51
97.95
73.11
89.11
95.42
75.17
Bi-BLIP4CIR
32.55
64.36
76.53
91.61
63.54
82.46
92.48
63.95
CoVR
39.76
70.15
80.89
95.01
72.46
87.86
94.77
71.30
Table 2: Retrieval quantitative performance comparison across different CIR models on the CIRR test dataset. Results are reported using Recall@K to evaluate global ranking accuracy over the entire image corpus and Recall subset @K to assess fine-grained target identification within the predefined candidate subset that contains the ground-truth image. The average metric jointly applies both global retrieval performance and subset-level discriminativeness.
Figure 4: Qualitative comparison of different CIR models on the FashionIQ validation dataset. Given a reference image and a modification text specifying “a blue short-sleeved shirt with white lettering”, we compare the top-ranked retrieval results produced by different CIR methods. While baseline models often retrieve images that partially match the query attributes, DQE-CIR retrieves images that correctly satisfy all specified modifications.
Figure 5: Qualitative comparison of DQE-CIR on the CIRR test dataset across different compositional categories. Retrieval results are conditioned on four categories of modifications: (a) Color, (b) Quantity, (c) Appearance, and (d) Form. Given a reference image and a modification text, the model retrieves images that apply the specified attribute changes while preserving the object.
Method
mAP@5
mAP@10
mAP@25
mAP@50
CoSMo
0.31
0.40
0.47
0.53
MGUR
0.14
0.17
0.25
0.30
CLIP4Cir
10.58
11.18
12.32
12.96
Bi-BLIP4CIR
4.74
4.97
5.69
6.10
CoVR
18.35
19.25
21.02
21.88
SPRC
17.57
18.48
20.14
20.98
Table 3: Zero-shot quantitative performance comparison of diverse CIR models on the CIRCO dataset, reported using mAP, which measures the average precision computed over the ranked retrieval results.
Mid-zone ( α – β )
Recall@K
Recall subset @K
K=1
K=5
K=10
K=50
K=1
K=2
K=3
0.30 – 0.70 (40%)
52.35
82.64
90.62
98.36
77.95
91.08
96.15
0.25 – 0.75 (50%)
53.22
83.42
91.12
98.52
79.02
91.84
96.78
0.20 – 0.80 (60%)
54.05
84.17
91.58
98.68
80.14
92.39
97.32
0.15 – 0.85 (70%)
53.34
83.50
91.08
98.55
79.18
91.92
96.86
0.10 – 0.90 (80%)
52.44
82.78
90.70
98.41
78.12
91.22
96.28
Table 4: Mid-zone range performance comparison on the CIRR test dataset. The mid-zone is defined by the ΔS interval [α,β] .
Figure 6: Cross-attention highlight visualization of DQE-CIR on the FashionIQ and CIRR datasets. The visualization highlights cross-attention maps corresponding to the composed queries, showing how the model attends to attribute-relevant regions in the retrieved images conditioned on the reference image and modification text.
Method
Backbone
Dress
Shirt
Toptee
Average
R@10
R@50
R@10
R@50
R@10
R@50
R@10
R@50
Bi-BLIP4CIR
BLIP
39.12
62.92
39.21
62.81
44.37
67.06
40.90
64.26
QuRe
BLIP
40.80
64.90
45.93
65.90
52.07
72.87
46.27
67.89
DQE-CIR
BLIP
41.87
65.92
47.16
66.83
53.58
74.24
47.54
69.03
CoVR-2
BLIP-2
46.41
69.51
49.75
67.76
51.86
72.46
49.34
69.91
QuRe
BLIP-2
46.80
69.81
53.53
72.87
57.47
77.77
52.60
73.48
Table 5: Retrieval performance comparison on the FashionIQ validation dataset with consistent vision–language model backbones.
Figure 7: Average TRNS negative set size across training on FashionIQ and CIRR. The negative set is redefined based on the target relative similarity distribution. The decreasing trend indicates that the distribution becomes progressively sharper, allowing the method to focus on a more informative set of negatives as training proceeds.
Composed Image Retrieval (CIR) aims to retrieve target images based on a reference image and modified texts. However, existing methods often struggle to extract the correct semantic cues from the reference image that best reflect the user's intent under textual modification prompts, resulting in interference from irrelevant visual noise. In this paper, we propose a novel Multi-level Vision Selection by Multi-modal Chain-of-Thought Reasoning (MCoT-MVS) for CIR, integrating attention-aware multi-level vision features guided by reasoning cues from a multi-modal large language model (MLLM). Specifically, we leverage an MLLM to perform chain-of-thought reasoning on the multimodal composed input, generating the retained, removed, and target-inferred texts. These textual cues subsequently guide two reference visual attention selection modules to selectively extract discriminative patch-level and instance-level semantics from the reference image. Finally, to effectively fuse these multi-granular visual cues with the modified text and the imagined target description, we design a weighted hierarchical combination module to align the composed query with target images in a unified embedding space. Extensive experiments on two CIR benchmarks, namely CIRR and FashionIQ, demonstrate that our approach consistently outperforms existing methods and achieves new state-of-the-art performance. Code and trained models are publicly released.
Xuri Ge, Chunhao Wang, Xindi Wang +3
Shandong University Jinan, China · Shandong University Qingdao, China
Composed Image Retrieval (CIR) uses a reference image and a modification text as a query to retrieve a target image satisfying the requirement of modifying the reference image according to the text instructions''. However, existing CIR methods face two limitations: (1) frequency bias leading to Rare Sample Neglect'', and (2) susceptibility of similarity scores to interference from hard negative samples and noise. To address these limitations, we confront two key challenges: asymmetric rare semantic localization and robust similarity estimation under hard negative samples. To solve these challenges, we propose the Modification frEquentation-rarity baLance neTwork MELT. MELT assigns increased attention to rare modification semantics in multimodal contexts while applying diffusion-based denoising to hard negative samples with high similarity scores, enhancing multimodal fusion and matching. Extensive experiments on two CIR benchmarks validate the superior performance of MELT. Codes are available at https://github.com/luckylittlezhi/MELT.
Guozhi Qiu, Zhiwei Chen, Zixu Li +4
School of Software, Shandong University · Southern University of Science and Technology
Composed image retrieval (CIR) aims to retrieve a desired target image from a query consisting of a reference image and a modification text. This task exhibits an unusual representational asymmetry: the modification text specifies a transition from the reference state, whereas retrieval candidates depict completed target states. This creates a representation mismatch for zero-shot methods that query pretrained vision-language spaces directly with transformation-oriented language. We study this mismatch and reformulate zero-shot composed image retrieval as target-state reconstruction followed by retrieval. We instantiate this formulation with ASAP-CIR, a training-free framework that reconstructs a static target representation using a frozen multimodal large language model (MLLM). The representation combines multiple holistic descriptions with a variable set of importance-weighted atomic semantics, thereby preserving both overall target identity and fine-grained visual constraints. Retrieval then integrates holistic state alignment, atomic constraint grounding, and calibrated target-state evidence aggregation. A controlled text-only diagnostic shows that target-side static query formulations achieve more reliable retrieval than dynamic composed query formulations, particularly when source-state semantics must be suppressed or transformed. Experiments on FashionIQ, CIRR, and CIRCO further characterize the effectiveness and limitations of this representation principle, with the clearest gains on the multi-target CIRCO benchmark. These results show that how composed intent is represented before retrieval is a consequential design choice, distinct from the choice of retrieval backbone itself.
Yihe Zhao, Songhe Feng
School of Computer Science and Technology, Beijing Jiaotong University, Beijing, 100044, China.