Organizations: Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · National University of Singapore · School of Artificial Intelligence, University of Chinese Academy of Sciences · Minzu University of China · Wuhan AI Research
Zero-Shot Composed Image Retrieval (ZS-CIR) aims to retrieve target images given a multimodal query (comprising a reference image and a modification text), without training on annotated triplets. Existing methods typically convert the multimodal query into a single modality-either as an edited caption for Text-to-Image retrieval (T2I) or as an edited image for Image-to-Image retrieval (I2I). However, each paradigm has inherent limitations: T2I often loses fine-grained visual details, while I2I struggles with complex semantic modifications. To effectively leverage their complementary strengths under diverse query intents, we propose WISER, a training-free framework that unifies T2I and I2I via a "retrieve-verify-refine" pipeline, explicitly modeling intent awareness and uncertainty awareness. Specifically, WISER first performs Wider Search by generating both edited captions and images for parallel retrieval to broaden the candidate pool. Then, it conducts Adaptive Fusion with a verifier to assess retrieval confidence, triggering refinement for uncertain retrievals, and dynamically fusing the dual-path for reliable ones. For uncertain retrievals, WISER generates refinement suggestions through structured self-reflection to guide the next retrieval round toward Deeper Thinking. Extensive experiments demonstrate that WISER significantly outperforms previous methods across multiple benchmarks, achieving relative improvements of 45% on CIRCO (mAP@5) and 57% on CIRR (Recall@1) over existing training-free methods. Notably, it even surpasses many training-dependent methods, highlighting its superiority and generalization under diverse scenarios. Code will be released at https://github.com/Physicsmile/WISER.
Figures & tables
Figure 1: Comparison of existing ZS-CIR methods. (a) T2I may fail to preserve visual details from the reference image, while (b) I2I often struggles with complex modifications. In contrast, (c) WISER successfully adapts to diverse modification intents through a “retrieve–verify–refine” pipeline.
Figure 2: Overview of the proposed WISER framework. (1) Wider Search. We leverage an editor to produce text and image queries for dual-path retrieval, aggregating the top- K results into a unified candidate pool. (2) Adaptive Fusion. We employ a verifier to assess the candidates with confidence scores, applying a multi-level fusion strategy for high-confidence results and triggering refinement for low-confidence ones. (3) Deeper Thinking. For uncertain retrievals, we leverage a refiner to analyze unmet modifications and then feed targeted suggestions back to the editor, iterating until a predefined limit is reached.
Backbone
Method
Training [-2pt] -free
Shirt
Dress
Toptee
Avg.
R@10
R@50
R@10
R@50
R@10
R@50
R@10
R@50
ViT-B/32
PALAVRA (ECCV’22)
✗
21.49
37.05
17.25
35.94
20.55
38.76
19.76
37.25
SEARLE (ICCV’23)
✗
24.44
41.61
18.54
39.51
25.70
46.46
22.89
42.53
CIReVL (ICLR’24)
✓
28.36
47.84
25.29
46.36
31.21
53.85
28.29
49.35
LDRE (SIGIR’24)
✓
27.38
46.27
19.97
41.84
27.07
48.78
24.81
45.63
OSrCIR (CVPR’25)
✓
31.16
51.13
29.35
50.37
36.51
58.71
32.34
53.40
Table 2: Comparison with the state-of-the-art methods on the validation set of Fashion-IQ. The best results are in bold , while the second-best results are underlined . Our method is highlighted in grey.
Wider Search
Deeper Thinking
Fusion
Fashion-IQ-Avg
CIRCO
T2I
I2I
R@10
R@50
mAP@5
mAP@10
mAP@25
mAP@50
-
✓
-
-
22.65
38.84
7.00
7.46
8.40
8.95
-
✓
✓
-
23.58
40.10
7.57
8.05
9.04
9.62
✓
-
-
-
28.59
49.18
17.28
17.94
19.64
20.51
✓
-
✓
-
29.22
49.94
17.64
18.30
20.01
20.94
✓
✓
-
AVG
33.40
52.92
13.53
14.30
15.96
16.77
Table 3: Ablation study on the core components of WISER. “AVG” denotes average and “ADA” denotes Adaptive Fusion.
Figure 3: Qualitative results on (a) Fashion-IQ, (b) CIRR, and (c) CIRCO datasets. Red indicates wrong, green represents correct and the gray arrow points to refined results.
Editor
Verifier
Refiner
k=5
k=10
k=25
k=50
Bagel [ 18 ]
Qwen2.5-VL-7B [ 3 ]
Qwen-Turbo [ 2 ]
32.80
33.56
35.21
35.79
GPT-3.5-Turbo [ 7 ]
32.57
33.49
35.13
35.63
GPT-4o-Mini [ 7 ]
32.21
33.06
34.74
35.28
GPT-4o [ 1 ]
32.23
33.18
34.82
35.35
Bagel [ 18 ]
Qwen2-VL-7B [ 53 ]
GPT-4o [ 1 ]
25.50
26.26
28.41
29.12
Qwen2.5-VL-3B [ 3 ]
27.50
28.45
30.16
30.79
Table 4: Ablation study on the choice of editor, verifier, and refiner models on the CIRCO dataset, evaluated with mAP@k.
Figure 4: Sensitivity analysis on confidence threshold τ and refinement iteration N on CIRCO.
Figure 5: Comparison between fixed fusion strategies and WISER on CIRCO. λ controls the T2I weight in the fixed fusion ( λ for T2I and 1- λ for I2I). Our WISER method achieves superior performance over all λ values, highlighting the limitation of static weighting.
Wider Search
Fusion
Deeper Thinking
Fashion-IQ-Avg
CIRCO
T2I
I2I
R@10
R@50
mAP@5
mAP@10
mAP@25
mAP@50
-
✓
-
-
22.65
38.84
7.00
7.46
8.40
8.95
-
✓
-
✓
23.58
40.10
7.57
8.05
9.04
9.62
-
✓
RAK
-
32.64
38.84
20.40
20.12
20.63
20.73
✓
-
-
-
28.59
49.18
17.28
17.94
19.64
20.51
✓
-
-
✓
29.22
49.94
17.64
18.30
20.01
20.94
Table 5: More ablation study on the core components of WISER on the Fashion-IQ and CIRCO datasets. “RAK” denotes rerank, “AVG” denotes average fusion at the similarity level, and “ADA” denotes Adaptive Fusion.
Method
k=5
k=10
k=25
k=50
w/ Gemini 2.5 Pro
32.92
33.65
35.4
35.95
w/ DeepSeek-R1
33.08
33.85
35.51
36.05
WISER (Ours)
32.23
33.18
34.82
35.35
Table 6: Further ablations of the refiner on CIRCO.
Figure 6: Prompt for the verifier. The prompt guides the verifier to answer a binary question given the reference image, the candidate image and the modification text.
Figure 7: More Qualitative Results on CIRR. Green checkmark indicates correct target retrieval. WISER successfully retrieves the target image at top-1 across various scenarios.
Figure 8: More Qualitative Results on CIRCO. Green checkmark indicates correct target retrieval. WISER successfully retrieves the target image at top-1 across various scenarios.
Figure 9: More Qualitative Results on Fashion-IQ. Green checkmark indicates correct target retrieval. WISER successfully retrieves the target image at top-1 across various scenarios.
Figure 10: Failure cases from CIRCO, CIRR, Fashion-IQ datasets. We demonstrate three challenging cases where WISER struggles to retrieve the target image at top-1.
Figure 11: Prompt for the refiner of T2I. The prompt guides the refiner to perform structured self-reflection for uncertain retrievals, given the reference image caption, modification text, and pseudo target caption from T2I. The generated suggestions are then fed to the editor to refine the edited caption for T2I.
Figure 12: Prompt for the refiner of I2I. The prompt guides the refiner to perform structured self-reflection for uncertain retrievals, given the reference image caption, modification text, and pseudo target caption from I2I. The generated suggestions are then fed to the editor to refine the edited image for I2I.
Zero-shot composed image retrieval (ZS-CIR) aims to retrieve a target image from a multimodal query consisting of a reference image and an edit text describing the desired modification. Recent ZS-CIR studies have relied on projection-based methods that map a reference image into pseudo-word tokens in the text embedding space. However, such methods require additional projection and re-encoding steps, increasing training complexity, reducing efficiency, and introducing a discrepancy between training and inference. In this paper, we propose DiCE-CIR, a direct composition learning method that predicts composed query representations by directly composing a reference image and an edit text. To enable scalable training without manually annotated triplets, we automatically construct compositional training samples from large-scale image-caption pairs using a large language model. Based on these samples, we train a lightweight composition module with objectives that promote alignment with the target, edit-consistent semantic transformation, and retrieval discriminability. We conduct extensive experiments on ZS-CIR benchmarks and show that DiCE-CIR achieves state-of-the-art performance on CIRCO and competitive performance on CIRR while maintaining high computational efficiency.
Gwang-Ho Na, Ho-Joong Kim, Seong-Whan Lee
Department of Artificial Intelligence, Korea University, Anam-dong, Seongbuk-ku, Seoul 02841, Korea
Zero-shot composed image retrieval (ZS-CIR) aims to retrieve a target image that reflects an intended textual modification while maintaining semantic consistency with a reference image. Existing training-free ZS-CIR methods typically rely on fine-grained generated target descriptions, which often overemphasize reference-specific visual details and lead to appearance-biased retrieval. Moreover, since these descriptions are generated through multimodal large language models (MLLMs), the direct influence of the original image-text query can be weakened in the final retrieval representation. In this paper, we propose DCR-TSF, a training-free ZS-CIR framework that combines Dual-level Compositional Reasoning (DCR) and Triadic Spherical Fusion (TSF). DCR generates fine- and coarse-grained descriptions from the reference image and modification text, capturing complementary target-side semantics across different granularities. To preserve the original image-text query information, TSF constructs a composed-query anchor by spherically interpolating the reference image feature and modification text feature, and fuses it with the fine- and coarse-grained representations using a weighted Karcher mean on the unit hypersphere. The resulting final query representation balances detailed visual evidence, high-level semantic abstraction, and direct preservation of the original composed query. Extensive experiments on CIRCO, CIRR, and Fashion-IQ demonstrate that our method achieves state-of-the-art performance among training-free methods.
Gwang-Ho Na, Ho-Joong Kim, Seong-Whan Lee
Dept. of Artificial Intelligence, Korea University, Seoul, Korea
Zero-shot composed image retrieval (ZS-CIR) aims to retrieve a target image based on a reference image and modification text. Recent approaches commonly rely on large vision-language models to generate target image captions for retrieval. However, these captions may include redundant semantic details, introducing noise when retrieving the target image and leading to suboptimal retrieval results. To address this issue, we propose Less-is-More CIR, a two-stage training-free framework that combines Concise Retrieval and Critic Re-ranking. In the first stage, concise target-oriented captions are generated that preserve only the most discriminative modification semantics, thereby reducing semantic noise in the retrieval. In the second stage, a critic re-rank module further evaluates the retrieved candidates from the perspectives of modification consistency and reference preservation. Extensive experiments demonstrate the effectiveness and robustness of the proposed framework for the ZS-CIR. Our code is available at: https://github.com/dxlabskku/LM-CIR.
Yuqian Zheng, Hyoju Ahn, Hyeongjin Ahn +1
Department of Applied Artificial Intelligence, Sungkyunkwan University, Seoul, Republic of Korea · Department of Immersive Media Engineering, Sungkyunkwan University, Seoul, Republic of Korea · Jaume I University, Castellon, Spain