MCoT-MVS: Multi-level Vision Selection by Multi-modal Chain-of-Thought Reasoning for Composed Image Retrieval

Xuri Ge; Chunhao Wang; Xindi Wang; Zheyun Qin; Zhumin Chen; Xin Xin

cs.CVMar 18, 2026

MCoT-MVS: Multi-level Vision Selection by Multi-modal Chain-of-Thought Reasoning for Composed Image Retrieval

Authors: Xuri Ge, Chunhao Wang, Xindi Wang, Zheyun Qin, Zhumin Chen, Xin Xin

Organizations: Shandong University Jinan, China · Shandong University Qingdao, China

Abstract

Composed Image Retrieval (CIR) aims to retrieve target images based on a reference image and modified texts. However, existing methods often struggle to extract the correct semantic cues from the reference image that best reflect the user's intent under textual modification prompts, resulting in interference from irrelevant visual noise. In this paper, we propose a novel Multi-level Vision Selection by Multi-modal Chain-of-Thought Reasoning (MCoT-MVS) for CIR, integrating attention-aware multi-level vision features guided by reasoning cues from a multi-modal large language model (MLLM). Specifically, we leverage an MLLM to perform chain-of-thought reasoning on the multimodal composed input, generating the retained, removed, and target-inferred texts. These textual cues subsequently guide two reference visual attention selection modules to selectively extract discriminative patch-level and instance-level semantics from the reference image. Finally, to effectively fuse these multi-granular visual cues with the modified text and the imagined target description, we design a weighted hierarchical combination module to align the composed query with target images in a unified embedding space. Extensive experiments on two CIR benchmarks, namely CIRR and FashionIQ, demonstrate that our approach consistently outperforms existing methods and achieves new state-of-the-art performance. Code and trained models are publicly released.

Figures & tables

Figure 1. Compared with the traditional CIR model (B) and cross-modal attention-based CIR model (C), in this paper, we introduce multimodal chain-of-thought reasoning to explicitly extract the relevant and irrelevant contents for reference visual information selection (A). In this way, while effectively focusing on fine-grained useful references, we can significantly avoid the attention noise interference driven by visual-linguistic correlation as in method (C).

Figure 2. Illustration of our proposed MCoT-MVS. It primarily consists of the Multimodal Chain-of-Thought (CoT) Reasoning module, Patch-level Visual Reference Selection (PVRS), the Instance-level Visual Reference Selection module (IVRS), and a Weighted Hierarchical Combination (WHC). The multimodal CoT reasoning module first reasons the composed query into retained and deleted contents, as well as the potential target context, based on a pre-trained MLLM. Then, the useful patch-level and instance-level reference visual representations are semantically selected by the explicitly reasoned retained and deleted texts to retain the correct user intent and to remove visual noise. Finally, the WHC fuses the selected representations with multiple target texts into an attention-aware query, aligning with the target image.

Traditional Model-Based Methods
Method	R@1	R@5	R@10	R@50	Avg
TIRG ( Vo et al., 2019 ) ${}_{(CVPR^{\prime}19)}$	14.61	48.37	64.08	90.03	54.27
CIRPLANT ( Liu et al., 2021 ) ${}_{(ICCV^{\prime}19)}$	19.55	52.55	68.39	92.38	58.22
ARTEMIS ( Delmas et al., 2022 ) ${}_{(ICLR^{\prime}22)}$	16.96	46.10	61.31	87.73	53.03
VLP Model-Based Methods
CLIP4CIR ( Baldrati et al., 2023 ) ${}_{(TOMM^{\prime}23)}$	38.53	69.98	81.86	95.93	71.58

Table 1. Performance comparison on CIRR from the open-world scene. The best and the second-best scores are marked in bold and underlined fonts, respectively.

	Dresses		Shirts		Tops&Tees		Avg
Method	R@10	R@50	R@10	R@50	R@10	R@50	R@10	R@50
Traditional Model-Based Methods
TIRG ( Vo et al., 2019 ) ${}_{(CVPR^{\prime}19)}$	14.87	34.66	18.26	37.89	19.08	39.62	17.40	37.39
CIRPLANT ( Liu et al., 2021 ) ${}_{(ICCV^{\prime}19)}$	17.45	40.41	17.53	38.81	21.64	45.38	18.87	41.53
ARTEMIS ( Delmas et al., 2022 ) ${}_{(ICLR^{\prime}22)}$	27.16	52.40	21.78	43.64	29.20	54.83	26.05	50.29
VLP Model-Based Methods

Table 2. Performance comparison on FashionIQ from fashion products. The best and the second-best scores are marked in bold and underlined fonts, respectively.

NO.	Variant							Performance
NO.	WHC	Sum	Combiner	M-T	T-T	PVRS	IVRS	Recall@1	Recall@5	Recall@10	Recall@50	Mean
1	$\surd$	-	-	$\surd$	$\surd$	$\surd$	$\surd$	55.33	84.75	91.45	98.55	82.52
2	-	$\surd$	-	$\surd$	$\surd$	$\surd$	$\surd$	52.02	83.01	90.80	98.36	81.05
3	-	-	$\surd$	-	$\surd$	$\surd$	$\surd$	45.00	76.48	85.35	96.87	75.93
4	-	-	$\surd$	$\surd$	-	$\surd$	$\surd$	54.50	83.69	91.28	98.56	81.98
5		-	$\surd$	$\surd$	-	$\surd$	-	53.49	84.07	91.21	98.55	81.83

Table 3. Ablation study for MCoT-MVS on CIRR. “M-T" and “T-T" indicate the modification texts and the reasoned target texts.

Figure 3. Visualization of the learned attention weights from PVRS and IVRS modules guided by the inferred explicit retain or delete modification intents from the MLLM-based multimodal CoT reasoning, as well as the weight distributions of weighted hierarchical combination modules (best viewed in color).

Model	R@1	R@5	R@10	R@50
w/o Multi-step MCot	54.29	84.45	91.15	98.60
w/ Multi-step MCot	55.33	84.75	91.45	98.55

Table 4. Retrieval performance (%) of MCoT-MVS with different reasoning patterns.

Model	R@1	R@5	R@10	R@50
Llama-3.2-11B 1 1 1 Llama-3.2-11B: https://huggingface.co/meta-llama/Llama-3.2-11B-Vision	54.31	84.26	91.78	98.69
Qwen2.5-VL-7B 2 2 2 Qwen2.5-VL-7B: https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct	54.53	84.31	91.13	98.50
Qwen2.5-VL-32B	55.33	84.75	91.45	98.55

Table 5. Retrieval performance (%) of MCoT-MVS with different reasoning models.

Explore similar work

FIRE-CIR: Fine-grained Reasoning for Composed Fashion Image Retrieval
Apr 10, 2026François Gardères, Camille-Sovanneary Gauthier, Jean Ponce +1Composed Image Retrieval VLM Reasoning
Louis Vuitton · Inria, École normale supérieure, CNRS, PSL Research University · Courant Institute of Mathematical Sciences and Center for Data Science, New York University
MELT: Improve Composed Image Retrieval via the Modification Frequentation-Rarity Balance Network
Mar 31, 2026Guozhi Qiu, Zhiwei Chen, Zixu Li +4Composed Image Retrieval Cross-Modal Feature Matching
School of Software, Shandong University · Southern University of Science and Technology
From Transformation to Target State: Rethinking Query Representation for Zero-Shot Composed Image Retrieval
Oct 5, 2026Yihe Zhao, Songhe FengCross-Modal Alignment Composed Image Retrieval
School of Computer Science and Technology, Beijing Jiaotong University, Beijing, 100044, China.

Apr 10, 2026J/K move · Enter open · S save

FIRE-CIR: Fine-grained Reasoning for Composed Fashion Image Retrieval

François Gardères, Camille-Sovanneary Gauthier, Jean Ponce, Shizhe Chen

Louis Vuitton · Inria, École normale supérieure, CNRS, PSL Research University · Courant Institute of Mathematical Sciences and Center for Data Science, New York University

Composed Image Retrieval VLM Reasoning

Open paper AskPDF arXiv

Composed image retrieval (CIR) aims to retrieve a target image that depicts a reference image modified by a textual description. While recent vision-language models (VLMs) achieve promising CIR performance by embedding images and text into a shared space for retrieval, they often fail to reason about what to preserve and what to change. This limitation hinders interpretability and yields suboptimal results, particularly in fine-grained domains like fashion. In this paper, we introduce FIRE-CIR, a model that brings compositional reasoning and interpretability to fashion CIR. Instead of relying solely on embedding similarity, FIRE-CIR performs question-driven visual reasoning: it automatically generates attribute-focused visual questions derived from the modification text, and verifies the corresponding visual evidence in both reference and candidate images. To train such a reasoning system, we automatically construct a large-scale fashion-specific visual question answering dataset, containing questions requiring either single- or dual-image analysis. During retrieval, our model leverages this explicit reasoning to re-rank candidate results, filtering out images inconsistent with the intended modifications. Experimental results on the Fashion IQ benchmark show that FIRE-CIR outperforms state-of-the-art methods in retrieval accuracy. It also provides interpretable, attribute-level insights into retrieval decisions.

Abstract

Explore similar work

FIRE-CIR: Fine-grained Reasoning for Composed Fashion Image Retrieval

MELT: Improve Composed Image Retrieval via the Modification Frequentation-Rarity Balance Network

From Transformation to Target State: Rethinking Query Representation for Zero-Shot Composed Image Retrieval