FashionMV: Product-Level Composed Image Retrieval with Multi-View Fashion Data

Peng Yuan; Bingyin Mei; Hui Zhang

cs.CVApr 11, 2026

FashionMV: Product-Level Composed Image Retrieval with Multi-View Fashion Data

Authors: Peng Yuan, Bingyin Mei, Hui Zhang

Organizations: Tsinghua University Beijing, China

Abstract

Composed Image Retrieval (CIR) retrieves target images using a reference image paired with modification text. Despite rapid advances, all existing methods and datasets operate at the image level -- a single reference image plus modification text in, a single target image out -- while real e-commerce users reason about products shown from multiple viewpoints. We term this mismatch View Incompleteness and formally define a new Multi-View CIR task that generalizes standard CIR from image-level to product-level retrieval. To support this task, we construct FashionMV, the first large-scale multi-view fashion dataset for product-level CIR, comprising 127K products, 472K multi-view images, and over 220K CIR triplets, built through a fully automated pipeline leveraging large multimodal models. We further propose ProCIR (Product-level Composed Image Retrieval), a modeling framework built upon a multimodal large language model that employs three complementary mechanisms -- two-stage dialogue, caption-based alignment, and chain-of-thought guidance -- together with an optional supervised fine-tuning (SFT) stage that injects structured product knowledge prior to contrastive training. Systematic ablation across 16 configurations on three fashion benchmarks reveals that: (1) alignment is the single most critical mechanism; (2) the two-stage dialogue architecture is a prerequisite for effective alignment; and (3) SFT and chain-of-thought serve as partially redundant knowledge injection paths. Our best 0.8B-parameter model outperforms all baselines, including general-purpose embedding models 10x its size. The dataset, model, and code are publicly available at https://github.com/yuandaxia2001/FashionMV.

Figures & tables

Figure 1. A fashion product displayed from four viewpoints. The deep V-neckline is only visible from the front, while the open-back criss-cross straps are only visible from the back—no single image can capture both defining features, illustrating View Incompleteness .

Figure 2. Overview of the FashionMV dataset construction pipeline. Stage 1 : multi-view images from three data sources are fed to Kimi K2.5 in a single request to generate per-image captions, long captions, and short captions. Stage 2 : Qwen3.5-397B cross-checks directional descriptions against visual evidence, removing products with confirmed hallucinations. Stage 3 : multi-path candidate retrieval (visual, long-caption, and short-caption similarity) produces a candidate pool; Gemini 3.1 Flash Lite selects up to 2 best targets from 10 candidates and generates short and long modification texts.

Figure 3. Overview of the ProCIR training architecture. The two-stage dialogue decomposes the query into Turn 1 (visual perception $\rightarrow$ source embedding $\mathbf{s}$ ) and Turn 2 (modification reasoning $\rightarrow$ query embedding $\mathbf{q}$ ); Turn 2 inherits the full dialogue context from Turn 1, enabling the model to reason about modifications conditioned on the perceived visual content. The document encoder produces target embedding $\mathbf{d}$ from multi-view target images. Caption-based alignment encodes source and target captions into $\mathbf{t}_{\text{src}}$ and $\mathbf{t}_{\text{tgt}}$ . The training loss combines $\mathcal{L}_{\text{cir}}$ , $\mathcal{L}_{\text{src}}$ , and $\mathcal{L}_{\text{doc}}$ , all based on SymInfoNCE. An optional SFT stage injects structured product knowledge prior to contrastive training. Two alternative knowledge injection paths are shown at the bottom: SFT pre-training (left) and Chain-of-Thought with progressive removal (right).

Dataset	Split	Triplets	Products
DeepFashion	Train	16,399	8,856
DeepFashion	Val	5,188	2,791
Fashion200K	Train	98,800	56,960
Fashion200K	Val	18,499	10,720
FashionGen	Train	72,816	42,612
FashionGen	Val	9,031	5,292

Table 1. FashionMV CIR triplet splits.

			DeepFashion		F200K		FashionGen		Average
Model	Params	Encoding	R@5	R@10	R@5	R@10	R@5	R@10	R@5	R@10	Avg
CLIP4CIR	0.25B	Joint	–	–	–	–	–	–	–	–	–
		MeanPool	28.0	39.3	13.3	19.3	16.2	23.6	19.2	27.4	23.3
		MaxSim	25.7	36.6	11.9	17.7	17.1	25.0	18.2	26.4	22.3
SPRC	1.2B	Joint	–	–	–	–	–	–	–	–	–
		MeanPool	53.4	65.1	32.9	42.3	38.5	48.7	41.6	52.0	46.8

Table 2. Comparison with existing models on multi-view CIR under three encoding strategies. Joint : all views encoded jointly; MeanPool : per-view embeddings averaged; MaxSim : per-view retrieval with max-score selection. “–”: unsupported or unavailable. Bold : best per column; underline : second best.

				DeepFashion		F200K		FashionGen		Average
MT	Align	CoT	SFT	R@5	R@10	R@5	R@10	R@5	R@10	R@5	R@10	Avg	Cost
Pretrained base
				77.0	87.8	61.4	73.6	59.7	72.5	66.0	78.0	72.0	1.01 $\times$ (7h25m)
	$\surd$			78.7	88.7	61.5	74.1	62.6	74.5	67.6	79.1	73.4	1.12 $\times$ (8h13m)
		$\surd$		77.9	87.9	60.5	71.6	56.3	68.6	64.9	76.0	70.5	1.16 $\times$ (8h31m)
	$\surd$	$\surd$		77.5	87.4	61.8	73.1	57.4	69.6	65.6	76.7	71.2	1.27 $\times$ (9h20m)

Table 3. Ablation study on the validation set. MT = two-stage dialogue, Align = caption-based alignment, CoT = chain-of-thought. Cost = relative wall-clock time with absolute duration in parentheses; 1.00

\times

corresponds to the fastest variant; SFT rows include SFT pre-training overhead (4h 29m). Bold : best in each section; underline : second best.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Checked	Errors	Retained	Error Rate
DeepFashion	12,711	487	12,224	3.83%
Fashion200K	77,106	3,607	73,499	4.68%
FashionGen-train	48,476	1,942	46,534	4.01%
FashionGen-val	6,086	271	5,815	4.45%
Total	144,379	6,307	138,072	4.37%

Appendix

Table 4. Directional hallucination detection results per dataset.

Dataset	Per-Image	Long Cap.	Short Cap.
DeepFashion	115	219	39
Fashion200K	115	220	39
FashionGen-train	115	222	40
FashionGen-val	115	223	40

Appendix

Table 5. Average caption word counts per dataset.

View Combination	Count	Percentage
back + front	180,040	81.6%
front + side	27,556	12.5%
back + front + side	6,390	2.9%
back + side	4,636	2.1%
back + detail + front	416	0.2%
Other combinations	1,695	0.8%

Appendix

Table 6. View combination distribution in CIR triplets.

Panel of Figure 4 — Figure 4. Distribution of the number of multi-view images per product, broken down by sub-dataset.

Panel of Figure 5 — Figure 5. Word-count distributions for all five text types in FashionMV. (a) Short captions peak around 40 words (median=40). (b) Short modification texts are tightly concentrated at 23–24 words (median=23). (c) Long captions are normally distributed around 220 words (median=222). (d) Long modification texts peak around 82–84 words (median=84). (e) Per-image captions concentrate at 105–115 words (median=115).

Parameter	Value
Backbone	Qwen3.5-0.8B
Hidden dimension	1024
Transformer layers	24
Image resolution	$128{\times}128$ – $512{\times}512$
Max views per product	5
Optimizer	AdamW

Appendix

Table 7. Full training hyperparameters.

Task	Full Size	Used (20%)
Caption Generation	587,050	117,610
CIR Triplet Generation	939,700	187,940
Total	1,526,750	305,550

Appendix

Table 8. SFT training data statistics (full dataset; 20% sampled for training).

				DeepFashion		F200K		FashionGen
MT	Align	CoT	SFT	R@1	R@5	R@1	R@5	R@1	R@5
Pretrained base
				60.2	89.6	57.8	81.8	53.1	78.6
	$\surd$			71.1	94.4	64.1	85.9	52.7	78.1
		$\surd$		44.2	73.3	28.9	53.5	21.7	43.1
	$\surd$	$\surd$		50.9	79.7	35.8	61.7	23.8	46.9

Appendix

Table 9. I2T retrieval R@1/R@5 (%) with short captions . MT = two-stage dialogue, Align = caption-based alignment, CoT = chain-of-thought, SFT = supervised fine-tuning. Bold : best per column; underline : second best.

				DeepFashion		F200K		FashionGen
MT	Align	CoT	SFT	R@1	R@5	R@1	R@5	R@1	R@5
Pretrained base
				71.2	93.5	60.3	83.5	57.3	81.5
	$\surd$			75.9	95.6	66.3	87.0	56.8	81.3
		$\surd$		56.6	85.0	41.3	67.4	38.5	64.9
	$\surd$	$\surd$		65.6	90.9	54.6	78.9	45.8	72.4

Appendix

Table 10. T2I retrieval R@1/R@5 (%) with short captions . MT = two-stage dialogue, Align = caption-based alignment, CoT = chain-of-thought, SFT = supervised fine-tuning. Bold : best per column; underline : second best.

Composed Image Retrieval

Composed image retrieval finds images that match a reference image together with a text instruction describing the desired change.

See all 65 papers

Jun 18, 2026

Exploring Multi-Modal Large Language Models and Two-Stage Fine-Tuning for Fashion Image Retrieval

Composed image retrieval retrieves a target image using a composed query of a reference image and a modified text description. In the fashion domain, this task requires understanding subtle attribute variations such as color, pattern, and texture. However, existing approaches face limitations due to scarce annotated data and simplistic negative sampling. We propose a novel framework that integrates a multi-modal large language model (LLaVA) to generate attribute-aware triplets and introduces a two-stage fine-tuning strategy to enhance contrastive learning. We leverage pretrained vision-language models, such as CLIP-ViT/B32, to generate and concatenate sentence-level prompts with the relative caption and to scale the number of negatives using static representations. Experimental results demonstrate enhanced compositional reasoning and improved fine-grained retrieval behavior, underscoring the feasibility and potential of the proposed framework for fashion retrieval.

Nguyen Cao Hoang, Hoang Bui Le, Nam Vo Hoang +1

University of Science, VNU-HCM, Ho Chi Minh, Vietnam · Vietnam National University, Ho Chi Minh, Vietnam

Multimodal Large Language Models Image-Text Retrieval Multimodal Contrastive Learning Composed Image Retrieval

Ask PDF

Apr 10, 2026

FIRE-CIR: Fine-grained Reasoning for Composed Fashion Image Retrieval

Composed image retrieval (CIR) aims to retrieve a target image that depicts a reference image modified by a textual description. While recent vision-language models (VLMs) achieve promising CIR performance by embedding images and text into a shared space for retrieval, they often fail to reason about what to preserve and what to change. This limitation hinders interpretability and yields suboptimal results, particularly in fine-grained domains like fashion. In this paper, we introduce FIRE-CIR, a model that brings compositional reasoning and interpretability to fashion CIR. Instead of relying solely on embedding similarity, FIRE-CIR performs question-driven visual reasoning: it automatically generates attribute-focused visual questions derived from the modification text, and verifies the corresponding visual evidence in both reference and candidate images. To train such a reasoning system, we automatically construct a large-scale fashion-specific visual question answering dataset, containing questions requiring either single- or dual-image analysis. During retrieval, our model leverages this explicit reasoning to re-rank candidate results, filtering out images inconsistent with the intended modifications. Experimental results on the Fashion IQ benchmark show that FIRE-CIR outperforms state-of-the-art methods in retrieval accuracy. It also provides interpretable, attribute-level insights into retrieval decisions.

François Gardères, Camille-Sovanneary Gauthier, Jean Ponce +1

Louis Vuitton · Inria, École normale supérieure, CNRS, PSL Research University · Courant Institute of Mathematical Sciences and Center for Data Science, New York University

Composed Image Retrieval VLM Reasoning

Ask PDF

Mar 18, 2026

MCoT-MVS: Multi-level Vision Selection by Multi-modal Chain-of-Thought Reasoning for Composed Image Retrieval

Composed Image Retrieval (CIR) aims to retrieve target images based on a reference image and modified texts. However, existing methods often struggle to extract the correct semantic cues from the reference image that best reflect the user's intent under textual modification prompts, resulting in interference from irrelevant visual noise. In this paper, we propose a novel Multi-level Vision Selection by Multi-modal Chain-of-Thought Reasoning (MCoT-MVS) for CIR, integrating attention-aware multi-level vision features guided by reasoning cues from a multi-modal large language model (MLLM). Specifically, we leverage an MLLM to perform chain-of-thought reasoning on the multimodal composed input, generating the retained, removed, and target-inferred texts. These textual cues subsequently guide two reference visual attention selection modules to selectively extract discriminative patch-level and instance-level semantics from the reference image. Finally, to effectively fuse these multi-granular visual cues with the modified text and the imagined target description, we design a weighted hierarchical combination module to align the composed query with target images in a unified embedding space. Extensive experiments on two CIR benchmarks, namely CIRR and FashionIQ, demonstrate that our approach consistently outperforms existing methods and achieves new state-of-the-art performance. Code and trained models are publicly released.

Xuri Ge, Chunhao Wang, Xindi Wang +3

Shandong University Jinan, China · Shandong University Qingdao, China

Composed Image Retrieval VLM Reasoning

Ask PDF

Abstract

Composed Image Retrieval

Explore similar work

Exploring Multi-Modal Large Language Models and Two-Stage Fine-Tuning for Fashion Image Retrieval

FIRE-CIR: Fine-grained Reasoning for Composed Fashion Image Retrieval

MCoT-MVS: Multi-level Vision Selection by Multi-modal Chain-of-Thought Reasoning for Composed Image Retrieval