HINT: Composed Image Retrieval with Dual-path Compositional Contextualized Network

Mingyu Zhang; Zixu Li; Zhiwei Chen; Zhiheng Fu; Xiaowei Zhu; Jiajia Nie; Yinwei Wei; Yupeng Hu

cs.CVMar 27, 2026

HINT: Composed Image Retrieval with Dual-path Compositional Contextualized Network

Authors: Mingyu Zhang, Zixu Li, Zhiwei Chen, Zhiheng Fu, Xiaowei Zhu, Jiajia Nie, Yinwei Wei, Yupeng Hu

Organizations: School of Software, Shandong University

Abstract

Composed Image Retrieval (CIR) is a challenging image retrieval paradigm. It aims to retrieve target images from large-scale image databases that are consistent with the modification semantics, based on a multimodal query composed of a reference image and modification text. Although existing methods have made significant progress in cross-modal alignment and feature fusion, a key flaw remains: the neglect of contextual information in discriminating matching samples. However, addressing this limitation is not an easy task due to two challenges: 1) implicit dependencies and 2) the lack of a differential amplification mechanism. To address these challenges, we propose a dual-patH composItional coNtextualized neTwork (HINT), which can perform contextualized encoding and amplify the similarity differences between matching and non-matching samples, thus improving the upper performance of CIR models in complex scenarios. Our HINT model achieves optimal performance on all metrics across two CIR benchmark datasets, demonstrating the superiority of our HINT model. Codes are available at https://github.com/zh-mingyu/HINT.

Figures & tables

Figure 1: An example of the CIR task and HINT’s main idea.

Figure 2: HINT framework: (a) Dual Context Extraction, (b) Quantification of Contextual Relevance, (c) Dual-Path Consistency Constraints.

Method	FashionIQ								CIRR
	Dresses		Shirts		Tops&Tees		Avg		R@ $k$				R subset @ $k$			Avg
	R@ $10$	R@ $50$	R@ $10$	R@ $50$	R@ $10$	R@ $50$	R@ $10$	R@ $50$	$k$ = $1$	$k$ = $5$	$k$ = $10$	$k$ = $50$	$k$ = $1$	$k$ = $2$	$k$ = $3$	Avg
TG-CIR [ 25 ] (ACM MM’23)	45.22	69.66	52.60	72.52	56.14	77.10	51.32	73.09	45.25	78.29	87.16	97.30	72.84	89.25	95.13	75.57
SSN [ 28 ] (AAAI’24)	34.36	60.78	38.13	61.83	44.26	69.05	38.92	63.89	43.91	77.25	86.48	97.45	71.76	88.63	95.54	74.51
MMF [ 20 ] (ICASSP’24)	40.45	65.29	44.85	66.29	49.26	70.98	44.85	67.52	43.80	77.37	86.94	97.71	71.39	88.16	95.15	74.38
CALA [ 12 ] (SIGIR’24)	42.38	66.08	46.76	68.16	50.93	73.42	46.69	69.22	49.11	81.21	89.59	98.00	76.27	91.04	96.46	78.74

Table 1: Performance comparison on FashionIQ and CIRR relative to R@

k

(%). The overall best results are in bold, while the second-best results are underlined. The Avg metric in CIRR denotes (R@

5

+ R subset @

1

) / 2.

Method	FashionIQ				CIRR
Method	R@ $10$	$\Delta$	R@ $50$	$\Delta$	Avg	$\Delta$
Dual Context Extraction (DCE)
w/o VCM	56.86	-1.62	77.03	-0.90	80.16	-1.41
w/o CCM	56.66	-1.82	77.07	-0.86	80.21	-1.36
w/o DCE	55.35	-3.13	75.91	-2.02	79.96	-1.61
Quantification of contextual relevance (QCR)

Table 2: Ablation study on FashionIQ and CIRR datasets. We compute Avg-R@

10

, Avg-R@

50

for FashionIQ, and Avg (mean of R@

5

and R subset @

1

) for CIRR, respectively.

Figure 3: Case study on CIRR and FashionIQ.

Explore similar work

MELT: Improve Composed Image Retrieval via the Modification Frequentation-Rarity Balance Network
Mar 31, 2026Guozhi Qiu, Zhiwei Chen, Zixu Li +4Composed Image Retrieval Cross-Modal Feature Matching
School of Software, Shandong University · Southern University of Science and Technology
Do Composed Image Retrieval Benchmarks Require Multimodal Composition?
May 14, 2026Matteo Attimonelli, Alessandro De Bellis, Aryo Pradipta Gema +8Benchmark Auditing Image Retrieval
Politecnico di Bari
HABIT: Chrono-Synergia Robust Progressive Learning Framework for Composed Image Retrieval
Apr 20, 2026Zixu Li, Yupeng Hu, Zhiwei Chen +4Learning with Noisy Labels Composed Image Retrieval
School of Software, Shandong University

Mar 31, 2026J/K move · Enter open · S save

MELT: Improve Composed Image Retrieval via the Modification Frequentation-Rarity Balance Network

Guozhi Qiu, Zhiwei Chen, Zixu Li, Qinlei Huang+3

School of Software, Shandong University · Southern University of Science and Technology

Composed Image Retrieval Cross-Modal Feature Matching

Open paper AskPDF arXiv

Composed Image Retrieval (CIR) uses a reference image and a modification text as a query to retrieve a target image satisfying the requirement of

modifying the reference image according to the text instructions''. However, existing CIR methods face two limitations: (1) frequency bias leading to

Rare Sample Neglect'', and (2) susceptibility of similarity scores to interference from hard negative samples and noise. To address these limitations, we confront two key challenges: asymmetric rare semantic localization and robust similarity estimation under hard negative samples. To solve these challenges, we propose the Modification frEquentation-rarity baLance neTwork MELT. MELT assigns increased attention to rare modification semantics in multimodal contexts while applying diffusion-based denoising to hard negative samples with high similarity scores, enhancing multimodal fusion and matching. Extensive experiments on two CIR benchmarks validate the superior performance of MELT. Codes are available at https://github.com/luckylittlezhi/MELT.

Abstract

Explore similar work

MELT: Improve Composed Image Retrieval via the Modification Frequentation-Rarity Balance Network

Do Composed Image Retrieval Benchmarks Require Multimodal Composition?

HABIT: Chrono-Synergia Robust Progressive Learning Framework for Composed Image Retrieval