Preserve-and-Compose Training for Composed Image Retrieval

Sehyun Kwon

cs.CVSep 25, 2026

Preserve-and-Compose Training for Composed Image Retrieval

Authors: Sehyun Kwon

Organizations: Hanyang University ERICA

Abstract

Composed image retrieval (CIR) aims to retrieve images that satisfy a user-specified modification while preserving relevant visual content from a reference image. Collecting target images for this purpose is costly, motivating zero-shot CIR methods that instead use target captions as supervision. However, target captions may omit source details that should be preserved. We therefore propose, Preserve-and-Compose Training, which complements target-caption supervision with visual evidence from the source image. PACT learns from image--text--text (ITT) triplets without target images or gallery updates, aligning composed queries with target captions while preserving source evidence through visual supervision. We further introduce Chord scoring, which combines target similarity with source-relative directional agreement in the frozen image space. Results across four ZS-CIR benchmarks show that combining target-caption supervision with source-image evidence leads to strong retrieval performance across datasets, backbone scales, and external galleries. The code is available on https://github.com/sehyunkwon/PACT.

Figures & tables

Figure 1: Supervise both the requested change and the available source evidence. Training triplets pair a source image with a modification instruction and a target caption, providing a modification target together with a real visual teacher. The diagram characterizes supervision signals, not measured failure rates of all methods in each family.

Figure 2: PACT. (a) We construct ITT training data from a source image, modification instructions, target captions, and optional negative captions, without target images. (b) During training, target captions supervise the composition, source images provide visual supervision for information shared across modifications, and negative captions discourage failed modifications. (c) At inference, Chord scoring combines query–candidate similarity with reference-relative change for retrieval.

Method	CIRR test			CIRCO test			FashionIQ val
	R@1	R@5	R@10	mAP@5	mAP@10	mAP@25	Avg R@10	Avg R@50
Pic2Word (CVPR 2023)	23.90	51.70	65.30	8.72	9.51	10.64	24.70	43.70
SEARLE-XL (ICCV 2023)	24.24	52.48	66.29	11.68	12.73	14.33	25.56	46.23
Context-I2W (AAAI 2024)	25.60	55.10	68.50	13.00	14.60	16.10	27.80	48.90
CIReVL (GPT-3.5) (ICLR 2024)	24.55	52.31	64.92	18.57	19.01	20.89	28.55	48.57
LinCIR (CVPR 2024)	25.04	53.25	66.68	12.59	13.58	15.00	26.28	46.49

Table 1: CIRR/CIRCO test and FashionIQ validation, ViT-L/14. Published baselines and official PACT test results; FashionIQ reports category-average recall.

Method	Focus			Change			Attribute			Object
	R@1	R@2	R@3	R@1	R@2	R@3	R@1	R@2	R@3	R@1	R@2	R@3
Pic2Word (CVPR 2023)	13.36	24.84	34.66	13.95	24.67	34.51	14.93	26.27	35.87	12.37	23.24	33.29
SEARLE-XL (ICCV 2023)	14.29	25.39	36.09	14.36	25.00	33.67	16.81	27.56	37.59	11.84	22.83	32.17
CIReVL (GPT-3.5) (ICLR 2024)	15.90	26.80	36.25	15.80	27.45	36.40	16.95	28.90	38.60	14.75	25.35	34.05
LinCIR (CVPR 2024)	14.55	26.18	35.71	15.08	26.77	36.53	16.37	29.17	38.61	13.27	23.78	33.62
RTD (ICCV 2025)	15.38	26.17	36.61	15.86	27.38	36.74	17.47	29.42	39.91	13.78	24.13	33.44

Table 2: GeneCIS validation, ViT-L/14. Local and published baseline results; – denotes unreported metrics.

Chord scoring
Training configuration	CIRR	CIRCO	FIQ	Mean
	R@1	mAP@5	R@10
PACT (full)	32.50	24.18	32.43	29.70
$\mathcal{L}_{\rm compose}$ only	29.47	23.80	29.19	27.49
$\mathcal{L}_{\rm preserve}$ only	28.58	16.12	29.45	24.72
$\mathcal{L}_{\rm compose}+\alpha\mathcal{L}_{\rm visual}$	30.40	23.64	30.47	28.17

Table 3: Ablation of PACT training objectives and retrieval scoring. Validation performance with ViT-L/14. The mean averages CIRR R@1, CIRCO mAP@5, and FashionIQ R@10. The cosine comparison uses the same checkpoints as the corresponding Chord results.

Figure 3: Chord scoring improves retrieval and resolves ambiguous cosine rankings. (a) Mean improvement over cosine across CIRR R@1, CIRCO mAP@5, and FashionIQ R@10, using one validation-selected

\lambda

per model. (b) Correction rate for positive–candidate pairs misordered by cosine similarity at a common

\lambda=0.34

. Q1–Q5 correspond to decreasing cosine deficits; shading denotes 95% reference-cluster bootstrap intervals. Appendix D provides full details.

Figure 4: Performance versus model size. PACT remains strong with a compact inference model.

Figure 5: Reference-dependent retrieval on MovieNet. Selected top-1 disagreements between PACT and RTD. PACT better preserves the referenced identity while following scene- and background-level modifications.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Value
Identity / joint updates	333 / 1,000
Composition / preservation batch size	512 / 512
Replay microbatch size	32
Warmup mapper learning rate	$10^{-4}$
Joint mapper / text LoRA learning rate	$3\times 10^{-5}$ / $10^{-5}$
Weight decay	0.001

Appendix

Table 5: ViT-L/14 training configuration.

Method	Evaluation source
Pic2Word ( Saito et al., 2023 )	Local evaluation of the authors’ released checkpoint
SEARLE-XL ( Baldrati et al., 2023 )	Local evaluation of the authors’ released checkpoint
LinCIR ( Gu et al., 2024 )	Local evaluation of a released checkpoint
RTD ( Byun et al., 2025 )	Local evaluation of a checkpoint reproduced with official code
CIReVL ( Karthik et al., 2024 )	Summaries computed from published task-wise scores
MoTa-Adapter ( Li et al., 2026 )	Published R@1 summary (16.7)

Appendix

Table 6: Sources of GeneCIS baseline results.

Benchmark and metric	ViT-B/32	ViT-L/14	ViT-G/14
CIRR R@1	0.268	0.386	0.319
CIRCO mAP@5	0.344	1.072	0.592
FashionIQ category-average R@10	0.115	0.340	0.327
FashionIQ category-average R@50	0.298	0.476	0.395
GeneCIS four-task mean R@1	0.144	0.384	0.434

Appendix

Table 7: Standard deviation across five training seeds. Validation results with Chord scoring. Values are sample standard deviations, in percentage points.

Benchmark	Metric	ViT-B/32	ViT-G/14
CIRR test	R@50	$92.34\pm 0.19$	$95.68\pm 0.05$
	Subset R@1	$63.46\pm 0.65$	$70.46\pm 0.43$
	Subset R@2	$82.42\pm 0.38$	$87.22\pm 0.35$
	Subset R@3	$91.24\pm 0.16$	$93.61\pm 0.17$
CIRCO test	mAP@50	$19.94\pm 0.23$	$35.70\pm 0.75$

Appendix

Table 8: Additional PACT retrieval metrics across five training seeds. Chord scoring; means are in percent and sample standard deviations are in percentage points.

Category	R@10	R@50
Dress	$27.86\pm 0.49$	$49.70\pm 0.42$
Shirt	$34.15\pm 0.53$	$52.25\pm 0.20$
Toptee	$35.28\pm 0.30$	$54.94\pm 0.89$
Category average	$32.43\pm 0.34$	$52.29\pm 0.48$

Appendix

Table 9: FashionIQ validation across five training seeds. ViT-L/14 with Chord scoring; mean

\pm

sample standard deviation, in percentage points.

Method	CIRR test			CIRCO test			FashionIQ val
	R@1	R@5	R@10	mAP@5	mAP@10	mAP@25	Avg R@10	Avg R@50
Pic2Word (2023) †	13.64	37.45	52.22	2.85	3.24	3.89	11.73	26.30
SEARLE (2023) †	23.71	53.30	66.84	8.90	9.42	10.64	22.64	41.85
Context-I2W (2024)	–	–	–	–	–	–	–	–
CIReVL (2024)	23.94	52.51	66.00	14.94	15.42	17.00	28.29	49.35
LinCIR (2024) †	18.87	45.66	58.43	6.25	6.74	7.62	18.14	36.20

Appendix

Table 10: CIRR/CIRCO test and FashionIQ validation, ViT-B/32. PACT cells show the five-seed mean above

\pm

sample standard deviation (%p). Bold marks the highest mean in each column; – indicates no available result for this backbone and metric.

Method	CIRR test			CIRCO test			FashionIQ val
	R@1	R@5	R@10	mAP@5	mAP@10	mAP@25	Avg R@10	Avg R@50
Pic2Word (2023) ‡	30.41	58.12	69.23	5.54	5.59	6.68	31.28	51.89
SEARLE (2023) ‡	34.80	64.07	75.11	13.20	13.85	15.32	34.81	55.71
Context-I2W (2024)	–	–	–	–	–	–	–	–
CIReVL (2024)	34.65	64.29	75.06	26.77	27.59	29.96	32.19	52.36
LinCIR (2024) ‡	35.25	64.72	76.05	19.71	21.01	23.13	45.11	65.69

Appendix

Table 11: CIRR/CIRCO test and FashionIQ validation, ViT-G/14. PACT cells show the five-seed mean above

\pm

sample standard deviation (%p). Bold marks the highest mean in each column; – indicates no available result for this backbone and metric.

Method	Average R@1	Average R@2	Average R@3
Pic2Word (2023)	11.13	21.08	31.05
SEARLE (2023)	12.19	22.56	32.03
CIReVL (2024) ‡	15.85	26.83	36.78
LinCIR (2024)	12.23	21.29	30.80
Pic2Word + RTD (2025)	12.03	21.61	31.09
SEARLE + RTD (2025)	12.82	22.97	32.44

Appendix

Table 12: GeneCIS four-task averages with ViT-B/32. RTD-evaluated baselines and a separately sourced CIReVL row. PACT uses Chord scoring and five-seed means. Bold marks column maxima; – indicates no available result for this backbone and metric.

Method	Focus Attribute			Change Attribute			Focus Object			Change Object
	R@1	R@2	R@3	R@1	R@2	R@3	R@1	R@2	R@3	R@1	R@2	R@3
Pic2Word (2023)	12.45	23.40	33.65	11.74	21.88	30.87	9.90	19.34	27.35	8.57	18.16	26.12
SEARLE (2023)	16.30	29.40	40.70	16.15	27.32	35.46	10.77	18.16	27.91	8.27	15.56	25.77
CIReVL (2024)	20.50	34.00	44.50	16.10	28.60	39.40	14.70	25.20	33.00	18.10	31.20	41.00
LinCIR (2024)	19.05	33.00	42.30	17.57	30.16	38.07	10.10	19.08	28.06	7.91	16.33	25.71
RTD (2025)	–	–	–	–	–	–	–	–	–	–	–	–

Appendix

Table 13: GeneCIS task-level results with ViT-G/14. Published baselines and five-seed PACT means. Bold indicates the highest value in each column among the compared methods; – indicates no available task-level result.

Model	$\lambda$	CIRR R@1	CIRCO mAP@5	FashionIQ R@10
Pic2Word	0.16	$22.82\to 23.44$	$8.10\to 9.02$	$25.56\to 25.76$
LinCIR	0.34	$25.09\to 27.36$	$10.65\to 12.90$	$26.05\to 27.30$
RTD	0.08	$27.12\to 27.77$	$13.23\to 14.03$	$29.93\to 29.83$
PACT	0.34	$25.76\to 32.50$	$18.68\to 24.18$	$28.29\to 32.43$

Appendix

Table 14: Cosine and Chord retrieval on validation data. Entries show cosine

\rightarrow

Chord with one selected coefficient per model. PACT corresponds to the full-model rows in Table 3 .

Figure 6: Chord pair correction by benchmark. Shared cosine-deficit quintiles and rank standardization follow the procedure described above. All models use

\lambda=0.34

. Q5 contains the smallest deficits; shading denotes 95% reference-cluster bootstrap intervals.

	Identity-preservation evaluation	Qualitative comparison
Queries	200	1,151
Distinct reference images	200	90
Films represented in queries	62	4
Gallery images	6,162	100,000
Films represented in gallery	66	104

Appendix

Table 15: MovieNet evaluation sets. Query-film counts refer to the films represented by the references; gallery-film counts refer to the retrieval collections.

Figure 7: MovieNet Top-1 comparisons S1–S2. S1: composition-only retrieves a different character in a pale outfit. S2: it matches the beside-a-woman relation but loses the reference character. PACT satisfies both requests. “Compose-only” denotes $\mathcal{L}_{\rm compose}$ -only training.

Figure 8: MovieNet Top-1 comparisons S3–S4. S3: composition-only matches the bright sky with a different character. S4: it retrieves a portrait instead of mirror reflections. PACT satisfies both requests. “Compose-only” denotes $\mathcal{L}_{\rm compose}$ -only training.

Figure 9: MovieNet Top-1 comparisons S5–S6. S5: PACT retrieves the requested pale pillow behind the head. S6: both composition-only and PACT retrieve the reference character beside a damaged car. “Compose-only” denotes $\mathcal{L}_{\rm compose}$ -only training.

Figure 10: MovieNet Top-1 comparisons S7–S8. Both composition-only and PACT satisfy the ornate-room (S7) and work-table (S8) requests. “Compose-only” denotes $\mathcal{L}_{\rm compose}$ -only training.

Figure 11: MovieNet failures F1 and F2. In F1, both methods retrieve the same martial-arts frame, but the requested two-fist pose is not visible. In F2, both recover suit and interior cues without the action of looking down at a wrist. Valid targets are illustrative gallery positives, not unique correct frames. All model results are Top-1.

Explore similar work

DiCE-CIR: Direct Composition Learning for Efficient Zero-Shot Composed Image Retrieval
Jul 6, 2026Gwang-Ho Na, Ho-Joong Kim, Seong-Whan LeeCross-Modal Alignment Compositional Representation Learning
Department of Artificial Intelligence, Korea University, Anam-dong, Seongbuk-ku, Seoul 02841, Korea
From Transformation to Target State: Rethinking Query Representation for Zero-Shot Composed Image Retrieval
Oct 5, 2026Yihe Zhao, Songhe FengCross-Modal Alignment Composed Image Retrieval
School of Computer Science and Technology, Beijing Jiaotong University, Beijing, 100044, China.
HABIT: Chrono-Synergia Robust Progressive Learning Framework for Composed Image Retrieval
Apr 20, 2026Zixu Li, Yupeng Hu, Zhiwei Chen +4Learning with Noisy Labels Composed Image Retrieval
School of Software, Shandong University

Jul 6, 2026, cs.CVJ/K move · Enter open · S save

DiCE-CIR: Direct Composition Learning for Efficient Zero-Shot Composed Image Retrieval

Gwang-Ho Na, Ho-Joong Kim, Seong-Whan Lee

Department of Artificial Intelligence, Korea University, Anam-dong, Seongbuk-ku, Seoul 02841, Korea

Cross-Modal Alignment Compositional Representation Learning Composed Image Retrieval

Open paper AskPDF arXiv

Zero-shot composed image retrieval (ZS-CIR) aims to retrieve a target image from a multimodal query consisting of a reference image and an edit text describing the desired modification. Recent ZS-CIR studies have relied on projection-based methods that map a reference image into pseudo-word tokens in the text embedding space. However, such methods require additional projection and re-encoding steps, increasing training complexity, reducing efficiency, and introducing a discrepancy between training and inference. In this paper, we propose DiCE-CIR, a direct composition learning method that predicts composed query representations by directly composing a reference image and an edit text. To enable scalable training without manually annotated triplets, we automatically construct compositional training samples from large-scale image-caption pairs using a large language model. Based on these samples, we train a lightweight composition module with objectives that promote alignment with the target, edit-consistent semantic transformation, and retrieval discriminability. We conduct extensive experiments on ZS-CIR benchmarks and show that DiCE-CIR achieves state-of-the-art performance on CIRCO and competitive performance on CIRR while maintaining high computational efficiency.

Abstract

Explore similar work

DiCE-CIR: Direct Composition Learning for Efficient Zero-Shot Composed Image Retrieval

From Transformation to Target State: Rethinking Query Representation for Zero-Shot Composed Image Retrieval

HABIT: Chrono-Synergia Robust Progressive Learning Framework for Composed Image Retrieval