Organizations: Institute of Information Engineering, Chinese Academy of Sciences, Beijing 100093, China · Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD 21218, USA
Training-free zero-shot composed image retrieval finds a target image in a gallery from a reference image and a text edit without learning from task-specific image triplets. Existing methods typically describe the target as a whole and match this description with a global image representation. This global matching can mix different semantic cues and lose fine- grained details. We propose MULVEC, a role-aware method whose compiler produces a structured query record that is mapped to four retrieval roles: Global describes the full target, Desired states what should appear, Preserve states what should remain, and Forbidden states what should disappear. Frozen encoders map the query to one target description vector and role-specific probe vectors, while each candidate is represented by one global visual vector and a bank of local visual vectors. The retrieval roles then use this shared evidence for their respective purposes, and a fixed weighted sum of their scores ranks the entire gallery in a single retrieval pass. Across CIRCO, CIRR, and FashionIQ and three backbone scales, MULVEC improves CIRCO mAP@5 by up to 23.0% over the strongest compared method and gives the best CIRR and FashionIQ results in our comparison.
Figures & tables
Fig. 1: Motivation on CIRR validation query 37730. Left: CoTMR’s global description matching [ 13 ] encodes one precise target description and compares it with one global visual vector per candidate, but its retrieval misses the requested birds and painting style. Right: TAPR factorizes the request into Target, Add, Preserve, and Remove factors, splits their constraints into atoms, and checks factor-specific probes against global and local candidate evidence to retrieve the ground-truth target at rank one.
Fig. 2: The TAPR pipeline on CIRCO validation query 45. Atomic Edit Factorization maps the reference image and modification to a factorized edit specification: Target describes an Arabic stop sign at night, Add specifies Arabic script and nighttime, Preserve keeps the octagonal stop-sign structure, and Remove suppresses English “STOP” and daylight. Frozen encoders produce one target description vector, factor-specific probe vectors, a candidate evidence bank of global and local visual vectors, and reference local visual vectors. Fine-grained multi-vector matching uses a different operator for each factor: Target performs global matching, Add rewards local evidence, Preserve checks reference continuity, and Remove penalizes local evidence. Factor-wise aggregation ranks the gallery; all five displayed results are ground-truth positives.
Backbone
Method
CIRCO
CIRR
mAP@ k
Recall@ k
Recall sub @ k
5
10
25
50
1
5
10
1
2
3
ViT-B
CIReVL [ 4 ] (ICLR’24)
14.94
15.42
17.00
17.82
23.94
52.51
66.00
60.17
80.05
90.19
OSrCIR [ 6 ] (CVPR’25)
18.04
19.17
20.94
21.85
25.42
54.54
68.19
62.31
80.86
91.13
CoTMR [ 13 ] (ICCV’25)
22.23
22.78
24.68
25.74
31.50
60.80
73.04
66.61
84.50
92.55
CIReVL + SoFT [ 15 ] (AAAIW’26)
19.21
20.04
21.86
22.75
32.94
62.92
74.17
70.31
86.58
93.78
TABLE I: Direct single-pass training-free ZS-CIR on the CIRCO and CIRR test sets, both evaluated online with hidden test annotations. Bold and underlined values are the best and second best among the listed rows within each backbone block.
Backbone
Method
Shirt
Dress
Toptee
Average
R@10
R@50
R@10
R@50
R@10
R@50
R@10
R@50
ViT-B
CIReVL [ 4 ] (ICLR’24)
28.36
47.84
25.29
46.36
31.21
53.85
28.29
49.35
OSrCIR [ 6 ] (CVPR’25)
31.16
51.13
29.35
50.37
36.51
58.71
32.34
53.40
CoTMR [ 13 ] (ICCV’25)
33.42
53.93
31.09
54.54
38.40
61.14
34.30
56.54
CIReVL + SoFT [ 15 ] (AAAIW’26)
31.26
50.98
27.52
49.13
34.37
58.44
31.05
52.85
SDR-CIR [ 16 ] (WWW’26)
36.41
57.02
36.84
58.85
43.14
64.71
38.80
60.19
TABLE II: Direct single-pass training-free ZS-CIR on FashionIQ validation. Average is the unweighted category mean. Bold and underlined values are the best and second best among the listed rows within each backbone block.
Fig. 3: Cross-method retrieval examples on CIRR and CIRCO. For CIRR, CoTMR [ 13 ] matches the dog but does not reject each unmet edit separately; TAPR verifies the Add atoms and penalizes Remove evidence for photographic content. For CIRCO, CIReVL [ 4 ] matches the people but not their relation direction; TAPR checks who acts on whom with a factor-specific probe. Red crosses and green checks mark incorrect and correct rank-1 results, respectively.
Candidate evidence
Factor-wise matching
CIRCO mAP@5
CIRR Recall@1
FashionIQ Macro R@10
Global Vector
×
26.71
30.14
24.15
Global Vector
✓
33.79
35.41
40.61
Global + Local
×
31.58
32.78
30.87
Global + Local
✓
37.17
38.28
43.23
TABLE III: Effect of candidate-evidence granularity and factor-wise matching on ViT-L validation.
Fig. 4: One success case and one failure case from the matched control on CIRCO validation. Each row fixes one ground-truth image across all four configuration columns. For CIRCO 60, image 23304 moves from ranks 5 to 2 and from 4 to 1. For CIRCO 22, image 148408 remains outside the Top-50 in every cell.
Harbin Institute of Technology (Shenzhen) Shenzhen, China · Southern University of Science and Technology Shenzhen, China · Pengcheng Laboratory Shenzhen, China +1