Organizations: Department of Natural Language Processing Mohamed Bin Zayed University of Artificial Intelligence · Department of Computer Science and Engineering Indian Institute of Technology Jodhpur
Fashion Image Captioning (FIC) plays a vital role in enhancing user experience and product search in e-commerce platforms. Unlike natural scene image captioning, FIC requires fine-grained visual reasoning and knowledge of domain-specific terminology to capture subtle attributes such as neckline and closure types, graphic patterns, and dress silhouettes. Moreover, as fashion inventories evolve rapidly with new trends, styles, and frequently emerging vocabulary, developing training-free captioning solution becomes essential for scalability and real-world adaptability. Instruction-tuned vision-language models (VLMs) offer a promising solution to fashion image captioning dueto their strong zero-shot capabilities and natural language fluency. However, these general-purpose models often lack attribute-level coverage and precision, and tend to hallucinate or misidentify fine-grained fashion details, making them less suitable for high-fidelity applications like product cataloging or personalized recommendations. To address this, we propose RA-CoA (Retrieval-Augmented Chain-of-Attributes), a novel, training-free framework that disentangles fashion image captioning into two interpretable stages: (i) retrieval of relevant attribute sets from a product knowledge base, and (ii) attribute-level reasoning to generate the final caption. RA-CoA is a model-agnostic approach that works with frozen VLMs to improve fine-grained attribute precision in product captions without the need for fine-tuning. Extensive evaluations across diverse VLM model families under different prompting paradigms demonstrate that RA-CoA significantly improves caption quality, achieving an average gain of 26.3% METEOR score over zero-shot captioning. We make our code publicly available.
Figures & tables
Figure 1: Task Overview: Given a fashion image and the product name, existing training-free prompting paradigms may fail to provide attribute-rich, coherent, hallucination-free captions. Captioning with (a) Zero-shot: Produces only a generic description, missing key fine-grained attributes. (b) In-context learning Dong et al. (2022) : Improves coherence via exemplars but tends to copy attributes. (c) Chain-of-Thought Wu et al. (2025) : Performs stepwise reasoning yet fails in capturing all fine-grained attributes correctly. (d) RA-CoA (Ours): A retrieval-augmented chain-of-attributes framework that aggregates key attributes from similar catalog items and sequentially predicts their values. The predicted attributes, combined with ICL exemplars, guide caption generation toward coherent and attribute-faithful captions.
Figure 2: Overview of RA-CoA . Ⓐ Retrieval-augmented Chain-of-Attributes - For a query fashion image, we crop the target product region using Florence-2, retrieve top-K similar products from ProductKB, and aggregate unique attributes. Each attribute is paired with the image and product name and fed to VLM for value prediction. Ⓑ Caption Generation - The resulting attribute-value pairs and in-context exemplars (retrieved products) enable the VLM to generate fine-grained coherent captions. ⓒ ProductKB Construction - We construct our ProductKB by extracting structured attribute-value pairs from FashionGen captions using LLaMA-3.2-V-11B.
Model (# params)
Paradigm
BLEU-1
Avg. BLEU
Rouge-1
Rouge-2
Rouge-L
METEOR
LLM-Judge
TinyLLaVA (3B)
Zero-shot
12.3
3.5
17.1
2.4
11.4
12.2
17.2
CoT-i
11.0
3.3
18.6
2.7
12.6
11.0
10.1
CoT-e
7.8
2.3
12.8
1.8
8.7
8.6
8.1
ICL
13.1
4.8
24.5
4.9
18.7
13.7
36.5
ICL+CoT-e
2.9
1.2
9.2
2.3
7.5
7.9
15.2
RA-CoA
22.1
11.50
32.5
10.8
25.2
24.6
50.9
Table 1: Comparison of RA-CoA against different prompting paradigms across VLMs of varying scales. Δ (gray rows) represents the gain of RA-CoA over Zero-shot.
Figure 3: A selection of RA-CoA ’s results. Our proposed approach RA-CoA generates coherent captions with minimal hallucinations that align well with the human-annotated ground truth descriptions.
Model (# params)
Top-K
BLEU-1
Avg. BLEU
Rouge-1
Rouge-2
Rouge-L
METEOR
TinyLLaVA (3B)
1
22.1
11.50
32.5
10.8
25.2
24.6
2
20.0
9.8
30.5
9.4
23.2
22.1
3
18.7
8.9
29.2
8.6
21.9
21.0
Qwen-2.5VL (3B)
1
26.8
14.5
35.3
11.7
28.3
28.6
2
31.3
16.6
37.7
12.6
29.3
31.3
3
25.6
13.1
33.1
9.8
25.1
28.3
Table 2: Ablation study to quantify the impact of top-K retrievals for CoA in our proposed RA-CoA method.
Model (# params)
Method variant
BLEU-1
Avg.BLEU
Rouge-1
Rouge-2
Rouge-L
METEOR
TinyLLaVA (3B)
RA-CoA
22.1
11.50
32.5
10.8
25.2
24.6
RA-CoA w/o ICL exemplars
16.8
5.3
19.8
2.5
12.8
17.9
RA-CoA with Oracle Attributes
22.9
11.2
34.1
10.5
26.4
24.7
RA-CoA with Oracle Attribute-Values
53.9
44.9
74.0
60.4
71.3
71.6
Qwen-2.5VL (3B)
RA-CoA
26.8
14.5
35.3
11.7
28.3
28.6
RA-CoA w/o ICL exemplars
17.7
5.7
22.6
2.8
14.2
19.8
Table 3: Ablation study to (1) quantify the importance of retrieval-augmented ICL exemplars, and (2) quantify the gap of RA-CoA with respect to oracle variants.
Figure 4: Comparison of CLIP-only and Florence-CLIP for retrieval in RA-CoA (Section 3.2.1 ).
Paradigm
Input image
BLEU-1
Avg.BLEU
Rouge-1
Rouge-2
Rouge-L
METEOR
Zeroshot
Cropped Product
9.6
2.7
19.1
2.9
12.6
11.7
CoT-i
13.0
3.8
18.0
2.9
12.6
15.0
CoT-e
7.0
2.1
16.8
2.4
12.4
13.2
ICL
29.8
17.3
35
13.3
28.7
31.6
RA-CoA
39.4
24.3
41.8
18.0
36.7
38.8
RA-CoA
Full Image
38.6
23.7
41.0
17.4
36.0
38.1
Table 4: Comparison of using full image vs. Florence-2–cropped product region as VLM input for captioning. Cropping offers a modest gain while the full image already yields strong results. All results are reported for InternVL2 (8B) as base VLM.
ProductKB size
BLEU-1
Avg.BLEU
ROUGE-1
ROUGE-2
ROUGE-L
METEOR
10K
36.2
21.1
38.8
15.1
33.4
35.5
20K
37.6
22.7
40.3
16.6
35.2
37.1
40K
38.5
23.5
41.2
17.2
35.9
37.9
60K (Full)
38.1
23.7
41.0
17.4
36.0
38.1
Table 5: Effect of ProductKB size on captioning performance using InternVL2-8B within the RA-CoA framework. Performance improves with KB size, but gains saturate beyond 20K entries, indicating that RA-CoA remains data-efficient even with smaller knowledge bases.
% corrupted
BLEU-1
Avg.BLEU
ROUGE-1
ROUGE-2
ROUGE-L
METEOR
0%
38.1
23.7
41.0
17.4
36.0
38.1
25%
37.0
21.8
39.5
16.1
34.5
36.5
50%
36.9
21.3
39.2
15.3
33.9
35.9
75%
35.7
19.9
37.7
13.8
32.4
34.3
Table 6: Effect of noisy ProductKB entries on captioning performance using InternVL2-8B within the RA-CoA framework. Performance decreases with increasing noise in attribute values, suggesting that RA-CoA is relatively robust to corrupted ProductKB entries.
% key-value pairs dropped
BLEU-1
Avg.BLEU
ROUGE-1
ROUGE-2
ROUGE-L
METEOR
0%
38.1
23.7
41.0
17.4
36.0
38.1
25%
34.4
20.5
39.2
16.4
34.3
33.9
50%
28.4
17.1
37.6
16.4
33.1
29.7
75%
21.1
13.1
35.9
16.7
31.6
25.5
Table 7: Effect of sparse ProductKB entries on captioning performance using InternVL2-8B within the RA-CoA framework. Performance degrades as sparsity in the ProductKB increases, as it affects retrieval quality and attribute guidance.
Method
Fine-tuning
BLEU-1
Avg.BLEU
Rouge-1
Rouge-2
ROUGE-L
METEOR
UniFashion
✓
11.4
3.9
16.5
4.1
12.5
16.2
RA-CoA (Ours)
✗
22.0
7.7
26.1
5.7
18.3
20.2
Table 8: Quantitative comparison of RA-CoA with UniFashion Zhao et al. (2024) (prior SOTA supervised method) showcasing real-world generalization of RA-CoA despite being completely training-free.
Method
BLEU-1
Avg.BLEU
Rouge-1
Rouge-2
ROUGE-L
METEOR
LLM-Judge
Zero-shot
4.0
1.4
22.8
2.0
17.4
9.2
65.7
CoT-i
38.6
17.7
36.5
11.2
28.3
27.4
75.1
ICL
71.0
55.0
68.2
44.8
67.5
72.8
34.2
RA-CoA (Ours)
74.5
60.6
74.8
51.3
71.4
73.2
77.2
Table 9: Quantitative comparison of RA-CoA with other VLM paradigms on the Furniture Dataset, showcasing generalization of our approach beyond fashion domain.
Method
Preference (%)
Zero-shot
3.7
CoT-i (Implicit CoT)
6.7
CoT-e (Explicit CoT)
7.0
In-Context Learning (ICL)
14.0
RA-CoA (Ours)
68.6
Table 10: User preference study over 300 unique test images using InternVL2-8B as the backbone. RA-CoA is preferred in 68.6% of cases, reflecting higher perceived coherence, completeness, and usefulness.
Figure 5: Error analysis into RA-CoA ’s generations. The text in red denotes incorrect/hallucinated features. The text in blue in the GT caption indicates features missed in the generated caption.
Step
Time (seconds)
ROI crop using Florence-2
0.30
CLIP embedding extraction for ROI
0.04
Retrieval from ProductKB (Section 3.2.1 )
0.01
Chain-of-attributes (Section 3.2 )
1.40
Attribute-value aware captioning (Section 3.2.4 )
1.80
Average inference time per sample (Total)
3.5
Table 11: Computational latency analysis of RA-CoA ’s components.