Early-stage product design retrieval often requires more than category recognition: designers may need reference examples that match both a short semantic intent and a rough structural cue. Existing product-image resources and generic image--text retrieval benchmarks rarely separate category retrieval from within-category structural fit. We present Sketch2Inspire, built from a curated subset of Amazon Berkeley Objects with aligned text queries, edge-based sketch-proxy queries, and fused text--sketch queries. The resource separates broad category-level retrieval from structure-sensitive within-category retrieval and includes a human-graded reference protocol for calibration. We evaluate a lightweight reference system based on pretrained CLIP-family encoders, comparing text-only retrieval, sketch-only retrieval, weighted late fusion, and text-first reranking without updating model weights. Under broad relevance, late fusion obtains the highest score (nDCG = 0.9962). Under automatic structure-sensitive relevance, late fusion again obtains the highest score (nDCG = 0.7015), exceeding text-only retrieval (nDCG = 0.5912). In the human-graded results, late fusion obtains the highest nDCG@10 (0.9133), while text-only retrieval ranks second (0.9030). These results show that the retrieval gain from multimodal input depends on how relevance is defined. Sketch2Inspire therefore provides a diagnostic resource for evaluating modality contribution and supports the development of structure-aware product-retrieval protocols with independent human annotation.
Figures & tables
Figure 1: Sketch2Inspire overview. The evaluation resource pairs controlled text and sketch-proxy queries with a curated product gallery, then evaluates text-only, sketch-only, late-fusion, and reranking retrieval under broad category-level, structure-sensitive, and human-graded protocols.
Setting
Cats.
Img./cat.
Total
Query coverage
Expanded Setting
300
30
9000
text / sketch / fusion
Structure-Sensitive Setting
260
15
3900
text / sketch / fusion
Table 1: Evaluation resources used in Sketch2Inspire. The human-graded protocol is reported separately from the automatic structure-sensitive setting.
Method
Recall@5
Recall@10
mAP
nDCG
text-only
0.2500
0.4983
0.9839
0.9698
sketch-only
0.2117
0.3983
0.7686
0.8984
late fusion ( α=0.6 )
0.2417
0.4683
0.9047
0.9962
Table 2: Automatic results on Expanded Setting under broad category-level relevance.
Setting
Method
Recall@5
Recall@10
mAP
nDCG
category-level
text-only
0.3500
0.6881
0.9606
0.9866
category-level
sketch-only
0.3048
0.5810
0.8132
0.9117
category-level
late fusion
0.3548
0.6810
0.9548
0.9858
category-level
text-first reranking
0.3452
0.6571
0.9160
0.9581
fine-grained structural
text-only
0.3778
0.7667
0.3902
0.5912
fine-grained structural
sketch-only
0.5889
0.7333
0.5070
0.6785
Table 3: Automatic results on Structure-Sensitive Setting. The late-fusion row uses the highest-scoring fine-grained structural setting ( α=0.40 ), and text-first reranking uses a text candidate pool of 20 images.
Variant
Recall@5
Recall@10
mAP
nDCG
α=0.05
0.4111
0.8000
0.4092
0.6035
α=0.10
0.4556
0.8444
0.4374
0.6276
α=0.20
0.5111
0.9000
0.4519
0.6333
α=0.30
0.5778
0.9222
0.4821
0.6613
α=0.40
0.6111
0.9000
0.5290
0.7015
α=0.50
0.6222
0.8778
0.5252
0.6962
Table 4: Alpha sweep on Structure-Sensitive Setting under fine-grained structural relevance.
Method
manual nDCG@10
manual mAP@10
manual Recall@5
text-only
0.9030
0.9479
0.5336
sketch-only
0.8888
0.8945
0.4722
late fusion
0.9133
0.9196
0.5206
Table 5: Results on Human-Graded Reference Protocol. This human-graded evaluation is reported as a separate manual check and is not part of the automatic structure-sensitive protocol.
Figure 2: Qualitative retrieval comparison on representative cases from Structure-Sensitive Setting. Each row shows a sketch-proxy query and text prompt, followed by top retrieved examples from text-only retrieval, sketch-only retrieval, late fusion, and text-first reranking. Borders indicate stored proxy annotations used for visualization. The figure illustrates why text can dominate broad category matching while sketch cues can help with within-category structural drift.