cs.CVMar 5, 2026
SaveNaiLIA: Multimodal Nail Design Retrieval Based on Dense Intent Descriptions and Palette Queries
Organizations: Keio University
Abstract
We focus on the task of retrieving nail design images based on dense intent descriptions, which represent multi-layered user intent for nail designs. This is challenging because such descriptions specify unconstrained painted elements and pre-manufactured embellishments as well as visual characteristics, themes, and overall impressions. In addition to these descriptions, we assume that users provide palette queries by specifying zero or more colors via a color picker, enabling the expression of subtle and continuous color nuances. Existing vision-language foundation models often struggle to incorporate such descriptions and palettes. To address this, we propose NaiLIA, a multimodal retrieval method for nail design images, which comprehensively aligns with dense intent descriptions and palette queries during retrieval. Our approach introduces a relaxed loss based on confidence scores for unlabeled images that can align with the descriptions. To evaluate NaiLIA, we constructed a benchmark consisting of 10,625 images collected from people with diverse cultural backgrounds. The images were annotated with long and dense intent descriptions given by over 200 annotators. Experimental results demonstrate that NaiLIA outperforms standard methods.
Figures & tables
Figure 1: A typical use case for our task. The user inputs a dense intent description and an optional palette query. The palette query allows the user to select zero or more colors through a color picker interface. The model should rank the leftmost image higher than the middle image (painted in a darker purple than specified by the palette query) and the rightmost image (featuring realistic shell ornaments rather than the intended shell-inspired design). Details of this task are provided in Section 3.1 .
Figure 2: Architecture of NaiLIA. A language-palette representation is extracted from and by the Intent-Palette Fusion Module, and a visual representation is extracted from by the Visual Design Fusion Module. In the Confidence-based Relaxed Alignment Module, unlabeled positives are assigned, and the loss is calculated considering the unlabeled positives. Here, Enc. denotes the encoder.
Figure 3: Examples of estimated by the MLLM for the following : “Please put long fake nails on my nails and make them pink only at the base and the rest should be a fancy design with strawberries.” Positives, unlabeled positives, and negatives are framed in green, yellow, and red, respectively.
| Method | w/ | NAIL-STAR | Marqo Fashion200K | ||||
|---|---|---|---|---|---|---|---|
| MRR↑ [%] | R@1↑ [%] | R@10↑ [%] | MRR↑ [%] | R@1↑ [%] | R@10↑ [%] | ||
| CLIP [ 46 ] | N | 25.2 0.1 | 15.5 0.3 | 44.4 0.3 | 61.7 0.5 | 47.6 0.7 | 88.8 0.1 |
| FashionViL [ 12 ] | N | 20.8 0.6 | 11.1 0.9 | 40.9 1.4 | 61.7 0.5 | 47.3 0.5 | 89.1 0.5 |
| FAME-ViL [ 13 ] | N | 45.1 0.3 | 31.7 0.7 | 71.8 0.3 | 65.3 0.5 | 51.9 0.8 | 90.5 0.3 |
| BEiT-3 [ 57 ] | N | 53.9 0.4 | 40.6 0.6 | 79.8 0.2 | 66.2 0.8 | 52.8 0.9 | 90.9 0.5 |
| BLIP-2 [ 27 ] | N | 33.3 1.0 | 20.8 0.9 | 58.7 1.2 | 75.3 0.1 | 65.2 0.1 | 94.9 0.1 |
Table 1: Quantitative comparison of NaiLIA and baseline methods on the NAIL-STAR and Marqo Fashion200K [ 38 ] benchmarks. All models were fine-tuned on their respective training sets. Here, w/ indicates whether the model takes palette queries as input. The scores present the average and standard deviation of five trials. The highest score for each metric is in bold , and the second-highest score is underlined . The results in the zero-shot setting are presented in Appendix H.1 .
Figure 4: Qualitative results of the proposed method (NaiLIA) and a baseline method (SigLIP) [ 67 ] for the following and : (a) “I’d like my nails to have a cute, teenage vibe. I’d love a pink base with floral patterns and maybe some character accessories. Can we do a long nail shape?” with (#ffd3e5). (b) “I would like a fairy-tale nail design based on purple and pink.” with (#fff4f4) and (#bfc5ff). (c) “I’d like a colorful and flashy nail design. Please add a large flower nail stone to the ring finger. The tips of the nail tips should be square-shaped.” in the setting without . (d) “red dress with a high waist. the dress is made of a stretchy material and has a flowy skirt. the dress is a formal style and is suitable for a special occasion.” with (#ff1f35). The top-5 retrieved images are shown. Positives and unlabeled positives are enclosed in green and yellow frames, respectively. Examples (a), (b), and (c) show successful cases of NaiLIA on the NAIL-STAR benchmark, whereas (d) is a successful case on the Marqo Fashion200K benchmark. Additional qualitative results, including cases where consists of more than three colors, are presented in Appendix K .
| Model | IPFM | VDFM | CRAM | MRR↑ [%] | R@1↑ [%] | R@10↑ [%] | ||||
| loss | ||||||||||
| (a) | ✓ | ✓ | ✓ | ✓ | ✓ | CRC | 67.6 0.5 | 56.4 0.7 | 88.0 0.3 | |
| (b) | ✓ | ✓ | ✓ | ✓ | CRC | 66.0 0.5 | 54.9 0.7 | 87.1 0.2 | ||
| (c) | ✓ | ✓ | ✓ | ✓ | CRC | 65.6 0.8 | 54.5 0.9 | 87.2 0.6 | ||
| (d) | ✓ | ✓ | ✓ | CRC | 63.0 0.5 | 51.6 0.6 | 84.8 0.5 | |||
| (e) | ✓ | ✓ | ✓ | ✓ | CRC | 66.4 0.4 | 55.1 0.7 | 87.3 0.3 | ||
Table 2: Ablation study on the core modules in our method. The average and standard deviation from five trials are reported. The highest score for each metric is in bold . The additional ablation study is presented in Appendix H.3 .
| Model | MRR↑ [%] | R@1↑ [%] | R@10↑ [%] |
|---|---|---|---|
| CLIP [ 46 ] w/ InfoNCE | 25.2 0.1 | 15.5 0.3 | 44.4 0.3 |
| CLIP [ 46 ] w/ CRC loss | 25.9 0.1 | 16.5 0.3 | 44.8 0.6 |
Table 3: Quantitative comparison of CLIP fine-tuned with InfoNCE vs. CRC loss on the NAIL-STAR benchmark.
| Method | w/ | MRR↑ [%] | R@1↑ [%] | R@10↑ [%] |
|---|---|---|---|---|
| CLIP [ 46 ] | N | 25.2 0.1 | 15.5 0.3 | 44.4 0.3 |
| CLIP [ 46 ] | Y | 29.7 0.2 | 21.3 0.3 | 48.7 0.2 |
| SigLIP [ 67 ] | N | 58.8 0.1 | 47.5 0.1 | 82.1 0.2 |
| SigLIP [ 67 ] | Y | 63.3 0.1 | 53.4 0.1 | 86.2 0.1 |
Table 4: Quantitative results of applying palette queries to CLIP and SigLIP on the NAIL-STAR benchmark.
Figure 5: Examples of estimated by the MLLM in CRAM. Positive, unlabeled positive, and negative labels are framed in green, yellow, and red, respectively, with unlabeled positives identified based on .
Figure 6: Hue histogram of nail design images. We compute the distribution of hue values (HSV space, 0–360°) using all pixels within segmented nail regions in the NAIL-STAR benchmark. To visualize hue independent of saturation and value, the color of each bar corresponds to the bin center with fixed parameters ( ). The vertical axis indicates the pixel proportion relative to the total segmented area across the dataset.
Figure 7: Description annotation interface. The annotators were requested to input a dense intent description for the displayed nail design image into the text box below the image.
Figure 8: Palette query annotation interface. The annotators were requested to specify a palette query for the displayed nail design by using a color picker, selecting any number of colors (0–5).
Figure 9: The word frequency distribution of the NAIL-STAR benchmark. The frequency distributions of the 100 most common nouns, adjectives, and all words are presented. Singular and plural forms of nouns, comparative and superlative forms of adjectives, and present and past participles of verbs were counted independently. Stopwords were removed.
| Method | MRR↑ [%] | R@1↑ [%] | R@10↑ [%] |
|---|---|---|---|
| CLIP [ 46 ] | 12.4 | 6.5 | 23.7 |
| FashionViL [ 12 ] | 2.3 | 0.6 | 4.8 |
| FAME-ViL [ 13 ] | 20.2 | 11.2 | 39.3 |
| BEiT-3 [ 57 ] | 34.9 | 23.5 | 57.9 |
| BLIP-2 [ 27 ] | 14.4 | 7.5 | 28.0 |
| SigLIP [ 67 ] | 31.4 | 21.3 | 51.6 |
Table 5: Quantitative results of baseline methods on the NAIL-STAR benchmark in the zero-shot settings. The scores are reported based on a single trial.
| Token Length | # | MRR↑ [%] | R@1↑ [%] | R@10↑ [%] |
|---|---|---|---|---|
| 0–19 | 781 | 57.9 | 45.3 | 82.5 |
| 20–29 | 560 | 60.0 | 48.0 | 81.4 |
| 30–39 | 187 | 68.4 | 58.3 | 89.3 |
| 40–69 | 72 | 78.6 | 69.4 | 94.4 |
| Total | 1600 | 60.8 | 48.9 | 83.4 |
Table 6: Retrieval performance of NaiLIA across subsets grouped by description token length on the NAIL-STAR benchmark, evaluated in the description-only setting without . The scores are reported based on a single trial.
| Model | MRR↑ [%] | R@1↑ [%] | R@10↑ [%] |
|---|---|---|---|
| (a) NaiLIA (full) | 82.5 | 74.5 | 96.6 |
| (b) w/o | 81.6 | 73.3 | 95.7 |
| (c) w/o | 41.1 | 28.5 | 65.7 |
| (d) w/o | 78.8 | 69.7 | 95.1 |
Table 7: Ablation study of VDFM on the Marqo Fashion200K. The scores are reported based on a single trial.
Figure 10: Sensitivity analysis of the CRC loss with respect to the confidence threshold and the balance terms and . For each curve, we varied one hyperparameter over while keeping the other two fixed at their default values ( , , ). The scores are reported based on a single trial.
| Errors Type | Description | #Error (desc-only) | #Error (full) |
| Comprehension | Comprehension errors in patterns and decorative elements | 19 | 20 |
| Comprehension errors in design themes | 17 | 19 | |
| Comprehension errors in color-related expressions | 13 | 6 | |
| Comprehension errors in nail shape and decoration placement | 11 | 11 | |
| Comprehension errors in the correspondence between fingers and design | 5 | 7 | |
| Ambiguity | Ambiguity in color description | 18 | 9 |
Table 8: Number of failure cases in each category for NaiLIA in the description-only and full settings on the NAIL-STAR benchmark. The largest count in each setting is highlighted in bold .
Figure 11: Additional qualitative results of NaiLIA in the description-only setting, where only is provided. The top-5 retrieved images are shown. Positives and unlabeled positives are enclosed in green and yellow frames, respectively.
Figure 12: Additional qualitative results of NaiLIA in the setting where both and are provided. The top-5 retrieved images are shown. Positives and unlabeled positives are enclosed in green and yellow frames, respectively.
Figure 13: Additional qualitative results of NaiLIA in the setting where both and are provided. The top-5 retrieved images are shown. Positives and unlabeled positives are enclosed in green and yellow frames, respectively.
More in this topic
Image Retrieval
Image retrieval finds and ranks images from visual, textual, or combined queries across large and specialized collections.
See all 85 papers