cs.CVOct 22, 2026

Class-Grounded Batik Captioning under Scarce Text Annotation: Label Retention and Source-Specific Error Propagation

Authors: Novanto Yudistira, Muhammad Dzaka Aufa Fadhillah, Irawati Nurmala Sari

Organizations: Universitas Brawijaya, Malang, Indonesia

Abstract

Class labels offer compact supervision for describing cultural-heritage images when expert text annotations are scarce. We investigate a batik-captioning cascade that integrates a LoRA-adapted DINOv2 classifier with Qwen2.5-VL-7B, adapted using LoRA and loaded in 4-bit NF4. A 3×23\times2 evaluation combines three captioner conditions with two inference prompts to examine adaptation and predicted-class injection. Across 150 caption-evaluation images, all three injected conditions achieve a ground-truth class-label mention rate of 0.9067. The classifier's separate test yields 96.10% pooled accuracy, with 98.33% on 300 public-source images and 75.76% on 33 local images. Supported-class macro-F1 and balanced accuracy on the local subset are 0.8493 and 0.8302, respectively. The classifier and captioner use separate partitions, and their training--evaluation overlap remains unquantified. Differences in label semantics and training support also affect interpretation of the source-specific accuracy gap. Reference-based caption metrics and image--text compatibility scores produce different model rankings. Qualitative examples show both retention of correctly predicted identities and propagation of classification errors into fluent descriptions. These findings characterize how class conditioning preserves label information and transfers upstream errors into batik captions under scarce text annotation.

Figures & tables

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Asymmetric Idiosyncrasies in Multimodal Models

    Feb 26, 2026Muzi Tao, Chufan Shi, Huijuan Wang +2T2I GenerationT2I Generation Evaluation

  2. How Do Diffusion Classifiers Decide? A Bias-Centric Evaluation

    Jul 4, 2026Saba Fathi, Fardin Ayar, Maryam Abdolali +3ClassificationZero-Shot Image Classification

  3. Zero-Shot Captioning for Cultural Heritage: Automated Image Analysis of Traditional Indonesian Clothing

    Jun 11, 2026Anugrah Aidin Yotolembah, Novanto Yudistira, Gembong Edhi SetyawanRetrieval-Augmented GenerationImage Captioning