Class labels offer compact supervision for describing cultural-heritage images when expert text annotations are scarce. We investigate a batik-captioning cascade that integrates a LoRA-adapted DINOv2 classifier with Qwen2.5-VL-7B, adapted using LoRA and loaded in 4-bit NF4. A 3×2 evaluation combines three captioner conditions with two inference prompts to examine adaptation and predicted-class injection. Across 150 caption-evaluation images, all three injected conditions achieve a ground-truth class-label mention rate of 0.9067. The classifier's separate test yields 96.10% pooled accuracy, with 98.33% on 300 public-source images and 75.76% on 33 local images. Supported-class macro-F1 and balanced accuracy on the local subset are 0.8493 and 0.8302, respectively. The classifier and captioner use separate partitions, and their training--evaluation overlap remains unquantified. Differences in label semantics and training support also affect interpretation of the source-specific accuracy gap. Reference-based caption metrics and image--text compatibility scores produce different model rankings. Qualitative examples show both retention of correctly predicted identities and propagation of classification errors into fluent descriptions. These findings characterize how class conditioning preserves label information and transfers upstream errors into batik captions under scarce text annotation.
Figures & tables
Figure 1: Batik images and generated caption excerpts from Dataset A. Bold class names indicate the model’s cultural attribution. The Malang identity in (a) agrees with the annotation; the Trenggalek identities in (b) and (c) conflict with the annotated Tulungagung and Lamongan classes. All outputs use the captioner adapted with class labels and a predicted-class inference prompt (CIFT-CI, defined in Section 4.2 ).
Table 1: Data allocation and supervision for the classifier and captioner. The two components use separately constructed partitions.
Figure 2: Batik-captioning cascade with optional class injection. The classifier prediction provides contextual information to the captioner and forms a pathway for downstream identity errors.
Subset
Test images
Correct
Accuracy
Pooled
333
320
0.9610
B
300
295
0.9833
A
33
25
0.7576
Table 2: Classifier accuracy by source. Both image sources are represented during training, and the source subsets use different cultural-label inventories.
True label
Support
Precision
Recall
F1
Lamongan
10
0.8889
0.8000
0.8421
Malang
16
0.9167
0.6875
0.7857
Trenggalek
6
0.7143
0.8333
0.7692
Tulungagung
1
1.0000
1.0000
1.0000
Supported-class mean
33
0.8800
0.8302
0.8493
Predictions to B labels: Betawi 1, Dayak 1, Parang 1, Sekarjagad 1.
Table 3: Dataset-A metrics computed from the confusion matrix. The averaging set comprises the four supported regional classes; the final rows summarize predictions into other labels and the total error count.
Baseline
PFT
CIFT
Metric
NCI
CI
NCI
CI
NCI
CI
BLEU-1
.4531
.4438
.6058
.4490
.4444
.4447
BLEU-4
.2004
.2639
.4415
.2675
.2020
.2698
ROUGE-L
.4296
.4674
.6775
.4696
.4218
.4761
METEOR
.5419
.5502
.7307
.5781
.5424
.5738
SPICE
.3152
.3701
.5249
.3761
.3160
.3714
Table 4: Reference-based caption metrics on 150 Dataset-A images. Columns compare captioner adaptation and inference-time class injection under the evaluation protocol in Section 5 .
Baseline
PFT
CIFT
Diagnostic
NCI
CI
NCI
CI
NCI
CI
MMR
.0000
.9067
.3800
.9067
.0000
.9067
CLIPScore
.3396
.3375
.3233
.3303
.3393
.3310
Table 5: Class-label mention and image–text compatibility on the caption-evaluation set. MMR measures ground-truth class-string inclusion; CLIPScore is shown in the evaluation’s numerical scale.
Figure 3: Generated caption excerpts across three model conditions. Tasik Madu illustrates correct identity retention under CI and unsupported visual nouns under PFT-NCI. Bunga Nirwana 2 and Melati Deret illustrate propagation of an incorrect Trenggalek prediction. The cases are selected to illustrate these mechanisms.
Memory (GB)
NF4
BF16 / hybrid
Model allocation
5.4946
9.6551
Peak allocation
7.2415
13.1161
Numerical status
Completed
NaN failure
NF4 latency, mean ± SD seconds per image:
Image I/O
0.2303±0.1026
Classifier
0.6225±0.3110
Table 6: Memory allocation and component latency. NF4 completes inference, while the BF16/hybrid configuration encounters a NaN failure. Latency is summarized as mean and standard deviation across 30 images.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Class
P
R
F1
Class
P
R
F1
Betawi
1
1
1
Sidoluhur
1
.87
.93
Bokor Kencono
1
1
1
Sidomukti
1
1
1
Buketan
1
1
1
Sidomulyo
.88
.93
.90
Dayak
.94
1
.97
Singa Barong
1
1
1
Jlamprang
.94
1
.97
Srikaton
1
1
1
Kawung
1
.93
.97
Tribusono
1
1
1
Appendix
Table 7: Dataset-B precision, recall, and F1 at the two-decimal precision of the classification report. Each class contains 15 test images.