Predicting product quality from multimodal item information is critical in cold-start scenarios, where user interaction history is unavailable and predictions must rely on images and textual metadata. However, existing vision-language models typically depend on large architectures and/or extensive external datasets, resulting in high computational cost. To address this, we propose EffiMiniVLM, a compact dual-encoder vision-language regression framework that integrates an EfficientNet-B0 image encoder and a MiniLM-based text encoder with a lightweight regression head. To improve training sample efficiency, we introduce a weighted Huber loss that leverages rating counts to emphasize more reliable samples, yielding consistent performance gains. Trained using only 20% of the Amazon Reviews 2023 dataset, the proposed model contains 27.7M parameters and requires 6.8 GFLOPs, yet achieves a CES score of 0.40 with the lowest resource cost in the benchmark. Despite its small size, it remains competitive with significantly larger models, achieving comparable performance while being approximately 4x to 8x more resource-efficient than other top-5 methods and being the only approach that does not use external datasets. Further analysis shows that scaling the data to 40% alone allows our model to overtake other methods, which use larger models and datasets, highlighting strong scalability despite the model's compact design.
Figures & tables
Figure 1: Model architecture of the proposed EffiMiniVLM.
Method
Model Size (M)
Train PLCC
Valid PLCC
Baseline
27.7
0.3336
0.3205
Weighted Huber Loss
27.7
0.3649
0.3306
Table 1: Effect of weighted Huber loss on EffiMiniVLM
Data Size
Model
PLCC
CES
Workshop development set
195k (1%)
MiniLM-L6-H384
0.30
0.33
470k (5%)
MiniLM-L6-H384
0.32
0.36
871k (10%)
MiniLM-L6-H384
0.33
0.36
Workshop test set
1.6M (20%)
MiniLM-L6-H384
0.35
0.39
Table 2: Performance comparison of different data scales and text encoders on the workshop development and test sets
Metric
Value
Total Parameters (M)
27.7
Active Parameters (M)
27.7
FLOPs (G)
6.8
Hardware
Dell Pro Max 16 (NVIDIA RTX PRO 2000 Blackwell GPU)
Batch size
32
Timing scope
End-to-end
Table 3: Efficiency metrics of EffiMiniVLM
Figure 2: Extrapolation of the scaling behaviour with increasing training data
Model
Data Used
Custom Val PLCC
Custom Val CES
Test PLCC
Test CES
MiniLM-L6-H384-uncased
871K (10%)
0.3511
0.3862
0.33
0.36
MiniLM-L6-H384-uncased
1.6M (20%)
0.3635
0.3999
0.35
0.39
MiniLMv2-L6-H384-distilled-from-BERT-Large
1.6M (20%)
0.3648
0.4013
0.36
0.40
MiniLMv2-L6-H384-distilled-from-BERT-Large
3.3M (40%)
0.3884
0.4274
–
–
Table 4: Post-Challenge Exploration of the Impact of Dataset Scaling on a Custom Train/Validation Split