Predicting product quality from multimodal item information is critical in cold-start scenarios, where user interaction history is unavailable and predictions must rely on images and textual metadata. However, existing vision-language models typically depend on large architectures and/or extensive external datasets, resulting in high computational cost. To address this, we propose EffiMiniVLM, a compact dual-encoder vision-language regression framework that integrates an EfficientNet-B0 image encoder and a MiniLM-based text encoder with a lightweight regression head. To improve training sample efficiency, we introduce a weighted Huber loss that leverages rating counts to emphasize more reliable samples, yielding consistent performance gains. Trained using only 20% of the Amazon Reviews 2023 dataset, the proposed model contains 27.7M parameters and requires 6.8 GFLOPs, yet achieves a CES score of 0.40 with the lowest resource cost in the benchmark. Despite its small size, it remains competitive with significantly larger models, achieving comparable performance while being approximately 4x to 8x more resource-efficient than other top-5 methods and being the only approach that does not use external datasets. Further analysis shows that scaling the data to 40% alone allows our model to overtake other methods, which use larger models and datasets, highlighting strong scalability despite the model's compact design.
Figures & tables
Figure 1: Model architecture of the proposed EffiMiniVLM.
Method
Model Size (M)
Train PLCC
Valid PLCC
Baseline
27.7
0.3336
0.3205
Weighted Huber Loss
27.7
0.3649
0.3306
Table 1: Effect of weighted Huber loss on EffiMiniVLM
Data Size
Model
PLCC
CES
Workshop development set
195k (1%)
MiniLM-L6-H384
0.30
0.33
470k (5%)
MiniLM-L6-H384
0.32
0.36
871k (10%)
MiniLM-L6-H384
0.33
0.36
Workshop test set
1.6M (20%)
MiniLM-L6-H384
0.35
0.39
Table 2: Performance comparison of different data scales and text encoders on the workshop development and test sets
Metric
Value
Total Parameters (M)
27.7
Active Parameters (M)
27.7
FLOPs (G)
6.8
Hardware
Dell Pro Max 16 (NVIDIA RTX PRO 2000 Blackwell GPU)
Batch size
32
Timing scope
End-to-end
Table 3: Efficiency metrics of EffiMiniVLM
Figure 2: Extrapolation of the scaling behaviour with increasing training data
Model
Data Used
Custom Val PLCC
Custom Val CES
Test PLCC
Test CES
MiniLM-L6-H384-uncased
871K (10%)
0.3511
0.3862
0.33
0.36
MiniLM-L6-H384-uncased
1.6M (20%)
0.3635
0.3999
0.35
0.39
MiniLMv2-L6-H384-distilled-from-BERT-Large
1.6M (20%)
0.3648
0.4013
0.36
0.40
MiniLMv2-L6-H384-distilled-from-BERT-Large
3.3M (40%)
0.3884
0.4274
–
–
Table 4: Post-Challenge Exploration of the Impact of Dataset Scaling on a Custom Train/Validation Split
Vision-language models (VLMs) are increasingly attractive for multimodal quality assessment, but their default reliance on autoregressive text generation and dynamic visual processing is poorly matched to scalar regression under strict latency budgets. We present a bounded-compute adaptation of SmolVLM2-256M-Video-Instruct for product-rating prediction in the LoViF 2026 Efficient VLM challenge. Motivated by recent multimodal engagement-prediction results showing that feature-based regression can outperform token-based score generation, we replace the language-modeling head with a lightweight two-layer MLP fed by pooled decoder states, and we enforce deterministic inputs through fixed 384x384 images and truncated metadata. Across controlled ablations, static global image processing slightly outperforms dynamic tiling, and scaling from 100K to 16M training examples substantially improves validation correlation. Under the official held-out evaluation, our 228M-parameter model achieves 0.39 PLCC and 0.40 CES, providing a strong and reproducible baseline for resource-constrained multimodal regression.
Deploying vision-language models (VLMs) in resource-constrained settings demands low latency and high throughput, yet existing compact VLMs often fall short of the inference speedups their smaller parameter counts suggest. To explain this discrepancy, we conduct an empirical end-to-end efficiency analysis and systematically profile inference to identify the dominant bottlenecks. Based on these findings, we develop optimization recipes tailored to compact VLMs that substantially reduce latency while preserving accuracy. These techniques cut time to first token (TTFT) by 53% on InternVL3-2B and by 93% on SmolVLM-256M. Our recipes are broadly applicable across both VLM architectures and common serving frameworks, providing practical guidance for building efficient VLM systems. Beyond efficiency, we study how to extend compact VLMs with structured perception outputs and introduce the resulting model family, ArgusVLM. Across diverse benchmarks, ArgusVLM achieves strong performance while maintaining a compact and efficient design.
Vision-language modeling is rapidly increasing in popularity with an ever expanding list of available models. In most cases, these vision-language models have parameters in the tens of billions, which is necessary for some needs, but in many cases smaller models are necessary (e.g., on edge devices or independent robotic platforms). Unfortunately, there is little research in producing light-weight models or in training them with small datasets. Inspired by the language learning progression and data sparsity in child development, in this paper, we address both of these goals in a systematic fashion. We show that two-tower encoder models are superior to one-tower encoders in low-resource settings for discriminative English tasks. We show also that incorporating traditional convolutional networks into the two-tower transformer architecture can help produce parameter efficient vision-language models. Finally, we show that the cross-modal fusion module of two-tower encoders can vary significantly in shape and size while producing the same results. In addition, we present ESsEN, a compact vision-language model that can be trained end-to-end with relatively few resources that performs as well on several tasks with only a fraction of the parameters compared to other models. The experimental results and the tools we present here make vision-language modeling more accessible to a wider variety of researchers.
Clayton Fields, Casey Kennington
Department of Computer Science · Boise State University