cs.CVMar 2, 2026

Leveraging Model Soups to Classify Intangible Cultural Heritage Images from the Mekong Delta

Authors: Quoc-Khang Tran, Minh-Thien Nguyen, Nguyen-Khang Pham

Organizations: College of Information and Communication Technology, Can Tho University · School of Graduate, Can Tho University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

The classification of Intangible Cultural Heritage (ICH) images in the Mekong Delta poses unique challenges due to limited annotated data, high visual similarity among classes, and domain heterogeneity. In such low-resource settings, conventional deep learning models often suffer from high variance or overfit to spurious correlations, leading to poor generalization. To address these limitations, we propose a robust framework that integrates the hybrid CoAtNet architecture with model soups, a lightweight weight-space ensembling technique that averages checkpoints from a single training trajectory without increasing inference cost. CoAtNet captures both local and global patterns through stage-wise fusion of convolution and self-attention. We apply two ensembling strategies - greedy and uniform soup - to selectively combine diverse checkpoints into a final model. Beyond performance improvements, we analyze the ensembling effect through the lens of bias-variance decomposition. Our findings show that model soups reduces variance by stabilizing predictions across diverse model snapshots, while introducing minimal additional bias. Furthermore, using cross-entropy-based distance metrics and Multidimensional Scaling (MDS), we show that model soups selects geometrically diverse checkpoints, unlike Soft Voting, which blends redundant models centered in output space. Evaluated on the ICH-17 dataset (7,406 images across 17 classes), our approach achieves state-of-the-art results with 72.36% top-1 accuracy and 69.28% macro F1-score, outperforming strong baselines including ResNet-50, DenseNet-121, and ViT. These results underscore that diversity-aware checkpoint averaging provides a principled and efficient way to reduce variance and enhance generalization in culturally rich, data-scarce classification tasks.

Figures & tables

Figure 1: An illustrative example of the challenging nature of this task: class 4 (left) and class 8 (right) share highly similar visual contexts, making them difficult to distinguish.
Figure 2: Overview of CoAtNet architecture. The model comprises five stages, gradually transitioning from convolutional blocks (MBConv) to Transformer blocks (self-attention). Each stage reduces spatial resolution while increasing the number of channels. Image credit: [ 2 ] .
No.Category# Images
1Đờn ca Tài tử Nam Bộ Art of Don ca Tai tu music and song in southern Viet Nam513
2Nghệ thuật Chầm riêng chà pây của người Khmer Khmer Cham rieng cha pay art185
3Nghề dệt chiếu Mat weaving642
4Lễ hội cúng biển Mỹ Long My Long Sea worship festival398
5Nghệ thuật sân khấu Dù Kê của người Khmer Du Ke (or Lakhon Bassac) Theater Arts of the Khmer people404
6Lễ hội Ok Om Bok của người Khmer Khmer Ok Om Bok festival465
Table 1: Details for the ICH-17 dataset
ClassTrainValidationTestTotal
14204152513
21511519185
35255265642
43263240398
53303341404
63803847465
Table 2: Number of images per class in the training, validation, and test sets of the ICH-17 dataset
Model# Params (M)
ResNet-5023.54
DenseNet-1216.97
ViT85.81
CoAtNet-026.68
CoAtNet-272.86
Table 3: Number of parameters for each model
ModelAccuracyPrecisionRecallF1
ResNet-5065.5564.7262.3362.80
DenseNet-12164.3560.3360.3360.02
ViT70.0968.7966.9967.41
Do et al. [ 5 ]65.32 ∗–––
Tran et al. [ 12 ]66.76 ∗–––
CoAtNet-070.6367.6666.1765.98
Table 4: Classification results on the ICH-17 test set. Asterisk ( ∗ ) indicates results reported in the original paper, and – denotes missing results. The values located in the upper right corner of the entries in the rows corresponding to uniform soup and greedy soup indicate the difference compared to the corresponding model without applying model soups.
ClassAccuracyPrecisionRecallF1
bestsoupsbestsoupsbestsoupsbestsoups
188.4688.4680.7085.1988.4688.4684.4086.79
247.3747.3775.0069.2347.3747.3758.0656.25
393.8592.3195.3195.2493.8592.3194.5793.75
422.5025.0028.1233.3322.5025.0025.0028.57
565.8565.8564.2971.0565.8565.8565.0668.35
Table 5: Results for each class of the CoAtNet-2 model, where best corresponds to the highest result among the saved checkpoints (see 4.2 ), and soups corresponds to applying a model soup (either uniform or greedy). The bolded values correspond to the highest value in the column for the respective class.
Figure 3: Comparison of validation and test accuracy (%) between individual CoAtNet-2 models and their combinations via model soups techniques. The ensemble models – greedy soup and uniform soup – outperform all individual models, demonstrating the effectiveness of model soups in enhancing generalization
Figure 4: MDS visualization of model predictions on the validation set. Each point represents a model projected via MDS using cross-entropy-based pairwise distances. The ingredient models (blue) are broadly distributed, while ensemble models – Greedy (red), Uniform (green), and Soft Voting (orange) – cluster near the center. This spatial structure confirms that model soups leverages predictive diversity more effectively than Soft Voting. The value above each point represents the accuracy on the validation set for the corresponding model.
AspectSoft VotingModel Soups
Inference CostHigh (all models active)Low (single averaged model)
Memory FootprintScales with number of modelsConstant (single model)
Deployment SimplicityRequires parallel model storageSimple (one model checkpoint)
Bias ReductionLimitedSlight increase (depends on soup strategy)
Variance ReductionModerate (correlated models)High (diverse checkpoint averaging)
Table 6: Comparison of Model Soups and Soft Voting on key ensemble properties.
ModelAccuracyPrecisionRecallF1
CoAtNet-049.1346.3645.3144.85
CoAtNet-251.2748.8748.2147.81
CoAtNet-2 + uniform52.07↑0.8052.07^{\uparrow 0.80}49.27↑0.4049.27^{\uparrow 0.40}48.82↑0.6148.82^{\uparrow 0.61}47.99↑0.1847.99^{\uparrow 0.18}
CoAtNet-2 + greedy51.80↑0.5351.80^{\uparrow 0.53}48.72↓0.1548.72^{\downarrow 0.15}48.77↑0.5648.77^{\uparrow 0.56}48.11↑0.3048.11^{\uparrow 0.30}
Table 7: Performance of CoAtNet models trained from random initialization on the ICH-17 test set. The values located in the upper right corner of the entries in the rows corresponding to uniform soup and greedy soup indicate the difference compared to the corresponding model without applying model soups .
More in this topic

Image Classification

Image classification assigns semantic labels to images, enabling recognition of objects, scenes, conditions, and fine-grained categories.

See all 390 papers

Explore similar work

CardsList
Aug 13, 2026

MLLM-Routed Heterogeneous Ensembles for Robust Cross-Dataset Image Classification

Modern image classification models excel when trained on single task-specific datasets but often struggle to generalize across domains and difficulty levels. We propose ARMDIL, an Adaptive Router for Multi-Domain Image Classification with LLMs. ARMDIL is an ensemble that uses a multimodal large language model (MLLM) agent to dynamically route each image to the most suitable vision backbone. Our diverse ensemble employs convolutional neural networks (ResNets), self-supervised representation learners (SSL), and vision language models (VLMs), each trained on a unified label space constructed from multiple image datasets with differing distributions and characteristics. Empirical evaluations illuminate the distinct capabilities and vulnerabilities of each architecture across disparate visual domains. Crucially, we show that ARMDIL effectively navigates these tradeoffs, performing competitively with specialized training-based routers. Furthermore, it drastically improves adaptability by allowing new information to be integrated via simple prompt modifications, while enhancing interpretability through natural language reasoning traces. These advances in cross-dataset image classification pave the way for more reliable general-purpose vision systems such as AI assistants and autonomous robots.

Daniel Perkins, John Squires, Janou Milligan +2

University of Tennessee, Knoxville · The Bredesen Center for Interdisciplinary Research and Graduate Education

Adaptive Model RoutingEnsemble LearningCross-Dataset GeneralizationImage Classification
AskPDF
Jul 10, 2026

A Coreset Selection Framework with Ensemble Aggregation for Image Classification

The rapid growth of image data has produced large-scale datasets, raising concerns about the time and memory costs of model training. Selecting representative training subsets, however, remains challenging: individual sample contributions are unclear, and model behavior varies across datasets and runs. We address these challenges with a framework that combines coreset selection with an ensemble aggregation over multiple runs. For coreset selection, we propose SCOre-Stratified Selection (SCOSS), which partitions the training data into intervals based on a chosen score and samples from each interval. The ensemble combines predictions from multiple runs, each performed on an independently sampled training subset. As baselines, we use moderate and random selection, each in original and class-balanced versions. We assess the framework with Simple Graph Convolution (SGC) and Support Vector Machine (SVM) classifiers under different sampling ratios. Experiments show that SCOSS is competitive with baselines, often the best choice for SGC, and enables favorable trade-offs between accuracy and efficiency. On the fine-grained dataset, SGC with SCOSS outperforms SVMs when using fewer labeled samples. The code and supplementary materials are publicly available at http://scoss.lucasvalem.com.

Pedro Rocha Dantas, Lucas Pascotti Valem

Institute of Mathematics and Computer Science (ICMC) University of S˜ao Paulo (USP) S˜ao Carlos – SP – Brazil

Training Data SelectionCoreset SelectionImage Classification
AskPDF
Mar 1, 2026

VP-Hype: A Hybrid Mamba-Transformer Framework with Visual-Textual Prompting for Hyperspectral Image Classification

Accurate classification of hyperspectral imagery (HSI) is often frustrated by the tension between high-dimensional spectral data and the extreme scarcity of labeled training samples. While hierarchical models like LoLA-SpecViT have demonstrated the power of local windowed attention and parameter-efficient fine-tuning, the quadratic complexity of standard Transformers remains a barrier to scaling. We introduce VP-Hype, a framework that rethinks HSI classification by unifying the linear-time efficiency of State-Space Models (SSMs) with the relational modeling of Transformers in a novel hybrid architecture. Building on a robust 3D-CNN spectral front-end, VP-Hype replaces conventional attention blocks with a Hybrid Mamba-Transformer backbone to capture long-range dependencies with significantly reduced computational overhead. Furthermore, we address the label-scarcity problem by integrating dual-modal Visual and Textual Prompts that provide context-aware guidance for the feature extraction process. Our experimental evaluation demonstrates that VP-Hype establishes a new state of the art in low-data regimes. Specifically, with a training sample distribution of only 2%, the model achieves Overall Accuracy (OA) of 99.69% on the Salinas dataset and 99.45% on the Longkou dataset. These results suggest that the convergence of hybrid sequence modeling and multi-modal prompting provides a robust path forward for high-performance, sample-efficient remote sensing.

Abdellah Zakaria Sellam, Fadi Abdeladhim Zidi, Salah Eddine Bekhouche +4

Institute of Applied Sciences and Intelligent Systems (ISASI), CNR, 73100 Lecce, Italy · Dept. of Engineering for Innovation, University of Salento, 73100 Lecce, Italy · VSC Laboratory, Department of Electronics and Automation, University of Biskra, Biskra, Algeria +4

Hybrid Mamba-Transformer ArchitecturesHyperspectral Image ClassificationHyperspectral Imaging
AskPDF