Model Ensemble

Model ensembles, combining predictions from multiple individual models, aim to improve prediction accuracy, robustness, and uncertainty quantification in diverse machine learning applications. Current research focuses on optimizing ensemble strategies, including exploring diverse model architectures (e.g., U-Nets, Swin Transformers, YOLOv8) and employing techniques like Bayesian optimization and active learning to enhance performance and efficiency. This approach has shown significant impact across various fields, from medical image analysis (e.g., brain tumor segmentation, retinal vascular analysis) and climate modeling to natural language processing and agricultural pattern recognition, demonstrating improved results compared to single-model approaches.

Papers

May 2, 2023

Single-model uncertainty quantification in neural network potentials does not consistently outperform model ensembles
Aik Rui Tan, Shingo Urata, Samuel Goldman, Johannes C. B. Dietschreit, Rafael Gómez-Bombarelli
Uncertainty Quantification Interatomic Potential Atomistic Simulation Model Ensemble Neural Network Potential Based NIDS

March 16, 2023

Rethinking Model Ensemble in Transfer-based Adversarial Attacks
Huanran Chen, Yichi Zhang, Yinpeng Dong, Xiao Yang, Hang Su, Jun Zhu
Adversarial Attack Adversarial Example Model Ensemble Adversarial Prediction Transfer Based Adversarial Attack

February 2, 2023

Is Model Ensemble Necessary? Model-based RL via a Single Model with Lipschitz Regularized Value Function
Ruijie Zheng, Xiyao Wang, Huazhe Xu, Furong Huang
Model Based Reinforcement Learning Model Ensemble Single Model Bellman Operator Lipschitz Regularization

November 12, 2022

Improving Deep Facial Phenotyping for Ultra-rare Disorder Verification Using Model Ensembles
Alexander Hustinx, Fabio Hellmann, Ömer Sümer, Behnam Javanmardi, Elisabeth André, Peter Krawitz, Tzung-Chien Hsieh
Model Ensemble Face Image Datasets Rare Disease Disorder Classification Performance

October 23, 2022

Model ensemble instead of prompt fusion: a sample-specific knowledge transfer method for few-shot prompt tuning
Xiangyu Peng, Chen Xing, Prafulla Kumar Choubey, Chien-Sheng Wu, Caiming Xiong
Full Model Ensemble Learning Knowledge Transfer Prompt Tuning Soft Prompt Model Ensemble Prompt Fusion Shot Prompt Tuning

September 30, 2022

Learning Robust Kernel Ensembles with Kernel Average Pooling
Pouya Bashivan, Adam Ibrahim, Amirozhan Dehghani, Yifei Ren
Adversarial Attack Adversarial Example Model Ensemble Ensemble Kernel

July 4, 2022

Solutions for Fine-grained and Long-tailed Snake Species Recognition in SnakeCLEF 2022
Cheng Zou, Furong Xu, Meng Wang, Wen Li, Yuan Cheng
Fine Grained Solution Path Tail Class Model Ensemble Snake Specie

May 31, 2022

FiLM-Ensemble: Probabilistic Deep Learning via Feature-wise Linear Modulation
Mehmet Ozgur Turkoglu, Alexander Becker, Hüseyin Anil Gündüz, Mina Rezaei, Bernd Bischl, Rodrigo Caye Daudt, Stefano D'Aronco, Jan Dirk Wegner, Konrad Schindler
Epistemic Uncertainty Model Ensemble Probabilistic Deep Learning Feature Wise Linear Modulation

May 20, 2022

SafeNet: The Unreasonable Effectiveness of Ensembles in Private Collaborative Learning
Harsh Chaudhari, Matthew Jagielski, Alina Oprea
Diverse Ensemble Private Learning Model Ensemble Adversarial Machine Learning Secure Multiparty Computation Multi Party Computation

April 13, 2022

CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing
Chen Liang, Pengcheng He, Yelong Shen, Weizhu Chen, Tuo Zhao
Consistency Regularization Model Ensemble Weight Sharing Level Perturbation BERT Base

April 11, 2022

Improved Training of Physics-Informed Neural Networks with Model Ensembles
Katsiaryna Haitsiukevich, Alexander Ilin
Physic Informed Neural Network Partial Differential Equation Model Ensemble PINN Method PINN Model Improved Training

March 25, 2022

Single Model Ensemble for Subword Regularized Models in Low-Resource Machine Translation
Sho Takase, Tatsuya Hiraoka, Naoaki Okazaki
Neural Machine Translation Model Model Ensemble Subword Segmentation Subword Regularization

February 9, 2022

Active Learning Improves Performance on Symbolic RegressionTasks in StackGP
Nathan Haut, Wolfgang Banzhaf, Bill Punch
Active Learning System Performance Symbolic Regression Model Ensemble Feynman Kac

January 1, 2022

AutoDES: AutoML Pipeline Generation of Classification with Dynamic Ensemble Strategy Selection
Yunpu Zhao, Rui Zhang, Xiaqing Li
Classification Code Model Ensemble Ensemble Selection Ensemble Strategy AutoML Pipeline

Model Ensemble

Papers

Single-model uncertainty quantification in neural network potentials does not consistently outperform model ensembles

Rethinking Model Ensemble in Transfer-based Adversarial Attacks

Is Model Ensemble Necessary? Model-based RL via a Single Model with Lipschitz Regularized Value Function

Improving Deep Facial Phenotyping for Ultra-rare Disorder Verification Using Model Ensembles

Model ensemble instead of prompt fusion: a sample-specific knowledge transfer method for few-shot prompt tuning

Learning Robust Kernel Ensembles with Kernel Average Pooling

Solutions for Fine-grained and Long-tailed Snake Species Recognition in SnakeCLEF 2022

FiLM-Ensemble: Probabilistic Deep Learning via Feature-wise Linear Modulation

SafeNet: The Unreasonable Effectiveness of Ensembles in Private Collaborative Learning

CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing

Improved Training of Physics-Informed Neural Networks with Model Ensembles

Single Model Ensemble for Subword Regularized Models in Low-Resource Machine Translation

Active Learning Improves Performance on Symbolic RegressionTasks in StackGP

AutoDES: AutoML Pipeline Generation of Classification with Dynamic Ensemble Strategy Selection