Ensemble Learning

Ask about this topic

Overview

Ensemble learning studies how to combine predictions from multiple models, from Random Forests and boosting to stacking, voting, and mixtures of experts. Methods may average outputs, learn weights from validation data, or select and route among specialists for each example, with applications ranging from tabular prediction and medical imaging to weather forecasting and language models. Combining models can reduce errors and quantify uncertainty, but gains depend on whether their mistakes are complementary rather than correlated. Current work also weighs accuracy against calibration, computational cost, distribution shifts, and leakage-safe evaluation, while developing theoretical guarantees for aggregation and boosting.

Latest papers

CardsList

278 papers25 papers in the last four weeks, down 24% on the four weeks before. 0.1% of all new papers.

Mar 19, 2026

An Optimised Greedy-Weighted Ensemble Framework for Financial Loan Default Prediction

Accurate prediction of loan defaults is a central challenge in credit risk management, particularly in modern financial datasets characterised by nonlinear relationships, class imbalance, and evolving borrower behaviour. Traditional statistical models and static ensemble methods often struggle to maintain reliable performance under such conditions. This study proposes an Optimised Greedy-Weighted Ensemble framework for loan default prediction that dynamically allocates model weights based on empirical predictive performance. The framework integrates multiple machine learning classifiers, with their hyperparameters first optimised using Particle Swarm Optimisation. Model predictions are then combined via a regularised greedy weighting mechanism. At the same time, a neural-network-based meta-learner is employed within stacked-ensemble to capture higher-order relationships among model outputs. Experiments conducted on the Lending Club dataset demonstrate that the proposed framework improves predictive performance compared with individual classifiers. The BlendNet ensemble achieved the strongest results with an AUC of 0.80, a macro-average F1-score of 0.73, and a default recall of 0.81. Calibration analysis further shows that tree-based ensembles such as Extra Trees and Gradient Boosting provide the most reliable probability estimates, while the stacked ensemble offers superior ranking capability. Feature analysis using Recursive Feature Elimination identifies revolving utilisation, annual income, and debt-to-income ratio as the most influential predictors of loan default. These findings demonstrate that performance-driven ensemble weighting can improve both predictive accuracy and interpretability in credit risk modelling. The proposed framework provides a scalable data-driven approach to support institutional credit assessment, risk monitoring, and financial decision-making.

Ezekiel Nii Noye Nortey, Jones Asante-Koranteng, Marcellin Atemkeng +4

Department of Statistics and Actuarial Science, University of Ghana, P. O. Box LG 115, Legon, Accra, Ghana. · Department of Statistics, Tali Graduate school, Dominion University College, P. O. Box LG 80, Legon, Accra,Ghana. · Department of Mathematics, Rhodes University, P.O. Box 94, Makhanda, 6140, Grahamstown, South Africa. +4

Ensemble LearningCredit Scoring
AskPDF
Mar 18, 2026

EvoGuard: An Extensible Agentic RL-based Framework for Practical and Evolving AI-Generated Image Detection

The rapid proliferation of AI-Generated Images (AIGIs) poses severe misinformation risks, making AIGI detection critical yet challenging. Traditional detection paradigms mainly rely on low-level features, whereas recent research increasingly focuses on leveraging the general understanding ability of Multimodal Large Language Models (MLLMs) to achieve better generalization, yet it still suffers from limited extensibility and expensive data annotations. Instead of building yet another detector, we recast AIGI detection as learned, reasoning-based evidence synthesis over a pool of heterogeneous off-the-shelf detectors, realized through EvoGuard, a novel agentic framework. A capability-aware selection mechanism profiles each detector and gathers complementary evidence per sample; a dynamic orchestration mechanism then reasons over heterogeneous outputs across multiple rounds, cross-validating conflicting or low-confidence signals before concluding. This design exploits the complementary strengths among heterogeneous detectors, transcending the limits of any single model. Furthermore, optimized by a GRPO-based Agentic Reinforcement Learning algorithm using only low-cost binary labels, it eliminates the reliance on fine-grained annotations. Extensive experiments demonstrate that this learned reasoning paradigm outperforms single-detector and static ensembling, achieving SOTA accuracy while mitigating the bias between positive and negative samples. More importantly, it allows the plug-and-play integration of new detectors to boost overall performance in a train-free manner, offering a highly practical, long-term solution to ever-evolving AIGI threats. Source code will be publicly available upon acceptance.

Chenyang Zhu, Maorong Wang, Jun Liu +2

The University of Tokyo Tokyo, Japan · National Institute of Informatics Tokyo, Japan

Ensemble LearningTool-Using AgentsAI-Generated Image Detection
AskPDF
Mar 15, 2026

EARCP: Self-Regulating Coherence-Aware Ensemble Architecture for Sequential Decision Making -- Ensemble Auto-Regule par Coherence et Performance

We present EARCP (Ensemble Auto-Régulé par Cohérence et Performance), a novel ensemble architecture that dynamically weights heterogeneous expert models based on both their individual performance and inter-model coherence. Unlike traditional ensemble methods that rely on static or offline-learned combinations, EARCP continuously adapts model weights through a principled online learning mechanism that balances exploitation of high-performing models with exploration guided by consensus signals. The architecture combines theoretical foundations from multiplicative weight update algorithms with a novel coherence-based regularization term, providing both theoretical guarantees through regret bounds and practical robustness in non-stationary environments. We formalize the EARCP framework, prove sublinear regret bounds of O(sqrt(T log M)) under standard assumptions, and demonstrate its effectiveness through empirical evaluation on sequential prediction tasks including time series forecasting, activity recognition, and financial prediction. The architecture is designed as a general-purpose framework applicable to any domain requiring ensemble learning with temporal dependencies. An open-source implementation is available at https://github.com/Volgat/earcp and via PyPI (pip install earcp).

Mike Amega

Independent Researcher, Windsor, Ontario, Canada

Ensemble LearningDynamic Regret Minimization
AskPDF
Mar 15, 2026

BROTHER: Behavioral Recognition Optimized Through Heterogeneous Ensemble Regularization for Ambivalence and Hesitancy

Recognizing complex behavioral states such as Ambivalence and Hesitancy (A/H) in naturalistic video settings remains a significant challenge in affective computing. Unlike basic facial expressions, A/H manifests as subtle, multimodal conflicts that require deep contextual and temporal understanding. In this paper, we propose a highly regularized, multimodal fusion pipeline to predict A/H at the video level. We extract robust unimodal features from visual, acoustic, and linguistic data, introducing a specialized statistical text modality explicitly designed to capture temporal speech variations and behavioral cues. To identify the most effective representations, we evaluate 15 distinct modality combinations across a committee of machine learning classifiers (MLP, Random Forest, and GBDT), selecting the most well-calibrated models based on validation Binary Cross-Entropy (BCE) loss. Furthermore, to optimally fuse these heterogeneous models without overfitting to the training distribution, we implement a Particle Swarm Optimization (PSO) hard-voting ensemble. The PSO fitness function dynamically incorporates a train-validation gap penalty (lambda) to actively suppress redundant or overfitted classifiers. Our comprehensive evaluation demonstrates that while linguistic features serve as the strongest independent predictor of A/H, our heavily regularized PSO ensemble (lambda = 0.2) effectively harnesses multimodal synergies, achieving a peak Macro F1-score of 0.7465 on the unseen test set. These results emphasize that treating ambivalence and hesitancy as a multimodal conflict, evaluated through an intelligently weighted committee, provides a robust framework for in-the-wild behavioral analysis.

Alexandre Pereira, Bruno Fernandes, Pablo Barros

University of Pernambuco, Recife, Brazil

Ambivalence and Hesitancy RecognitionAffective ComputingMultimodal FusionEnsemble Learning
AskPDF
Mar 13, 2026

Exploring label correlations using decision templates for ensemble of classifier chains

The use of ensemble-based multi-label methods has been shown to be effective in improving multi-label classification results. One of the most widely used ensemble-based multi-label classifiers is Ensemble of Classifier Chains. Decision templates for Ensemble of Classifier Chains (DTECC) is a fusion scheme based on Decision Templates that combines the predictions of Ensemble of Classifier Chains using information from the decision profile for each label, without considering information about other labels that might contribute to the classified result. Based on DTECC, this work proposes the Unconditionally Dependent Decision Templates for Ensemble of Classifier Chains (UDDTECC) method, a classifier fusion method that seeks to exploit correlations between labels in the fusion process. In this way, the classification of each label in the problem takes into account the label values that are considered conditionally dependent and that can lead to an improvement in the classification performance. The proposed method is experimentally compared with two traditional classifier fusion strategies and with a stacking-based strategy. Empirical evidence shows that using the proposed Decision Templates adaptation can improve the performance compared to the traditionally used fusion schemes on most of the evaluated metrics.

Victor F. Rocha, Alexandre L. Rodrigues, Thiago Oliveira-Santos +1

Department of Statistics, Federal University of Espírito Santo, Vitória, Espírito Santo, 29075-910, Brazil

Multi-Label ClassificationEnsemble Learning
AskPDF
Mar 13, 2026

FraudFox: Adaptable Fraud Detection in the Real World

The proposed method (FraudFox) provides solutions to adversarial attacks in a resource constrained environment. We focus on questions like the following: How suspicious is Smith', trying to buy \$500 shoes, on Monday 3am? How to merge the risk scores, from a handful of risk-assessment modules (oracles') in an adversarial environment? More importantly, given historical data (orders, prices, and what-happened afterwards), and business goals/restrictions, which transactions, like the Smith' transaction above, which ones should we pass', versus send to human investigators? The business restrictions could be: at most $x$ investigations are feasible', or at most $yy lost due to fraud'. These are the two research problems we focus on, in this work. One approach to address the first problem (`oracle-weighting'), is by using Extended Kalman Filters with dynamic importance weights, to automatically and continuously update our weights for each 'oracle'. For the second problem, we show how to derive an optimal decision surface, and how to compute the Pareto optimal set, to allow what-if questions. An important consideration is adaptation: Fraudsters will change their behavior, according to our past decisions; thus, we need to adapt accordingly. The resulting system, \method, is scalable, adaptable to changing fraudster behavior, effective, and already in production at Amazon. FraudFox augments a fraud prevention sub-system and has led to significant performance gains.

Matthew Butler, Yi Fan, Christos Faloutsos

Amazon.com, Inc., 410 Terry Ave N, Seattle WA 98109, U.S.A. · Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, U.S.A.

Financial Fraud DetectionEnsemble LearningCost-Sensitive Learning
AskPDF
Mar 11, 2026

HAPEns: Hardware-Aware Post-Hoc Ensembling for Tabular Data

Ensembling is commonly used in machine learning on tabular data to boost predictive performance and robustness, but larger ensembles often lead to increased hardware demand. We introduce HAPEns, a post-hoc ensembling method that explicitly balances accuracy against hardware efficiency. Inspired by multi-objective and quality diversity optimization, HAPEns constructs a diverse set of ensembles along the Pareto front of predictive performance and resource usage. Existing hardware-aware post-hoc ensembling baselines are not available, highlighting the novelty of our approach. Experiments on 83 tabular classification datasets show that HAPEns significantly outperforms baselines, finding superior trade-offs for ensemble performance and deployment cost. Ablation studies also reveal that memory usage is a particularly effective objective metric. Further, we show that even a greedy ensembling algorithm can be significantly improved in this task with static multi-objective weighting.

Jannis Maier, Lennart Purucker

Helmholtz-Zentrum Berlin · TU Dortmund · Prior Labs +1

Ensemble LearningTabular Prediction
AskPDF
Mar 9, 2026

InFusionLayer: a CFA-based ensemble tool to generate new classifiers for learning and modeling

Ensemble learning is a well established body of methods for machine learning to enhance predictive performance by combining multiple algorithms/models. Combinatorial Fusion Analysis (CFA) has provided method and practice for combining multiple scoring systems, using rank-score characteristic (RSC) function and cognitive diversity (CD), including ensemble method and model fusion. However, there is no general-purpose Python tool available that incorporate these techniques. In this paper we introduce InFusionLayer, a machine learning architecture inspired by CFA at the system fusion level that uses a moderate set of base models to optimize unsupervised and supervised learning multiclassification problems. We demonstrate InFusionLayer's ease of use for PyTorch, TensorFlow, and Scikit-learn workflows by validating its performance on various computer vision datasets. Our results highlight the practical advantages of incorporating distinctive features of RSC function and CD, paving the way for more sophisticated ensemble learning applications in machine learning. We open-sourced our code to encourage continuing development and community accessibility to leverage CFA on github: https://github.com/ewroginek/Infusion

Eric Roginek, Jingyan Xu, D. Frank. Hsu

Department of Computer and Information Science, Fordham University, New York, NY 10023.

Ensemble Learning
AskPDF
Mar 6, 2026

Ensemble Learning with Sparse Hypercolumns

Directly inspired by findings in biological vision, high-dimensional hypercolumns are feature vectors built by concatenating multi-scale activations of convolutional neural networks for a single image pixel location. Together with powerful classifiers, they can be used for image segmentation i.e. pixel classification. However, in practice, there are only very few works dedicated to the use of hypercolumns. One reason is the computational complexity of processing concatenated dense hypercolumns that grows linearly with the size NN of the training set. In this work, we address this challenge by applying stratified subsampling to the VGG16 based hypercolumns. Furthermore, we investigate the performance of ensemble learning on sparse hypercolumns. Our experiments on a brain tumor dataset show that stacking and voting ensembles deliver competitive performance, but in the extreme low-shot case of N≤20N \leq 20, a simple Logistic Regression classifier is the most effective method. For 10% stratified subsampling rate, our best average Dice score is 0.66 for N=20N=20. This is a statistically significant improvement of 24.53% over the standard multi-scale UNet baseline (pp-value = [3.07e−11][3.07e-11], Wilcoxon signed-rank test), which is less effective due to overfitting.

Julia Dietlmeier, Vayangi Ganepola, Oluwabukola G. Adegboro +3

Insight Research Ireland Centre for Data Analytics, Dublin City University, Glasnevin 9, Dublin, Ireland · Research Ireland Centre for Research Training in Machine Learning (ML-Labs), Dublin City University, Glasnevin 9, Dublin, Ireland · School of Computing, Dublin City University, Glasnevin 9, Dublin, Ireland

Few-Shot Medical Image SegmentationEnsemble LearningBrain Tumor Segmentation
AskPDF
Mar 4, 2026

Beyond Mixtures and Products for Ensemble Aggregation: A Likelihood Perspective on Generalized Means

Density aggregation is a central problem in machine learning, for instance when combining predictions from a Deep Ensemble. The choice of aggregation remains an open question with two commonly proposed approaches being linear pooling (probability averaging) and geometric pooling (logit averaging). In this work, we address this question by studying the normalized generalized mean of order r∈R∪{−∞,+∞}r \in \mathbb{R} \cup \{-\infty,+\infty\} through the lens of log-likelihood, the standard evaluation criterion in machine learning. This provides a unifying aggregation formalism and shows different optimal configurations for different situations. We show that the regime r∈[0,1]r \in [0,1] is the only range ensuring systematic improvements relative to individual distributions, thereby providing a principled justification for the reliability and widespread practical use of linear (r=1r=1) and geometric (r=0r=0) pooling. In contrast, we show that aggregation rules with r∉[0,1]r \notin [0,1] may fail to provide consistent gains with explicit counterexamples. Finally, we corroborate our theoretical findings with empirical evaluations using Deep Ensembles on image and text classification benchmarks.

Raphaël Razafindralambo, Rémy Sun, Frédéric Precioso +2

Université Côte d’Azur, Inria, CNRS, I3S/LJAD, Maasai, Nice, France · Julius-Maximilians-Universität Würzburg, Institute for Computer Science / CAIDAS, Würzburg, Germany

Ensemble LearningDeep Ensembles
AskPDF
Mar 3, 2026

ERP-RiskBench: Leakage-Safe Ensemble Learning for Financial Risk

Financial risk detection in Enterprise Resource Planning (ERP) systems is an important but underexplored application of machine learning. Published studies in this area tend to suffer from vague dataset descriptions, leakage-prone pipelines, and evaluation practices that inflate reported performance. This paper presents a rebuilt experimental framework for ERP financial risk detection using ensemble machine learning. The risk definition is hybrid, covering both procurement compliance anomalies and transactional fraud. A composite benchmark called ERP-RiskBench is assembled from public procurement event logs, labeled fraud data, and a new synthetic ERP dataset with rule-injected risk typologies and conditional tabular GAN augmentation. Nested cross-validation with time-aware and group-aware splitting enforces leakage prevention throughout the pipeline. The primary model is a stacking ensemble of gradient boosting methods, tested alongside linear baselines, deep tabular architectures, and an interpretable glassbox alternative. Performance is measured through Matthews Correlation Coefficient, area under the precision-recall curve, and cost-sensitive decision analysis using calibrated probabilities. Across multiple dataset configurations and a structured ablation study, the stacking ensemble achieves the best detection results. Leakage-safe protocols reduce previously inflated accuracy estimates by a notable margin. SHAP-based explanations and feature stability analysis show that procurement control features, especially three-way matching discrepancies, rank as the most informative predictors. The resulting framework provides a reproducible, operationally grounded blueprint for machine learning deployment in ERP audit and governance settings.

Sanjay Mishra

IEEE Member Raleigh, NC, USA

Stacked Ensemble LearningFinancial Fraud DetectionAnomaly DetectionEnsemble LearningData Leakage
AskPDF
Feb 28, 2026

GeNeX: Genetic Network eXperts framework for addressing Validation Overfitting

Excessive reliance on validation performance during model selection can lead to validation overfitting (VO), where models appear effective during development but fail at test time. This issue is further amplified in low-data regimes and under distribution shifts, where validation signals become unreliable. Although ensemble learning is widely used to improve robustness and generalization, most ensemble construction pipelines depend heavily on validation scores, leaving them vulnerable to VO and limiting their reliability in real-world deployment scenarios. To address this, we propose GeNeX (Genetic Network eXperts), a framework that mitigates validation overfitting at both model generation and ensemble construction stages. In the generation phase, GeNeX employs a dual-path strategy: gradient-based training is coupled with genetic model evolution. Offspring networks are created via crossover of trained parents, promoting structural diversity and weight-level regeneration without relying on validation feedback. This results in a candidate pool of robust, non-overfitted models. During ensemble construction, the candidate networks are clustered by prediction behavior to identify complementary model spaces. Within each cluster, multiple diverse experts are selected using criteria such as robustness and representativeness, and fused at the weight level to form compact prototype networks. The final ensemble aggregates these prototypes, with predictions optimized via Sequential Quadratic Programming for output-level synergy. To rigorously evaluate VO resilience, we introduce a VO-aware evaluation protocol that simulates realistic deployment scenarios by enforcing distributional divergence between training and test subsets.

Emmanuel Pintelas, Ioannis E. Livieris

Department of Statistics & Insurance Science, University of Piraeus, GR 18534, Greece · Department of Business & Organization Administration, University of Peloponnese, GR 24100, Greece

Ensemble LearningModel Merging
AskPDF
Feb 26, 2026

An Explainable Ensemble Framework for Alzheimer's Disease Prediction Using Structured Clinical and Cognitive Data

Early and accurate detection of Alzheimer's disease (AD) remains a major challenge in medical diagnosis due to its subtle onset and progressive nature. This research introduces an explainable ensemble learning Framework designed to classify individuals as Alzheimer's or Non-Alzheimer's using structured clinical, lifestyle, metabolic, and lifestyle features. The workflow incorporates rigorous preprocessing, advanced feature engineering, SMOTE-Tomek hybrid class balancing, and optimized modeling using five ensemble algorithms-Random Forest, XGBoost, LightGBM, CatBoost, and Extra Trees-alongside a deep artificial neural network. Model selection was performed using stratified validation to prevent leakage, and the best-performing model was evaluated on a fully unseen test set. Ensemble methods achieved superior performance over deep learning, with XGBoost, Random Forest, and Soft Voting showing the strongest accuracy, sensitivity, and F1-score profiles. Explainability techniques, including SHAP and feature importance analysis, highlighted MMSE, Functional Assessment Age, and several engineered interaction features as the most influential determinants. The results demonstrate that the proposed framework provides a reliable and transparent approach to Alzheimer's disease prediction, offering strong potential for clinical decision support applications.

Nishan Mitra

Computer Science & Engineering (AI&ML), Institute of Engineering & Management, Kolkata, India

Alzheimer's Disease ClassificationClinical PredictionEnsemble Learning
AskPDF
Feb 26, 2026

Model Agreement via Anchoring

Numerous lines of aim to control model disagreement\textit{model disagreement} -- the extent to which two machine learning models disagree in their predictions. We adopt a simple and standard notion of model disagreement in real-valued prediction problems, namely the expected squared difference in predictions between two models trained on independent samples, without any coordination of the training processes. We would like to be able to drive disagreement to zero with some natural parameter(s) of the training procedure using analyses that can be applied to existing training methodologies. We develop a simple general technique for proving bounds on independent model disagreement based on anchoring\textit{anchoring} to the average of two models within the analysis. We then apply this technique to prove disagreement bounds for four commonly used machine learning algorithms: (1) stacked aggregation over an arbitrary model class (where disagreement is driven to 0 with the number of models kk being stacked) (2) gradient boosting (where disagreement is driven to 0 with the number of iterations kk) (3) neural network training with architecture search (where disagreement is driven to 0 with the size nn of the architecture being optimized over) and (4) regression tree training over all regression trees of fixed depth (where disagreement is driven to 0 with the depth dd of the tree architecture). For clarity, we work out our initial bounds in the setting of one-dimensional regression with squared error loss -- but then show that all of our results generalize to multi-dimensional regression with any strongly convex loss.

Eric Eaton, Surbhi Goel, Marcel Hussing +4

Department of Computer and Information Sciences, University of Pennsylvania · Department of Computer Science, The Johns Hopkins University

Statistical Learning TheoryEnsemble Learning
AskPDF
Feb 25, 2026

Integrating Machine Learning Ensembles and Large Language Models for Heart Disease Prediction Using Voting Fusion

Cardiovascular disease is the primary cause of death globally, necessitating early identification, precise risk classification, and dependable decision-support technologies. The advent of large language models (LLMs) provides new zero-shot and few-shot reasoning capabilities, even though machine learning (ML) algorithms, especially ensemble approaches like Random Forest, XGBoost, LightGBM, and CatBoost, are excellent at modeling complex, non-linear patient data and routinely beat logistic regression. This research predicts cardiovascular disease using a merged dataset of 1,190 patient records, comparing traditional machine learning models (95.78% accuracy, ROC-AUC 0.96) with open-source large language models via OpenRouter APIs. Finally, a hybrid fusion of the ML ensemble and LLM reasoning under Gemini 2.5 Flash achieved the best results (96.62% accuracy, 0.97 AUC), showing that LLMs (78.9 % accuracy) work best when combined with ML models rather than used alone. Results show that ML ensembles achieved the highest performance (95.78% accuracy, ROC-AUC 0.96), while LLMs performed moderately in zero-shot (78.9%) and slightly better in few-shot (72.6%) settings. The proposed hybrid method enhanced the strength in uncertain situations, illustrating that ensemble ML is considered the best structured tabular prediction case, but it can be integrated with hybrid ML-LLM systems to provide a minor increase and open the way to more reliable clinical decision-support tools.

Md. Tahsin Amin, Tanim Ahmmod, Zannatul Ferdus +5

Faculty of Computer Science & Engineering, Patuakhali Science & Technology University, Bangladesh · Dept. of Electrical & Electronic Engineering, Rajshahi University of Engineering & Technology, Bangladesh · Dept. of Industrial & Production Engineering, Ahsanullah University of Science & Technology, Bangladesh +3

Cardiovascular Risk PredictionClinical Risk PredictionEnsemble Learning
AskPDF
Feb 24, 2026

LMU-Based Sequential Learning and Posterior Ensemble Fusion for Cross-Domain Infant Cry Classification

Decoding infant cry causes remains challenging for healthcare monitoring due to short nonstationary signals, limited annotations, and strong domain shifts across infants and datasets. We propose a compact acoustic framework that fuses mel-frequency cepstral coefficients (MFCCs), short-time Fourier transform (STFT) features, and fundamental-frequency (F0) contours within a multi-branch convolutional neural network (CNN) encoder, and models temporal dynamics using an enhanced Legendre Memory Unit (LMU). Compared to LSTMs, the LMU backbone provides stable sequence modeling with substantially fewer recurrent parameters, supporting efficient deployment. To improve cross-dataset generalization, we introduce calibrated posterior ensemble fusion with entropy-gated weighting to preserve domain-specific expertise while mitigating dataset bias. Experiments on Baby2020 and Baby Crying demonstrate improved macro-F1 under cross-domain evaluation, along with leakage aware splits and real-time feasibility for on-device monitoring.

Niloofar Jazaeri, Hilmi R. Dajani, Marco Janeczek +1

University of Ottawa, Ottawa, Canada · Crynostics Inc., Canada

Audio ClassificationEnsemble Learning
AskPDF
Feb 23, 2026

Align When They Want, Complement When They Need! Human-Centered Ensembles for Adaptive Human-AI Collaboration

In human-AI decision making, designing AI that complements human expertise has been a natural strategy to enhance human-AI collaboration, yet it often comes at the cost of decreased AI performance in areas of human strengths. This can inadvertently erode human trust and cause them to ignore AI advice precisely when it is most needed. Conversely, an aligned AI fosters trust yet risks reinforcing suboptimal human behavior and lowering human-AI team performance. In this paper, we start by identifying this fundamental tension between performance-boosting (i.e., complementarity) and trust-building (i.e., alignment) as an inherent limitation of the traditional approach for training a single AI model to assist human decision making. To overcome this, we introduce a novel human-centered adaptive AI ensemble that strategically toggles between two specialist AI models - the aligned model and the complementary model - based on contextual cues, using an elegantly simple yet provably near-optimal Rational Routing Shortcut mechanism. Comprehensive theoretical analyses elucidate why the adaptive AI ensemble is effective and when it yields maximum benefits. Moreover, experiments on both simulated and real-world data show that when humans are assisted by the adaptive AI ensemble in decision making, they can achieve significantly higher performance than when they are assisted by single AI models that are trained to either optimize for their independent performance or even the human-AI team performance.

Hasan Amin, Ming Yin, Rajiv Khanna

Purdue University, USA

Human-AI ComplementarityHuman-AI CollaborationHuman-AI Decision MakingEnsemble Learning
AskPDF
Feb 23, 2026

Ensemble Machine Learning and Statistical Procedures for Dynamic Predictions of Time-to-Event Outcomes

Dynamic predictions for longitudinal and time-to-event outcomes have become a versatile tool in precision medicine. Our work is motivated by the application of dynamic predictions in the decision-making process for primary biliary cholangitis patients. For these patients, serial biomarker measurements (e.g., bilirubin and alkaline phosphatase levels) are routinely collected to inform treating physicians of the risk of liver failure and guide clinical decision-making. Two popular statistical approaches to derive dynamic predictions are joint modelling and landmarking. However, recently, machine learning techniques have also been proposed. Each approach has its merits, and no single method exists to outperform all others. Consequently, obtaining the best possible survival estimates is challenging. Therefore, we extend the Super Learner framework to combine dynamic predictions from different models and procedures. Super Learner is an ensemble learning technique that allows users to combine different prediction algorithms to improve predictive accuracy and flexibility. It uses cross-validation and different objective functions of performance (e.g., squared loss) that suit specific applications to build the optimally weighted combination of predictions from a library of candidate algorithms. In our work, we pay special attention to appropriate objective functions for Super Learner to obtain the most optimal weighted combination of dynamic predictions. In our primary biliary cholangitis application, Super Learner presented unique benefits due to its ability to flexibly combine outputs from a diverse set of models with varying assumptions for equal or better predictive performance than any model fit separately.

Nina van Gerwen, Sten Willemsen, Bettina E. Hansen +9

Department of Biostatistics, Erasmus University Medical Center, the Netherlands · Department of Epidemiology, Erasmus University Medical Center, the Netherlands · Reference Center for Inflammatory Biliary Diseases and Autoimmune Hepatitis, European Reference Network on Hepatological Diseases (ERN Rare-Liver), Saint-Antoine Hospital, Assistance Publique - Hˆopitaux de Paris; Inserm UMR S938, Saint-Antoine Research Center, Sorbonne University, Paris, France +7

Survival AnalysisEnsemble Learning
AskPDF
Feb 21, 2026

(α,β)(α,β)-Stability for Boosting Vector-Valued Prediction

Despite the widespread use of boosting in structured prediction, a general theoretical understanding of aggregation beyond scalar prediction remains incomplete. We study vector-valued prediction under a target divergence and identify a geometric stability property under which aggregation amplifies weak guarantees into strong ones. We formalize this property as (α,β)(α,β)-stability by geometric median and show how it supports a boosting framework based on exponential reweighting and geometric-median aggregation. For vector-valued prediction, we characterize this stability property under several natural divergences: ℓ1\ell_1 and ℓ2\ell_2 distances for unconstrained vector-valued prediction, and TV, Hellinger, and KL for density estimation over finite probability vectors. Building on these results, we propose a generic boosting framework \geomedboost. Under a weak learner condition and (α,β)(α,β)-stability, we obtain exponential decay of the empirical divergence error, which then yields population guarantees through a generalization bound.

Jian Qian, Shu Ge

The University of Hong Kong · Independent Researcher

Ensemble LearningStatistical Learning Theory
AskPDF
Feb 19, 2026

A Hybrid Federated Learning Based Ensemble Approach for Lung Disease Diagnosis Leveraging Fusion of SWIN Transformer and CNN

The significant advancements in computational power cre- ate a vast opportunity for using Artificial Intelligence in different ap- plications of healthcare and medical science. A Hybrid FL-Enabled Ensemble Approach For Lung Disease Diagnosis Leveraging a Combination of SWIN Transformer and CNN is the combination of cutting-edge technology of AI and Federated Learning. Since, medi- cal specialists and hospitals will have shared data space, based on that data, with the help of Artificial Intelligence and integration of federated learning, we can introduce a secure and distributed system for medical data processing and create an efficient and reliable system. The proposed hybrid model enables the detection of COVID-19 and Pneumonia based on x-ray reports. We will use advanced and the latest available tech- nology offered by Tensorflow and Keras along with Microsoft-developed Vision Transformer, that can help to fight against the pandemic that the world has to fight together as a united. We focused on using the latest available CNN models (DenseNet201, Inception V3, VGG 19) and the Transformer model SWIN Transformer in order to prepare our hy- brid model that can provide a reliable solution as a helping hand for the physician in the medical field. In this research, we will discuss how the Federated learning-based Hybrid AI model can improve the accuracy of disease diagnosis and severity prediction of a patient using the real-time continual learning approach and how the integration of federated learn- ing can ensure hybrid model security and keep the authenticity of the information.

Asif Hasan Chowdhury, Md. Fahim Islam, M Ragib Anjum Riad +3

BRAC University, Dhaka-1212, Bangladesh

Medical DiagnosisChest X-Ray ClassificationHybrid CNN-Transformer ArchitecturesEnsemble LearningFederated Learning
AskPDF
Feb 14, 2026

StackingNet: Collective Inference Across Independent AI Foundation Models

Artificial intelligence built on large foundation models has transformed language understanding, computer vision, and reasoning, yet these systems remain isolated and cannot readily share their capabilities. Coordinating the complementary strengths of independently developed, black-box foundation models is essential for trustworthy intelligent systems, yet no established method exists. Here we show that such coordination can be achieved through a meta-ensemble framework termed StackingNet, which aggregates the output predictions of independent models at inference. StackingNet improves accuracy, reduces individual-model error and group-wise disparities, ranks model reliability, and identifies or prunes models that degrade performance, all without access to internal parameters or training data. Across language comprehension, visual attribute estimation, and academic paper rating, it consistently outperforms individual models and classic ensembles, with gains that persist when the base models are uniformly strong. These gains stem from variance reduction and consensus alignment among independent models rather than from any emergent group cognition, and they widen as the model pool grows more diverse. By turning model diversity from a source of inconsistency into a resource for cooperation, StackingNet offers a practical path toward coordinated artificial intelligence, where progress emerges not only from larger single models but from principled cooperation among many specialized ones.

Siyang Li, Chenhao Liu, Dongrui Wu +2

School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China. · School of Civil and Hydraulic Engineering, Huazhong University of Science and Technology, Wuhan 430074, China.

Ensemble LearningLanguage Model EnsemblesStacked Ensemble Learning
AskPDF
Feb 13, 2026

Probabilistic Wind Power Forecasting with Tree-Based Machine Learning and Weather Ensembles

Accurate production forecasts are essential for the integration of renewable energy sources into the power grid. This paper illustrates how to obtain probabilistic forecasts of wind power generation using gradient boosting trees and an ensemble of weather forecasts. To this end, we perform a comparative analysis across three state-of-the-art probabilistic prediction methods-conformalized quantile regression, natural gradient boosting and conditional diffusion models-all of which can be combined with tree-based machine learning. The methods are validated using four years of data for all Belgian offshore wind farms. We benchmark the models against the power curve and a calibrated wake model as well as a probabilistic method using stochastic variational Gaussian process regression. The tree-based models significantly reduce the mean absolute error in comparison to the deterministic baselines. Additionally, all three methods outperform the Gaussian process baseline in probabilistic skill, while two out of the three also improve point forecast accuracy. The conditional diffusion model attains the best performance, with improvements of 5% in mean absolute error and 12% in continuous rank probability score compared to the probabilistic baseline. Last, the results indicate an average improvement in point forecast accuracy of 17% by using an ensemble of weather forecasts instead of a single provider.

Max Bruninx, Diederik van Binsbergen, Timothy Verstraeten +2

Vrije Universiteit Brussel, Brussels, Belgium · Norwegian University of Science and Technology, Trondheim, Norway

Gradient-Boosted Decision TreesEnsemble LearningTime Series ForecastingConformal PredictionWind Power ForecastingConditional Diffusion Models+2
AskPDF
Jan 23, 2026

Is BatchEnsemble a Single Model? On Calibration and Diversity of Efficient Ensembles

In resource-constrained and low-latency settings, uncertainty estimates must be efficiently obtained. Deep Ensembles provide robust epistemic uncertainty (EU) but require training multiple full-size models. BatchEnsemble aims to deliver ensemble-like EU at far lower parameter and memory cost by applying learned rank-1 perturbations to a shared base network. We show that BatchEnsemble not only underperforms Deep Ensembles but closely tracks a single model baseline in terms of accuracy, calibration and out-of-distribution (OOD) detection on CIFAR10/10C/SVHN. A controlled study on MNIST finds members are near-identical in function and parameter space, indicating limited capacity to realize distinct predictive modes. Thus, BatchEnsemble behaves more like a single model than a true ensemble.

Anton Zamyatin, Patrick Indri, Sagar Malhotra +1

TU Wien

Ensemble LearningEnsemble DiversityDeep EnsemblesModel Calibration
AskPDF
Dec 27, 2025

Collaborative Optimization of Multiclass Imbalanced Learning: Density-Aware and Region-Guided Boosting

Numerous studies on Boosting attempt to mitigate classification bias caused by class imbalance. However, existing studies have yet to explore the collaborative optimization of imbalanced learning and model training. This constraint hinders further performance improvements. To bridge this gap, this study proposes a collaborative optimization Boosting model of multiclass imbalanced learning. By integrating the density factor and the confidence factor, this model implements a noise-resistant weight update mechanism alongside a dynamic sampling strategy. Rather than functioning as independent components, these modules are tightly integrated to orchestrate weight updates, sample region partitioning, and region-guided sampling. Thus, this study proposes the collaborative optimization of imbalanced learning and model training. Extensive experiments on 40 public imbalanced datasets demonstrate that the proposed model significantly outperforms seven state-of-the-art baselines. The code and datasets for this paper are available at: https://github.com/ChuantaoLi/DARG.

Chuantao Li, Zhi Li, Jiahao Xu +2

School of Automation Engineering, University of Electronic Science and Technology of China, Chengdu 611731, Sichuan, China · School of Mathematics and Computer, Guangdong Ocean University, Zhanjiang 524088, Guangdong, China

Class-Imbalanced LearningEnsemble LearningMulticlass Classification
AskPDF
Nov 27, 2025

PISA: Prioritized Invariant Subgraph Aggregation for Out-of-Distribution Generalization on Graphs

Invariant learning on graphs aims to build predictors that rely on causal substructures rather than on environment-specific shortcuts. Current methods extract either a single invariant subgraph (CIGA) or, more recently, a set of diverse invariant subgraphs (SuGAr). We observe that the second half of the multi-subgraph pipeline, "how the extracted subgraphs are combined" has gone essentially unexamined; existing methods average branch predictions uniformly or select one greedily. We show that this is precisely where accuracy is lost. A minimum-variance combination argument establishes that uniform averaging is optimal only when every branch carries the same total error covariance with the ensemble---a knife edge balance that second order exchangeability guarantees and that the diversity regularizer used to obtain multiple subgraphs is explicitly designed to break. We then show that invariance is closed under post composition; because the branches are frozen and read the graph only through their invariant subgraphs, any measurable aggregator of their outputs remains invariant, and the aggregate's environment gap is bounded by the branches' residual invariance violation uniformly over aggregators. Learning the combination rule by plain empirical risk minimization is therefore safe, and aggregator capacity trades off against estimation error rather than robustness. PISA instantiates this with a lightweight instance-conditional MLP over stacked branch logits, trained on frozen branches at negligible cost. Across 15 synthetic and real benchmarks, PISA improves the average over the strongest prior method by 2.96 points on synthetic shifts and 1.74 points on realistic shifts, with per-dataset gains up to 5.90 points, and nearly doubles SuGAr's improvement over ERM on the six DrugOOD tasks (from +2.76 to +5.07 ROC-AUC).

Ali Ghasemi, Farooq Ahmad Wani, Maria Sofia Bucarelli +1

Sapienza University of Rome · Université Côte d’Azur, CNRS, Inria, I3S

OOD GeneralizationInvariant Representation LearningEnsemble Learning
AskPDF
Oct 12, 2025

Provable Anytime Ensemble Sampling Algorithms in Nonlinear Contextual Bandits

We provide a unified algorithmic framework for ensemble sampling in nonlinear contextual bandits and develop corresponding regret bounds for two most common nonlinear contextual bandit settings: Generalized Linear Model Ensemble Sampling (GLM-ES) for generalized linear contextual bandits and Neural Ensemble Sampling (Neural-ES) for neural contextual bandits. Both methods maintain multiple estimators for the reward model parameters via maximum likelihood estimation on randomly perturbed data. We prove high-probability frequentist regret bounds of O~(d3/2T+d4)\widetilde{\mathcal{O}}(d^{3/2} \sqrt{T} + d^{4}) for GLM-ES and O~(d~3/2T)\widetilde{\mathcal{O}}(\widetilde{d}^{3/2} \sqrt{T}) for Neural-ES, where dd is the dimension of feature vectors, d~\widetilde{d} is the effective dimension of a neural tangent kernel (NTK) matrix and TT is the number of rounds. The regret bound of GLM-ES matches the state-of-the-art result of randomized exploration algorithms in generalized linear bandit setting. In the theoretical analysis, we introduce techniques that address challenges specific to nonlinear models. Practically, we remove fixed-time horizon assumption by developing anytime versions of our algorithms, suitable when TT is unknown. Finally, we empirically evaluate GLM-ES, Neural-ES and their anytime variants, demonstrating strong performance. Overall, our results establish ensemble sampling as a provable and practical randomized exploration approach for nonlinear contextual bandits.

Jiazheng Sun, Weixin Wang, Pan Xu

Duke University

Ensemble LearningContextual BanditsGeneralized Linear ModelsGeneralized Linear Bandits
AskPDF
Oct 10, 2025

CRPS-LAM: Probabilistic Regional Weather Forecasting with Continuous Ranked Probability Score

Limited-Area Models (LAMs) enable weather forecasting over regional domains at higher resolutions than what is computationally feasible for global models. At such high resolutions, machine learning approaches for weather prediction increasingly rely on ensemble methods to produce probabilistic forecasts. However, existing machine learning LAMs are not scalable due to relying on computationally costly diffusion models or inefficient graph neural networks. We tackle this by introducing a new hybrid CNN/GNN architecture, tailored to the LAM weather forecasting problem. Using this architecture, we construct the DET-LAM deterministic model, producing LAM forecasts both more efficiently and accurately than its graph-based competitor. We then tackle the ensemble forecasting problem, by using this architecture as a backbone for the generative model CRPS-LAM. CRPS-LAM is trained using a Continuous Ranked Probability Score (CRPS) objective, enabling efficient training and sampling in a single forward pass. This yields a speedup of ≈×39\approx \times 39 compared to diffusion-based baselines. We evaluate our approach on regional domains in northern Europe, demonstrating that CRPS-LAM produces skillful and well-calibrated forecasts across a range of atmospheric variables.

Erik Larsson, Joel Oskarsson, Tomas Landelius +1

Linköping University · ETH AI Center · SMHI & Linköping University

Graph Neural NetworksEnsemble LearningProbabilistic ForecastingSpatiotemporal ForecastingWeather Forecasting
AskPDF
Oct 8, 2025

Robust and Efficient Finetuning of Vision Foundation Models via Implicit Ensembling

Robust finetuning of foundation models seeks to improve out-of-distribution (OOD) generalization during in-distribution (ID) adaptation. We revisit Mixout, a stochastic regularizer that intermittently replaces finetuned parameters with a reference anchor, and study its effectiveness for robust finetuning of vision foundation models. We reinterpret Mixout as a single-run, weight-sharing implicit ensemble and analyze its expected OOD error through a bias-variance-covariance-locality (BVCL) lens. This perspective reveals three key factors that govern the ID-OOD trade-off: the choice of masking anchor, the mask resampling frequency, and mask sparsity. Guided by this analysis, we propose GMixout, which generalizes Mixout in three ways. First, it explicitly controls the masking period through a resampling-frequency hyperparameter. Second, it replaces the fixed pretrained anchor with an exponential moving average snapshot that adapts throughout training. Third, it uses sparse kernels to update only a small subset of parameters at each iteration, introducing no inference-time overhead and enabling large-model finetuning on consumer-grade GPUs. Experiments across five vision benchmarks show that GMixout consistently outperforms Mixout, Model Soups, and strong parameter-efficient finetuning baselines under both covariate shift and class imbalance.

Masih Aminbeidokhti, Heitor Rapela Medeiros, Eric Granger +1

Vision Foundation Model AdaptationOOD GeneralizationEnsemble LearningParameter-Efficient Fine-Tuning
AskPDF
Oct 6, 2025

Semantics-Aware Hierarchical Consensus Learning for Remote Sensing Image Classification

Deep learning has become increasingly important in remote sensing image classification due to its ability to extract semantic information from complex data. Classification tasks often include predefined label hierarchies that represent the semantic relationships among classes. However, these hierarchies are frequently overlooked, and most approaches focus only on fine-grained classification schemes. In this paper, we present a novel Semantics-Aware Hierarchical Consensus (SAHC) approach that integrates hierarchical-level-specific classification heads within a deep network architecture and combines their output through cross-level probability projectors. Direct and projected predictions are fused into a geometric consensus distribution, which is used for self-consistent training and optional hierarchy-aware inference. This mechanism acts as a geometric ensemble that leverages the inherent structure of the hierarchical classification task. The projectors are initialized from the user-defined taxonomy (i.e., the hierarchical label structure), and can be adaptively refined during optimization. The proposed SAHC method is evaluated on two benchmark datasets with different degrees of hierarchical complexity on different tasks, considering varying spectral and spatial resolutions. Experimental results show both the effectiveness of the proposed approach in guiding network learning and the robustness of the hierarchical consensus for remote sensing image classification tasks. The source code is available at https://github.com/rslab-unitrento/sahc.

Giulio Weikmann, Gianmarco Perantoni, Lorenzo Bruzzone

Department of Information Engineering and Computer Science, University of Trento, 38123 Trento, Italy

Hierarchical ClassificationRemote SensingRemote Sensing Image UnderstandingEnsemble Learning
AskPDF
Aug 25, 2025

Huracan: A skillful end-to-end data-driven system for ensemble data assimilation and weather prediction

Over the past few years, machine learning-based data-driven weather prediction has been transforming operational weather forecasting by providing more accurate forecasts while using a mere fraction of computing power compared to traditional numerical weather prediction (NWP). However, those models still rely on initial conditions from NWP, putting an upper limit on their forecast abilities. A few end-to-end systems have since been proposed, but they have yet to match the forecast skill of state-of-the-art NWP competitors. In this work, we propose Huracan, an observation-driven weather forecasting system which combines an ensemble data assimilation model with a forecast model to produce highly accurate forecasts relying only on observations as inputs. Huracan is not only the first to provide ensemble initial conditions and end-to-end ensemble weather forecasts, but also the first end-to-end system to achieve an accuracy comparable with that of ECMWF ENS, the state-of-the-art NWP competitor, despite using a smaller amount of available observation data. Notably, Huracan matches or exceeds the continuous ranked probability score of ECMWF ENS on 80.2% of the variable and lead time combinations. Our work is a major step forward in end-to-end data-driven weather prediction and opens up opportunities for further improving and revolutionizing operational weather forecasting.

Zekun Ni, Jonathan Weyn, Hang Zhang +7

Microsoft Corporation. · Department of Earth System Science, Ministry of Education Key Laboratory for Earth System Modeling, Institute for Global Change Studies, Tsinghua University, Beijing, China.

Ensemble LearningData AssimilationProbabilistic ForecastingWeather Forecasting
AskPDF