Ensemble Learning

Follow this topic Ask about this topic

Momentum

18 papers in the last four weeks, up 38% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 166

Aug 13, 2026cs.CV

MLLM-Routed Heterogeneous Ensembles for Robust Cross-Dataset Image Classification

Modern image classification models excel when trained on single task-specific datasets but often struggle to generalize across domains and difficulty levels. We propose ARMDIL, an Adaptive Router for Multi-Domain Image classification with LLMs. ARMDIL is an ensemble that uses a multimodal large language model (MLLM) agent to dynamically route each image to the most suitable vision backbone. Our diverse ensemble employs convolutional neural networks (ResNets), self-supervised representation learners (SSL), and vision-language models (VLMs), each trained on a unified label space constructed from multiple image datasets with differing distributions and characteristics. Empirical evaluations illuminate the distinct capabilities and vulnerabilities of each architecture across disparate visual domains. Crucially, we show that ARMDIL effectively navigates these trade-offs, performing competitively with specialized training-based routers. Furthermore, it drastically improves adaptability by allowing new information to be integrated via simple prompt modifications, while enhancing interpretability through natural language reasoning traces. These advances in cross-dataset image classification pave the way for more reliable general-purpose vision systems such as AI assistants and autonomous robots.

Daniel Perkins, John Squires, Janou Milligan +2

University of Tennessee, Knoxville · The Bredesen Center for Interdisciplinary Research and Graduate Education

Adaptive Model Routing Ensemble Learning Cross-Dataset Generalization Image Classification

Aug 12, 2026cs.CV

Automated binary classification of hazelnut X-ray images: A deep-learning benchmark for quality assessment

Non-destructive X-ray imaging can reveal internal hazelnut defects that are difficult to detect by external inspection alone; however, automated interpretation remains challenging because of subtle radiographic differences among classes, marked class imbalance, and limited annotated data. Here, we present a benchmark for binary hazelnut quality classification (healthy versus defective) based on 799 segmented single-kernel X-ray images (224 x 224 pixels, grayscale), grouped into 101 acquisition units. Seven single-model configurations and ten probability-aggregation ensembles were evaluated using a group-wise split-rotation protocol across five data splits generated using different random seeds. Decision thresholds were selected on the validation set, and performance was assessed deterministically on validation and test sets. Under the expert-reassessed annotation condition, the average-probability ensemble of the binary cross-entropy-trained convolutional neural network and frozen Swin Transformer achieved the highest mean balanced accuracy (86.3% +/- 1.8%, five seeds), with several other ensembles providing comparable performance. Across methods, substantial split-to-split variability was observed, indicating that multi-split evaluation is essential for reliable model comparison at this dataset scale. Expert reassessment of ambiguous samples improved the performance of all 17 evaluated methods by 2.8-8.1 percentage points, while having only a limited effect on cross-split variance. The results highlight both the potential of deep learning for automated X-ray-based hazelnut quality assessment and the importance of rigorous evaluation and label curation in small, imbalanced agricultural imaging datasets.

Giancarlo Sportelli, Nicola Belcari, Roberta Pace +4

Department of Physics “E. Fermi”, University of Pisa, Largo Bruno Pontecorvo 3, 56127 Pisa, Italy · National Institute for Nuclear Physics (INFN), Pisa Division, Largo Bruno Potencorvo 3, 56127 Pisa, Italy · Institute for Agriculture and Forestry Systems in the Mediterranean, National Research Council of Italy (ISAFOM-CNR), Piazzale Enrico Fermi 1, Località Porto del Granatello, 80055 Portici, Naples, Italy +1

Agricultural Image Analysis Ensemble Learning Binary Classification Image Classification

Aug 11, 2026cs.CV

Gaussian Meta-Space Augmentation for Stacking Ensembles in Multimodal IPMN Risk Stratification

Pancreatic cancer is among the most lethal malignancies; risk stratification of intraductal papillary mucinous neoplasms (IPMNs) offers a crucial opportunity for early intervention but typically requires invasive tissue biopsy. Dominant vision-based approaches, including radiomics and deep learning, provide promising but initially separate discrimination opportunities. Similarly, multisequence MRI (T1W/T2W) and anatomically decomposed (head, body and tail) analysis of the pancreas provide additional and potentially complementary signals. Effective fusion of this information is crucial in ordinal IPMN dysplasia risk prediction and can be accomplished via a meticulously regularized and calibrated ensemble stacking combiner. We present cUPMI, a class-conditional Gaussian augmentation of a combiner's log-probability meta-features, and test it on various prediction paradigms. In our multi-center analysis, we find cUPMI adds limited value to properly regularized L2-logistic binary classification stacks, but consistently regularizes higher-capacity tree combiners in the binary and radiomics-only setting (RF +0.015 and XGBoost +0.024 binary AUC, positive in all seeds). Its cleanest ordinal benefit appears for XGBoost on an 8-stream radiomics task (3-class no < low < high, +0.022 QWK in all seeds). Separately, fold-locked fusion of radiomics and 2.5D CNN streams yields the strongest overall model, an RF stack reaching QWK 0.595 (95% CI [0.54, 0.64]) and binary AUC 0.839, surpassing radiomics, 2.5D ResNet, and 3D DenseNet-121 baselines.

Max A. Nelson, Eminenur Sen Tasci, Zhixiang Wang +12

Machine & Hybrid Intelligence Lab, Department of Radiology, Northwestern University, Chicago, IL, USA · Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA · Istanbul Faculty of Medicine, Istanbul University, Istanbul, Turkey +2

Ensemble Learning Ordinal Regression Oncology Radiomics Multimodal Fusion Clinical Risk Prediction

Aug 11, 2026cs.CV

CLEAR: Class-wise Expert Aggregation with Structured Sampling for Long-Tailed Classification

Long-tailed classification poses a reliability challenge because models trained on imbalanced data are unevenly reliable across frequent and underrepresented classes. While existing methods address imbalance through re-balancing, adjustment, representation learning, or multi-expert modeling, they rarely estimate which expert should be trusted for each class. This paper proposes CLEAR (Class-wise reLiability-aware Expert Aggregation for long-tailed Recognition), a modular ensemble framework for long-tailed classification. CLEAR generates diverse experts through threshold-based structured sampling while preserving the full label space, then estimates a class-wise trust score for each expert using a smoothed class-wise precision formulation. During inference, expert predictions are combined through class-wise generalized product-of-experts aggregation, allowing different experts to be emphasized for different classes. Experiments on CIFAR-100-LT, ImageNet-LT, and Places-LT across multiple backbones show that CLEAR achieves competitive overall accuracy and particularly strong few-shot performance. These results support class-wise expert reliability as a useful design principle for long-tailed ensemble learning.

Gawon Lim

School of Information Sciences University of Illinois Urbana-Champaign

Long-Tailed Classification Ensemble Learning Classification Mixture-of-Experts Models

Aug 11, 2026cs.CL

EVIL-Detect for NLPCC 2026 Shared Task 6: LLM-Generated Text Detection

The rapid development of large language models (LLMs) has increased the need for reliable detection of LLM-generated text, especially in realistic Chinese scenarios involving human-written text (HWT), LLM-generated text (LGT), and LLM-refined text (HLT). This paper presents EVIL-Detect, a multi-signal ensemble framework with conflict-aware fusion for NLPCC 2026 Shared Task 6. The system integrates edit-extent regression, zero-shot likelihood-contrast signals, lexical statistics, and conservative text rules. With calibrated decision boundaries and conflict-aware integration, our system improves robustness under strong out-of-distribution shifts, achieving a macro-F1 score of 0.8888 and ranking first in the official evaluation. Our code is available at https://github.com/bbbbhrrrr/evildetect.

Hongrui Bao, Hangyu Rong, Zhuoshang Wang +2

Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China

AI-Generated Text Detection Language Model Generation Evaluation Ensemble Learning Robustness of Machine-Generated Text Detection

Aug 10, 2026cs.LG

REATS: LLM Reasoning-based Ensemble Learning for Adaptive Time Series Forecasting

Due to the diversity of real-world time series, no single forecasting model consistently dominates across all samples. Ensemble learning addresses this by combining complementary model strengths, yet existing methods rely on fixed rules or black-box models based solely on numerical inputs, failing to leverage LLM reasoning for interpretable weighting decisions. We propose REATS, which leverages LLM reasoning capabilities as an intelligent ensemble router that jointly processes textual temporal pattern descriptions and numerical features to produce interpretable, sample-adaptive ensemble weights through chain-of-thought reasoning. To enable effective LLM-based ensembling, we study its key design choices and propose: (i) a structured input pipeline that transforms raw time series into hybrid textual--numerical representations with fixed token cost, enabling rule-based chain-of-thought construction without API dependency, augmented with retrieved similar-sample priors; (ii) a diverse multi-row weight supervision scheme coupled with a token-efficient percentage-table format that reduces numerical complexity and mitigates LLM hallucinations; and (iii) a two-stage fine-tuning framework combining SFT with GRPO, where a reciprocal reward mapping transforms the continuous unbounded MSE gap into bounded signals with amplified near-oracle sensitivity, addressing the uniform sensitivity and outlier-dominated advantage compression inherent in naive reward designs for regression-based GRPO. Experiments on eight benchmarks demonstrate that REATS outperforms competitive ensemble baselines while providing natural language explanations and demonstrating strong transfer learning and out-of-domain generalization to unseen candidate models.

Xu Zhang, Chang Xu, Hui Sun +5

Fudan University Shanghai, China · Microsoft Research Beijing, China · Jilin University Jilin, China

RL for Language Models Explainable Time Series Forecasting Time Series Reasoning Ensemble Learning Time Series Forecasting LLM-Based Time Series Forecasting

Aug 10, 2026cs.LG

Full-Feature versus Limited-Input Machine Learning for Residential Energy Estimation: A Comparative Analysis of RECS and ResStock Under Realistic Input Constraints

Residential energy estimates are often needed before detailed envelope characteristics, equipment efficiencies, infiltration, sensor, or billing data are available. This study quantifies the trade-off between predictive accuracy and input accessibility using two nationally representative U.S. residential-energy datasets: the survey-based Residential Energy Consumption Survey (RECS) and the simulation-based ResStock dataset. Full-feature models were first used to establish dataset-specific performance benchmarks. For total-energy estimation, the models were subsequently restricted to ten low-burden variables obtainable from occupants, administrative records, or location-based weather data without an on-site energy audit. Among CatBoost, XGBoost, LightGBM, Random Forest, and Neural Networks, CatBoost consistently achieved the highest predictive performance for the full-feature analysis, reaching R2 = 0.90 for ResStock and R2 = 0.73 for RECS. When the feature set was restricted to ten homeowner-accessible inputs to simulate realistic deployment conditions, model performance converged to R2 = 0.61 for RECS and R2 = 0.62 for ResStock, showing that algorithmic complexity cannot fully compensate for missing physical and behavioral information. However, for a more homogeneous ResStock cohort consisting of single-family detached, natural-gas-heated homes in Climate Zone 6A constructed between 2000 and 2010, a reduced-input model improved accuracy to R2 = 0.85, demonstrating the value of targeted modeling for homogeneous populations. The results indicate that tree-based ensemble models can serve as high-fidelity emulators of national-scale residential energy datasets. However, careful consideration of feature availability, dataset origin (empirical vs. synthetic), and applicable use cases are also important.

Aditya Ramnarayan, Fatih Evren, Patti Gunderson +1

PhD Intern, Pacific Northwest National Laboratory · Researcher, Building Systems Group at Pacific Northwest National Laboratory · Data Scientist, Building Systems Group at Pacific Northwest National Laboratory

Surrogate Modeling Ensemble Learning Tabular Prediction

Aug 9, 2026cs.LG

A Domain-Structured Ensemble Framework for Perioperative Outcome Prediction Using Electronic Health Record Data

Perioperative risk prediction models are often limited by narrow surgical populations, incomplete intraoperative data, poor calibration, and limited interpretability. We present a domain-structured ensemble framework for perioperative outcome prediction using routinely collected electronic health record (EHR) data. Predictors are organized into patient-related, surgery-related, and anesthetics-related domains. Domain-specific gradient boosting models generate independent risk estimates that are integrated through a logistic regression meta-learner. We demonstrate the framework using postoperative delirium (POD) in a case-control sample of 5,386 surgical encounters (2,693 cases, 2,693 controls) from a statewide health information exchange. POD required both delirium-related ICD codes and a positive Confusion Assessment Method screening within seven postoperative days; patients with preexisting dementia were excluded. The stacked meta-learner achieved AUROC 0.899 (95% CI: 0.891-0.906), precision-recall AUC 0.881, and Brier score 0.126, compared with AUROC 0.849 for the best single-stage model. Domain ablation showed improved discrimination and calibration over a surgery-only model (AUROC 0.879, Brier 0.140). Temporal validation on held-out post-2017 data yielded AUROC 0.915. Calibration was excellent, with intercept -0.006 (95% CI: -0.083 to 0.070) and slope 1.035 (95% CI: 0.982 to 1.088). Decision curve analysis, corrected for case-control sampling, showed positive net benefit across clinically plausible thresholds. The modular framework supports alternative outcomes, extension of predictor domains, and dynamic risk updating, providing a scalable foundation for interpretable, calibration-aware perioperative clinical decision support.

Shikhar Shukla, Cristina Barboi

BioHealth Informatics Indiana University Indianapolis Indianapolis, USA · Department of Anesthesiology Indiana University School of Medicine Indianapolis, USA

Ensemble Learning Clinical Outcome Prediction Clinical Risk Prediction

Aug 8, 2026cs.LG

Accurate Ensembles, Fragile Narratives: Multi-Scale Stacking and a Fidelity Audit of LLM-Generated Explanations for Credit Risk

Credit scoring increasingly relies on models whose decision logic cannot be read off their parameters, in tension with supervisory expectations that adverse decisions be explainable. A common proposal closes that gap with a language model: compute feature attributions, hand them to an LLM, and let it write the rationale. We build such a system end to end and test whether the second half of the promise holds. The predictive component is a multi-scale stacking ensemble fusing four differently regularised gradient-boosting learners with a residual network through a neural meta-learner trained on out-of-fold predictions. On a public 32,581-application credit dataset it reaches test ROC-AUC 0.9539 (95% CI [0.9462, 0.9616]) and PR-AUC 0.9137, beating the best single model by Delta-AUC = 0.0143 (p = 0.016 under a conservative independence assumption). Our central finding is asymmetric. The ranking gain is real but operationally small: at the F1-optimal threshold the ensemble avoids only six additional missed defaults out of 1,422 against a tuned random forest, cutting cost-weighted loss by under 2%. The narrative layer fails in a way prompt engineering alone does not fix. In an audited case the model named three factors as risk-increasing that the supplied attributions scored as risk-reducing, omitted the dominant driver, and introduced a feature never given to it. We trace this to properties we measure rather than assume: SHAP and LIME agree on which features matter (overlap@10 = 0.80) but not on their order (tau = 0.43, p = 0.18), and the attribution sign for the model's most sensitive input is near a coin flip across applicants (modal-sign share 0.53). Calibration (ECS = 0.117) and perturbation stability (DPD = 0.078) both fall short of our own thresholds. Constrained prompting is necessary but not sufficient: grounding must be verified after generation, not assumed.

Gregorius Reynaldi Pratama, Kuo-Kun Tseng

School of Computer Science and Technology Harbin Institute of Technology, Shenzhen Shenzhen, China

Feature Attribution Faithfulness of Language Model Explanations Ensemble Learning LLM Interpretability Credit Scoring Explainability Evaluation

Aug 7, 2026cs.LG

Uncertainty-Aware Ensemble Deep Randomized Neural Networks for Classification

The current state-of-the-art (SOTA) deep randomized neural networks, such as deep Random Vector Functional Link (dRVFL) and ensemble deep RVFL (edRVFL), treat all training samples uniformly, which limits their robustness and effectiveness when applied to real-world datasets containing noise and outliers. Furthermore, the propagation of contaminated features across hidden layers negatively influences the decision-making capability of these models. To overcome these limitations, we propose intuitionistic fuzzy dRVFL (IF-dRVFL) and intuitionistic fuzzy edRVFL (IF-edRVFL) frameworks that enhance model robustness. The proposed models unify intuitionistic fuzzy theory to exploit sample neighborhood information in the kernel space by jointly considering membership and non-membership degrees for each sample. Membership degrees are computed based on the distance of samples from their respective class centroids, while non-membership degrees quantify sample heterogeneity within local neighborhoods. These measures are employed to assign adaptive weights to training samples, enabling effective discrimination among clean, noisy, and outlier data points. Extensive experiments conducted on UCI and KEEL benchmark datasets, with and without the presence of Gaussian noise, demonstrate the superiority of the proposed IF-dRVFL and IF-edRVFL models over existing SOTA fuzzy and non-fuzzy approaches. The source code is available at https://github.com/mtanveer1/IF-edRVFL.

M. Sajid, A. Quadir, A. Rahaman +2

Indian Institute of Technology Indore, Simrol, Indore, India · Qatar University, Qatar

Ensemble Learning Neural Network Robustness Random Feature Methods Fuzzy Logic Learning with Noisy Labels

Aug 6, 2026cs.AI

Hybrid Machine Learning Framework for Herd-Level Cattle Growth Pattern and Weight Gain Forecasting in Grazing-Based Production Systems

Commercial grazing systems yield irregular livestock observations, which challenge cattle growth forecasting. This study developed a hybrid machine learning framework for herd level cattle weight forecasting using automated sensing observations collected between 2022 and 2024 in southeastern Australia. Weekly live weight observations, demographic variables, and lagged environmental predictors were integrated into structured forecasting datasets. Herd level forecasting trajectories were generated through temporal aggregation of animal level predictions. Four hybrid architecture families were evaluated, including residual, stacked, cascade, and ensemble assisted frameworks. ARIMA, LSTM, and GRU models were used as comparative baselines. Independent testing demonstrated strong predictive agreement across multiple forecasting horizons. The cascade GB to RF to NN architecture achieved the best performance, with a test R^2 of 0.889, RMSE of 21.319 kg, and MAE of 15.462 kg. Hybrid architectures maintained greater robustness than recurrent sequential models under sparse observation conditions. Forecasting error increased progressively across extended prediction horizons. Feature importance analysis identified animal age, rainfall, and temperature as dominant predictors influencing herd level growth forecasting. The proposed framework may support feed allocation, grazing management, and livestock marketing decisions under heterogeneous sensing environments.

Muhammad Riaz Hasib Hossain, Rafiqul Islam, Shawn R. McGrath +2

a School of Computing, Mathematics and Engineering, Charles Sturt University, Wagga Wagga, NSW 2650, Australia · b School of Computing, Mathematics and Engineering, Charles Sturt University, Albury, NSW 2640, Australia · c Gulbali Institute for Agriculture, Water and Environment, Charles Sturt University, Wagga Wagga, NSW 2678, Australia +4

Irregular Time-Series Modeling Multivariate Time Series Forecasting Ensemble Learning Time Series Forecasting

Aug 6, 2026cs.CV

Bayesian adaptively-weighted ensembles for few-shot abdominal segmentation

Few-shot learning has emerged as a promising approach for anatomical segmentation when labelled data are scarce. However, different few-shot learning algorithms exhibit complementary strengths and weaknesses, with performance varying across anatomical targets and institutions. Existing few-shot segmentation ensembles, that combine predictions from multiple algorithms, typically employ fixed weighting schemes and therefore cannot adjust model contributions according to the target domain. In this work, we propose a Bayesian adaptively-weighted ensemble framework for segmentation under label scarcity and domain shift. Multiple few-shot segmentation algorithms are first adapted using a small labelled support set. Bayesian optimisation is then used to automatically identify ensemble weights that maximise segmentation performance on a target-domain validation set. The learned weights are subsequently fixed and applied to combine predictions on previously unseen query images from the target domain. The proposed framework is evaluated on the Cross-institution Male Pelvic Structures dataset using held-out anatomical structures and institutions to simulate simultaneous label scarcity and institutional domain shift. Results demonstrate statistically significant improvements over individual few-shot learners, fixed-weight ensembles, training-from-scratch baselines and recent state-of-the-art ensembling approaches. By adapting model contributions to the target anatomy and institutional domain, the proposed framework provides a practical mechanism for deploying segmentation systems to new clinical sites under severe annotation constraints.

Abbas Al-Sabbagh, Shalom F. Mushtaq, Tomás M. da Silva +7

Centre for Bioengineering, School of Engineering and Materials Science, Queen Mary University of London, London, United Kingdom · Digital Environment Research Institute, Queen Mary University of London, London, United Kingdom · Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, United Kingdom +1

Ensemble Learning Medical Image Segmentation Few-Shot Medical Image Segmentation

Aug 4, 2026cs.AI

Adversarially Robust Abductive Fusion of Pre-trained Transformer-based Perception Models

Deploying pre-trained perception models in novel environments degrades their accuracy under distributional shift, and assembling them alone does not recover it: combiners such as majority voting trade recall for precision and are brittle to coordinated failures. Prior metacognitive methods learn logical rules that flag a model's errors, but rely on hand-authored domain-knowledge cues (object-size priors, segmentation masks) that do not transfer to genuinely novel scenes. We show that this metacognitive layer can be learned without any domain knowledge by exploiting vector-space geometry: per-model Label Vector Pools (LVP), built from each model's own training embeddings, yield error-detection rules from the geometry of detections relative to training-determined prototypes, reaching parity with domain-knowledge rules to within

0.002

every F1 on test set. Because the approach remains neurosymbolic, these geometric rules share a single logical framework and can still be complemented by domain knowledge when available. We frame the fusion of multiple imperfect ViT-based detectors as a consistency-based abduction problem solved at test time by an exact Integer Program (IP) and a polynomial-time heuristic. On an aerial-imagery benchmark of 15 weather-shifted test sets and six ViT detectors, our domain-knowledge-free layer matches the strongest majority-vote variant on clean data (within

0.005

F1) and, unlike every majority-vote baseline, retains its performance under a coordinated label-flipping attack: at a

90\%

flip rate it averages

0.42

F1 versus

0.35

for MV-Plurality (a

22\%

relative gain) and attains the highest F1 on \emph{every} test set once the flip rate exceeds

0.4

Mario Leiva, Yue Ma, Qinru Qiu +2

1DCIC, Universidad Nacional del Sur (UNS) & ICIC (UNS-CONICET), Bahía Blanca, Argentina · 2Syracuse University, Syracuse, NY USA

Distribution Shift Abductive Reasoning Ensemble Learning Adversarial Robustness Object Detection

Aug 4, 2026cs.CV

Frequency-Decorrelated Temporal Ensembles for EEG--fNIRS Imagined-Handwriting Decoding

Imagined handwriting offers a temporally rich paradigm for non-invasive neural decoding, yet reliable recognition across unseen participants remains difficult because scalp EEG is noisy and internally generated stroke sequences vary across individuals. The Multimodal Brain-Computer Interface Grand Challenge provides synchronized EEG and fNIRS for four-class subject-independent handwriting-trajectory classification. We propose FRED, a task-adapted system that models imagined handwriting as a multi-second motor sequence and trains a compact multi-scale temporal network on three complementary EEG frequency views. With three seeds per view, cross-band members produce substantially less-correlated errors than same-band replicas, yielding a clean nine-member ensemble accuracy of 0.8076/0.7242/0.7492 on the public/private/overall test partitions without test-set adaptation or output constraints. The submitted pipeline further incorporates transductive pseudo-label training, three EEG-Conformer members, posterior aggregation, and a paradigm-aware decoder. Because every 12-trial randomization block contains three instances of each class, the final predictions are obtained by Hungarian assignment under the known block quota. On one fixed posterior pool, independent, session-constrained, and block-constrained decoding achieve 0.7600, 0.7758, and 0.7952 overall accuracy, respectively. The complete system reaches 0.8498/0.7718/0.7952, ranking fourth on the private split. A modality audit finds fNIRS-only decoding at chance (0.2511 overall), while adding fNIRS to EEG changes accuracy by only +0.0025. These results identify frequency-diverse temporal EEG modeling and protocol-matched structured inference as the principal sources of performance in this sparse-montage EEG--fNIRS setting. The source code is available at https://github.com/XiuFan719/EEG-fNIRS-fuse-method-for-MM-challenge.

Xiao Fan, Hongbin Guo, Yubo Han +1

Xidian University

Cross-Subject EEG Decoding Brain-Computer Interfaces Ensemble Learning

Aug 3, 2026cs.RO

Why Does Action Chunking Improve Behavioral Cloning Performance in Robotic Control?

Action chunking---predicting and executing multiple actions instead of a single action---has proven to be a critical component for learning effective robotic control policies. However, our precise understanding of why action chunking improves performance has remained limited. In this work we seek to close this gap. Through rigorous experimental evaluations in both simulated and real-world settings, we show that existing hypotheses for the success of action chunking---temporal consistency, horizon reduction, and representation learning---fail to explain the success of action chunking. Instead, we find that action chunking benefits from greater non-Markovian expressivity and reduced compounding error compared to Markovian policies, but, in many settings of interest, these effects can be fully captured by delayed policies, which at each step predict a single action based on the observation

k

steps in the past. We then show that there exists an additional benefit of action chunking that we refer to as implicit ensembling. In particular, by learning a diversity of temporal relationships (that is,

a_t | o_t, a_t | o_{t-1}, \ldots

), action-chunked policies exhibit behavior matching that of a model ensemble, increasing their robustness and generalization ability over policies that only learn a single temporal relationship. Building on these insights, we show that in simulated and real-world robotic control settings, we can match the performance of action chunking without action chunking---by deploying an action chunking policy as an ensemble of policies with randomized delays. Furthermore, we propose a policy class that amplifies the benefits of action chunking by explicitly instantiating an ensemble, and which we show significantly improves over the performance of action chunking in many domains.

Filippo Lazzati, Kyle Stachowicz, William Chen +3

Politecnico di Milano · UC Berkeley

Ensemble Learning Robot Imitation Learning Action Chunking Robot Policy Generalization

Aug 3, 2026cs.LG

An AI-Based Decision-Support Pipeline for Day-Ahead Photovoltaic Forecasting

Reliable photovoltaic (PV) forecasts can support low-carbon energy systems, but deployed sites may have only short and incomplete records. Physical and hybrid methods can be sensitive to weather inputs, calibration, and timestamp-alignment, while individual machine learning models may capture different parts of the forecasting problem. We study hourly day-ahead PV forecasting at a United Kingdom charging station using one year of inverter measurements, with 9.25% of hours missing. The pipeline checks timestamp-alignment, derives solar and clearness features, adds short-term weather context, and combines five complementary models using non-negative least squares stacking, with the combination fitted only on validation observations. We compare against smart persistence, a weather-scaled baseline that carries the previous day's PV behaviour forward using target-day irradiance. With retrospective weather, the combined model reduces daylight normalised root mean square error (RMSE) by 31.2% under random day-fold evaluation and by 2.9% under rolling-origin evaluation, although the latter improvement is not robust across days. It also improves by 3.0% over the single model selected from validation performance. Replacing retrospective weather with a public product sampled at a constant 24-hour lead increases daylight RMSE by 13.1% and 4.2% under the two protocols, while retaining positive skill over smart persistence.

Fariba Dehghan, Sebastian Stein, Vahid Yazdanpanah +2

University of Southampton

Ensemble Learning Time Series Forecasting Solar Power Forecasting Power Systems

Aug 2, 2026cs.CV

ST-LoRA: Single Trajectory LoRA Ensemble for Uncertainty Aware Agricultural Segmentation

Reliable decision support in digital agriculture requires not only accurate predictions but also well-calibrated uncertainty estimates, particularly for dense prediction tasks such as semantic segmentation. Ensembles provide strong uncertainty quantification but are computationally and memory demanding, while single-model approximations often sacrifice uncertainty quality. We propose ST-LoRA, a parameter-efficient ensemble that builds diverse members from a single training trajectory by combining Low-Rank Adaptation (LoRA) with snapshot ensembling. All members share a frozen pretrained backbone and differ only in lightweight low-rank adapters, which sharply reduces trainable parameters, checkpoint storage, and I/O overhead. We evaluate SegFormer, Mask2Former, and EoMT on GrowliFlower-L (cauliflower, open field) and BUP20 (sweet pepper, glasshouse), covering in-distribution performance, calibration under covariate shift, and near- and far-out-of-distribution (OoD) detection, with BUTom21 (tomato) as near-OoD data. Extensive ablations show that feed-forward layers, not attention projections, are the critical LoRA target for dense prediction, and that the scaling ratio

α/r

governs an accuracy--calibration trade-off. Against full-rank snapshot ensembles, ST-LoRA is competitive in segmentation quality, with architecture-dependent training time and energy savings. Against MC Dropout, DDU, and six post-hoc calibrators, it achieves the strongest far-OoD image-level detection and near-OoD pixel-level localization with low cross-seed variance, although full-rank ensembles remain better calibrated. These results show that LoRA-based ensembling offers a compelling efficiency--performance trade-off for agricultural vision systems.

Mohamed Farag, Genc Hoxha, Yahia Maleki +2

Machine Learning in Agriculture Lab, Institute of Geodesy and Geoinformation, University of Bonn, Bonn, Germany · Commonwealth Scientific and Industrial Research Organisation (CSIRO), Australia · Lamarr Institute for Machine Learning and Artificial Intelligence, University of Bonn, Bonn, Germany

Image Segmentation Ensemble Learning Low-Rank Adaptation Deep Ensembles OOD Detection Segmentation Uncertainty Estimation

Aug 2, 2026cs.LG

Factorized AdaBoost.MH Achieves the Same Convergence Rate as AdaBoost.MH

{AdaBoost.MH} reduces multi-class classification to a collection of binary subproblems and enjoys the classical boosting-type convergence guarantee under a weak learning condition. A more structured variant, Factorized {AdaBoost.MH}, uses base classifiers of the form

\mathbf{h}(x)=α\mathbf{v} \bm{\varphi}(x)

, where a single binary classifier

\bm{\varphi}

is shared across all classes and the label dependence is carried by a vote vector

\mathbf{v} \in\{\pm1\}^K

. This factorization is algorithmically attractive and achieves better performance in practice, but its convergence depends on whether one can always choose a vote vector with sufficiently large induced binary weight mass. Previous work resolved this question with a lower bound

\max\{1/n,1/\sqrt{2K}\}

, which still leaves a dimension-dependent slowdown relative to the original {AdaBoost.MH} analysis. In this paper, we sharpen this combinatorial step. For the minimax quantity

\mathfrak{W}_{n,K}

governing the factorized edge, we prove

\mathfrak{W}_{n,K} = C_{\min\{n+1,K\}}

, where

C_q=1

for

q=1

,

C_q=q/(3q-4)

for even

q\ge2

, and

C_q=(q+1)/(3q-1)

for odd

q\ge2

. Since

C_q\downarrow 1/3

, our bounds show that

\mathfrak{W}_{n,K}=Θ(1)

uniformly over

n

and

K

. Consequently, Factorized {AdaBoost.MH} achieves the same boosting-type convergence rate as {AdaBoost.MH} up to a universal constant factor, removing the previously suggested additional dependence on

n

or

K

in the number of boosting rounds.

Xin Zou, Jingyuan Xu

Ensemble Learning Multiclass Classification Classification Statistical Learning Theory

Aug 2, 2026cs.LG

Breaking Diversity Collapse in Spiking Pseudo-Ensembles for Efficient OOD Detection in Remote Sensing

Spiking Neural Networks (SNNs) are attractive for resource-constrained remote-sensing systems, but reliable out-of-distribution (OOD) detection remains challenging. Deep ensembles provide strong predictive uncertainty, yet require multiple complete models and backbone evaluations. We propose an efficient spiking pseudo-ensemble that attaches multiple lightweight classification heads to a frozen SNN backbone. Naively training these heads with cross-entropy can lead to diversity collapse, where independently parameterized heads may produce correlated predictions. To address this, we introduce an agree--disagree objective that preserves correct predictions on clean in-distribution samples while encouraging diversity on structured, uncertainty-inducing transformations of the same inputs. This provides a diversity-promoting training signal without requiring external OOD data. Experiments with Spikformer and ResNet19-SNN on EuroSAT demonstrate consistent improvements over conventionally trained pseudo-ensembles. Using three backbones with five heads each matches or improves upon a five-model deep ensemble on UCM and AID, while requiring approximately 38% fewer parameters and 40% fewer backbone evaluations. These results show that explicit diversity promotion can recover useful ensemble-style uncertainty at substantially lower deployment cost.

Srinivas Anumasa, Rushi Shah, Qiran Zou +1

National University of Singapore

Efficient Neural Network Inference Ensemble Learning Remote Sensing Spiking Neural Networks OOD Detection Ensemble Diversity

Jul 31, 2026cs.CV

CALM-AH: An ABAW11-Calibrated Multimodal Ensemble with Reliability-Gated Multi-Expert Consensus for Video-Level Ambivalence and Hesitancy Recognition

Ambivalence and hesitancy (A/H) are subtle behavioural states that may be expressed through language, voice, facial activity, and other non-verbal cues. The ABAW11 A/H Video Recognition Challenge asks systems to assign a binary A/H label to each naturalistic interview video. Performance is measured using Macro-F1 so that recognition of both A/H and No-A/H samples receives equal importance. We present CALM-AH, a multimodal ensemble that combines textual, acoustic, visual, and derived behavioural-statistical features. We construct 15 non-empty combinations of these feature branches. For each combination, we select the best of three classifier families using validation binary cross-entropy and optimise its decision threshold for validation Macro-F1. The resulting binary decisions are combined using fixed hard-voting weights transferred from BROTHER. We further introduce Reliability-Gated Multi-Expert Consensus(RG-MEC), an anchor-preserving decision-level ensemble that combines an initial prediction with three complementary correction experts: CALM-AH, AffectGPT, and a GPT-based semantic verifier. The initial system provides the default prediction. Its label is overridden only when all three correction experts unanimously support the same alternative class; otherwise, the anchor prediction is retained. This unanimity-gated design limits the influence of isolated expert errors while permitting bidirectional correction when task-specific, multimodal-affective, and semantic-pragmatic evidence are fully consistent. On the participant-disjoint ABAW11 dataset, CALM-AH achieves a Macro-F1 of 0.7525, and the complete RG-MEC system achieves 0.7771.

Wenzhuo Sun, Mingjian Liang, Richard Attfield +3

Monash University, Melbourne, Australia · Monash Suzhou Research Institute, Suzhou, China

Ensemble Learning Multimodal Emotion Recognition Ambivalence and Hesitancy Recognition Multimodal Learning

Jul 30, 2026cs.LG

Semi-Supervised Learning for Molecular Graphs via Ensemble Consensus

Machine learning is transforming molecular sciences by accelerating property prediction, simulation, and the discovery of new molecules and materials. Acquiring labeled data in these domains is often costly and time-consuming, whereas large collections of unlabeled molecular data are readily available. Standard semi-supervised learning methods often rely on label-preserving augmentations, which are challenging to design in the molecular domain, where minor changes can drastically alter properties. In this work, we show that semi-supervised methods that rely on an ensemble consensus can boost predictive accuracy across a diverse range of molecular datasets, task types, and graph neural network architectures. We find that training with an ensemble consensus objective increases robustness in models and exhibits an effect similar to knowledge distillation; an individual member of an ensemble trained this way outperforms a full ensemble trained in a traditional supervised fashion in almost all cases. In addition, this type of semi-supervised training reduces calibration error.

Rasmus Tirsgaard, Laurits Fredsgaard, Marisa Wodrich +2

Cognitive Systems, Department of Applied Mathematics and Computer Science, Technical University of Denmark.

Ensemble Learning Graph-Based Semi-Supervised Learning Molecular Property Prediction Semi-Supervised Learning Molecular Representation Learning

Jul 29, 2026stat.ML

Feature Bagging Provides Stability

We study feature bagging through the lens of algorithmic stability. Feature bagging is an ensemble strategy that aggregates base learners trained on randomly subsampled feature subsets, possibly in a data-dependent manner. We introduce feature instability (FI), the feature-axis analogue of instance instability (II), which measures sensitivity to removing a single feature. Smaller values of II or FI correspond to stronger stability, and our experiments show that FI captures generalization-relevant information complementary to II. Within this framework, we analyze feature bagging in both a parametric linear model and a model-free setting inspired by recursive feature subsampling in random forests. In both settings, we establish formal guarantees showing that feature bagging improves the relevant stability relative to its non-bagged counterpart, with larger improvements under more aggressive subsampling. We further show that a modest number of bagging rounds is sufficient to approach the infinite-bagging stability level.

Yuheng Ma, Qiang Sun

School of Statistics, East China Normal University · University of Toronto · MBZUAI

Ensemble Learning Algorithmic Stability Statistical Learning Theory

Jul 29, 2026cs.LG

Tight Generalization Bound for AdaBoost

In this paper we show that the generalization error of AdaBoost is

Θ\big(\tfrac{d\ln(nγ^{2}/d)}{nγ^2}+\tfrac{\ln(1/δ)}{n}\big)

, where

γ

is the advantage guaranteed by the weak learner,

d

is the VC-dimension of the class containing the weak hypotheses,

n

is the sample size, and

δ

is the confidence parameter. The contribution of this paper is the upper bound; the matching lower bound follows from prior work. The upper bound proof follows by combining the known fact that AdaBoost outputs a voting classifier whose voting function has zero empirical

γ/2

-margin loss with what is, to the best of our knowledge, a new margin-based generalization bound for voting classifiers.

Mikael Møller Høgsgaard

Department of Statistics, University of Oxford

Ensemble Learning Generalization Bounds

Jul 28, 2026cs.CV

Leak-Free Cross-Validated Stacking with Per-Architecture Calibration for Sand-Boil Segmentation in Earthen Levees

Sand boils, points where water seeping beneath an earthen levee re-emerges at the surface, are early warnings of internal erosion, and deep segmentation networks are increasingly used to find them in inspection photographs. Annotated examples are scarce, and two common ways of working around that scarcity quietly inflate reported accuracy: tuning ensemble weights on the same images later used to score them, and training on synthetic images derived from the very photographs held out for testing. We present a sand-boil segmentation framework that closes both loopholes. Every synthetic image carries a pointer to its real parent, and a per-fold filter excludes any image whose parent is held out; five encoder-decoder backbones are trained under five-fold cross-validation, calibrated by one temperature scalar each, and combined by a per-pixel meta-learner fitted only on out-of-fold predictions. On the held-out test set the proposed Updated SandBoilNet reaches an intersection-over-union of 0.707 over three seeds, against 0.608 for the published original re-evaluated on the same split. Under the stacking protocol the calibrated stack reaches 0.681 against 0.694 for the strongest fold-averaged member, so it does not improve on the best single model; eight meta-learner families reproduce that outcome, which we trace to a mean pairwise error correlation of 0.894 among members. A synthetic pool filtered for label fidelity lifts the champion to 0.718 over three seeds against a 0.707 control. We also introduce a mask-conditioned synthesis route that makes the conditioning mask the label by construction, giving labelled training images at zero annotation cost.

Padam Jung Thapa, Anav Katwal, Ayon Dey +4

The Center for Advanced Computer Studies (CACS), School of Computing and Informatics, University of Louisiana at Lafayette, Lafayette, LA 70504, USA · Department of Computer Science, LSU New Orleans, New Orleans, LA 70148, USA · US Army Corps of Engineers, Engineer Research and Development Center, Vicksburg, MS 39180, USA

Image Segmentation Ensemble Learning Industrial Inspection

Jul 27, 2026cs.LG

Calibrated Tree-Neural Fusion for Fine-Grained Vegetation Community Classification

Accurate vegetation-community classification is essential for ecological monitoring, habitat assessment, and evidence-based environmental management in heterogeneous landscapes. Existing studies often rely on standalone tree ensembles or generic neural networks, although fine-grained ecological classes frequently exhibit overlapping spectral, topographic, and structural characteristics. Many frameworks also provide limited protection against stacking leakage, insufficient probability calibration, weak minority-class evaluation, and little evidence of stability across repeated data splits. To address these limitations, this study proposes Calibrated EcoTreeFuseNet-Plus, a tree-neural probability-fusion framework that combines out-of-fold tree probabilities, EcoFuseNet-V2 outputs, validation-selected meta-learning, and post-hoc temperature scaling. Raster values from six LiDAR-derived terrain and canopy variables and two hyperspectral vegetation indices were extracted at coordinate-based reference locations. Quality control removed 26 samples with missing elevation and one sample with non-finite NDWI, producing 1,833 complete records across 29 vegetation and non-vegetation classes. On the held-out test set, the proposed model achieved an accuracy of 0.8000, a macro F1-score of 0.7768, a balanced accuracy of 0.7903, and an MCC of 0.7903. Calibration reduced the expected calibration error from 0.3866 to 0.0651 without changing class predictions. Five-seed evaluation yielded a macro F1-score of 0.7717 +/- 0.0112, indicating stable performance across repeated splits. The results demonstrate a reliable discrimination-calibration trade-off for small-sample, fine-grained ecological classification.

Dristi Datta, Md Khalid Hasan Sakib, Manoranjan Paul

School of Information Technology and Engineering, Sydney Metropolitan Institute of Technology (SydneyMet), Sydney, NSW, 2000, Australia · Department of Electrical & Electronic Engineering, Rajshahi University of Engineering & Technology, Rajshahi, Bangladesh · School of Computing, Mathematics and Engineering, Charles Sturt University, Bathurst, NSW, 2795, Australia

Remote Sensing Image Understanding Ensemble Learning Remote Sensing

Jul 27, 2026cs.LG

MAPLE: Efficient and Diverse Multi-Alpha Generation for Portfolio Construction

Classical alpha mining achieves strong risk-adjusted returns by combining many low-correlated predictive signals, yet deep learning stock-ranking methods typically produce a single alpha per stock, rely on increasingly complex architectures with diminishing gains, and obtain diversity only through separate models or implicit routing, without explicitly controlling inter-alpha correlation. We introduce MAPLE (Multi-Alpha Position-aware Listwise Ensembling), a backbone-agnostic framework that recovers this diversity principle within a single training pass. MAPLE combines a unified, capacity-scaled prediction head with an extreme-rank weighted listwise ranking loss and a diversity regularizer that explicitly penalizes pairwise correlation across alphas. Across four equity markets spanning the US, China, and Japan, MAPLE achieves the best average Sharpe and Calmar ratios among nine baselines, using up to 55x fewer parameters and 2.5x less training time, and generalizes across five backbone architectures with Sharpe and Calmar Ratio gains of 10-23% and 17-43%, respectively. Behavioral analysis further shows why each component works: the unified head already reduces inter-alpha correlation before any diversity loss is applied, and the extreme-rank loss lets diversity regularization improve rather than erode per-alpha ranking quality as capacity scaling sustains this balance at scale. These results show that principled loss design and capacity allocation, rather than architectural complexity, drive diverse and effective multi-alpha generation.

Yu-Chen Den, Kuan-Yu Chen, Kendro Vincent +1

SinoPac Holdings · Taipei, Taiwan · National Chengchi University

Learning to Rank Ensemble Learning Portfolio Optimization Finance

Jul 27, 2026cs.LG

Variational Boosting for Physics-Informed Neural Networks

Physics-Informed Neural Networks (PINNs) solve differential equations by minimizing the residual of a nonlinear operator over a neural parameterization of the solution. However, monolithic PINNs often suffer from ill-conditioning, spectral bias, and optimization instability. We introduce a variational boosting framework in which solutions are constructed additively in function space. Each stage trains a weak learner whose converged correction satisfies a local orthogonality condition, equivalent to a projected functional gradient descent step onto the tangent space of the network's function manifold. Because each correction network is deliberately small, the restricted minimization admits full Newton or conjugate gradient updates, which are typically infeasible in large PINNs. The resulting method separates global nonlinear refinement into a sequence of well-conditioned subproblems while preserving the full variational structure of the operator. This framework provides a geometric interpretation of multi-stage PINNs as projected functional gradient descent and enables stable second-order optimization for nonlinear differential equations.

Kaylee Vo, Pavlos Protopapas

Harvard University

Ensemble Learning PDE Solving Second-Order Optimization Physics-Informed ML

Jul 26, 2026cs.LG

Controllable Diversity in Normalization-Based Implicit Ensembles via Softmax-Temperature Modulation

Deep ensembles provide the most reliable uncertainty estimates in deep learning, but their cost grows linearly with the number of members. Implicit ensembles lower this cost by sharing a single backbone across members. Member diversity is a primary determinant of ensemble quality, yet no implicit ensemble can shape it during training; existing methods fix it at initialisation or build it into the architecture. We introduce

σ

N-Ens, a normalisation-based implicit ensemble that treats each member as a task in a multi-task architecture and modulates the shared backbone through sigmoid-bounded scalers. We also introduce a softmax-temperature regulariser, which shapes the equilibrium level of sharing between members and traces the accuracy-calibration frontier. Because only normalisation layers are replicated, the mechanism can wrap convolutional and transformer backbones alike, also allowing pretrained models to be adapted through a short fine-tune. We frame the epistemic uncertainty such an ensemble expresses as modulation uncertainty, and explain why its calibration holds under input corruption, and why its out-of-distribution detection is weaker. Our method is evaluated across ResNets and transformers on CIFAR-10/100, ImageNet and SST-2.

σ

N-Ens matches or outperforms deep ensembles at a fraction of their parameter cost, scales with ensemble size where partitioning methods collapse, and maintains calibration under distribution shift.

Mihai Suteu, Ovidiu Serban

Data Science Institute, Imperial College London

Ensemble Learning Ensemble Diversity Uncertainty Calibration Epistemic Uncertainty

Jul 26, 2026cs.LG

Covariance Last-Layer Ensembles: Function-Space Diversity for Efficient Uncertainty Quantification

A Last-Layer Ensemble (LLE),

K

linear units on one shared frozen feature map, is an efficient single-pass approach to the disagreement-based epistemic uncertainty for out-of-distribution (OOD) detection. Its weakness is that members share the backbone gradient and can converge toward the same function, collapsing the inter-member diversity the signal depends on. Whether last-layer diversity can be restored, and what mitigates the collapse, is an open question. The weight-orthonormality defining Orthonormal Certificates (OC), the weight-orthonormal special case of the LLE, is only an indirect correction; it decorrelates the weights of the members, not their predictions. Here, we instead target the collapse directly in function space, with a Covariance Last-Layer Ensemble (cov-LLE) that places a direct covariance penalty on member activations. Cov-LLE restores the function-space diversity that weight-orthonormality cannot, and at matched

K

recovers much of the diversity and calibration of a deep ensemble at

1\times

backbone cost (in-distribution prediction variance

0.05\!\to\!9.3

vs.

22.1

(

\times10^{-3}

), and ECE

0.135\!\to\!0.090

vs.

0.035

, for a

K\times

-cost deep ensemble), at no cost to accuracy. Viewing OC as a last-layer ensemble also organizes detectors into a two-axis taxonomy (by how their units are trained and how their outputs are scored) and exposes the OC score as a magnitude, motivating a scale-invariant, label-free direction score that repairs its near-OOD failure, adding

+0.16

to

+0.18

ROC AUC on every backbone.

H. Martin Gillis, Isaac Xu, Gabriel Spadon +1

Dalhousie University Halifax, Nova Scotia, Canada

Uncertainty Quantification Ensemble Learning Deep Ensembles OOD Detection

Jul 24, 2026cs.LG

A Leakage-Free Stacked Ensemble Method for Multiclass Classification

Multiclass classification is a fundamental problem across a wide range of domains. It is still challenging due to possession of high inter-class similarity, class imbalance datasets, and variability in data distributions. Rule-based classifiers such as XGBoost often achieve stronger performance on structured features, but they are limited in capturing smooth functional relationships among variables. Similarly, neural network models can represent complex nonlinear interactions but frequently suffer from overfitting and generalization issues. To address these limitations, we propose LFS-FRAME, a Leakage-Free Stacked ensemble framework that integrates functional learning using Kolmogorov-Arnold Networks (KAN) and rule-based learning via XGBoost for robust multiclass classification. The proposed framework constructs unbiased meta-features by employing a strict out-of-fold stacking strategy to ensure complete isolation between training and validation data hence preventing performance leakage. By learning over probabilistic outputs from heterogeneous base learners, the meta-classifier effectively exploits both global functional patterns and sharp decision boundaries present in the complex data. Experimental evaluations on multi-class datasets demonstrate that LFS-FRAME improves performance metrics, and overall accuracy is 89.85% in identifying major families and 81.74% in identifying sub-families relative to strong single-model baselines. These results highlight the effectiveness of leakage-free functional and rule-based stacking for reliable and generalizable multiclass classification.

S. P. Sharmila, Aruna Tiwari

Indian Institute of Technology Indore, Madhya Pradesh, India. · Siddaganga Institute of Technology, Tumakuru, Karnataka, India.

Ensemble Learning Multiclass Classification