cs.LGMar 11, 2026

HAPEns: Hardware-Aware Post-Hoc Ensembling for Tabular Data

Authors: Jannis Maier, Lennart Purucker

Organizations: Helmholtz-Zentrum Berlin · TU Dortmund · Prior Labs · University of Freiburg

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Ensembling is commonly used in machine learning on tabular data to boost predictive performance and robustness, but larger ensembles often lead to increased hardware demand. We introduce HAPEns, a post-hoc ensembling method that explicitly balances accuracy against hardware efficiency. Inspired by multi-objective and quality diversity optimization, HAPEns constructs a diverse set of ensembles along the Pareto front of predictive performance and resource usage. Existing hardware-aware post-hoc ensembling baselines are not available, highlighting the novelty of our approach. Experiments on 83 tabular classification datasets show that HAPEns significantly outperforms baselines, finding superior trade-offs for ensemble performance and deployment cost. Ablation studies also reveal that memory usage is a particularly effective objective metric. Further, we show that even a greedy ensembling algorithm can be significantly improved in this task with static multi-objective weighting.

Figures & tables

Figure 1: Illustration of three ensemble selection strategies: a standard method ignoring hardware constraints, a naive hardware-aware variant that sacrifices accuracy, and an advanced hardware-aware method that balances accuracy and efficiency. Box size reflects model resource usage; the red dashed line indicates the hardware resource constraint.
Figure 2: Overview of the main research areas. HW-NAS (red) is shown as a parallel area, while the others (orange) directly influence HAPEns. This work focuses solely on tabular data (blue).
Figure 3: The HAPEns search process. Ensembles are sampled from bins over memory footprint and average loss correlation, then evolved via crossover and mutation to explore the behavior space.
Figure 4
Figure 5
Figure 8: Critical difference plots for the hypervolume across different hardware-aware objectives.
Figure 9: Comparison of constructed ensembles when including cost metrics in the ensembling process. The baselines and the hardware-aware methods in the density plots, produce a clear trend, where the ensembles of the latter methods are more condensed toward the x-axis.
Figure 8
Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

GroupIDDescription
Main ResultsEXP1Evaluation of HAPEns and baseline methods using the HV indicator, calculated from the ROC AUC of the test (inverted to a loss) and the averaged normalized cost metrics: inference time, memory usage, disk space usage.
DetailsEXP2Comparison of HV and IGD+ values to assess robustness between indicators.
 EXP3HV evaluation with respect to each individual cost metric (inference time, memory, disk space).
 EXP4Examining the differences in the construction of the ensemble between HAPEns and the baseline methods.
AblationEXP5Analysis of alternative cost metrics in the behavior space of HAPEns.
 EXP6Sensitivity analysis of different weightings in the Multi-GES method.
Appendix
Table 1: Overview of experiments
Figure 12: Normalized improvement over the single-best baseline method.
Figure 13: Overall number of ensembles generated per method and how many of these were unique when comparing the specific weight vectors.
Figure 14: The average number of Pareto ensembles created vs the total number of ensembles created per method on average. The averages are per seed and task to reflect real-world yields of these methods.
Figure 15: Density of ensembles produced by Multi-GES transitions to less expensive ensembles when increasing the time weight. The sub-plot titled Multi-GES shows the weighting we chose for our experiments: 0.68.
More in this topic

Ensemble Learning

Ensemble learning combines multiple predictors to improve accuracy, robustness, calibration, or uncertainty estimates.

See all 272 papers

Explore similar work

CardsList

  1. TabPack: Efficient Hyperparameter Ensembles for Tabular Deep Learning

    Jul 6, 2026Yury Gorishniy, Akim Kotelnikov, Ivan Rubachev +1Multilayer PerceptronsEnsemble Learning

    1Yandex · 2HSE University

  2. Choosing a parallel heterogeneous ensemble method for tabular classification

    Jul 6, 2026Vassili Maillet, Gustavo, Angulo +1Ensemble LearningTabular Classification

    Jesus · Mines Paris, PSL University, France

  3. TabLoRA: Parameter-Efficient Low-Rank Ensemble Learning for Large-Scale Tabular Data

    Jul 11, 2026Jiaqi Luo, Shixin XuEnsemble LearningLow-Rank Adaptation

    School of Mathematical Sciences, Soochow University, No.1 Shizi Street, Suzhou, 215006, Jiangsu Province, China · Digital innovation research center, Duke Kunshan University, No.8 Duke Avenue, Kunshan, 215000, Jiangsu Province, China