Robust Satisficing Ensemble: Scalable Model Aggregation Under Distribution Shifts
Organizations: Dept. of Electrical and Electronics Eng., Bilkent University, Ankara, Türkiye
Abstract
Standard ensemble methods can improve robustness by mitigating individual model stochasticity, but they frequently fail to generalize beyond the training distribution, leaving systems vulnerable to out-of-distribution (OOD) data. We introduce the Robust Satisficing Ensemble (RSE), a scalable probabilistic aggregation framework designed to secure model combinations by minimizing their fragility against distributional changes. By bridging a robust satisficing objective with a quadratic risk surrogate, RSE derives an analytical inner adversary. This fundamental reformulation enables highly efficient weight updates without the computational bottleneck of standard minimax optimization, crucially decoupling optimization complexity from sample size to scale effectively to large datasets. In particular, we prove a geometrically decaying truncation error for the fragility approximation via Lagrange-Burmann expansion and establish stationarity guarantees for RSE. We evaluate RSE across a comprehensive spectrum of distribution shifts, including adversarial label shift (SST-5, TREC), natural subpopulation and temporal shifts (CivilComments, HuffPost). Our results demonstrate that RSE consistently outperforms strong robust baselines, including GroupDRO, IRM, and SRM. These findings establish RSE as a practical, theoretically grounded safeguard for reliable model aggregation in dynamic environments.
Figures & tables
| Method | Objective | Ambiguity-Set Free | Analytical Inner Solution | -Indep. Update |
|---|---|---|---|---|
| ERM | Average Risk | N/A | ||
| GroupDRO | Worst-Case | (Gradient) | ||
| SRM | Worst-Case | (Iterative) | ||
| KL-RS | Fragility | (Bisection) | ||
| RSE (Ours) | Fragility |
| Scenario | ERM | IRM | DRO | KL-RS | SRM | RSE |
|---|---|---|---|---|---|---|
| Original (8 Models) | 71.3 | 71.1 | 72.8 | 71.1 | 71.7 | 74.1 |
| Leave-one-out | 70.7 | 70.4 | 72.4 | 70.5 | 71.4 | 73.7 |
| Random Subset | 68.4 | 68.3 | 69.4 | 68.1 | 69.1 | 70.5 |
| Original + Correlated | 71.2 | 71.2 | 72.7 | 71.0 | 71.6 | 74.1 |
| Original + Uncalibrated | 71.8 | 71.5 | 74.0 | 71.9 | 72.5 | 74.0 |
| Original + Weak Models | 71.1 | 70.6 | 72.8 | 70.9 | 71.5 | 74.1 |
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
| RSE (Ours) | DRO | IRM | SRM | KL-RS | ERM | |
|---|---|---|---|---|---|---|
| 0.00 | ||||||
| 0.31 | ||||||
| 0.45 | ||||||
| 0.56 | ||||||
| 0.65 | ||||||
| 0.73 |
| Shift ( ) | RSE (Ours) | DRO | IRM | SRM | KL-RS | ERM |
|---|---|---|---|---|---|---|
| 0.00 | 77.6 1.8 | 77.5 1.6 | 78.6 1.6 ∗∗ | 78.3 1.8 ∗∗ | 78.5 1.7 ∗∗ | 78.5 1.8 ∗∗ |
| 0.32 | 75.9 2.7 | 75.2 2.4 ∗∗ | 74.9 2.5 ∗∗ | 75.1 2.5 ∗∗ | 74.8 2.5 ∗∗ | 74.9 2.6 ∗∗ |
| 0.46 | 75.1 3.1 | 74.0 2.5 ∗∗ | 73.2 2.8 ∗∗ | 73.5 2.7 ∗∗ | 73.2 2.8 ∗∗ | 73.2 2.9 ∗∗ |
| 0.57 | 74.6 3.5 | 73.4 3.3 ∗∗ | 72.1 3.6 ∗∗ | 72.6 3.3 ∗∗ | 72.1 3.6 ∗∗ | 72.2 3.6 ∗∗ |
| 0.66 | 74.4 3.7 | 72.9 3.3 ∗∗ | 71.2 3.8 ∗∗ | 71.9 3.4 ∗∗ | 71.2 3.8 ∗∗ | 71.4 3.8 ∗∗ |
| 0.74 | 73.6 4.1 | 71.9 3.7 ∗∗ | 69.8 4.0 ∗∗ | 70.5 3.6 ∗∗ | 69.9 4.0 ∗∗ | 70.1 4.0 ∗∗ |
| Method | Shift Magnitude ( ) | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.00 | 0.31 | 0.45 | 0.56 | 0.65 | 0.73 | 0.80 | 0.86 | 0.92 | 0.96 | 1.00 | |
| Optimization Baselines | |||||||||||
| RSE | 48.0 | 44.2 | 42.5 | 41.5 | 40.5 | 39.5 | 38.4 | 38.0 | 37.2 | 36.8 | 36.3 |
| DRO | 50.0 | 44.5 | 42.0 | 40.2 | 38.9 | 37.2 | 35.8 | 35.0 | 34.1 | 33.4 | 32.5 |
| IRM | 52.9 | 45.2 | 41.6 | 39.2 | 37.1 | 34.7 | 32.7 | 31.6 | 30.2 | 29.1 | 28.0 |
| SRM | 52.2 | 45.1 | 41.7 | 39.5 | 37.7 | 35.5 | 33.7 | 32.7 | 31.5 | 30.4 | 29.3 |
| Method | Shift Magnitude ( ) | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.00 | 0.32 | 0.46 | 0.57 | 0.66 | 0.74 | 0.81 | 0.87 | 0.92 | 0.96 | 1.00 | |
| Optimization Baselines | |||||||||||
| RSE | 77.6 | 75.9 | 75.1 | 74.6 | 74.4 | 73.6 | 73.5 | 73.0 | 73.0 | 72.5 | 72.0 |
| DRO | 77.5 | 75.2 | 74.0 | 73.4 | 72.9 | 71.9 | 71.7 | 71.2 | 71.1 | 70.6 | 69.9 |
| IRM | 78.6 | 74.9 | 73.2 | 72.1 | 71.2 | 69.8 | 69.2 | 68.7 | 68.3 | 67.7 | 66.7 |
| SRM | 78.3 | 75.1 | 73.5 | 72.6 | 71.9 | 70.5 | 70.0 | 69.6 | 69.3 | 68.6 | 67.7 |
| Method | Overall Accuracy | Worst-Group Accuracy (WGA) |
|---|---|---|
| RSE (Ours) | 72.1 0.1 | 59.7 0.7 |
| DRO | 70.6 0.1 ∗∗ | 57.7 0.8 ∗∗ |
| IRM | 69.8 0.1 ∗∗ | 56.5 0.8 ∗∗ |
| SRM | 68.8 0.1 ∗∗ | 55.1 0.8 ∗∗ |
| KL-RS | 69.6 0.1 ∗∗ | 56.3 0.8 ∗∗ |
| ERM | 70.1 0.1 ∗∗ | 56.8 0.8 ∗∗ |
| Method | Black | Christian | LGBTQ | Muslim | White | Female | Male | Other Rel. |
|---|---|---|---|---|---|---|---|---|
| Optimization Baselines | ||||||||
| RSE | 65.4 | 77.2 | 68.0 | 60.8 | 59.7 | 65.6 | 73.7 | 67.1 |
| DRO | 64.3 | 76.2 | 66.3 | 59.1 | 57.8 | 64.2 | 72.2 | 64.8 |
| IRM | 63.5 | 75.4 | 65.5 | 57.8 | 56.5 | 63.1 | 71.5 | 63.8 |
| SRM | 62.8 | 74.3 | 64.2 | 56.9 | 55.1 | 62.1 | 70.5 | 61.8 |
| KL-RS | 63.3 | 75.1 | 65.0 | 57.8 | 56.3 | 62.9 | 71.2 | 63.5 |
| Method | Average Yearly Accuracy (AYA) | Worst-Year Accuracy (WYA) |
|---|---|---|
| RSE (Ours) | 78.8 0.3 | 77.2 0.4 |
| DRO | 76.3 0.3 ∗∗ | 75.5 0.4 ∗∗ |
| IRM | 78.4 0.3 ∗∗ | 76.9 0.4 ∗ |
| SRM | 78.4 0.3 ∗∗ | 76.9 0.4 ∗ |
| KL-RS | 78.1 0.3 ∗∗ | 76.6 0.4 ∗∗ |
| ERM | 78.1 0.3 ∗∗ | 76.7 0.4 ∗∗ |
| Method | 2016 | 2017 | 2018 |
|---|---|---|---|
| Optimization Baselines | |||
| RSE | 77.2 | 78.1 | 81.0 |
| DRO | 76.1 | 75.6 | 77.2 |
| IRM | 77.0 | 77.5 | 80.6 |
| SRM | 77.0 | 77.7 | 80.5 |
| KL-RS | 76.7 | 77.3 | 80.4 |
| Scenario | ERM | IRM | DRO | KL-RS | SRM | RSE |
|---|---|---|---|---|---|---|
| Original (8 Models) | 36.9 | 36.6 | 38.6 | 37.1 | 37.3 | 40.3 |
| Leave-one-out | 36.4 | 36.3 | 38.1 | 36.5 | 37.1 | 39.4 |
| Random Subset | 35.0 | 35.0 | 36.0 | 35.1 | 35.4 | 37.2 |
| + Correlated | 36.7 | 36.6 | 38.5 | 37.0 | 37.1 | 40.2 |
| + Uncalibrated | 35.2 | 35.2 | 37.8 | 35.5 | 36.4 | 38.5 |
| + Weak Models | 36.9 | 36.5 | 38.7 | 37.1 | 38.4 | 40.2 |
| Moment (s) | Opt. (s) | Stats (KB) | Cache (MB) | |
|---|---|---|---|---|
| 2 | 0.5 | 0.8 | ||
| 4 | 1.6 | 1.6 | ||
| 8 | 5.6 | 3.1 | ||
| 16 | 21.3 | 6.2 | ||
| 32 | 82.5 | 12.3 | ||
| 64 | 324.6 | 24.5 |
| Moment (s) | Opt. (s) | Stats (KB) | Cache (MB) | |
|---|---|---|---|---|
| 5 | 10.2 | 3.1 | ||
| 10 | 21.3 | 6.2 | ||
| 20 | 43.0 | 12.3 | ||
| 50 | 106.5 | 30.6 | ||
| 100 | 213.0 | 61.1 |
| Method | Overall Accuracy (%) | Significance |
| RSE (Ours) | 86.7 1.5 | – |
| Optimization Baselines | ||
| DRO | 81.7 1.7 | ∗∗ |
| KL-RS | 80.1 1.7 | ∗∗ |
| ERM | 80.1 1.7 | ∗∗ |
| SRM | 79.3 1.8 | ∗∗ |
Distribution Shift Robustness
Distribution shift robustness studies how to keep machine-learning predictions reliable when deployment data differ from training data.
See all 301 papers