NCCL-BOT: Closing the Loop on NCCL's Cost Model with Online Bandits
Organizations: NVIDIA
Abstract
NCCL picks an algorithm-protocol pair for every collective from an analytic cost model (AUTO) that never sees a measured latency. The model is a strong prior but a brittle final decision: priced on forced-arm runs, its choice is exact on most contexts and badly wrong in a few. We built NCCL-BOT, a tuner plugin that closes the loop: a per-context bandit, warm-started from the model, learns from measured latencies. We replay a forced-arm H100 sweep, tuned at 4-8 nodes and scored at 16-32, and price the model's choice on the same forced arms; that sweep has no stock-NCCL run. The bandit cuts the contexts where the model's choice loses more than 10% from 19 to 1 (p99 103% to 9.9%, mean 6.1% to 1.6%; p=0.047 over nine scenarios after Holm correction). The prior and continued learning both matter: cold-started bandits, and committing after one probe of each arm, lose more on the mean in every scenario. Supervised selectors fit once on the same data and tuned with the same budget match the mean, and their heavier tail is not significant (7 against 1 contexts above 10%, p=0.06); at seen scales they match or beat online selection. With hyperparameters frozen from H100, UCB1 cuts AUTO's p99 from 60% to 23% and from 56% to 15% on two GB200 fabrics, but its contexts above 10% go from 16 to 8 on one and from 10 to 13 on the other, and scenario-level tests do not separate the two. Live, on two latency-emulated node sets of one H100 system with one run per policy, the learners repair most of a gross AUTO error on one (selection loss 31% to 6-8% for deterministic Greedy and UCB1; average all-reduce bandwidth 1.18-1.24x stock) but average 0.94-0.99x stock on the other, and on both lose to stock below 1 MiB (on the first through plugin overhead); on H100 the cost model is exact there offline, so small sizes can be left to it. We do not show a net end-to-end win.
Figures & tables
| Selection TRLT | Realized TRLT | |||
|---|---|---|---|---|
| Policy | mean (%) | mean (%) | p99 (%) | |
| NCCL AUTO | 5.99 1.58 | 6.08 1.58 | 103.3 | 19 |
| Supervised GBM+cost, fixed | 1.76 0.51 | 1.78 0.50 | 32.2 | 10 |
| Supervised GBM+cost, tuned | 1.30 0.42 | 1.37 0.43 | 27.3 | 7 |
| ETC ( , ) | 2.70 0.49 | 4.74 0.75 | 63.0 | 11 |
| Softmax | 0.93 0.11 | 1.75 0.22 | 11.9 | 5 |
| Cold (no prior) | Warm (prior) | Warm cold | ||||
|---|---|---|---|---|---|---|
| Family | mean (%) | p99 (%) | mean (%) | p99 (%) | mean (pp) | Wilcoxon |
| UCB1 | 8.82 1.42 | 98.7 | 1.61 0.19 | 9.9 | -7.20 | 0.001 |
| Softmax | 6.71 0.90 | 49.5 | 1.75 0.22 | 11.9 | -4.96 | 0.001 |
| KL-UCB | 4.94 0.73 | 40.8 | 2.11 0.36 | 22.7 | -2.83 | 0.001 |
| MNNVL-off | MNNVL-on | |||||||
|---|---|---|---|---|---|---|---|---|
| Policy | sel. mean | real. mean | real. p99 | sel. mean | real. mean | real. p99 | ||
| NCCL AUTO | 3.43 | 3.80 | 60.1 | 16 | 2.02 | 2.65 | 56.3 | 10 |
| ETC ( ) | 3.42 | 5.47 | 40.5 | 20 | 2.55 | 3.60 | 27.1 | 9 |
| Softmax | 1.22 | 2.86 | 22.8 | 7 | 1.63 | 2.61 | 25.8 | 5 |
| UCB1 | 1.07 | 2.59 | 23.1 | 8 | 2.46 | 3.54 | 15.2 | 13 |
| KL-UCB | 1.03 | 2.58 | 20.3 | 5 | 2.21 | 3.35 | 23.2 | 7 |
Appendix figures & tables63 assets
Supplementary material from the paper’s appendix.
Appendix
| symbol / acronym | meaning |
|---|---|
| MAB | multi-armed bandit |
| AUTO | NCCL’s static cost-model selector (baseline) |
| KPI | key performance indicator |
| SEM | standard error of the mean |
| ETC | explore-then-commit |
| FPL | follow-the-perturbed-leader |
| TRLT (%) | ||||||
| policy | train | test | test p99 [95% CI] | |||
| Fixed | ||||||
| NCCL AUTO (cost model) | ||||||
| Random | ||||||
| Supervised | ||||||
| Linear regression + cost | ||||||
| component | specification |
|---|---|
| Node | Dell HGX H100 Gen-11; 8 H100 |
| Intra-node | NVLink |
| NIC | BlueField-3 SuperNIC |
| Network | Spectrum-X Ethernet |
| NCCL | 2.30.4 (CUDA 13.0) |
| Scale | 4–32 nodes (32–256 GPUs) |
| TRLT (%) | |||
| warm start | |||
| policy | false | true | (true false) |
| Softmax | |||
| UCB1 | |||
| KL-UCB | |||
| Gaussian Thompson | |||
| Policy | stationary | abrupt | gradual |
|---|---|---|---|
| NCCL AUTO (static) | 5.99 1.58 | 45.01 3.93 | 44.60 3.89 |
| Softmax (plain) | 0.93 0.11 | 31.57 2.63 | 31.99 2.65 |
| Softmax ( ) | 1.07 0.12 | 5.96 0.29 | 13.11 1.17 |
| UCB1 (plain) | 1.00 0.13 | 6.60 0.61 | 10.93 0.87 |
| UCB1 ( ) | 1.16 0.13 | 5.88 0.54 | 10.68 0.87 |
| TRLT (%) | |||
| policy | updates | mean | p99 [95% CI] |
| NCCL AUTO (cost model) | — | ||
| UCB1, update every collective | 200 | ||
| UCB1, update every ms | 51 | ||
| policy | conditional TRLT (%) |
|---|---|
| NCCL AUTO | 26.08 |
| Softmax (warm) | 2.38 |
| UCB1 (warm) | 2.82 |
| Gaussian Thompson (warm) | 4.76 |
| Hierarchical EXP3/Softmax (warm) | 7.76 |
| epoch length | ||||
| policy | 10 | 20 | 50 | 100 |
| NCCL AUTO (cost model) | 5.99 | 5.99 | 5.99 | 5.99 |
| Random | 46.83 | 44.52 | 44.94 | 45.22 |
| Softmax (warm) | 5.76 | 3.22 | 1.52 | 0.93 |
| UCB1 (warm) | 3.73 | 2.73 | 1.42 | 1.00 |
| Stationary (seq., cf. Table 5 ) | Abrupt | Gradual | ||||
|---|---|---|---|---|---|---|
| Policy | mean | p99 | mean | p99 | mean | p99 |
| NCCL AUTO (static) | 5.99 1.58 | 103.25 | 45.01 3.93 | 197.46 | 44.60 3.89 | 195.36 |
| Softmax (plain) | 0.93 0.11 | 6.59 | 31.57 2.63 | 128.39 | 31.99 2.65 | 133.57 |
| Softmax ( ) | 1.07 0.12 | 5.34 | 5.96 0.29 | 15.70 | 13.11 1.17 | 58.12 |
| UCB1 (plain) | 1.00 0.13 | 7.97 | 6.60 0.61 | 33.71 | 10.93 0.87 | 44.31 |
| UCB1 ( ) | 1.16 0.13 | 7.59 | 5.88 0.54 | 30.32 | 10.68 0.87 | 43.15 |
| config | T50 | T100 | Kmin | Kmed | Kmax | Kavg |
|---|---|---|---|---|---|---|
| On ( ) | 1.42 | 1.00 | 2 | 2.00 | 5 | 2.56 |
| Off | 1.41 | 1.01 | 2 | 3.00 | 7 | 4.01 |
| TRLT (%) | |||
|---|---|---|---|
| robust estimation | |||
| policy | false | true | (true false) |
| UCB1 | |||
| Softmax | |||
| Gaussian Thompson | |||
| UCB-tuned | |||
| TRLT (%) | |||
|---|---|---|---|
| adaptive observed-max | |||
| policy | false | true | (true false) |
| Softmax | |||
| UCB1 | |||
| KL-UCB | |||
| Gaussian Thompson | |||
| policy | best trial | train TRLT default best (%) | search changes (init best) |
|---|---|---|---|
| Softmax | 144 | 29.8 0.82 (37 ) | 1 0.089; decay F T; pc 1.24; warm F T |
| UCB1 | 135 | 9.9 0.84 (12 ) | robust F T; clip 0.389; lr 0.468; 0.467; 0.05 0.0928; pc 2.89; warm F T |
| KL-UCB | 146 | 2.4 0.85 (3 ) | c 0.217 0.0373; pc 1.56; warm F T |
| Gaussian Thompson | 123 | 4.0 0.99 (4 ) | ign- 0 1; pv 0.002 0.000301; pc 0.491; warm F T; AOM F T |
| -greedy | 53 | 1.4 1.07 (1 ) | pc 0.892; warm F T; AOM F T |
| Madan RR-ETC | 48 | 11.1 2.44 (5 ) | ign- 0 1; 10 1; gate 0.05 0.0102; pc 0 |
| Policy | 1 MiB | 1–8 MiB | 16 MiB–16 GiB |
|---|---|---|---|
| Greedy | 0.64 (0.54–0.89) | 1.27 (1.23–1.42) | 0.98 (0.93–1.03) |
| ETC ( ) | 0.65 (0.61–0.90) | 1.31 (1.20–1.45) | 0.97 (0.82–1.07) |
| UCB1 | 0.65 (0.56–0.88) | 1.29 (1.27–1.41) | 0.97 (0.86–1.02) |
| Softmax | 0.61 (0.42–0.87) | 1.22 (1.19–1.40) | 0.97 (0.11–1.03) |
| Thompson | 0.59 (0.33–0.87) | 1.29 (1.26–1.40) | 0.98 (0.91–1.01) |
| -greedy | 0.64 (0.45–0.89) | 1.21 (1.00–1.24) | 0.99 (0.90–1.03) |
| configuration | state sync | bus bw (GB/s) | overhead (%) |
|---|---|---|---|
| Stock NCCL | — | 73.0 | — (ref.) |
| NCCL AUTO | off | 71.5 | 2.1 |
| AUTO + sync/4 | 4 | 67.5 | 7.5 |
| AUTO + sync/16 | 16 | 70.5 | 3.5 |
| AUTO + sync/64 | 64 | 71.3 | 2.3 |
| Size bin | no sync | sync/4 | sync/16 | sync/64 | 1 ms flush: coll./sync | |
| 1 MiB | 11 | +0.6 (0) | +61.6 (1) | +13.4 (1) | +4.3 (1) | 3 |
| 1–32 MiB | 6 | +1.4 (5) | +41.8 (32) | +10.1 (19) | +3.8 (13) | 2 |
| 64–512 MiB | 4 | +5.7 (80) | +13.0 (40) | +7.6 (58) | +6.0 (69) | 1 |
| 1 GiB | 5 | +0.4 (14) | +2.9 (27) | +1.2 (22) | +0.7 (17) | 1 |
| avg. bus bw drop | 26 | 2.1 | 7.5 | 3.5 | 2.3 | |
| run-to-run, 64 MiB | 9 | 0.4 | 1.5 | 1.3 | 0.5 |
| Component | Campaign | Finding | Measured |
|---|---|---|---|
| selection quality | H100-A (64 ranks) | best learner epsilon-greedy: 2.86% mean / 8.64% p99 selection TRLT | ✓ |
| H100-B (64 ranks) | best learner greedy (epsilon=0): 0.59% mean / 1.98% p99 selection TRLT | ✓ | |
| plugin / sync overhead | H100-A (64 ranks) | 2.1% average bus bandwidth lost vs stock with profiling only (AUTO selection, state sync off; one run per row); sync every 4 collectives is costliest and the ladder falls toward stock as the sync period grows | ✓ |
| H100-B (64 ranks) | not measured: this campaign has no plugin runs with AUTO selection fixed | – | |
| one-time setup toll | H100-A (64 ranks) | 6.6 s median (6.4–7.8 s over 10 runs) NVLS_TREE_SIMPLE first-use stall (6.1% median of run wall); 123,717 median steady collectives to amortize at 1% | ✓ |
| H100-B (64 ranks) | 5.8 s median (5.7–5.9 s over 6 runs) NVLS_TREE_SIMPLE first-use stall (6.2% median of run wall); 172,261 median steady collectives to amortize at 1% | ✓ |
| TRLT (%) | bus bw | ||||
|---|---|---|---|---|---|
| policy | mean | worst | rank | GB/s | vs stock |
| -greedy | 2.85 | 9.1 | 1.6 | 90.7 | 1.24 |
| Greedy | 6.43 | 19.0 | 2.8 | 86.1 | 1.18 |
| UCB1 | 8.28 | 12.5 | 4.0 | 89.3 | 1.22 |
| Thompson | 8.49 | 20.4 | 3.9 | 88.0 | 1.21 |
| Softmax | 9.00 | 15.6 | 4.0 | 90.0 | 1.23 |
| Policy | none | 1 s | 3 s | 5.8 s |
|---|---|---|---|---|
| NCCL AUTO | 6.1 | 98.9 | 285.1 | 545.8 |
| Supervised GBM+cost, fixed | 1.8 | 11.8 | 32.2 | 60.9 |
| Supervised GBM+cost, tuned | 1.4 | 12.2 | 34.4 | 65.5 |
| ETC ( , ) | 4.7 | 595.6 | 1779.2 | 3436.2 |
| Softmax | 1.8 | 411.5 | 1231.9 | 2380.6 |
| UCB1 | 1.6 | 209.7 | 626.6 | 1210.3 |
| NVLS offered | NVLS excluded | Pooled, 5.8 s once per comm. | |||||
|---|---|---|---|---|---|---|---|
| Policy | mean | p99 | mean | p99 | |||
| NCCL AUTO | 6.08 | 103.3 | 6.27 | 111.4 | 15.82 | 4.14 | 2.98 |
| Supervised GBM+cost | 1.78 | 32.2 | 1.79 | 32.2 | 14.08 | 2.41 | 1.24 |
| ETC ( ) | 4.74 | 63.0 | 3.26 | 13.8 | 13.76 | 2.08 | 0.92 |
| Softmax | 1.75 | 11.9 | 1.22 | 8.6 | 13.50 | 1.82 | 0.66 |
| UCB1 | 1.61 | 9.9 | 1.23 | 11.2 | 13.60 | 1.93 | 0.76 |
| Policy | mean | p99 |
|---|---|---|
| NCCL AUTO | 5.99 | 103.3 |
| Random | 45.73 | 194.0 |
| Supervised GBM+cost | 1.76 | 36.9 |
| Bank UCB1, cold | 7.68 | 102.8 |
| Bank Softmax, cold | 5.26 0.49 | 56.7 |
| Bank UCB1, warm | 1.00 | 8.0 |
| policy | TRLT |
|---|---|
| UCB1 | 0.22 |
| NCCL cost model | 1.08 |
| Dataset | Platform / coverage | Role |
|---|---|---|
| H100 sweep | HGX H100, Spectrum-X Ethernet, NCCL 2.30.4; 3 collectives 4/8/16/32 nodes 26 sizes | primary ground truth |
| GB200 sweep | GB200, 2 rank placements (MNNVL off: one rank per physical node; on: whole allocation); 310 contexts/slice | cross-architecture |
| CCL-Bench | Llama3 trace, nine contexts | workload weighting |
| H100-A live | 8 HGX H100 (64 GPUs), Spectrum-X Ethernet, NCCL 2.30.7 (development build), 2 queue pairs per connection (10 long-haul), long-haul latency emulation; 22 runs, AllReduce | live |
| H100-B live | 8 HGX H100 (64 GPUs), Spectrum-X Ethernet over RoCE, NCCL 2.32.0a4, 1 queue pair per connection, long-haul latency emulation; 6 runs, AllReduce, on the nodes and configuration of a forced-arm sweep with one stock run | live, stock control |
| approach | day | online | CM | bench |
|---|---|---|---|---|
| NCCL AUTO | 0 | No | Selector | Yes |
| Offline tuning | 1 | No | No | No |
| STAR-MPI Faraj et al. (2006) | 2 | Yes | No | No |
| AutoCCL Xu et al. (2025) | 2 | Early iters | No | No |
| Madan RR-ETC | 2 | Yes | Gate | Yes † |
| This work | 2 | Live + replay | Prior/filter | Yes † |
| Coll. | Size | 4 nodes | 8 nodes | 16 nodes | 32 nodes |
|---|---|---|---|---|---|
| AG | 128 MiB | 207 / 267 | 90 / 250 | 51 / 209 | 32 / 120 |
| AG | 256 MiB | 289 / 288 | 163 / 282 | 68 / 259 | 62 / 212 |
| AG | 512 MiB | 314 / 288 | 258 / 296 | 126 / 292 | 92 / 271 |
| AG | 1 GiB | 321 / 300 | 309 / 293 | 217 / 303 | 166 / 296 |
| AG | 2 GiB | 322 / 307 | 327 / 303 | 284 / 297 | 262 / 305 |
| RS | 128 MiB | 203 / 266 | 91 / 250 | 51 / 204 | 33 / 127 |
| AUTO picks | best arm | collectives, sizes | AUTO | UCB1 | GBM+c | ||
|---|---|---|---|---|---|---|---|
| max realized TRLT in the group (%) | |||||||
| Ring_Simple | Ring_LL128 | 9 | AG/RS, 128 MiB–2 GiB | 0.08–0.18 | 134 | 2.9 | 40 |
| NVLSTree_Simple | Tree_LL128 | 6 | AR, 16 MiB–128 MiB | 0.06–0.57 | 65 | 13.5 | 0 |
| Ring_LL128 | Ring_LL | 4 | AG/RS, 4 MiB–8 MiB | 0.20–0.21 | 26 | 9.1 | 26 |
| NCCL AUTO | UCB1 | GBM+cost | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Scenario | loss share | mean | max | mean | max | mean | max | ||
| AG, 4 nodes | 3 | 3% | 9.5 | 29 | 1 | 0.67 | 1.3 | 0.08 | 0.2 |
| AG, 16 nodes | 24 | 21% | 8.1 | 131 | 3 | 1.08 | 3.3 | 1.83 | 39.2 |
| AG, 32 nodes | 23 | 12% | 5.0 | 78 | 3 | 1.19 | 3.7 | 1.63 | 20.6 |
| AR, 4 nodes | 2 | 0% | 0.0 | 0 | 0 | 0.00 | 0.0 | 0.00 | 0.0 |
| AR, 16 nodes | 26 | 18% | 6.3 | 65 | 3 | 2.81 | 9.9 | 0.94 | 11.7 |
| Test | Comparison | Units | Statistic | |
|---|---|---|---|---|
| sign-flip, scenario means | AUTO -UCB1 mean | 9 | 3.99 | 0.016 |
| wild-cluster bootstrap- | AUTO -UCB1 mean | 9 | 5.09 | 0.016 |
| sign-flip, exceedance diff. | AUTO -UCB1 | 9 | 18.00 | 0.016 |
| sign-flip, scenario means | AUTO -Softmax mean | 9 | 3.90 | 0.031 |
| wild-cluster bootstrap- | AUTO -Softmax mean | 9 | 4.08 | 0.031 |
| sign-flip, exceedance diff. | AUTO -Softmax | 9 | 14.00 | 0.047 |
| Policy | contexts | independent-contexts interval | scenarios with [interval] |
|---|---|---|---|
| AUTO | 19 (12.3%) | [7.6, 18.6]% | 7/9 [40, 97]% |
| GBM+cost | 10 (6.5%) | [3.2, 11.6]% | 6/9 [30, 93]% |
| UCB1 | 1 (0.6%) | [0.0, 3.6]% | 1/9 [0, 48]% |
| Softmax | 5 (3.2%) | [1.1, 7.4]% | 2/9 [3, 60]% |
| KL-UCB | 4 (2.6%) | [0.7, 6.5]% | 2/9 [3, 60]% |
| ETC | 11 (7.1%) | [3.6, 12.4]% | 2/9 [3, 60]% |
| Comparison | nine scenarios | six 16/32-node | ||
| Holm | Holm | |||
| AUTO UCB1, mean | 0.016 | 0.047 | 0.031 | 0.125 |
| AUTO UCB1, contexts 10% | 0.016 | 0.047 | 0.031 | 0.125 |
| cold-tuned warm UCB1, mean | 0.004 | 0.016 | 0.031 | 0.125 |
| tuned GBM+cost UCB1, contexts 10% | 0.062 | 0.062 | 0.062 | 0.125 |
| UCB1 tuning | mean (%) | p99 (%) | 10% | |
| Policy | mean | p50 | p90 | p95 | p99 | CVaR 5% | |||
|---|---|---|---|---|---|---|---|---|---|
| NCCL AUTO | 6.08 | 0.00 | 17.7 | 39.6 | 103.3 | 77.2 | 19 | 12 | 6 |
| Supervised GBM+cost | 1.78 | 0.00 | 3.8 | 14.6 | 32.2 | 25.7 | 10 | 4 | 0 |
| ETC ( ) | 4.74 | 2.53 | 8.3 | 12.1 | 63.0 | 35.1 | 11 | 3 | 3 |
| KL-UCB | 2.11 | 1.09 | 4.2 | 6.2 | 22.7 | 16.3 | 4 | 2 | 0 |
| Softmax | 1.75 | 0.87 | 4.0 | 6.4 | 11.9 | 11.1 | 5 | 0 | 0 |
| UCB1 | 1.61 | 0.98 | 4.1 | 6.2 | 9.9 | 9.1 | 1 | 0 | 0 |
| Contrast | statistic | iid contexts | scenarios (9) | size blocks (41) |
|---|---|---|---|---|
| AUTO UCB1 | mean | 4.5 [1.8, 7.8] | 4.5 [2.8, 6.3] | 4.5 [1.3, 8.3] |
| AUTO UCB1 | p99 | 93.4 [36.8, 124.9] | 93.4 [53.4, 128.1] | 93.4 [36.9, 125.0] |
| AUTO UCB1 | CVaR5% | 68.1 [32.8, 102.0] | 68.1 [45.4, 92.2] | 68.1 [32.0, 102.3] |
| GBM+cost UCB1 | mean | 0.2 [-0.8, 1.2] ∘ | 0.2 [-0.8, 1.2] ∘ | 0.2 [-0.7, 1.2] ∘ |
| GBM+cost UCB1 | p99 | 22.3 [7.3, 31.3] | 22.3 [10.9, 35.9] | 22.3 [8.5, 31.3] |
| GBM+cost UCB1 | CVaR5% | 16.6 [5.2, 25.6] | 16.6 [7.7, 27.3] | 16.6 [5.8, 25.7] |
| Split | AUTO | GBM+cost | ETC | Softmax | KL-UCB | UCB1 | |
|---|---|---|---|---|---|---|---|
| scale-OOD (main) | 154 | 6.08 / 103.3 | 1.78 / 32.2 | 4.74 / 63.0 | 1.75 / 11.9 | 2.11 / 22.7 | 1.61 / 9.9 |
| reverse scale (train 16/32, test 4/8) | 154 | 3.96 / 60.7 | 1.99 / 23.8 | 4.04 / 16.9 | 1.51 / 16.5 | 1.38 / 11.7 | 1.20 / 11.7 |
| random 50/50, seed 42 | 150 | 4.43 / 74.2 | 0.75 / 13.3 | 5.07 / 51.3 | 2.46 / 27.2 | 1.56 / 9.5 | 1.29 / 11.8 |
| random 50/50, seed 43 | 150 | 5.80 / 104.0 | 0.93 / 13.0 | 4.66 / 63.2 | 1.80 / 16.0 | 2.14 / 12.7 | 1.69 / 15.2 |
| random 50/50, seed 44 | 150 | 4.62 / 74.2 | 1.10 / 10.0 | 3.73 / 18.2 | 1.32 / 15.6 | 1.37 / 9.0 | 1.24 / 9.5 |
| leave out all-gather | 98 | 4.67 / 79.9 | 0.45 / 8.2 | 2.73 / 9.0 | 0.88 / 14.6 | 1.26 / 5.5 | 0.89 / 4.5 |
| Policy | mean (%) | p99 (%) | (pp) | |||
|---|---|---|---|---|---|---|
| GBM+cost (tuned) | 1.46 | 38.6 | 9 | +0.38 | 0.422 | 0.031 |
| GBM+cost (fixed) | 1.79 | 32.2 | 12 | +0.31 | 0.406 | 0.031 |
| AUTO | 6.27 | 111.4 | 20 | +4.36 | 0.016 | 0.016 |
| UCB1 (champion) | 1.23 | 11.2 | 2 | – | – | – |
| Arm set | Policy | train sel. | mean (%) | p99 (%) | |||
|---|---|---|---|---|---|---|---|
| NVLS offered | AUTO | 4.04 | 6.08 1.58 | 103.3 | 19 | 0.016 | 0.016 |
| Probe ( , ) | 2.42 | 4.74 0.75 | 63.0 | 11 | 0.004 | 0.500 | |
| UCB1 (champion) | 0.86 | 1.61 0.19 | 9.9 | 1 | – | – | |
| NVLS excluded | AUTO | 2.17 | 6.27 1.66 | 111.4 | 20 | 0.016 | 0.016 |
| Probe ( , ) | 2.36 | 3.21 0.22 | 11.6 | 2 | 0.004 | 1.000 | |
| UCB1 (champion) | 0.56 | 1.23 0.18 | 11.2 | 2 | – | – |
| Comparison ( ) | mean | 5% | 10% | 20% |
|---|---|---|---|---|
| AUTO UCB1 | +3.99 (0.016) | 25 vs 13 (0.062) | 19 vs 1 (0.016) | 15 vs 0 (0.016) |
| GBM+cost (tuned) UCB1 | +0.07 (0.914) | 11 vs 13 (0.812) | 7 vs 1 (0.062) | 2 vs 0 (0.500) |
| GBM+cost (tuned) Softmax | -0.03 (0.965) | 11 vs 13 (1.000) | 7 vs 5 (0.844) | 2 vs 1 (1.000) |
| GBM+cost (tuned) KL-UCB | -0.25 (0.734) | 11 vs 12 (1.000) | 7 vs 4 (0.531) | 2 vs 2 (1.000) |
| Comparison ( ) | Statistic | (pp) | Scenario 95% CI | Context 95% CI |
|---|---|---|---|---|
| GBM+cost (tuned) UCB1 | mean | -0.24 | [-0.96, +0.65] | [-1.03, +0.73] |
| p99 | +17.46 | [+2.62, +35.88] | [+1.75, +31.12] | |
| CVaR5% | +11.31 | [+3.28, +21.58] | [+0.68, +22.57] | |
| GBM+cost (tuned) Softmax | mean | -0.38 | [-1.65, +0.76] | [-1.24, +0.61] |
| p99 | +15.41 | [-6.12, +36.35] | [-5.79, +30.44] | |
| CVaR5% | +9.39 | [-0.51, +22.45] | [-1.94, +21.18] |
| Supervised | Online | |||||||
|---|---|---|---|---|---|---|---|---|
| Split | GBM cost fixed | GBM cost tuned | RF cost tuned | Softmax | KL-UCB | UCB1 | mean / | |
| scale-OOD (main) | 154 | 1.78 / 32.2 / 10 | 1.37 / 27.3 / 7 | 3.23 / 76.7 / 9 | 1.75 / 11.9 / 5 | 2.11 / 22.7 / 4 | 1.61 / 9.9 / 1 | 0.91 / 0.06 |
| reverse scale (train 16/32, test 4/8) | 154 | 1.99 / 23.8 / 12 | 1.97 / 30.7 / 7 | 1.30 / 16.0 / 6 | 1.51 / 16.5 / 10 | 1.38 / 11.7 / 3 | 1.20 / 11.7 / 3 | 0.38 / 0.12 |
| random 50/50, seed 42 | 150 | 0.75 / 13.3 / 3 | 0.64 / 13.3 / 3 | 0.69 / 16.7 / 3 | 2.46 / 27.2 / 8 | 1.56 / 9.5 / 1 | 1.29 / 11.8 / 2 | 0.01 / 1.00 |
| random 50/50, seed 43 | 150 | 0.93 / 13.0 / 5 | 1.62 / 17.7 / 11 | 0.76 / 12.7 / 4 | 1.80 / 16.0 / 10 | 2.14 / 12.7 / 5 | 1.69 / 15.2 / 6 | 0.80 / 0.25 |
| random 50/50, seed 44 | 150 | 1.10 / 10.0 / 2 | 0.77 / 9.6 / 1 | 0.75 / 9.6 / 1 | 1.32 / 15.6 / 5 | 1.37 / 9.0 / 0 | 1.24 / 9.5 / 2 | 0.10 / 1.00 |
| Cold, warm knobs | Cold, own search | Warm (champion) | ||||
|---|---|---|---|---|---|---|
| Family | mean | p99 | mean | p99 | mean | p99 |
| KL-UCB | 4.94 | 40.8 | 4.27 | 33.9 | 2.11 | 22.7 |
| Softmax | 6.71 | 49.5 | 4.77 | 63.0 | 1.75 | 11.9 |
| UCB1 | 8.82 | 98.7 | 4.84 | 63.0 | 1.61 | 9.9 |
| Gaussian Thompson | 4.93 | 64.5 | 4.93 | 64.5 | 2.17 | 17.8 |
| -greedy | 6.63 | 75.0 | 4.97 | 63.5 | 2.12 | 23.3 |
| NVLS offered | NVLS excluded | ||||
| Policy | realized | p99 | selection | realized | p99 |
| ETC, tuned champion ( , ) | 4.74 | 63.0 | 2.70 | 3.21 | 11.6 |
| ETC , | 7.14 | 97.6 | 5.97 | 5.98 | 73.6 |
| ETC , | 14.25 | 66.6 | 12.10 | 12.76 | 43.4 |
| ETC , | 16.69 | 72.8 | 14.57 | 15.13 | 53.0 |
| ETC, , train-selected (offered ; excluded ) | 4.74 | 63.0 | 2.70 | 3.21 | 11.6 |
| selection mean | realized | |||||
|---|---|---|---|---|---|---|
| NVLS | Family | warm | cold | warm cold [95% CI] | p99 w/c | w/c |
| offered | UCB1 | 1.00 | 2.78 | -3.22 [-4.78, -1.97] | 9.9/63.0 | 1/10 |
| offered | Softmax | 0.93 | 2.71 | -3.02 [-4.42, -1.93] | 11.9/63.0 | 5/10 |
| offered | KL-UCB | 1.18 | 2.78 | -2.16 [-2.94, -1.49] | 22.7/33.9 | 4/8 |
| excluded | UCB1 | 0.77 | 2.24 | -2.08 [-2.58, -1.60] | 11.2/12.0 | 2/2 |
| excluded | Softmax | 0.70 | 2.19 | -2.04 [-2.44, -1.65] | 8.6/12.0 | 1/2 |
| Family | sampler seed | / | train sel. obj. | test realized mean / p99 | ||
|---|---|---|---|---|---|---|
| UCB1 | 42 | 0.0033 | 1 | 2.41 | 4.75 / 63.0 | 10 |
| Softmax | 42 | 0.0005 | 1 | 2.41 | 4.93 / 63.0 | 11 |
| UCB1 | 43 | 0.0030 | 1 | 2.41 | 4.76 / 63.0 | 10 |
| Softmax | 43 | 0.2072 | 0 | 2.26 | 5.36 / 33.1 | 21 |
| UCB1 | 44 | 0.0029 | 1 | 2.41 | 4.76 / 63.0 | 10 |
| Softmax | 44 | 0.0007 | 1 | 2.41 | 4.93 / 63.0 | 11 |
| H100 | GB200 MNNVL-off | GB200 MNNVL-on | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Policy | mean | p99 | mean | p99 | mean | p99 | |||
| AUTO | 4.80 | 76.2 | 9 | 3.38 | 51.4 | 8 | 0.23 | 5.4 | 0 |
| ETC | 4.73 | 33.7 | 7 | 4.63 | 28.1 | 5 | 2.72 | 12.4 | 1 |
| Softmax | 2.30 | 14.5 | 4 | 2.56 | 13.9 | 1 | 1.61 | 7.9 | 0 |
| UCB1 | 1.89 | 11.0 | 1 | 2.31 | 18.1 | 4 | 2.76 | 11.7 | 6 |
| KL-UCB | 3.06 | 34.7 | 3 | 2.19 | 16.6 | 2 | 2.47 | 10.7 | 2 |
| Policy (NVLS excluded) | gain | train sel. | test mean | p99 | |
| AUTO | – | 2.17 | 6.27 | 111.4 | 20 |
| Supervised GBM+cost | – | 0.04 | 1.79 | 32.2 | 12 |
| UCB1 champion | – | 0.56 | 1.23 | 11.2 | 2 |
| Softmax champion | – | 0.68 | 1.22 | 8.6 | 1 |
| KL-UCB champion | – | 0.60 | 1.90 | 27.5 | 6 |
| KL-UCB constant gain | 0.1 | 0.73 | 1.90 | 19.1 | 3 |
| Arm set | vs. | pooled | scen.-mean | scen. / better | (sign-flip) | (wild- ) | / | ( ) |
|---|---|---|---|---|---|---|---|---|
| NVLS excluded | AUTO vs. UCB1 (champion) | +5.04 | +4.36 | 1/7 | 0.016 | 0.016 | 20/2 | 0.016 |
| NVLS excluded | GBM+cost vs. UCB1 (champion) | +0.56 | +0.31 | 3/5 | 0.406 | 0.141 | 12/2 | 0.031 |
| NVLS excluded | AUTO vs. UCB1 (recommended) | +5.06 | +4.34 | 1/7 | 0.016 | 0.016 | 20/0 | 0.016 |
| NVLS excluded | GBM+cost vs. UCB1 (recommended) | +0.58 | +0.28 | 4/4 | 0.461 | 0.195 | 12/0 | 0.031 |
| NVLS excluded | UCB1 (champion) vs. UCB1 (recommended) | +0.02 | -0.03 | 5/2 | 0.891 | 0.922 | 2/0 | 1.000 |
| NVLS excluded | GBM+cost vs. UCB1 (recommended, 1 ms flush) | +0.68 | +0.38 | 3/5 | 0.281 | 0.195 | 12/0 | 0.031 |
| Scenario | AUTO UCB1, offered | GBM+cost UCB1, offered | AUTO UCB1, excl. | GBM+cost UCB1, excl. | GBM+cost rec., excl. | GBM+cost rec.+flush, excl. | |
|---|---|---|---|---|---|---|---|
| AG, 16 nodes | 24 | +6.98 | +0.75 | +6.98 | +0.75 | +0.27 | +0.14 |
| AG, 32 nodes | 23 | +3.84 | +0.43 | +3.84 | +0.43 | -0.19 | -0.11 |
| AG, 4 nodes | 3 | +8.86 | -0.60 | +8.86 | -0.60 | -0.83 | -0.88 |
| AR, 16 nodes | 26 | +3.50 | -1.87 | +4.78 | +0.10 | +0.62 | +0.92 |
| AR, 32 nodes | 26 | +1.69 | -0.49 | +3.80 | -0.14 | +0.98 | +1.29 |
| AR, 4 nodes | 2 | +0.00 | +0.00 | +0.00 | +0.00 | +0.00 | +0.00 |
| Gate threshold | train sel. mean | test mean | test p99 | |
| none | 0.86 | 1.61 | 9.9 | 1 |
| 64 KiB | 0.82 | 1.46 | 9.9 | 1 |
| 256 KiB | 0.80 | 1.35 | 9.9 | 1 |
| 1 MiB ⋆ | 0.63 | 1.15 | 9.5 | 1 |
| 4 MiB | 1.02 | 1.03 | 9.5 | 1 |
| 16 MiB | 1.29 | 1.36 | 20.8 | 5 |
| target-scale labels (sizes, samples per arm) | none | 1/8, | 1/4, | 1/2, | 1/2, | |
|---|---|---|---|---|---|---|
| H100 | labelled (context, arm) cells | 0/590 | 74/590 | 148/590 | 295/590 | 295/590 |
| GBM cost | 1.87 / 33 | 1.25 / 28 | 1.01 / 25 | 0.67 / 8 | 0.68 / 8 | |
| UCB1 (no labels) | 1.68 / 10 | 1.68 / 10 | 1.68 / 10 | 1.68 / 10 | 1.68 / 10 | |
| GB200 MNNVL-off | labelled (context, arm) cells | 0/672 | 84/672 | 168/672 | 336/672 | 336/672 |
| GBM cost | 4.80 / 83 | 3.78 / 49 | 3.40 / 39 | 2.96 / 35 | 2.31 / 28 | |
| UCB1 (no labels) | 2.63 / 25 | 2.64 / 27 | 2.64 / 28 | 2.63 / 26 | 2.63 / 26 |
| Policy | stationary | abrupt | gradual |
|---|---|---|---|
| AUTO | 5.99 (131.7) | 45.01 (200.4) | 44.60 (199.8) |
| UCB1 champion | 0.87 (6.5) | 6.60 (34.4) | 10.93 (47.7) |
| UCB1, | 1.05 (6.3) | 5.88 (30.6) | 10.68 (46.8) |
| UCB1, gain 0.1 | 1.20 (7.9) | 4.38 (15.5) | 9.35 ( 37.4 ) |
| UCB1, gain 0.2 | 0.96 (6.5) | 2.92 ( 9.6 ) | 9.13 (41.3) |
| Softmax champion | 0.75 (9.4) | 31.57 (134.6) | 31.99 (139.4) |
| Tuned on selection-priced train TRLT (main) | Tuned on realized train TRLT | |||||||
| Family | train | test | p99 | train | test | p99 | ||
| UCB1 | 1.39 | 1.61 | 9.9 | 1 | 1.17 | 1.62 | 11.4 | 3 |
| Softmax | 1.43 | 1.75 | 11.9 | 5 | 1.57 | 1.63 | 19.6 | 6 |
| Gaussian Thompson | 1.66 | 2.17 | 17.8 | 11 | 1.57 | 1.80 | 16.3 | 5 |
| KL-UCB | 1.35 | 2.11 | 22.7 | 4 | 1.68 | 2.20 | 13.0 | 3 |
| ETC | 4.16 | 4.74 | 63.0 | 11 | 4.15 | 4.74 | 63.0 | 11 |
| train | test (154 contexts) | |||||
|---|---|---|---|---|---|---|
| to bandit | mean | mean | p99 | max | ||
| ( AUTO ) | 0% | 3.98 | 6.08 | 103.3 | 134 | 19 |
| 0.1 | 13% | 2.44 | 2.83 | 41.4 | 50 | 13 |
| 0.2 | 24% | 1.57 | 2.01 | 37.6 | 50 | 7 |
| 0.3 | 40% | 1.08 | 1.59 | 27.6 | 43 | 4 |
| 0.5 ∗ | 52% | 1.01 | 1.43 | 17.9 | 33 | 3 |
| train | test, realized TRLT (%) | |||||
|---|---|---|---|---|---|---|
| Policy | mean | mean SEM | p99 | p99 95% CI | ||
| Supervised GBM+cost, tuned | – | 1.37 0.43 | 27.3 | [12.7, 40.1] | 7 | 0 |
| UCB1 | 1.39 | 1.61 0.19 | 9.9 | [7.6, 13.5] | 1 | 0 |
| Softmax | 1.43 | 1.75 0.22 | 11.9 | [7.5, 22.2] | 5 | 0 |
| Supervised GBM+cost | 0.04 | 1.78 0.50 | 32.2 | [18.3, 40.1] | 10 | 0 |
| KL-UCB | 1.35 | 2.11 0.36 | 22.7 | [8.1, 38.5] | 4 | 0 |
High-Performance Computing
High-performance computing (HPC) uses parallel systems and optimized software to solve computationally demanding scientific and engineering problems.
See all 152 papers