Are Stochastic Multi-objective Bandits Harder than Single-objective Bandits?
Authors: Changkun Guan, Mengfan Xu
Organizations: H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology · Mechanical and Industrial Engineering, University of Massachusetts Amherst
Multi-objective bandits have attracted increasing attention for their broad applicability, with d-dimensional reward vectors inducing Pareto regret. There has been a subtle debate over whether this added structure makes the problem fundamentally harder than single-objective bandits. We answer this by showing that, in terms of Pareto regret, it is surprisingly no harder: Pareto regret scales inversely with g†, the largest objective-wise suboptimality gap, and thus matches the smallest objective-wise classical regret. We formalize this idea via a novel method with upper and lower confidence-bound estimators for every arm-objective pair. It uses top-two races to compare arms within each objective and an uncertainty-greedy rule to allocate exploration toward the largest objective-wise gap g†, until the corresponding Pareto-optimal arm is committed to. We prove that it achieves Pareto regret of O(\nicefraclogTg†), where T is the horizon, with \emph{no dependence on d}. A matching lower bound of Ω(\nicefraclogTg†) implies optimality. We evaluate the method on synthetic and real-world datasets, confirming the theory and achieving order-of-magnitude reductions in Pareto regret over baselines. Real-world results further show that our method commits to a Pareto optimal arm, possibly at the cost of empirical fairness, suggesting a potential hardness absent in single-objective bandits.
Figures & tables
Subset type
Method
Pareto regret ↓
Pr(aT∈A⋆)↑
Fairness regret ↓
Spotify d6
Misleading near-front
Width-guided
3.05±1.99
1.000
1.27×105±8.86×104
Misleading near-front
Pareto UCB1
2584.77±60.31
1.000
1.67×105±1.10×104
Misleading near-front
Annealing-Pareto
337.59±1504.82
0.983
1.37×105±2.53×104
Misleading near-front
Scalarized UCB
1435.03±258.58
1.000
1.07×105±2.14×104
Friendly near-front
Width-guided
0.12±0.07
1.000
1.40×105±9.26×104
Table 1: Real-world benchmark on held-out Spotify and KuaiRec subsets at T=106 . Subsets are grouped before policy simulation. The main target is Pareto regret and detection of at least one Pareto-optimal arm; fairness is reported as an empirical measure.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 1: Synthetic controlled family at T=106 with 20 repetitions. The first panel decreases the Pareto gap δ with one near-front dominated arm; the second increases the number m of near-front dominated arms at fixed δ=0.02 . The first-certification scale g†=0.55 is fixed throughout.
δ
m
CPUCB
Pareto UCB1
Width-guided
Cert. rate
Panel A: vary δ , m=1
0.120
1
21.05
2490.0±46.8
458.5±27.2
100%
0.080
1
21.99
2564.8±70.3
445.6±28.7
100%
0.040
1
24.81
2773.7±97.3
433.2±26.3
100%
0.020
1
30.45
3048.1±104.4
427.4±23.1
100%
0.010
1
41.72
3185.8±146.7
430.6±30.9
100%
Appendix
Table 2: Synthetic family with fixed certification gap and small Pareto gaps at T=106 ; g†=0.55 throughout and δ=ΔminP . Regret entries report mean ± standard deviation over 20 runs.
Figure 2: Synthetic regret and certification trajectories for two representative settings. Width-guided certifies in every run and then stops accumulating Pareto regret, while Pareto UCB1 continues to sample from a broader optimistic front.
Dataset
Validation seed
Test seed
Validation
Test
Selected c
Spotify d6, misleading/friendly
20260604
20260704
3×3
3×20
0.02
Spotify d6, easy
20260604
20260704
3×3
3×20
0.00894
KuaiRec d26, all types
20260604
20260704
3×3
3×20
0.02
Appendix
Table 3: Validation/test split and selected Width-guided coefficient. “Subsets × seeds” is per dataset and subset type. All held-out rows use the same test seed; Spotify easy uses a separately frozen geometry-selected easy suite.
Dataset
Subset type
Method
Front precision ↑
Front recall ↑
Coverage entropy ↑
Spotify d6
Misleading
Width-guided
0.760
0.947
0.001
Spotify d6
Misleading
Pareto UCB1
1.000
1.000
1.000
Spotify d6
Misleading
Annealing-Pareto
0.782
0.840
0.848
Spotify d6
Misleading
Scalarized UCB
1.000
1.000
0.812
Spotify d6
Friendly
Width-guided
0.850
0.921
0.002
Spotify d6
Friendly
Pareto UCB1
0.971
1.000
1.000
Appendix
Table 4: Front-identification and coverage diagnostics on the held-out real-data subsets.
H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of Technology · Mechanical and Industrial Engineering University of Massachusetts Amherst